probablyisjordan

[probablyisjordan]

It's 2015 and we still can't reliably run screens

Barely a day goes by when I don't spot a broken screen somewhere in London. I find it incredible that this still happens so much. We can deploy and operate infrastructure better than ever before, but we still struggle to keep a screen displaying a slideshow or a dashboard up and running.

A short while ago, Matt Webb and I spent some time looking at this problem within the context of government. The GDS office has tons of screens, and central government must have thousands, displaying performance data, meeting room bookings, or promotional material. Across wider government, there must be tens of thousands: in job centres, driving test centres, at the border, and elsewhere.

Usually, each screen is run by a standalone computer at best - or expensive specialist hardware at worst - and we only know if it goes down if someone spots the screen is off (and knows how to fix it). Even if the screen's on, we can't be sure that there isn't a software update dialog blocking the screen, if the screen saver's kicked in, or whether the device is still connected to the internet at all.

Displaying screens feels like something that should be provided by commodity technology already. Given your dashboard or display is hosted somewhere on the internet, it should be really damn easy to get a device which boots up out-of-the-box into a web browser, discovers what it should show, and pings a monitoring service to let it know its online.

In lieu of such a thing appearing to exist, we decided to prototype one. Matt's been doing a broader piece of work into how IoT concepts could be used in government, and it appeared a good fit. We used some Raspberry Pis and a clever online service called Resin, which allows you to easily deploy containerised applications across a fleet of small devices. I built an online service for the devices to talk to, and a web-based control panel where you can configure each screen and see which ones are online.

Given an SD card with the Resin base image, a new Raspberry Pi will start up, install our custom client application, run Chromium and then appear in the control panel ready to be configured (pictured above). A user can then choose where they want to point the screen to, and it will update automatically.

We had a bunch of problems with wifi, and some performance issues when we tried some of our more-animated performance dashboards, but it feels like there's definitely potential here to be explored.

I've posted the source code for our client app and web service prototypes on GitHub. I'm not sure what's happening next with the idea, but there's a few people at GDS interested in taking it forward. I'd also love to find out where others have tried to solve this problem.

[probablyisjordan]

Week 88

First week notes of the year. Happy new year!

This month, GOV.UK is having a 'firebreak'. I'm not sure where the name came from, but it's a really great idea. Most of us have downed tools for a month and are spending the time fixing tech debt, testing parts of our infrastructure and prototyping some new ideas. This week, I've been working with Mark, Charlotte, Vicky and Gaynor. We've been exploring how we can better model services on GOV.UK.

At the moment, content on GOV.UK is collected together in buckets of vaguely related things, such as this browse page about having children. We currently don't connect up all the content for a single service (eg. child tax credits), nor do we provide a high-level view of all the services which you might need as you go through a particular phase in life (eg. childcare).

We want to make this better. We first looked at all the mainstream content on GOV.UK relating to childcare and organised it into the phase at which a user would need the content (from pregnancy to schools). I then tried experimenting with a few different models for an individual service, using content about child benefit and child maintenance. Mark made it a bit simpler and we arrived on this model.

This was actually my final week on the GOV.UK team. From next week, I'm going to be working on the Transformation team at GDS, working with other departments building services. Due to sensitivity around the project I'm working on, I think it's going to be difficult to carry on writing week notes, but I'll see how it goes.

It was sad to see a few people leave GDS this week. Ralph, Camille and Steve L are all moving on - they'll be missed. Wishing all of you the best for whatever you're doing next.

[probablyisjordan]

Week 87

We released email alerts and the latest changes feed for sub-topics on Tuesday. For example, take a look at the most recent changes to content about PAYE. It was great to be able to hit our deadline on time, and for everything to be released without a hitch (we like making releases boring).

We also finished another transition blocker this week: merging the Schools and Colleges topic into the Children's Services topic, creating a new topic: Schools, colleges and children's services. This unblocked the transition of Ofqual to GOV.UK, and will also allow us to turn the Services and information page on soon for the Department for Education.

We've spent the rest of the week picking up fixes, writing documentation, and generally tidying things up. We've also been planning how we could build end-to-end monitoring for email alerts, to give us greater visibility on whether the entire email alert stack is working.

Finally, I wrote a blog post this week about the code club we've been running at GDS, and (more importantly) why it's important to have a basic understanding of code at all. It's live on the GDS blog, have a read.

[probablyisjordan]

Week 86

On Search and Browse this week, we've been ironing out a few bugs with email alerts and the latest feed. We're planning to turn both features on this coming week to support the HM Revenue and Customs transition to GOV.UK. However, every sub-topic will get them at the same time, so if you're interested in motorways, consumer protection or even large yachts, then real soon you'll be able to subscribe to receive any updates by email.

We've also spent much of the week writing reusable scripts to make it easier to retag content to different topics. We're doing this because we need to merge two large education topics together to support the Ofqual transition. We'll be finishing that up in the next few days and, now that we have the scripts, in the case that we don't have capacity to perform similar changes in future, other product teams will be better placed to help out.

On Monday, Giles interviewed me about some of the history of GDS. It was a good time to reflect on a busy three years. We noted that the organisation is so big now that we hardly know what we're all doing, and were proven right when we left the meeting room to stumble across a wall about the history of GDS which another team had been putting together.

This week's Leadership Academy session was really good. The theme was communication and there were some good things about diplomatic language, body language, writing good feedback, and asking the right questions. We also had a good discussion about digital etiquette.

And I went to the Science Museum's monthly Lates event on Wednesday. The event was brilliant (and so is the new Information Age bit).

[probablyisjordan]

Week 85

It's been a busy week.

We had our sprint planning for our final sprint before our deadline for the HM Revenue and Customs transition to GOV.UK. Whilst HMRC won't be fully transitioning for a little while, we're aiming to finish up email alerts and the latest feed by December 2nd so that we have enough time for a gradual release.

I've picked up a few stories this week: I've added links through to the email subscribe page, and the latest feed, from the sub-topic pages. I worked with Guy to tidy up the design and copy on the subscribe page, to be clearer to users about what they should expect to receive if they sign up for emails. I've also updated Rails in a lot of our apps, as I noticed a few had fallen behind a bit.

We had the third code club on Thursday: we looked at the <div>, <span> and <table> elements, and then everybody made their own GOV.UK calendar page. A few people have also started making their own websites outside of the weekly sessions, so we had a show-and-tell at the end.

And I went to the Good Law Hackathon on Saturday, at the Ministry of Justice. It was a really interesting hack day - I ended up making something to analyse the occurrences of duties and powers across the statute book. If you missed it, there's a good recap of the event on Storify.

[probablyisjordan]

Week 84

The second week in my flat has felt a bit more homely than the first. I've now moved in a lot of my things and my parents came and helped fix a bunch of things that needed doing.

On Search and Browse this week, I finished the work to add pagination to our new "latest changes" list for a sub-topic. (It's felt like the work was a lot more complex than it should have been.) As we're quite close to our deadline for releasing this work, along with the email alerts feature, I put up a 'skirmish wall' on Friday to help focus the team on the things we need to do.

We ran the second session of GDS' code club on Thursday. We had a good turnout again, and at the end of the session everyone was building their own GOV.UK organisation page. I'm really pleased with the format and I'm hoping it continues through the remaining four weeks.

And along with a cohort of GDS folks, I went to the Houses of Parliament on Wednesday for a guided tour. It's not my first time looking around Parliament, but I always find it fascinating to be able to walk through the House of Commons and the House of Lords and see all the tradition that still remains.

[probablyisjordan]

Week 83

I'll start with the big news: I've moved to London! It's only taken me three years of commuting to finally get a flat here. I've spent evenings this week visiting IKEA, buying towels, building furniture, and realising just how much I don't know about living.

I've been picking up a variety of things on Search and Browse this week: I fixed a bug in our new email alerts where links to changed content would link to the GOV.UK homepage, rather than the piece of content itself. I paired with Richard on a story to step through the email alerts stack, working out the potential failure states at each point. Richard's now writing up some stories for monitoring or mitigating each of these. I finished the week picking up a story to add pagination to our new feed of latest changes to content in a sub-topic.

We ran our first code club at GDS on Thursday evening. I was really pleased with how it went: it felt like we had a good atmosphere where people were asking lots of questions. We're teaching HTML and CSS - mainly building on the Codecademy course - and doing additional work to link this back into GOV.UK. Massive thanks to Rob, Barrie and Alex for helping out.

And I went along to the 'Leadership Academy' on Tuesday afternoon. It was the first of six sessions about leadership, being ran by the heads of the Efficiency and Reform Group. I was pleased that there were other GDS people there (about half the people in the room were GDS folks).

[probablyisjordan]

Week 82

I've spent most of my week continuing to build out the new tag management interface in the Collections Publisher app. I've been pairing with Richard a bit, which has been nice.

On Friday, Dai, Matt and I went to All Your Base 2014, a conference in Oxford about data and databases.

My favourite talk was Jeni Tennison's keynote about an impending Data Reformation analogous to the Protestant Reformation, made possible not by the dissemination of scriptures, but open data. I found her thoughts on equality interesting: in a world where people are empowered by access to data, the 1% are the people who know how to analyze and make things from it, and we have a duty to make it easier for everyone else to be able to.

I also enjoyed Laura Thompson's talk on how Mozilla monitor their complex infrastructure. It had a good run-down of what you may want to monitor for each function of your service, and had some good comments on how services should be designed degrade gracefully in response to issues, so that the critical core functionality keeps running. It's probably a good primer to anyone new to monitoring - worth a watch when the videos go online.

I was impressed at the diversity of the speaker line-up. As a community we mainly seem to call out bad examples of speaker diversity, so it's worth recognising when a conference does this well.

I'm off on holiday mid-next week, so there won't be week notes until November.

[probablyisjordan]

Week 81

We had our sprint planning and retrospective on Tuesday. Our team focus is still on building email alerts for subtopics, and everything that's needed to support it.

I've spent most of the week building out new functionality to manage tags in Collections Publisher. Whenever a tag is created or updated, Panopticon has to be kept up-to-date. Eventually, we'll also publish these tags into the content store. It's a huge story, and it's going to take quite a while.

I also picked up a small change to put gov.uk/coal live as a topic page, similar to gov.uk/transport from last week. I love the idea of top-level topic pages, but I'm a bit concerned that they're too focussed on specialist content, and not very useful to 'mainstream' users.

The progress on the GDS code club moved one step further this week: after looking at a handful of different online courses to use for the club, we've decided to go with Codecademy's HTML/CSS course. We'll use the session time to help people through the online lessons, or provide some extension activities if they've already completed it.

I finished reading Hunter S. Thompson's Fear and Loathing on the Campaign Trail '72 this week. I've been on-and-off with it all year, but glad I stuck it out. It's a brilliantly cynical account of the 1972 presidential election between Nixon and McGovern. It's a must-read if you're interested in American politics.

Matt kindly gave me a copy of Laurie Penny's Unspeakable Things, which I've started reading. (I've got such a backlog of unstarted books that I simply have to start reading more regularly.)

[probablyisjordan]

Week 80

At the start of the week, I made some changes to the Competition topic which had been requested by the Competition and Markets Authority. I also made a few changes so that gov.uk/transport is now also a topic, and not a redirect.

On our publishing tools this week, I've changed Panopticon so that we'll be able to make changes to tags through JSON requests - part of a broader piece of work to move the UI for tag creation out into Collections Publisher. To help out the team working on User Needs, who want to "validate" the quality of a couple of thousand needs in the Need API, I built a thing that's a bit reminiscent of the review-o-matic.

I organised our first meeting about the GDS Code Club this week. We decided what's in, what's out and what our next steps are. Since none of us have much free time at the moment, we're keen on picking a good existing HTML/CSS/JS course and running weekly sessions to help people through it.

And on Thursday, we were visited by the Ordnance Survey, who talked about some of the mapping services and APIs they provide - and we had an interesting chat about how their PSMA licensing scheme applies when we build APIs using OS data.

[probablyisjordan]

Week 79

Short notes this week.

I spent the start of the week prepping for planning, as we've just finished a slower sprint where we spent most of our time ensuring our upcoming work is in order.

I picked up a small bit of work to stop the tags for content published in the Whitehall publisher from being changed in Panopticon. We've got plans to do the same for content published in Mainstream publisher, once we build functionality for content to be tagged in the publishing tool itself. Both these changes will make the transition to the Content Store easier.

In the latter half of the week, I attended the Bertelsmann Creativity & Innovation conference, at the St Pancras Renaissance Hotel. I was part of a panel on Thursday afternoon with Zach Sims, Lily Cole and Nick D'Aloisio. Martha hosted it, and we were joined on stage by will.i.am at the end. It was the first panel I've taken part in and I thought it went quite well.

[probablyisjordan]

Week 78

We continued our planning work this week. Now that we have a good sense of what we should do next, we took each big 'feature', dug into the detail of each one, and wrote rough stories. By doing this, we hope to make the team more resilient: if any unexpected blockers arise on a piece of work, we still have other big things we can pick up.

I also carried on tidying up our team wiki. Confluence is terrible and breaks in bizarre ways. Despite this, all the pages for old features are now gone, and I've organised all the things we're working on into themes of work, giving a stronger narrative to our team. Ideally, this should help limit future work too, because if a feature doesn't fit into one of the following, then it's probably not something we should be doing:

Remove detailed guide categories and policy areas

Build a complete search index

Improve the quality of search results

Make search easier to use

Simplify navigation across GOV.UK

I've had a few conversations this week about the GDS Code Club. It's about time we get things moving - we held out longer than we planned in the hope we could get some more people signing up to help out, but it hasn't really happened. I'm fortunate to have been sent the feedback from the pilot of the Cabinet Office Code Club that took place earlier this year, which should help us avoid some easy mistakes.

We continued our work to iterate policies this week. To a group of people from across GOV.UK, I showed the prototypes I made last week and we talked more about what we know about the current policies format and who's using it (short answer: not much). If you use policies on GOV.UK, we'd like to talk with you, so please do get in touch.

[probablyisjordan]

Week 77

Our team spent most of this week re-prioritising, working out what the most important things are from our roadmap. I think it was a worthwhile effort and has helped us to get a good perspective of what we should be doing next.

I used that as a chance to clean up our team wiki a bit too - the IA was a bit confusing and we still had a lot of pages around for "features" planned very early this year but which have since been consolidated, scrapped, or picked up by other teams.

I made a couple of prototypes of what policies could look like in the future, after the discussion we had last week. I'm experimenting with the idea of splitting apart the intent of the government from the actual projects and programmes that the government has committed to doing.

And on Thursday, I went to Brussels to meet with Neelie Kroes. We talked about the Young Advisors Group and Code Week, and I gave some feedback on the group which I hope will help to shape it in the next Commission.

[probablyisjordan]

Week 76

I spent a lot of this week interviewing candidates for the junior developer roles we advertised a while back. It's the first time I've been on an interview panel, and I'm finding it really interesting.

In addition, I've had chance to pick up a couple of Search and Browse stories this week. I cleaned up the remnants of an old specialist sector in the search index, which will unblock the 'Services and information' page for the Marine Management Organisation going live. I also made a change to Panopticon so that it won't delete routes for content owned by the Whitehall app. This fixes a problem where an "unpublished" detailed guide showed a generic "gone" page instead of a custom note or redirect (eg. this "unpublished" guide).

We had another chat about the Content Store this week, and how it should handle tags. We've still got some unanswered questions around how curated lists of content might work and how we can link between content in a way that's resilient to changes in URLs. However, it's good to get more of this nailed down.

On Friday, Neil, Ben, Graham and I sat down to talk about the policies format and how it could look in the future. We know from analytics that it isn't working very well right now, and many of the existing policies will change in the next parliament (regardless of the outcome of the 2015 election). It also seems like there's some good opportunities to join up the work we've been doing building specialist sectors (which we're renaming to "topics"). Watch this space!

And this week saw the return of floor meetings to the GOV.UK Team. Floor meetings were a thing we used to do as an entire organisation until we grew too big for one floor. It's nice and informal and I think it's really good to bring people together under the guise of being part of the same team, which is something we've lacked a bit lately.

[probablyisjordan]

Week 75

With the bank holiday, it's been a short week.

We had our sprint planning on Tuesday. This sprint is mainly composed of lots of smaller stories, or "sand".

One of those stories was to include the page for the government's new Unimoney campaign in search, as we don't index campaign pages by default. The other story has been to investigate why a deleted specialist sector is still hanging around in the search index.

On Thursday, I was interviewing candidates for the new junior developer roles, with Rob and Brad. We're interviewing a lot more next week.

[probablyisjordan]

Interlude

I've not written week notes for two weeks. I burned out a bit at work, so I took a week off to do other things.

I went to Edinburgh for the Fringe festival, for the first time. My favourites were The Capone Trilogy, The Picture of Dorian Gray, and Sh*t-Faced Shakespeare.

Over the Bank Holiday weekend, I visited the Black Country and Gloucestershire, spending time with friends.

Also, I did write a little bit about my first week at GDS.

Normal service resumes this week. :)

[probablyisjordan]

My first week at GDS

Recently, Alice wrote about her first six months at the Government Digital Service. Reading it made me think back to my own first week.

GDS was my first real job: I'd never worked in an office before. There was about thirty or forty of us in early November 2011.

Until early this year, I kept a journal. I read back through to the things my 17-year-old self wrote at the time. I thought I'd publish them here.

Here's my first day:

I caught a very early train down to London this morning, and then headed straight to Holborn to Aviation House, where I got an ID card and met with James Stewart. He took me up to the office and I met everyone, and everyone was really friendly!

Mark O'Neil recognised me and gave me a pack of chocolate biscuits :-) and me, richard and james had a chat about the app architecture and things. For lunch a few people asked me to come with them to a vegetarian restaurant in covent garden which was really nice! Had like a jalfrezi meal :-) had a lovely chat with jamie arnold too!

This afternoon I got loads of work done on the calendars, ran into a few bugs though, and left about half 5.

As the week went on, I carried on working on the Calendars app:

Today i expanded the calendars to support other calendar scopes eg daylight saving. Richard was pleased :-)

By Thursday, I started talking with Gareth and Dai:

At work, finished calendars, spoke a lot to Gareth and Dafyydd and they were really nice and helpful! Also got to use the Done stamp :-)

And then Friday finished with a pub lunch:

For lunch, we went to a pub nearby and had ham, egg and chips, with Richard, James, Dave, Dafydd, Paul and a couple more :-)

This afternoon we did a show-and-tell for the end of the sprint, and I showed off my calendars app. It went down really well and I felt really proud :-)

We did a retrospective afterwards, and Josh and Dave put post-it notes up on the wall saying that me joining the team was something that went well this sprint :-) Josh said he likes me! :-)

I headed back to Euston and had a lovely train journey back up north! And got a tweet from the product lead, Tom Loosemore, saying he heard I 'delivered' on the project and 'nice job'.

I've been at GDS for more than two-and-a-half years now. In that time, we've changed a lot. But I'm glad that we still do a lot of the things that we did on my first week, like team lunches, show-and-tell and using the 'Done' stamp.