Weeknote 5 - 16 April 2021

On introducing agile, single source of truth and responding to more shootings in the US.

Doing

We’ve introduced more agile techniques to the teams running our finance and workforce systems. We aren’t forcing everyone to do the same things but, instead, are trying new ideas when the opportunities arise.

One team has started to adopt a sprint-based^[1] way of working and finished their latest sprint last week. We’ll plan the next sprint on Monday. We’ll need to take account of some absences in the team which will have a big impact on the work we can take on. We have also started to talk about running regular show and share sessions to help everyone keep up to date on the most important bits of work.

Other teams are trying different tools (for instance, building plans in Microsoft Teams or Azure DevOps, or even spreadsheets!) and techniques (e.g. one team is trying Kanban^[2]).

This reactive approach makes sense as the teams are doing such a wide range of work and it isn’t clear what will work best. Our teams operate services, handle user requests and develop new features but the balance of these different types of work varies a lot between teams and over time. For example, some teams have finished their financial year end-processing but for others this continues through April. Some teams only include permanent staff but others also include temporary contractors or partner staff for specific bits of work.

Thinking

All organisations have to manage a lot of data. Councils have to handle data for an unusually large range of services (more than any of the commercial or central government departments I have worked with) but their data management challenges will be familiar to most data professionals.

All the organisations I have worked with are seeking “a single source of the truth”. Taking action based on incorrect or out of date information can be expensive or dangerous so having one obvious place to get the right information is clearly a good thing, isn’t it? Unfortunately, it often isn’t achievable.

One of the problems is economics. A whole series of actions would be needed to reach a single source of the truth. Each action takes us closer by combining databases or increasing data quality but they also consume effort and money. Once the most obvious problem areas have been addressed you will have “a few sources of good-enough data” and there won’t be a strong enough case for spending more time and effort to go further.

The other problem is that the truth might be hard to pin down or might not exist.

There aren’t single, true answers to simple questions like: what is your address? The answer changes over time and for different purposes such as place of birth, delivery address and billing address. For the homeless or people with complex relationships and family lives it is even harder. To confuse matters the question is sometimes a proxy for something else. For example, we sometimes ask about billing address and town of birth to help establish someone’s identity. In these cases, the answers need to be consistent across our interactions with that person; the physical truth doesn’t actually matter and might get in the way.

So let’s have lists of addresses for different purposes, tracked over time and different levels of assurance. Will that suffice? Sadly, not for some types of data. As an extreme example, the relationship between two people is important for some public services. I’ve seen a guide to determine if two people are a couple which runs to 15 pages and even then the official version of the “truth” is subject to appeal and could need to be settled in a court. In other situations, when dealing with a single individual, it might not be legal or even possible to link information obtained through different services into a single source of truth about that person.

In many cases “single source of the truth” is a convenient label for ongoing work to improve data and data management but we shouldn’t fall into the trap of thinking about it too literally. Removing errors and other sorts of waste from our services is actually what we should be putting our scarce effort and money into.

Caring

My professional network is focused on the IT and digital industry so a lot of people reading this are likely to be white men. I hope many are allies and are thinking about how their colleagues might be feeling about recent events. The shocking way Adam Toledo was killed last month has hit the news in the UK and there are also headlines linked to Breonna Taylor and George Floyd who were killed last year. I could view these as just headlines from a distant, troubled country without much impact on me. That is a privileged viewpoint. I’m lucky to feel that I and my loved ones are safe and fairly treated here. That experience isn’t shared by everyone and these events might make others feel upset, angry or frightened.

Some might feel powerless to do much more than be sympathetic but IT and digital professionals can take some small but important actions to counter the systemic bias in our industry. IT is a highly paid profession so we wield a lot of economic power. In addition, the things we build are inceasingly influential but they can perpetuate our own biases. We should be really uncomfortable that the IT industry is so dominated by white men and does not reflect the diversity of the communities we serve. Fortunately, there are things we can do. Last week I joined a group of coaches at another codebar^[3] event to help people from under represented groups start or advance their career in IT. If you don’t feel comfortable being a coding coach there are numerous other groups you could help or support.

Accept you can’t do everything but please try to do something.

Footnotes: