Buried at the end of a paper from 1968 on how committees work, is one of the most important and influential statements on how companies design, develop and manage software projects, writes James Campanini, VP and GM EMEA, Sumo Logic.
Conway’s Law states the following: “Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization’s communication structure.”
Since then, it has been a core principle for software development teams to understand; Microsoft Research, for example, analysing the impact that a company’s structure has on its development work around Windows Vista.
Over the past few years, data projects have become important to companies of all sizes. How company data is handled and managed can have a significant impact on decisions that are made, and data goes through a similar development and management process to software. So will companies see Conway’s Law affect their approach to data too, or will this expanding role for data lead to companies operating in a different way?
Conway’s Law, Software Development and Understanding Communications
To start with, it’s important to understand what Conway’s Law means in practice. For businesses that have rigid hierarchies and centralised command and control infrastructures for their communications, software projects should follow suit. In practice, this would mean that all software development work would be based on hard rules on how projects get processed and how these business requirements are turned into new software.
This centralised approach would lead to well defined software and IT projects at the start, but would also run the risk of being slow to develop and get signed off. While traditional IT development processes based on Waterfall methodologies would continue to function well under this framework, the degree of control means that it can be harder to support innovation.
Conversely, organisations with very open infrastructures and flat hierarchies tend to be more happy to use DevOps methods and agile development to deliver software more quickly. When companies are open to communication between teams, they should also be more open to collaboration and fast delivery of software.
In practice, this openness should lead to faster software delivery across projects. However, it can risk teams developing what they are interested in rather than what the business needs or not standardising on what is required. This can itself create problems over time.
For software development teams, understanding how communication can shape activities as it takes place is a necessary ongoing consideration. Getting to the right mix of collaboration, control and supportiveness is a delicate balancing act.
The Data Science Pipeline – will Conway’s Law Still Apply?
Today, many companies are investing in data and analytics projects as part of their digital transformation initiatives. Gartner estimates that, by 2022, 90 percent of corporate strategies will call out information as a critical enterprise asset while analytics will be viewed as a similarly essential competency. Data has become critical to how businesses see their operations and running them differently.
Underneath this, companies are investing in data science teams to prepare and analyse all this data over time. From raw data coming in via multiple devices or applications through to analysed data presented back to the business via dashboards, this information will go through a pipeline of tools, analytics and cleansing in order to be useful. This process – taking raw data and making it useful to the business – relies on communication and conversation between teams to work.
So will these processes around data be subject to Conway’s Law too? Or can companies see data differently?
For companies with rigid communications hierarchies, will there be a risk that data will be only used when it reflects the existing preoccupations of the business? For companies with flat hierarchies and open communications approaches, will data be limited or only used in specific teams? Or will a third approach develop?
To answer this, it’s worth looking at data science in more detail. Data science follows the scientific method, where experiments are designed, built and tested to see how well they reflect the reality of a situation. These algorithms are then used on new sets of data to check how well they perform and what recommendations they can provide. Over time, the algorithms improve and provide better responses, which can then be used to improve decision making.
Because the algorithms and results are checked against data – and because the algorithms are continuously updated – these data projects should be less influenced by existing communications structures compared to software development. By bringing data sets from different applications and making it useful to different business teams, the emphasis should be on what the data has to say, rather than the specific communications approaches involved within a company. By seeing what the data has to say – rather than what we want the data to say – we should get more objective and better quality results.
However, the results of these data science projects still have to get shared out and used by the rest of the business. It’s here where the results of all this analytics work can be subject to Conway’s Law – are they shared and made available to everyone across a business? Or are they the preserve of a few people within the company? Do the results get shared out in ways that people can make use of and understand in their own ways, or are they instead sent out as static documents that can’t be changed?
Changing our Approaches Around Data
For the majority of companies, data should be a critical element in how they run their operations in the future. With information coming in from applications, services and infrastructure continuously, companies should now be able to get continuous intelligence on their operations. However, while this data should be made available to everyone, it is not often that this actually happens in practice. To break this cycle will involve making data pervasive within an organisation.
In practice, this means that everyone across a business should be able to get the results of data to use for their own purposes. Rather than relying on a central data science team to provide results – or for a centralised communications operation to deign to share out necessary data – every employee should be given access to data as it makes sense to them. Seeing data in this way is fundamental to new companies that want to compete against existing market leaders, and for those established brands threatened by newer startups entering their markets alike.
This pervasive adoption can be hugely positive for companies – for example, Samsung SmartThings has extended analytics around data to 95 percent of its staff in order for them to make better decisions. This implementation started with the company’s Internet of Things products, but the machine data generated here also gets analysed and used for a range of tasks across the business, from customer service, product insights, partner health metrics through to other operational tasks.
As more companies look at how to make the most of data across their operations, they will all have to consider how their existing communications strategies can affect these processes over time. Conway’s Law points to how businesses can affect their software development processes both positively and negatively through their communications models; understanding this can help data science projects be more influential and more successful over time.
See Also: Database Diversity: The Dirt, the Data
