Evaluation of Organisational data culture

 Data Culture:

A culture is a set of shared ideals that promotes certain behaviours and discourages certain other behaviours. It defines what is acceptable behaviour as a whole for a group of people. In data culture, this then defines how data is produced, stored, processed and consumed. 

The three major stakeholders of any data culture then are the producers, consumers and the regulators. If this sounds similar to a marketplace it is because this is in many ways similar and the forces that act here are not too different from those that apply in a marketplace. The producers in a data culture are the IT team, the consumers are the business team and the regulators here are the executives. 

Taking the example of market further and focussing on farming as a focus areas, we see that the executives play the role of regulators by providing right incentives and sponsoring long gestation projects, the producers (business team) create the demand through their tastes and preferences and ability to consume the produce and the creators  (IT) team develops abilities and infrastructure to cater to the demand.

Various categories of data cultures:

Consumer lead (Business lead) data culture: In these cultures, the business teams have native BI&A teams, often different business teams having their own BI&A teams. This generally leads to a division of responsibility where the IT responsibility is limited to upstream data sources, generally managing data lakes, data marts or data warehouses. The focus of captive BI&A teams are on business problems and have limited technical capabilities.

The advantage of this model is that for the problems predefined, you can ensure very high quality, speed and manageability. 

On the other hand the disadvantage can be the incremental problems which business would want to address. This will be delayed and potentially causing complaints related to unavailability of data to make decisions.

Producer lead (IT lead) data culture: In such an environment centralisation of data planning occurs. The BI&A teams have to plan for potential requirements from business and build capabilities and infrastructure that is common thus building efficiencies by sharing. The focus here is on efficiency, single version of truth and technologies used to implement them.

The advantage of this model is that the business gets a variety of data at a quicker pace.
The accompanying disadvantage is different teams looking at different data and potentially coming to conflicting conclusions.

Consumer owned and Producer lead data culture: This is the structure where division of work happens keeping in mind the flexibility requirements of business and technical considerations of IT. This means dividing the problem into two sets, a core aspect managed by IT, which is still developed based on solutions where the second part of the capacity facing the business provides required flexibility. This is called Managed self service solutions. Here the managed part refers to IT managing the core data, and self service enabling business to easily get any data required.

The advantage here is that it provides consistent data at a relatively quicker pace.

However the disadvantage is that learning is more. The IT team has to learn business and the Business team needs to learn the basics of IT tools. This is a more difficult culture to develop with much larger investment required from the executive team on training and laying out guidelines of responsibility.

Evaluation of data cultures:

The three cultures are a rough history of evolution of data teams. 

The first phase of this evolution was based on Business lead data culture. In this phase the data volumes were lower and IT tools were not that developed. An excel sheet was generally sufficient to do one's analysis. This led to demand for more data and specialised and efficient tools to manage it, bringing about the second phase.

As there was an increase of data and IT tools increased in complexity, there was a need for dedicated professionals to manage them. This brought the shift towards standardization with concepts like data warehouses, ETL and reporting and analytics tools. The success of this phase increased the demand for data further and business decisions started using data extensively, thus triggering the third phase.

In the age of Managed self service business was expected to be good with analysis and IT was expected to keep an eye on business developments. This meant closer collaboration, with IT being involved in business meetings as an observer and Business teams taking training to be able to do their own analysis. This need for great agility and variety of data led to a boom in big data, python, and cloud computing etc. This is the phase where most of the organisations currently are. With all stakeholders in the data ecosystem requiring a larger skill set this brought an increased focus on training and knowledge sharing.

This brings our focus to the cornerstones of the Business led IT owned data culture: knowledge sharing and training. The standard tools involved for implementing thisa are-

1. Business focus for all teams: All the employees of an organisation should have the same focus. Companies having great technology can not succeed without great business to back them up. IT teams need to align themselves with the larger goals of the organisation. This starts with larger participation in overall goals, identifying ways to contribute to the goals, and experimenting with ways to deliver better value. 

This translates into simple things like participating in business focussed meetings, IT teams creating robust channels of communication and feedback, and business teams being proactive in sharing their objectives and plans and bringing IT on board early on.

2. Regular IT lead training sessions: as technologies and resources change, there is a need to regularly refresh Business skills on using the tools necessary to consume data effectively. Regular sessions at predefined intervals to spread this knowledge ensures that lack of skill does not create bottlenecks and keeps the process efficient.

3. Community: To create a culture there is a need for leaders. In adoption of any change, there will be early adopters, followers and late adopters generally following a bell curve. To take maximum advantage of early adopters, there should be portals where consumers of IT infrastructure can reach out and post their questions. These questions will often require the expertise of the IT team, but in a lot of cases, existing and exciting use cases that business has come up with can be shared with the audience. 

A portal where consumers can reach out to get their answers also boosts the confidence of the business consumers that they will not have to run around to get replies. Additionally, based on frequently asked questions, standard documents can be developed to cater especially to these.

4. Executive sponsorship: All cultural behaviours need positive and negative feedback loops. There is always resistance to change and even degradation to a normal state which may not be most productive. Therefore, a culture needs thorough understanding of vision, roadblocks and required incentives. Accordingly long term changes must be sponsored by the executive teams to put in the required investment before the results start showing up.

A well defined data culture in a team with proper nurturing can go a long way in solving a lot of long term data issues. Each model has its advantages and disadvantages. But a metric understanding of what is being implemented and promoted also provides a good understanding of where one stands exactly and what pivot may be required with change in an organisational needs.