Making the Intangible Tangible: Measuring Culture, Psychological Safety and more…

We know that intentionally shaping organisational culture is critical to achieve meaningful and positive organisational change as discussed in my recent article.

But how do we truly understand the reasons behind numbers and people’s behaviours, feelings and thoughts?

How can we uncover the underlying drivers and preventers within organisations?

To answer these questions, we need to delve into how we effectively can measure social phenomena and concepts¹ like organisational culture, engagement, motivation, psychological safety etc.

Although most organisations I have encountered acknowledge the importance of social measurement, the practical implementation remains a challenge for them. Consequently, many organisations fall into two traps: either avoid social measurement altogether or due to poor research design they generate data that lacks practical value and/or is unreliable and invalid leading to incorrect conclusions.

To address these challenges this article offers practical strategies to help you understand what is happening in your organisation and why through establishing an effective research design specifically focusing on qualitative data.

Create a robust research design

A research design is a plan that outlines how we intend to collect, measure and analyse data. A good research design enables us to get meaningful insights that address the problem/opportunity as clearly, as accurately and as unbiased as possible. It should enhance the validity and reliability of our findings. In the case of measuring organisational culture it means that the research design needs to provide us with a roadmap that enables us to understand the behaviours of employees.

Key components of a research design

Establish a clear & shared purpose

The foundation of effective measurement lies in clearly defining why and what we intend to measure. This clarity directly informs our research design, making our efforts purposeful and impactful. Conversely, a lack of clear purpose can lead to the use of inappropriate metrics and unintended negative consequences. When dealing with complex social concepts such as culture, engagement and psychological safety, establishing a clear and shared understanding of what we mean is essential as there are no single universal definitions in social research. Defining a concept like culture is thus a crucial first step in making the otherwise abstract tangible and measurable.

Measuring human behaviour is an incredibly powerful tool when done right, however, potentially harmful if done incorrectly. You will find an overview common risks associated with measuring human behaviour and how to mitigate them in this article.

Determine the appropriate sample size

Determining the appropriate sample size is crucial to achieve in-depth findings and draw valid conclusions. In other words, we need to identify who and how many participants, data points, and/or observations etc that we need. There is no single right answer to what is an appropriate sample size as it is determined by the epistemological, methodological and practical context of the study. However, data saturation is a good indicator for an adequate sample size, which means that there is no new information being obtained from the data. While aiming for a sample that accurately represents what we are studying, we also need to consider what is feasible and practical within our constraints. For instance, when examining organisational culture we might not be able to observe everyone in the company but we might be able to survey everyone and interview selected representatives. A small pilot study is often helpful to refine the sample size estimates before undertaking a larger study.

Define the data collection method

If we only get a pulse check when measuring concepts like culture e.g. employees perceive the culture to be a 5 out of 10 does not help us to understand why that is and what we can do to improve it. As such, we can use quantitative methods to understand the status of something but not the reason behind the status and how to improve it. For that we need qualitative methods. They are essential to explore the fundamental root causes that help inform strategies that drive positive change.

Unlike non-grounded theory, which tests pre-existing ideas, grounded theory's inductive nature is useful to understand deep context-specific aspects such as company culture. It allows us to develop new insights through the ongoing collection and analysis of data. Therefore, studying organisational culture within a single company can be effectively done using the flexible grounded theory approach.

When measuring concepts like organisational culture it is recommended if possible to apply a combination of methods such as surveys, interviews and observations to ensure a holistic understanding.

When defining the data collection method it is important to consider what tools that will be used (e.g. survey tool). How and by whom will these tools be managed and how will we ensure that data access is restricted to relevant personnel. Finally, time frame and frequency are other important considerations when defining how data is collected.

While surveys are prevalent in organisations, the design of them often hinders the collection of useful data. The list below offers recommendations for designing surveys that enable organisations to understand the underlying ‘why’ behind numbers.

Survey design tips

Pilot survey to gather feedback and test the data for validity and reliability issues.

Fewer questions might increase the response rate but provide fewer insights. More questions provide greater and more granular insights, but might decrease response rate. Finding the right balance is key.
Make sure to address usability and inclusivity: consider language, disabilities and so on)
Open ended questions (especially exploratory such as how and why) allow for in depth feedback and understanding of participants’ attitudes, experiences and actions. This is crucial when measuring concepts like culture.
Minimise social desirability bias by asking non leading, neutral questions, and don’t share directly what you are researching but provide enough information to guide people through the survey.
Only ask one question at a time (avoid double, triple barreled questions) and use clear and simple language to ensure valid data and avoid survey fatigue.
Ensure anonymity by avoiding questions that prompt identification, secure who has access to the data and remove any personal information from data before analysis if respondents have added it. Explicitly state in the survey introduction that responses will be anonymous.
Don’t force people to answer questions to avoid invalid data.

Outline the analytical approach

Coding is the core part of a qualitative analysis: a process by which the researcher systematically categorises qualitative data such as open ended survey response and transcripts in order to find themes, meanings and patterns to generate useful insights about a social phenomenon and/or concept. While there are many sub coding approaches that can be applied depending on what we are studying, the below table outlines the key analytical steps based on the grounded theory approach.

In addition to pen and paper there are various online tools such as Dedoose and Delve that can help us in our qualitative data analysis. Visual collaboration platforms such as Miro can also be a helpful starting point.

The iterative approach of collecting and analysing qualitative data.

Ensure measurement quality

Just like models, no measure is exact but some measures are more accurate than others. There are a lot of things we can do to increase the quality of our measurement. Below table outlines the key quality criteria lack validity and reliability.

Conclusion

The ability to effectively measure social phenomena and concepts like organisational culture is a powerful tool in achieving sustainable long term organisational success.

Through quality research design organisations can gain invaluable insights into their culture that enable them to drive positive change. This deeper understanding of the otherwise ‘hidden’ why behind numbers, people’s behaviours, feelings, and perspectives provides companies with the power to effectively shape not only their culture but also other critical aspects of human behavior within the organisation.

Footnotes

Behavioural social phenomena are referred to as the effects of actions of other people that influence while concepts are the tools we use to think about and understand those behaviours. As such, culture can be understood as both a social phenomenon and a concept.