Dr Sylvia Dinius shares with us a promising approach to measure behavioural changes
As a data geek, I have always subscribed to the notion that data is king and in my work as an evaluator I hold fervently to the mantra that “the data must comes first before the story”. However, my work in education transformation involves changing human behaviors and gathering data to show changes in behavior that is often intangible proves to be challenging if not impossible. As William Bruce Cameron (not Albert Einstein) said: “Not everything that can be counted counts, and not everything that counts can be counted.” To manage effectively, you need to measure the right things: Those things that provide insight in the work performed, and that you are able to influence.
This drove me to investigate possible measurement for evaluating behavioral change and one of the most promising approach I came across is the most significant change stories (MSC) method developed by Rick Davies. I feel that this approach that often gives opportunity for beneficiaries who are from the vulnerable and marginalised group to participate in evaluation exercises is able to provide insights into changes in human behavior better than most quantitative tool can.
MSC subscribes to the constructivist perspective, which is concerned with learning in complex nonlinear environments by eliciting stakeholder voices using participatory methods (Armytage, 2011)1. Exponents of this approach, who are mostly concerned with accountability, will reinforce calls for more scientific and precise measurement involving more use of quantitative data due to their perceived greater validity and reliability.
Although some evaluators may view these two perspectives as binary, my view is that it should be complimentary because stories gathered through MSC coupled with good robust quantitative data as evidence will further strengthen the validity of our findings. At the end of the day, we must remember that evaluation is a human activity and measuring changes in human behavior is never easy. As such, it is important for us to be open to new possibilities of measurement and MSC is one of them.
References
- 1 Armytage, L. (2011). Evaluating aid: An adolescent domain of practice. Evaluation, 17(3), 261-276.
- You can read the summary of the MSC approach here: https://www.intrac.org/wpcms/wp-content/uploads/2017/01/Most-significant-change.pdf
- To better understand how MSC is operationalised on the ground you can watch this video (YT)