MES Blog

Malaysian Evaluation Society

By: Lim Kheng Joo, MES Adviser and Senior Associate at CeDRE International

I had the opportunity to attend the inaugural Asia-Pacific Evaluation Association (APEA) International Evaluation Conference which was conducted over five days from 21-25 November, 2016 in Hanoi, the capital city of Vietnam.  The theme of the Conference was "SDGs: Making a Difference Through Evaluation".  There was the usual conference sessions during the first two days which were then followed by another two days of post-conference workshops.  I would say the highlight of the conference was the final day Seminar titled as the “The Great Evaluation Debate” (GED).  The GED was specially tailored and geared towards identifying issues and challenges related to the institutionalization of evaluation in developing countries. As attendance at this special event was by invitation only, I would like to share some key takeaways with those who missed the GED.

Now, here are my six (6) key takeaways from the GED 2016.

1. Necessary to have a national evaluation policy but not necessarily be legislated.

There was much debate on the necessity of developing countries having a national legislation on evaluation in order to ensure institutionalisation of evaluation. Arguments against such a move drew examples of developed countries e.g. USA, Canada and European countries, not having any legislation on evaluation and yet evaluation is a culture flourished. On the other hand, those in favour argued that developing countries lack discipline and political will to carry out development programme evaluation and therefore, legislation is the best prescription.  The agreed position is that evaluation policy is necessary and need to be embedded into other existing national policies and regulations and the legislation of evaluation policy is an option for each developing country taking into context respective local environment and cultural factors.

2. Right strategy with right target group yields the right results.

The current strategy of VOPEs to reach out to Parliamentarians of respective nations to create the demand for evaluation may not be effective. Efforts to engage and educate Parliamentarians on the importance of evaluation in order to effect the legislation of national evaluation policy may not yield the desired results as Parliamentarians have many agendas on their minds. As they are answerable to their constituents who voted them into the national legislative assemblies, they naturally would pay more attention and listen to the demands of voters.  Notwithstanding the constant engagement and advocacy with Parliamentarians, it would be advisable for national VOPEs to also reach out, engage and educate members of local communities in evaluation capacity development so that they can become powerful allies in compelling their representatives to demand for evaluation in order to ensure delivery of desired programme results.

3. Evaluation is everybody’s business!

The demand for evaluation does not belong to the international donors/funding agencies nor does it belongs to the national governments. Similarly, national VOPE is not the custodian   of the supply of evaluation. It is not an issue of public sector versus civil societies. The professionalization, mainstreaming and institutionalisation of evaluation in a sustainable manner is everybody’s business. Addressing issues related to both the demand and supply sides of the evaluation equation requires an integrated multi-stakeholders and multi-sectoral approach. All development actors have to play their respective roles and at times, may even need to assume multiple roles and responsibilities.

4. Beware of Greeks bearing gifts.

An interesting issue was raised by one of the participants and attested to by other participants. Some international development agencies make it a point to include evaluation of programmes/projects as one of the funding conditions.  While these agencies emphasised on the imperative of developing local evaluation capacity and capability, it seems that they are only paying lip service. Most of the times, they would write the Terms of Reference (ToR) in such a manner that only their preferred consultants or evaluation service providers are qualified and selected. So it was not surprising that the same group of consultants/service providers are the ones being selected for the available evaluation jobs.

5. Government to take charge and be an Evaluation Champion.

In any developing country, the national government is the largest funder of development programmes and projects. Development is its core business. In the quest for excellent public programme performance results, the public sector should take charge and lead in the institutionalisation of evaluation. It is possible to carry out this leadership role and responsibility with the cooperation and collaboration of other development partners, public sector organisations would be seen as Evaluation Champions, thereby laying a strong foundation for the development of a culture of evaluation within and outside their organisations. They would also become exemplary role models for private sector organisations and civil society organisations to emulate.

6. Act with bravery and innovativeness and yet retaining convention.

The conduct of the event is a powerful message by itself.  The GED kicked off by a presentation of a position paper entitled “Strategies, systems, approaches and tools for sustainable institutionalization of  evaluation within developing countries”. The presentation was in an unrehearsed dialogue format between two experienced VOPE representatives, one from the developed North and the other another from the developing South. This brave act was then followed by a moderated debate session conducted in the conventional panel format. The breakout session ala World Café style was another innovation adopted to effectively elicit the thoughts and viewpoints of participants in a facilitated environment. The value of the GED lies in the summary findings, recommendations and action plan distilled from the facilitated breakout sessions.

Prepared by:

Lim Kheng Joo

Senior Associate

Centre for Development and Research in Evaluation (CeDRE), International

31 January 2017

You are here: Home Blog Key Takeaways from the Great Evaluation Debate at the 2016 APEA Evaluation Conference