Transforming Health Systems Initiative Mid-Term Evaluation—Conducted in 2011
Why this Initiative
While health spending has increased dramatically around the world, access to affordable, quality services has lagged. At the same time, national health systems in developing countries often lack the capacity to meet daily health needs, as well as anticipate, prepare for, and recover from health shocks down the line.
Launched in 2009, The Rockefeller Foundation’s Transforming Health Systems (THS) initiative was a five-year, $100 million program. Its aim was to catalyze innovations and actions to create broader access to affordable, quality health services in developing countries, and enhance resilience in these countries’ health systems and populations. The initiative’s activities included a global effort to promote universal health coverage, as well as a strong commitment to the use of mobile-based electronic health (e-health) tools to facilitate medical care.
Employing Monitoring & Evaluation to Increase Impact
In 2011, halfway through the scheduled time frame of our Transforming Health Systems initiative, the Foundation commissioned an independent, mid-term evaluation of the initiative’s strategy and impact to date, in order to identify potential opportunities for mid-course corrections. The external evaluation team reviewed all relevant strategic plans, reports, and initiative activities, debriefed current and past members of The Rockefeller Foundation’s initiative team as well as our senior leadership, and interviewed some 40 partners and grantees. The evaluation team also conducted a panel discussion that included the Foundation’s initiative team and outside experts, and surveyed a number of global health leaders.
The resulting report included detailed breakdowns on milestones achieved, cost-effectiveness, and perceived impact in each of the initiative’s activity areas. Overall, the evaluation confirmed that the initiative was well-matched to The Rockefeller Foundation’s strengths and had clear value, and that it was exerting substantial global influence. However, the evaluation team also suggested that the initiative’s activities could benefit from a greater degree of focus, and expressed concern that it was “trying to do too much with too little capacity and resources,” especially in regard to the initiative’s in-country work in its four target nations: Bangladesh, Ghana, Rwanda, and Vietnam.
Other potential areas for improvement cited by the external evaluation team included “an opportunity and a desire for greater coordination and communication among grantees”; “a need for greater/simpler clarity in how the work of THS… ties together into something that will synergistically affect health systems”; “room for improved team dynamics”; and “an opportunity for greater clarity around initiative-wide capacity strengthening efforts.”
Based on these and other observations, the external evaluators presented a number of recommendations for mid-course corrections. These included:
- Increasing the involvement of the initiative’s local grantees, and enhancing communication with all stakeholders around the initiative’s strategy and learnings
- Defining the initiative’s target end points and policy goals more clearly
- Defining the initiative’s in-country strategy in its four target nations more clearly, including how the work in each country is supporting the global efforts of THS
- Clarifying the role and benefits of the e-health tools the initiative was employing
- Continuing to build on the momentum of the Joint Learning Network for Universal Health Coverage (JLN)—the global consortium for knowledge sharing around universal health coverage that THS helped establish
As a result of this evaluation, and after consultation with the Board of Trustees, the Foundation decided to devote much of its attention in the THS initiative to promoting universal health coverage on the global agenda. It was also decided that THS would focus on strengthening the promising Joint Learning Network in order to maximize its sustainability in the future. Finally, the Board of Trustees agreed that THS would consolidate and complete work in the four THS focus countries, focusing on areas of greatest dynamism and traction in each country. Accordingly, an additional $15 million was appropriated toward these goals and the life of the initiative was extended through 2015.
In retrospect, some of the mid-term evaluation’s recommendations were more valuable in shaping the design of future initiatives than in changing the course of THS itself. The THS initiative was able to achieve a number of positive final outcomes—including its successful push for the United Nations resolution, passed unanimously in 2012, urging all member nations to develop affordable universal health coverage systems, as well as its founding of the JLN, which now thrives independently. The initiative also succeeded in including universal health coverage as a goal in the UN-approved Sustainable Development Goals in 2015. However, the Foundation ultimately agreed with the mid-term evaluation’s concerns that the stated ambitions of the initiative exceeded what could be accomplished with the available capacity and resources. This understanding played an important role in guiding The Rockefeller Foundation’s subsequent strategy, including the decision to implement our 100 Resilient Cities initiative on a robust global scale, with an appropriately high level of funding support.