In this way, replicability is at the heart of the grand project to give everyone a shot at a decent life.

But in economics, and the social sciences generally, replication doesn’t happen much. Why? No one ever won a Nobel Prize by copying others. And as I think John Kenneth Galbraith once observed about competition, people like replication until it happens to them.

But times are changing. The World Wide Web was invented in 1990 to help researchers share data, and it is slowly shifting norms about openness and collaboration. Communities converge faster to agreement, dare I say the “truth,” when analysis is public, collective, and iterative. Back when data were stored in punchcards and scientific discourse took place via ungainly bound volumes, statisticians could easily bar outside observers from their private digital laboratories. Now expectations of transparency are higher.

So here’s a cool thing. The International Initiative for Impact Evaluation (3ie) is seeking nominations by May 31 for impact studies that it should pay to have replicated. 3ie was born out of a recommendation in a CGD report led by Ruth Levine, Nancy Birdsall, and Bill Savedoff; its mission is to support new, high-quality research on what works in development, and at what cost. The just-launched replication program, on whose advisory board I serve, embodies an astute recognition that 3ie can also contribute by rigorously reexamining research done by others.

Are there studies in your field that you think are influential enough to deserve reexamination—and confirmation, contradiction, or something in between? How strong is the evidence base for the idea that girls’ education is an investment with particularly high returns? How sure should we be that, roughly speaking, men spend all their money on beer while women invest in the children? Or that financial sector expansion increases economic growth?

“Replication” has a range of meanings. I could replicate the Hyderabad microcredit impact study by performing a similar experiment in Mumbai, what 3ie calls “external replication.” That is important, but not what 3ie has in mind here. Or I could field my own survey team in Hyderabad to collect and analyze new data on the impacts of the microcredit experiment there, what psychologist John Hunter called statistical replication. Or, since I am lazy and lack managerial prowess, I could download the original study’s data and reanalyze it, for a “pure replication.” 3ie looks to support work of these latter kinds.

Actually, I’d draw a further distinction within the “pure replication” category. There is what I consider true, pure replication, which strives to exactly reproduce a published analysis in order to scrutinize it. Partly for lack of imagination, that’s what I usually do, as in my work with Jonathan Morduch on the Pitt & Khandker microcredit impact study. And there is what I could call reanalysis, which applies different methods to same data, as Jonathan did for Pitt & Khandker in 1998 and as Maren Duvendack and Richard Palmer-Jones (who also advises the 3ie program) did in 2011.

Perhaps the biggest challenge will be persuading the authors of the chosen studies to share their data. Should that problem arise, I hope it will be solved by the public exposure generated by this exercise.

Importantly, the expectation is that results will be posted regardless of outcome. Everyone loves a conflict, but confirmations matter as much as contradictions.

Send your nominations to replication@3ieimpact.org. But surely the list should be crowdsourced, so I encourage you to post your ideas below as well. And if you want to achieve riches by replicating (any starving grad students out there?), stay tuned.

A few days ago the World Health Statistics 2012 Report released its annual compendium of statistics. No doubt, it was a lot of work to compile—to verify every number in every cell, for each country and indicator. The WHO should be commended for providing this invaluable global public good. A sincere request: the Report would be more user-friendly and useful if the Report came with spreadsheets in downloadable tables (much like the World Malaria Report), and if the Report’s tables were consistent with their main database, the Global Health Observatory (GHO). For example, the coverage measures of oral rehydration therapy (ORT) which were included in the Report are absent from the GHO, as far as we can tell. (On an unrelated note, we did notice that the GHO recently added hand-washing as an indicator, perhaps in response to a recent blog—kudos to WHO!)

The Report also offers a glimpse into what is of current interest and priority among donors and countries. Global health donors generally have pet priorities and interests. These are reflected in part by how many countries report for a given indicator. For example, if one turns to the chapter on “Health Service Coverage” (beginning on page 98), we see that “immunization coverage” for measles and for diphtheria, pertussis and tetanus is available for the reporting 193 Member States. This is not surprising given the recent occurrence of measles epidemics in high-income countries as well as GAVI’s success and use of coverage levels as conditions of both eligibility and future funding. Similarly, when we consider the “big three” of AIDS, tuberculosis, and malaria, we see relatively high coverage of the relevant indicators.  For the indicator “antiretroviral therapy coverage among people with advanced HIV infection”, 44 of the 47 of sub-Saharan countries reported. When considering the “case-detection rate… of tuberculosis” indicator, 40 of the 47 countries reported the statistic. For the indicator on under-5 children with fever treated with anti-malarials, 38 of the 47 countries reported. These indicators seem to have higher-than-average coverage.

In contrast, when one considers the indicators for under-5 children (1) with diarrhea who received oral rehydration therapy (ORT), and (2) with pneumonia who receive antibiotics, the picture is bleaker. For the ORT indicator, 36 of the 47 reported. For the pneumonia indicator, less than half (23 of the 47) reported.  Moreover, under-5 children with diarrhea or with pneumonia who take zinc does not appear in the Report (or the GHO), even though zinc is in the WHO’s Model List of Essential Medicines and is widely recognized as an important intervention for preventing child mortality.

What explains the differences in coverage of these indicators? The differences in reporting reflect in part different global health priorities and with it, the money disbursed by donors. Where there is interest and money, there is an indicator, and the coverage of indicators is higher (the correlation is not perfect). Not surprisingly, the single disease category with the highest development aid in 2009 was AIDS (see here), and the ART indicator in the Report also has fairly high coverage. Malaria and tuberculosis are the other two leading single disease categories for development assistance, with the associated malaria indicator having slightly worse coverage than the tuberculosis indicator. However, when considering diarrhea and pneumonia (which are not listed as separate diseases in the IHME report despite causing 10% and 18% of all under-five deaths), less international aid is devoted to these diseases and not surprisingly the coverage of these indicators is lower, at least compared to the big three. While some have noted the tremendous dearth and low coverage of cause-of-death statistics, the phenomenon of low coverage is also true for health service indicators which are arguably easier to measure than cause of death.

Do these indicators matter? In her first address as director of the WHO, Dr. Margaret Chan quoted the axiom, “What gets measured gets done.” If what is measured gets done, then better measurement and reporting is urgently needed. However, if only what is of interest is what is measured — and only what is measured gets done — the question remains: For diseases which are of less interest, can we (the global health community and perhaps the WHO in particular) create the incentives for better measurement and reporting for those diseases?

One of the useful things the federal government does for the economy is produce information as a public good. And the American Community Survey is chock full of information that’s useful to researchers, companies, curious individuals, policymakers at different levels of government, etc. But House Republicans have decided that they want to kill it, and it seems clear that some of them have a passion for the cause that completely exceeds their understanding of the issue. Representative Daniel Webster, for example, is a sponsor of the anti-ACS survey in part because he thinks $70 per survey respondent is “not cost effective … especially since in the end this is not a scientific survey. It’s a random survey.” [Emphasis added.]

President Obama announced $3 billion in new private sector investments in agriculture in three African countries at a packed event in Washington, D.C., last Friday. The New Alliance for Food Security and Nutrition is the cornerstone of the United States’ 2012 G-8 commitments to development led by USAID and administrator Rajiv Shah. There’s a lot to like about the partnership: presidential leadership, a link between public and private investment, and a focus on policy change. But all eyes are on how the relatively modest investments will be implemented and whether they can reach the ambitious poverty reduction targets.

In his remarks at the Chicago Council’s symposium, President Obama said the new alliance would build on the G-8′s 2009 $22 billion global food security commitments. The White House fact sheet says:

The New Alliance for Food Security and Nutrition is a shared commitment to achieve sustained and inclusive agricultural growth and raise 50 million people out of poverty over the next 10 years by aligning the commitments of Africa’s leadership to drive effective country plans and policies for food security; the commitments of private sector partners to increase investments where the conditions are right; and the commitments of the G-8 to expand Africa’s potential for rapid and sustainable agricultural growth.

The new alliance focuses initially on Ethiopia, Tanzania, and Ghana. More than 45 multinational and small and local business have committed $3 billion in investments; the countries have committed to specific business environment policy reforms.

Here’s what to like about the announcement:

The president is using his bully pulpit to champion development. The president has been relatively quiet on global development since issuing his policy directive in September 2010. In his remarks on Friday, President Obama made a strong case for why global development and food security are in the United States’ moral, economic, and security interests. It’s still a big deal that development is on the president’s agenda and that he is using his bully pulpit to champion the issues (something the Bush administration understood and CGD president Nancy Birdsall urged prior to the 2008 elections).

It’s the PPD in action. The new alliance checks off just about all of the presidential policy directive on global development‘s key elements. It’s selective, country-led, focuses on economic growth and policy reforms, leverages the private sector and multilateral development agencies (African Development Bank, World Bank, International Fund for Agricultural Development, etc.), and emphasizes results and accountability.

Links private investment, public sector know-how and policy reform. The new alliance and specifically USAID administrator Rajiv Shah are exploiting U.S. convening power to leverage country policy reforms and private investments. ONE Campaign co-founder Bono went so far as to suggest the announcement could signal the death of the traditional donor-recipient relationship, presumably in part because of the country leadership, but also because of the mix of public and private sector roles.

Comments:

Some critics argue the United States–and the other G-8 members–are trying to replace missing in action public funds with smaller private investments. But few expected the United States (or others) would come up with new, large sums of money. And while it’s easier to experiment when times are flush (think creation of PEPFAR and MCC), it’s still a good thing if tight budgets push new approaches for better aid delivery. It just means the stakes are a lot higher for the administration to:

1. Move quickly to implementation. President Boni Yayi of Benin (and African Union chairman) put it this way: “Let’s go now to action!” The challenge for the administration (as is often the case for developing countries themselves) is execution. I’m eager to better understand how the letters of intent between the countries and the companies will work, what will happen if companies fail to invest or countries fall down on their side of policy reforms, what USAID’s role will be, how the investments relate to the African Union’s Grow Africa initiative already underway as well as Feed the Future and the Global Agriculture and Food Security Program, and the timeline for investments.

2. Show evidence that investments lead to poverty reduction. While most agree that private sector investment should follow aid, not everyone agrees that private sector investments will lead to poverty reduction. The new alliance will have to include clear, transparent measures to show whether or not the investments are reducing poverty.

3. Bring Congress along. Chicago Council symposium co-chair and former member of Congress Dan Glickman remarked Friday that the president proposes ideas, but if Congress doesn’t go along, those ideas don’t go anywhere. Bipartisan congressional support–and legislation–was critical to the creation of PEPFAR and MCC during the Bush administration. The Obama administration passed up the opportunity to work with the House (Rep. Howard Berman) on a new Foreign Assistance Act and the Senate on a bipartisan food security agenda (led by Sen. Richard Lugar whose leadership will be sorely missed and Sen. Robert Casey). For food security to become the Obama administration’s signature (and lasting) development issue, it must have Democratic and Republican support in Congress. Those prospects look a lot dimmer on both sides of the aisle today.

The star-studded symposium on Friday lit up much of the development community, but attention will quickly shift from show to substance. The new alliance includes a leadership council to drive and track implementation and report to the G-8 and African Union on progress. G-8 watchers are understandably nervous about the size of the commitments and that it looks and feels different. They’ll be looking to the leadership council, the White House and USAID to share information early and often to shed light on whether the administration’s agriculture focus will be a fleeting or permanent part of its development profile.

I’m eager to know what others think and hope the G-8 and food security experts among you will add your comments below.

The goal of the program remains the same: to provide a one-stop shop for timely, relevant analysis on U.S. development assistance. As in the past, the primary focus will be on reporting, analysis and commentary on the mission, mandate and organizational structure of U.S. aid agencies such as USAID and MCC and their interactions with other U.S. development actors, such as the State Department and Pentagon.  The program will also track presidential development initiatives like Feed the Future and the Global Health Initiative.

I also plan to look beyond aid, where appropriate, to consider related U.S. policies in areas such as trade, migration, investment, environment, security and technology. The U.S. government has embraced the view that development policy is more than aid (see the Presidential Policy Directive on U.S. Global Development Policy), an idea long championed by my colleagues here at CGD, so it seems only appropriate that our Rethinking U.S. Foreign Assistance Initiative should reflect this, too.

So in the months ahead, you’ll not only be hearing more from me but also from my CGD colleagues when their work touches on U.S. development policy, as with Vijaya Ramachandran and Julie Walz’s new policy paper on the (un)accountability of U.S. aid in Haiti. I’m happy that CGD policy analyst Casey Dunning and research assistant Will McKitterick will continue to bring their hard work, brains, and enormous energy to the program.

Most of all, Rethink’s content should be useful to you. What’s on your wish list for what we should cover? What should we drop? And how do you like to receive your U.S. development policy re-thinking? Blog? Email? Wonkcast? Tweets? Carrier pigeon? Share your suggestions as comments on this blog or send to me directly. I’m looking forward to hearing from you and navigating upcoming U.S. global development issues together over the coming months.

The other is on the site of the Whole Planet Foundation (you have to dig a bit on this page or go here). The Whole Planet Foundation’s mission is “poverty alleviation through microcredit in communities worldwide that supply Whole Foods Market stores with products.” I gather that when you check out at Whole Foods you can donate your change and then some to the foundation.

Reading the latter one, by Executive Program Director Steve Wanta, led me to reflect on what can make a review persuasive. I thought of two things. One is eminence. The reviewer’s achievements and experience and position—who he is—can make him credible when he describes a book as insipid or brilliant. I think of Amartya Sen’s review of Bill Easterly’s second book and Bill Clinton’s take on Robert Caro’s latest opus on LBJ. But that avenue is not open to Wanta, not so much because he lacks the stature of Sen or Clinton, but because he has a professional, vested interest in what I critique. Who he is should make him compelling only to the converted.

The other avenue is evidence. I.e., backing up statements like “Roodman does not demonstrate the same statistical rigor for the potential adverse effects as he does for the positive effects of community building reported by the industry” with specifics. I tried to do something like what I have in mind in my biting review of Dambisa Moyo’s Dead Aid, which the Whole Planet Foundation deems “Compelling reading and a must read for anyone interested in African development.” Or see Alex Counts’s thorough examination of my book, which I still think is the best so far. At any rate, I think the Whole Planet review of my book does not persuade through either eminence or evidence.

So, will West Africa be able to capture the benefits of a demographic dividend?  The answer to this question hinges on how fast West Africa can bring down its high fertility levels.

A new study by Jean-Pierre Guengant for the French Development Agency (AFD) provides a detailed analysis of long-term population and economic trends for 12 Western African countries, i.e., the eight countries of the West African Economic and Monetary Union (WAEMU), plus Ghana, Guinea, Mauritania and Nigeria.

The study acknowledges that economic growth has returned to the region since the mid-1990s.  However, revenues per capita have not increased much.  Assuming a yearly population growth of 2.5 percent and an economic growth of 5 percent per annum, it will take almost 30 years to just double the income per capita, which is already very low to start with.  Worse, total poverty levels ($1.25/day) remain high, between 30 and 50 percent of the population in all countries.  Poverty reduction is all the more challenging for ever growing populations, and so is the formation of human capital.

The AFD analysis offers new population projections, which are made consistent with increases in contraceptive use.  This methodology was proposed by Guengant and myself in a 2011 United Nations Expert Paper.  The intermediary scenario (Medium variant) of the UN population projections assumes sharp fertility declines that are not warranted given the slow increases in contraceptive coverage.  These are currently progressing at the snail pace of 0.5 percentage point per year in West Africa, which is not enough even to achieve the high scenario (High variant) of the UN projections (assuming a slower decline in fertility).

So what are the policy implications?

First, rapid declines in fertility spearheaded by strong family planning programs appear to be one of the prerequisites for sub-Saharan Africa to be able to reap the benefits of a demographic dividend.  The emphasis here is on rapid.

Second, human capital investments and poverty reduction efforts will require huge increases in education and health budgets, to the tune of at least 7 percent per year for the next two decades, a figure well above the current economic growth.

Third, West Africa and Africa at large need to capitalize on the ‘golden moment”, i.e., the boost of family planning efforts that will be discussed at the upcoming family planning summit scheduled in London on July 11.  The time has come for African governments to reenergize their family planning programs.

Too often African leaders and their partners want to believe that the demographic dividend is just around the corner.  This is wishful thinking.  The demographic dividend is not a given, and its chief precondition – the sharp decline in fertility – has not even started in earnest in many West African countries.

For the demographic dividend to present itself, voluntary family planning programs that respect human rights are necessary to trigger much needed fertility declines.  These programs will be useful in their own right as they will serve the needs of millions of African women who want to space and limit their children, but do not have the means to do so.  As these programs will help decrease fertility, they will also facilitate female universal education, which in turn will accelerate the fertility decline.  In short, stronger family planning programs and a faster fertility decline will help usher one of the necessary (but not sufficient) conditions for the countries to reap the benefit of a demographic dividend.

Now let’s compare U.S. assistance programs to the two bodies.  The CBO estimated that the moon program cost NASA about $170 billion in 2005 dollars.  OECD DAC data for U.S. ODA to Africa (North and South) from 1960 to 2010 was worth a cumulative $168 billion in 2010 dollars.  So, give or take, the U.S. spent as much sending technical experts (aka astronauts) to the moon as it did on all assistance to Africa over the past fifty years.  That would be somewhere over $10,000 per square mile.

And what about efficiency and effectiveness?  Whatever the horrible amount spent on per diems and airfares by beltway bandits, the travel and overhead portion of the Apollo missions budget was considerably higher, at approximately 100%.   It is also worth noting Africa has seen a considerable amount of development progress over those fifty years and some of it (think smallpox and rinderpest eradication) was pretty undoubtedly in part thanks to U.S. assistance.  Compare that to the moon, which hasn’t even managed to accomplish the basic underpinnings of development –like an atmosphere.

From the outside, KGFS seems like alchemy. No longer is a single, low-quality financial service like group microcredit mass-produced for the poor. Instead, poor people get customized financial services the middle class might envy. A client walks into a branch and meets with a KGFS employee who inventories her income and spending, assets and liability, worries and goals, punches them into a computer, and prints out a recommended portfolio of services that may include loans, insurance, savings, and even retirement savings. KGFS then provides all the recommended services, either on its own account, as with loans, or as an agent, as with insurance. The financial assessment, and presumably the service portfolio preferred, is updated every six months. Somehow, despite all this custom service and the forays into services most others have lost money on, KGFS turns a profit. When Nachiket Mor, who I think is the driving force behind IFMR, described it at a CGAP meeting last December, someone in the audience aptly commented that she would like access to something that good.

There is much that is refreshing about KGFS: taking client needs as the starting point; using of high tech to cut the cost of assessing and meeting those needs; partnering with insurance companies and other financial titans, with KGFS the retail agent; and delivering a set of services. The big question in my mind has been whether it can pay for itself and grow.

So far it seems to have. CGAP has just put out a report and a video interview with Bindu.

From the report, I learned that in the last few years the IFMR Trust has licensed the KGFS system to five independent for-profit entities, which have raised all their capital from private investors. Together they have reached 200,000 people. Even more impressive is that mature branches, ones approaching three years in age, reach nearly 70% of people in the vicinity, or “catchment area.”

I commend both the report and the video to you as introductions to the KGFS model. They are clear, accessible, not too long. Having met and talked shop with Bindu, as well as her CGAP coauthors on the report, Greg Chen and Stephen Rasmussen, I have great respect for judgment and intelligence of all those involved.

That said, you should recognize that these materials do not represent an independent inquiry into KGFS, which is what I, as someone already familiar with the model, was most hoping to see. Both are clean presentations of the KGFS story as the KGFS people want to tell it. That’s fine as long as it’s seen for what it is.

At this point, I have a couple of critical questions not confronted by the new CGAP report. Writing for CGAP, Rich Rosenberg has warned that demand for microcredit may be lower than is often assumed when people talk about the billions who still lack access to it. If forced to pick a number, I bet he’d estimate the typical demand at closer to 10% of poor households than 70%. Yet according to the new report, 87% of the 70% of people reached by a mature KGFS branch within its catchment area take loans. That works out to 61% of the population, way higher than the statistics we’ve seen for national microcredit penetration anywhere else. Should we worry about this spread of indebtedness? A lot of the loans are group loans, presumably not unlike those in traditional microfinance.

In the same vein, which products generate the profits? I think it’s fantastic that customizing multiple services to meet actual needs is hard-wired into KGFS. Yet if some products make money and others lose it, that generates incentives that are hard to ignore, and could easily push the practice, whatever the theory, toward credit. The “stylized branch income and expense statement” on page 14 of the report oddly displays net income from lending (so we see it’s profitable) and gross income from everything else (so we can’t tell what in the array of other services is profitable and what not).

I am also curious about who is investing in KGFS, and how many would consider themselves “social investors.” Beth Rhyne has argued that a core cause of the Andhra Pradesh debacle is India’s prohibition on non-profits owning parts of for-profits. So when Indian microcreditors went for-profit in order to raise capital and grow, mission-driven actors were structural excluded, unable to counterbalance the drive for profits. What does that imply for KGFS?

Not that I have reason to be deeply suspicious of KGFS. Indeed, I assume Greg and Stephen only lent their names to this report after serious, behind-the-scenes due diligence. But KGFS is so striking a departure from the past, so potentially important, that it deserves close, independent scrutiny, which would not be coauthored by one of the promoters. The model I have in mind is MicroSave’s tracking (with DFID funding?) of the roll-out of Grameen II a decade ago. They hired Stuart Rutherford to study it from the client point of view, and I assume gathered other kinds of evidence through examination of financial records and interviews with staff from Muhummad Yunus on down. The result was a credible, penetrating, balanced analysis with important lessons for the rest of the microfinance world.

Update: I forgot to mention how great it is that the IFMR Trust has commissioned a randomized evaluation of KGFS, to be led by Rohini Pande and Erica Field. Especially exciting is that the experiment will run for three years instead of the usual 12–18 months, giving us a glimpse of longer-term impacts.

The January 2010 earthquake that devastated Haiti, killed over 220,000 people, displaced several million, and flattened much of the capital, Port Au Prince, also unleashed a tsunami of outside assistance. In the 28 months since the earthquake official donors have disbursed almost $6 billion in aid to help the people of Haiti, the equivalent of $600 per person for a country where per capita annual income is just $670.

Where has all the money gone? On the second anniversary of the quake we set out to answer this question; our new CGD policy paper is the result.  The short answer is that the vast majority of the money so-far disbursed has been paid to international non-governmental organizations (NGOs) and private contractors. And while many of these organizations do excellent work, there is shockingly little information on how they used the funds.

The reliance on NGOs is understandable, given the limited capacity of the Haitian government and weak national institutions. But the proliferation of NGOs makes tracking the money extremely difficult. Estimates of the number of NGOs operating in Haiti range from a few hundred to more than 20,000.  Only a small proportion of the organizations working in Haiti are officially registered with the Ministry of Planning. Using a directory of organizations run by the UN Office of the Special Envoy for Haiti and Inter-American Development Bank, we constructed a data set of 980 NGOs operating in Haiti.

Post-quake aid to Haiti falls into two categories: humanitarian, for short-term relief efforts; and recovery, longer-term financing for reconstruction and development. Humanitarian agencies, NGOs, private contractors, and other non-state service providers received 99 percent of global humanitarian aid – less than one percent went to Haitian public institutions.  None of the $1.28 billion disbursed in humanitarian aid from the United States went to the Haitian government.

Figures 1 and 2 show the breakdown of U.S. Government funding to Haiti in the aftermath of the quake.  Other donors have provided some recovery funding to the Government of Haiti; of total recovery funding, somewhere between 14 and 21 percent of the money has been allocated to the government.
How have international NGOs and private contractors performed with regard to service delivery in Haiti?  The bottom line is that twenty-seven months after the earthquake, it is still very difficult to tell.

Two external organizations catalog evaluations and other reports from organizations operating in Haiti since the earthquake: ReliefWeb and the Active Learning Network for Accountability and Performance in Humanitarian Action (ALNAP).  Less than half of the reports were conducted entirely by an independent party and only slightly over half detail the methodology used.  More than one-third of the reports do not have specific project data.  Especially of concern is the lack of budget or cost data.  Only four program and organization reports have any detail about how the money was spent (how much tents cost, how much money was given per cash transfer, or what percentage of funds went to transport vs. logistics).  Only one of the reports has any discussion about providing the best value for money and what the alternatives might be to the program currently being implemented.  There are almost no publicly-available evaluations of private contractors operating in Haiti.

Despite all of this, given the continued weakness of the Haitian government and USAID’s operating model, it is likely that NGOs and private contractors will continue to dominate service provision in Haiti for some time to come. Fortunately, there are some simple steps to improve accountability for public money and private charitable contributions intended to help the people of Haiti, as a means to getting much greater value from post-quake aid. We recommend three ways to track and evaluate programs and make aid more effective:

[1] Require More and Better Independent Evaluations

There is a great need for systematic evaluation of the $6 billion spent in Haiti since the quake.  There are six key criteria that we would like to see in evaluations:

• Independence (should be carried out by a third party not the organization itself)
• Clear methodology, which explains how the evaluation was conducted
• Clear project data about the number of services provided and number of people benefiting
• Cost break-down or budget report
• Discussion of alternative programs, cost comparisons, or other uses for the money
• Recommendations for improvement

USAID and other US Government (USG) contracting agencies need to clarify reporting mechanisms for recipients of public money, and require evaluations that meet the above criteria for organizations that receive contracts from USG agencies.

[2] Share Data Through the International Aid Transparency Initiative

NGOs and private contractors can greatly improve the reporting of data on expenditures and outcomes.  The International Aid Transparency Initiative (IATI) is a multi-stakeholder initiative that has developed a standard for publishing information about aid spending.  Donors, partner countries, and civil society organizations can publically disclose information on volume, aid allocation, and results of development expenditure.  The UK Department for International Development (DFID) has spearheaded the process and is currently requiring the NGOs which get the biggest grants to implement IATI.  We strongly recommend that the United States, which signed on to IATI last November, adopts the IATI process in a timely manner and requires NGOs receiving public money to become IATI complaint (as in the UK).  Since implementation of standardized reporting requirements is likely to take time, USAID should render public the financial reports from primary contractors and grantees in Haiti and build the capacity needed to track grants and sub-grants, so as to provide some form of transparency in the interim.

 [3] Pilot Competitive Bidding

New Public Management-style contracts to provide services like transportation, health and school construction could help order the NGO landscape in Haiti through a competitive bidding process, while increasing service supply and efficiency. With careful design, they might also increase accountability between donors, NGOs and the Haitian government.  Although Haiti lacks a robust private sector, market competition is possible since there are several thousand NGOs to compete for contracts.  Donor funding for specific projects could be channeled through this model, and competition would help to eliminate the inefficient organizations.  Contracts can be contingent on IATI compliance.  Donors and the Government of Haiti should conduct a pilot of this model in an effort to bring order to the proliferation of NGOs and create a much-needed enforcement mechanism.

The original paper estimated that for every dollar that in developing-countries governments received in health-focused aid, they cut health spending from their own resources, such as tax revenues, by $0.43–1.14. Ergo, donors who thought they were investing in health were substantially financing something else unknown—road building? jets for dictators? That conclusion went for the noted private philanthropy that paid for the research. This uncomfortable factoid about fungibility was widely cited in academia, and in the press (e.g., the New York Times, where it cameoed, garbled, on the home page).

To the casual reader, the reanalysis by Rajaie Batniji and Eran Bendavid seems damning:

We…demonstrate that prior conclusions drawn from these data are unstable and driven by outliers. While government spending may be displaced by development assistance for health in some settings, the evidence is not robust and is highly variable across countries.

My advice: don’t trust the conclusions of either side.

Of the two studies, the original is of higher quality, not least because it embodies the Herculean effort to collect the data that the new study is quick to describe as flawed. It is always harder to create than destroy. Still, as I blogged when the Lancet paper appeared, there are strong technical reasons to doubt its confident claims. Sure, maybe countries receiving more aid for health spend less on it themselves, but what is causing what is less clear. It’s that old correlation-is-not-causation bugaboo. The IHME team battled the bugaboo with a fancy mathematical technique, but in a way that I am qualified to doubt as the author of a popular program to implement it.

The Stanford docs’ new scrutiny of the IHME health data set is healthy. But their own data undermine their conclusion that the Lancet conclusion is undermined by bad data. (In 2009, my colleague Mead Over took apart another study by Bendavid as tautological.)

You don’t need a Ph.D. in statistics to see how. The key table from the new paper is below. It shows the results of four sets of statistical runs, listed from top to bottom. The first set was done on the full IHME database, the next three after omitting certain extreme groups of data points in turn. Within each set of runs, the left side of the table uses health spending data from IMF and the right from WHO; the two sources disagree enough to significantly affect results. Also within each set, the first row uses the statistical method favored in the Lancet (“Arellano-Bover/Blundell Bond”—Did I mention I’m an expert on that? Yes I’m a highly expert expert on that) while the second uses the simpler method favored by Batniji and Bendavid (“Linear, country clustered”). Scan the table, then read my exegesis underneath:

The “coefficient” columns show the central estimates of fungibility. So, e.g., “–0.40″ suggests that for every dollar of health aid, receiving governments spent an average 40 cents less of their own money on health.

Notice that the p values in the rows for the IHME-favored “Arellano-Bover/Blundell Bond” method are almost all 0 or 0.001. “p” stands for probability, so these small numbers are saying that if the IHME authors are wrong, if health aid is not fungible, then it is highly improbable that we would see such high estimates of fungibility as are reported to the left of the p values. Turning that around, if you buy the IHME statistical approach, aid is probably fungible, and little in this table changes that conclusion. Contrary to the damning verdict I quote at the outset, the Lancet analysis is robust to the deletion of data points that the Stanford authors deem dubious.

What does change the numbers—producing lower fungibility estimates and higher p values—is the switch to the “Linear, country clustered” method that the authors of the new study prefer. So the big divide within this table is not between its top quarter the rest, but between the”Arellano-Bover/Blundell Bond” rows and the “Linear, country clustered” rows.

That is, what breaks the fungibility finding is not the changes to the data sample, but the change to statistical method. The authors have interpreted their results backwards.

To go slightly technical for a moment, there are other problems, which together persuade me that the Batniji and Bendavid have more to learn about short-panel econometrics than the IHME team did. The argument for fixed effects over random effects reads like a non sequitur, focusing on whether nations are diverse instead of whether their diversity fits a bell curve. The authors do not explain why their statistical approach is superior, thereby seeming to confuse issues of data, which they do discuss, with issues of method, which they essentially do not. Their preferred method is described in just one sentence, which contains a phrasing,”country fixed effects, clustered by country,” that I have never seen before, and I think reveals confusion. It appears to me that the estimator does not include fixed effects, but merely clusters standard errors by country. (A quick e-mail exchange with Batniji has strengthened my guess that Ordinary Least Squares was done without fixed effects, but I’m still not certain.)

Most importantly, if my educated guess is right, then the authors appear unaware that their favored method is known to be biased. Dynamic panel bias is exactly what motivated Manuel Arellano and Stephen Bond to design their famous alternative. (See page 4 of this paper by Bond.) In the case at hand, the naive method will overestimate the ability of last year’s domestic health spending to predict this year’s, which will leave less room for variables such as health aid receipts to explain that outcome. OLS without fixed effects can be expected to underestimate the negative relationship between health aid receipts and governments’ own health spending, which is consistent with what we see in the table above.

When I blogged two years ago, I reported that the Lancet authors had not responded to a request for their data and computer code. In effect, and as is still the norm in academia, they asked the world to trust their results—indeed, to use them to shape policy involving billions of dollars and millions of lives—while they kept the precise method behind those results secret.

This new article sits on the servers of the Public Library of Science, which aims to break the centuries-old lock of the journals on scientific research. Now, anyone can be a reviewer. Check out the dozen comments I’ve peppered throughout the web version of the article. (Look for the little “1″ icons. Did I go overboard?)

It appears to me that the old and new-style publishers have both let slip into their pages articles with problematic methods—or at least methods whose results are over-relied upon—probably for lack of reviewers expert in short-panel econometrics. But at least with the PLoS one, the crowd can flag the low quality, and even correct it at the margin (and in the margins).

In January, I gave a talk on my new microfinance book at the IHME. Toward the end, I described CGD’s new data and code transparency policy, which calls on our researchers to publicly post all the data sets and the lists of computer commands (code) needed to reproduce our statistical results. I argued that the IHME could learn from this policy. My hosts were extremely gracious, and soon sent me the data and code for the Lancet paper. Now, two years later, I am engaged in a constructive, private dialog with them, that I will say more about when the time is right.

Meanwhile PLoS, unlike the Lancet, has a policy that calls for data sharing in letter and code sharing in spirit. I mentioned that I’m not yet 100% certain about the methods in the new paper. But thanks to that policy, I’m optimistic that, much more quickly this time, I’ll get to the bottom of the latest research on health aid fungibility.

I have been honored to work with such a fine group of people who are doing important research on difficult issues.  During just ten years in existence, CGD has had major impacts on how the United States engages with the rest of the world on global development issues.  I look forward to watching its next decade of work.

I am proud of the work we’ve done in the Rethinking program.  From nudging the administration to live up to its commitments made in the President’s Policy Directive on Global Development and the Quadrennial Diplomacy and Development Review to our tracking of special initiatives to our work on the USAID Monitor, I hope that we have filled a gap in existing analysis.  It has been a delight to work with the many groups and individuals who care about these issues.

I leave with mixed feelings – a bit of sadness to not be involved in the day-to-day work of monitoring U.S. assistance programs, but also with much excitement to embark on several independent projects.

I also leave knowing that the program will be in the able hands of Sarah Jane Staats who will assume leadership of Rethink in June.  Together with the talented Casey Dunning and Will McKitterick, I am certain of its continued success.

Best.

The powers in Bangladesh did not appreciate Hillary’s contribution. Two top officials has gone after Clinton and Yunus, equally publicly. The rather less-famous finance minister who helped Yunus found the Grameen Bank, opined:

“I think this comment is very unwarranted,” finance minister AMA Muhith told reporters.

“Grameen Bank is a government institution. The government has created it and brought it thus far. It is because of the government that Mr. Yunus could come this far.”

Syed Ashraful Islam, general secretary of the ruling political party and Minister for Local Government and Rural Development & Co-operatives, attacked Yunus in the presence of his Prime Minister:

The country cannot be developed through multilevel marketing companies and NGOs based on illegal money from abroad. Microcredit won’t change our fates and it won’t be able to change Bangladesh in the next 100 years.
…
His basic subject was Economics. But he did not get the prize for Economics. He got the Nobel prize in Peace though he did not play any role in stopping a war.

Now, many people in our country know how to ‘get a Nobel prize’.

If you think about it, this is a pretty stark spectacle. It is hard for me to imagine two members of Obama’s cabinet attacking a single citizen in public, even ridiculing him in front of the President—much less one so highly celebrated. It’s clear that governance in Bangladesh, not just politics, is war by other means. I hope that as regards the safety of Yunus and the Grameen Bank staff, and the integrity of the institution, the threat from the government is offset by the equally unusual endorsement from a powerful foreign official.

In a recent working paper, Jacob Hughes, Walter Gwenigale and I describe Liberia’s unique experience in pooling donor funds for health in a post-conflict setting, with good results. We also describe a new and complementary agreement between Liberia and USAID, called the Fixed Amount Reimbursement Agreement (FARA). It’s been heartening to see USAID take this step towards implementing results-based aid in Liberia, but the process has also highlighted the problems that such aid faces in the ‘real world’.

In September 2011, the Government of Liberia and USAID signed a FARA for $42 million over 4 years. According to the agreement, “a FARA is a U.S. Government assistance mechanism whereby the host government implementing agency is reimbursed a fixed amount for the successful completion of specified activities or outputs with previously agreed upon specifications or standards.”

In the agreement, USAID will reimburse the Ministry of Health and Social Welfare (MOHSW) for the cost of implementing specific activities from the National Health Plan, namely performance-based contracting of NGOs for health service delivery and monitoring and evaluation of service delivery.

Reimbursement to the MOHSW is based on pre-determined amounts, irrespective of actual cost, and is contingent upon USAID verification and approval of each agreed deliverable. USAID has the right to withhold reimbursement until it verifies that each deliverable has been produced as per the verification criteria, but commits to thereafter completing the reimbursement within 45 days. The MOHSW agrees to keep USAID apprised of implementation progress through quarterly reports as well as to manage and monitor FARA supported activities. USAID source-origin policies for procurement of goods and services are waived in the agreement and all goods from the “Free World” are considered eligible (restricted countries include Cuba, Iran, North Korea and Syria). A limited number of additional terms and conditions are included such as pre-approval for drug procurement and requests for proposals that will exceed $1.5 million in value, based upon recommendations made during a 2010 USAID Office of Acquisition and Assistance Procurement System Assessment of the MOHSW. However, USAID accepts that reimbursable expenditure will be based on the MOHSW’s systems for planning, procurement and financial management.

The FARA replaces the previous arrangement whereby USAID funds for service delivery were provided through a cooperative agreement with the U.S.-based company, John Snow Incorporated. This change in approach reflects the USAID FORWARD Implementation and Procurement Reform, which commits USAID to “Strengthen[ing] partner country capacity to improve aid effectiveness and sustainability by increasing use of reliable partner country systems and institutions to provide support to partner countries.”

This is all quite revolutionary – USAID disbursing funds on achievement of outputs, supporting government health plans on-budget. This is a good news story that makes Global Health Initiative rhetoric on country ownership and sustainability a reality. It is a financial instrument that would allow USAID to go in the direction of COD Aid, conditioning disbursements on health coverage and outcomes.

But what about the pool fund? Why didn’t USAID join the pool to further increase the harmonization of aid and multilateral collaboration? The answer to this question says a lot about the trouble with aid in general.

The MOHSW did originally propose that USAID make the FARA reimbursement payments into the Health Sector Pool Fund mechanism. However, other donors to the pool fund felt that “. . .the concept was based on the premise that other donors underwrote USAID’s risk. Donors will find it difficult to advance money from the pool fund.”

In other words, the existing pool fund donors didn’t want to spend money on inputs when USAID would only spend on results.  While these donors should be credited with supporting the pool fund these many years while the United States went its own way, their objection is all about appearances (who looks most innovative? who produces results and how?) and not at all about delivering health aid in the most effective way possible.  The FARA contribution to the pool could have created complementary, virtuous incentives to achieve the pool fund results as efficiently as possible, which would be a credit to the Government and all the participating donors, and would set a precedent for USAID to participate in pooling in other countries.

Should donors really care whether their dollar is used ex-ante or ex-post if the hoped-for results on health are achieved? If donors are worried about pre-financing another donor’s support, why don’t more of them shift to funding results? If donors won’t “front the money” for each other, can low-income country governments like Liberia afford to do it?

Note: This blog reflects only the views of the authors of the blog. For sources on cited material in this blog, see the paper here.

Recently, the American Journal of Tropical Medicine & Hygiene published a paper by Shepard et al. evaluating the impact of HIV/AIDS funding on Rwanda’s health system. The headline of the press release was catchy and assertive: “Six-year Study in Rwanda Finds Influx of HIV/AIDS Funding Does Not Undermine Health Care Services for Other Diseases. Study Addresses Long-standing Debate about Funding Imbalances for Global Diseases.”

But after reading the report, we quickly assessed that a more accurate and appropriate press release headline for this paper would be “Some Differences Observed in General Healthcare Delivery between Facilities with and without HIV/AIDS Services in Rural Rwanda.” The study has serious limitations associated with its design and its generalizability that aren’t reflected in its catchy press release, and thus have unfortunately gone unrecognized. And because there is, in fact, an important and “long-standing debate about funding imbalances for global diseases” that this study does not sufficiently address, it’s important to examination the shortcomings of the study’s results:.

1. Internal Validity: Does the study do what it claims to do?

No. Treatment was not randomly assigned, while matching and control strategies do not mitigate the effects generated by non-random assignment. As a result, the study’s current comparisons between the treatment and comparison groups are problematic in validly testing the proposed hypothesis.

The paper analyzes a “randomly selected” intervention group of 25 health centers that provided HIV/AIDS services, which is then “perfectly matched” to a control group of 25 health centers that did not offer HIV/AIDS services. But in reality, the intervention group was “randomly” selected only in the sense that the authors chose to study them, not that the health centers in the intervention group were randomly assigned for treatment.

Indeed, why were these health centers chosen to receive HIV/AIDS funding in the first place, back in 2002 or whenever? It’s quite possible that the centers were assigned to have HIV/AIDS funding because the centers were already more likely to have better outcomes. For example, centers that received funding may have had more and better (or better paid) doctors, or perhaps they were located in areas with higher population density, or with higher HIV/AIDS prevalence rates. Similarly, the authors note that, unlike the rural areas that were the subject of the study, all urban health centers in Rwanda provide HIV/AIDS services; this fact alone suggests that treatment (HIV/AIDS funding) was initially assigned based on facility characteristics rather than a random assignment in a representative list of centers.

The authors attempt to address this issue by matching the 25 intervention health centers to 25 control health centers. But the authors match on just three characteristics – (1) health center ownership, (2) performance-based financing, and (3) district income in 2002; however, it is unclear that these were the criteria for initial assignment to treatment.

Further, the authors do not provide any information to reassure us that the intervention group and control group were comparable on a range of relevant characteristics prior to treatment that might otherwise explain differential performance.

2. External Validity: How generalizable are study’s claims?

Beyond the internal validity constraints, the generalizability of the study’s findings is very limited.

The study—and particularly the press release—claims to measure the effects of HIV/AIDS funding on non-HIV/AIDS health services. Such a claim, however, ignores the numerous channels by which HIV/AIDS funding can affect a health system besides funding HIV/AIDS treatment in existing facilities; for example, HIV/AIDS funding can lead to technical assistance at the national level, newly built facilities operated by international NGOs or other foreign organizations, as well as health promotion and preventive care at the community level. But the authors’ indicator for HIV/AIDS funding is simply a binary categorization of whether a facility offered HIV/AIDS treatment or not. Moreover, the paper does not discuss the magnitude of funding, the funding source (PEPFAR or Global Fund vs. Ministry of Health disbursements), or whether the facility received an earmarked funding stream specifically for HIV/AIDS rather than general funds which it then elected to spend on HIV service provision. The narrowly focused study does not consider the wide array of other system level effects created by HIV/AIDS funding that have been raised in the previous literature.

In particular, the study does not tell us anything about the effects of parallel NGO service delivery or the impact of new or dedicated facilities exclusively for HIV/AIDS, both of which are hot topics in the HIV/AIDS health systems debate. Indeed, in 2008, less than 5% of Rwanda’s PEPFAR funding was channeled through national institutions; the rest was delivered via a range of contractors, most of which were American NGOs or universities (Table 1). The paper makes no effort to address the consequences this funding arrangement and the presence of the 44 PEPFAR prime partners in Rwanda.

Table 1: Top Planned Recipients of PEPFAR Funding for Rwanda (USD), FY2008

What’s more, this particular country (Rwanda) is likely to be an outlier among HIV/AIDS funding recipients due to its exceptional national healthcare system, high quality HIV/AIDS service delivery, and innovative health initiatives like community-based health insurance. According to the World Bank’s World Wide Governance Indicators for 2009, Rwanda ranked 7th out of 45 Sub-Saharan African countries for government effectiveness, scoring more than one standard deviation above the mean. Moreover, HIV/AIDS funding in Rwanda accounted for about a fifth of total health spending, a percentage higher than 30 other countries in sub-Saharan Africa.

We understand that the authors likely suffered from significant data constraints; likewise, we recognize the enormous empirical challenges in demonstrating system-wide effects at the national level. Still, it remains important to carefully state qualify results and recognize the limitations of one’s research.

Bottom line: The jury is still out on whether HIV/AIDS funding has displaced or improved efforts on other disease control priorities.  Let the debate about funding imbalances for global diseases continue…

The U.S. approach to counter-terrorism cooperation desperately needs revisiting.  The Atlantic Council’s Peter Pham noted, in more diplomatic language than I’ll use here, that the total collapse of the Malian security forces (recall that a collection of Tuareg separatists and Jihadist elements took Gao, Kidal, and Timbuktu in just three days following the March 22 coup) suggests that something is very wrong about the U.S. approach to counterterrorism cooperation in the Sahel.

Indeed.  The two-pronged military-civilian strategy has been to (a) build security capacity of the Malian and other regional militaries to control territory and fight terrorists and (b) take steps to prevent the spread of violent extremism. The cornerstones of this approach are, since 2005, the State-led Trans Sahara Counterterrorism Partnership and the Pentagon’s Operation Enduring Freedom-Trans Sahara (sometimes referred to jointly, in elegance only possible in government circles, as TSCTP/OEF-TS).

After seven years (or a full decade if you count DoD’s Pan Sahel Initiative that predates OEF-TS), the USG apparently has made little headway in building the capacity of the Malian security forces. Mali’s army was losing badly to Tuareg rebels prior to the coup and then crumbled nearly overnight, leaving an area larger than Texas literally ungoverned.  Perhaps just as worrying, U.S. efforts and training (including that of the coup leader) also didn’t seem to result in the personal relationships with the Malian officer corps that would have given the U.S. better eyes into the barracks and perhaps levers to deter a military take-over just weeks before an election where the incumbent president was retiring anyway.

This is all a pretty big indictment of our approach so far. (Full disclosure:  I am partially to blame for this failure.  As Deputy Assistant Secretary for West Africa 2007-08, TSCTP was partly under my watch and, in hindsight, deserved much deeper scrutiny from me and my colleagues.) At a minimum, the Mali debacle should invite some serious reflection on what we think we are doing to promote security and contain violent extremism in places like the Sahel and East Africa.

Unfortunately, the early signs aren’t good that the USG really recognizes the scale of the problem. At the Heritage event, the State Dept rep followed her talking points by emphasizing that USG interagency engagement via TSCTP and existing tools will continue. In other words: what we are doing isn’t working, but we are planning on doing more of the same. Sigh.

U.S. policy dysfunction is at least part of the problem.  It’s certainly true that Mali’s problems are highly complex while U.S. policy instruments tend to be very blunt. It’s also possible that Mali would have plunged overnight from a seemingly-model democracy to a failed state regardless of what the U.S. did (I don’t believe this, but it’s not implausible).

But I absolutely know we can do better. Excessive stove-piping and the confused ‘whole of government’ mess that plagues all our overseas efforts don’t help. The inability of the interagency to clarify objectives, deploy tools, and monitor how things are progressing is all hard enough for a single sector in a single country; it’s nearly impossible for a regional multi-sectoral effort like TSCTP/OEF-TS.  Back at Heritage’s panel, Alexis Arieff of the Congressional Research Service provided an excellent example here.  In describing USG efforts in the Sahel, she charged that no one knows for sure what the U.S. has invested since the programming is spread across so many different budget lines and authorities. Put another way, for the signature USG counterterrorism initiative for West Africa, even insiders cannot figure out how much we are spending nor, by implication, what the money is for.  (The Washington Post figured this out this confusion some time ago too.)  No wonder it’s not working.

I’ve tried, in my civilian post-government life and with time to reflect on this, to figure out more about TSCTP, what are the underlying ideas, and how programs map to those assumptions, but have come up largely empty and frustrated. (There are exceptions, like these useful studies from USAID here and here.)  If we really believe that fighting terrorism in Africa is in our national interest, then the disaster still unfolding in Mali begs for an honest and aggressive rethinking of both the what and the how.

When President Obama released his FY2013 request for the MCC, I expressed concern that this low number was only the opening salvo, subject to reduction in congressional mark-up. So it’s good to see the House recognize the importance of the MCC’s mission and model and fulfill the request.  Assuming the $898 million request for MCC is appropriated, FY2013 will be the third consecutive year that the MCC has been funded at $898 million.

Also worth noting, the House subcommittee included language on the MCC’s new income categories that define “low income” as the poorest 75 countries. These new definitions first passed in FY2012 but will have to be legislated on an annual basis.

Pregnancy poses a substantial health risk for adolescents. The World Health Organization reports that health problems associated with adolescent pregnancy include increased maternal and neonatal mortality and increased incidence of preterm or low birth weight, among many other adverse health conditions. A study from the British Medical Journal found that complications from pregnancy and childbirth were the leading causes of death for young women between the ages of 15-19 in developing countries.

Yet beyond the health and rights consequences of teenage fertility, little work has systematized the extent of the non-health adverse effects associated with adolescent fertility and the effectiveness of the interventions used to date to reduce teenage fertility in low- and middle-income countries.  In a new paper, Kate McQueston, Rachel Silverman and I aim to fill this gap.

First, we explore trends between adolescent childbearing and socioeconomic outcomes. While the review finds strong correlations between adolescent fertility and school drop-out, the question of causation remains far more ambiguous, as effect sizes decrease sharply with more rigorous research methods. Moreover, the study also finds that in some contexts, high numbers of women continue education after child birth—suggesting that childbirth and education (and other related outcomes) may not be incompatible.

Similarly, the review of interventions to reduce adolescent fertility finds variation across studies, but also notes some general findings. The evidence base for conditional cash transfers, though somewhat variable, is by far the most robust when compared to the other interventions. Additionally, programs that lowered barriers to attending school or increased the opportunity costs of not attending school were also found to be effective—suggesting that education may substitute for adolescent fertility. Notably, the most effective interventions appeared to be outside of the typical reproductive health sphere.

The paper is limited by the quality and scope of the studies available. Further, adolescent fertility and its causes are complex, nuanced issues that are affected by a range of motivations and external factors. Nonetheless, this research reframes the conversation about adolescent fertility and the policies and interventions that might be used to reduce its frequency.

Among other findings, we suggest that adolescent fertility is more consequence than cause of socioeconomic disadvantage. While fertility is often correlated with school dropout, other factors – current school enrollment, marital status, anticipated economic returns to education, family attitudes, and other related context — drive both school continuation and the likelihood of experiencing an adolescent pregnancy. This finding suggests that a more holistic (and possibly complex) approach may be needed to address the foundational causes of adolescent fertility. Creating economic opportunities for women, reducing adolescent marriage, and changing gender norms are likely to be more effective in the quest to accelerate economic development than merely reducing adolescent fertility.

While interventions that focused on increasing knowledge and changing attitudes about sexual and reproductive health appeared successful in the short term, there was little evidence of any long term impact. On the contrary, interventions that encouraged school attendance proved more effective in reducing overall adolescent fertility. This evidence suggests that policymakers should expand educational opportunities for girls and create incentives for school continuation, such as conditional cash payments or the expectation of a worthwhile job following graduation. Reproductive health services are important for many young women, but increasing contraceptive access and uptake may not, on its own, be sufficient to change fertility trends.

As the world prepares for the upcoming family planning summit, it is worth pausing to examine the findings of the growing experimental literature on programs that aim to reduce teen pregnancy and to consider a broader and synergistic approach.

The potential end-of-year train wreck will be caused by the very large differences among White House, Senate, and House budgets.  The House adopted a budget resolution that is $19 billion less than the Senate’s.  In terms of the international affairs budget, this means that the House is operating with a $40.1 billion base allocation while the Senate comes in at $49.8 billion.  That’s a nearly 20 percent difference.  (Ouch.)

At these levels, it will be very difficult for the State Department and USAID to effectively run their foreign affairs portfolios, not just because of the stark spending cuts but also because of the numerous restrictions, directives, and earmarks that reduce agency flexibility.  I say this as a believer that the budget can be reoriented to accommodate cuts and improve effectiveness, but also as an advocate of greater flexibility and less micro-managing.

But what about OCO, you ask?  The Overseas Contingency Operations request was designed to accommodate temporary and extraordinary expenses related to operations in Afghanistan, Pakistan, and Iraq.  Last year’s potential budget disaster was ameliorated because appropriators shifted some base funding to OCO, which by the way, doesn’t count against spending caps.

So what did the House Committee do with OCO for 2013?  It fully funded the Administration’s request of $8.2 billion.  Of course, House appropriators rearranged what gets funded in OCO, cutting in half the amount for State Department operations and ESF funding, while increasing amounts for USAID operating expenses, international organizations, disaster assistance, transition initiatives, migration aid, and security assistance.  The problem?  As I’ve discussed before, a dangerous precedent is set if some of this money isn’t truly temporary and extraordinary.  When the time comes to put this money back in the base, there could be justified resistance.

But there’s another problem with how 2013 will play out.  While the House agreed to $8.2 billion for OCO, the Senate’s allocation is just $3.2 billion.  This is one of the reasons that the House’s base budget is so much lower than the Senate’s.  (Combining the base and OCO, the House number is still about 9 percent less than the Senate.)

The difficulty will come as the two chambers try to reconcile these differences.  We’ll know a little more next week when the Senate panel marks-up.  Don’t expect this to sort itself out prior to the elections.  This will be end-of-year calculus with way too many variables, including who wins the White House, and which party controls the House and Senate.  And then there’s the threat of sequestration.

So what does all this mean for development?  When the budget doesn’t get completed by the beginning of the fiscal year, the release of those funds is delayed and the period in which they must be spent is compressed.  As agencies struggle to ship the money out the door, I can’t help but think that some due diligence in oversight might be sidestepped.  The future of programs and projects in the field are left dangling and the United States is not seen as a reliable partner.

And don’t get me started on the hilarity of the U.S. focus on governance…

There were just six months left in Mali’s MCA compact when the military coup occurred. One might assume the impact of a slightly early finish would be minimal. But the MCA compacts invest an enormous amount of energy setting up systems with the partner country in the early years and much of the final disbursements and construction takes place in the final months. Of the $460 million compact, the MCA had expended, but not yet paid or invoiced, $350 million as of December 2011. That means Mali has lost upwards of $100 million in MCA money and the anticipated benefits (most of which occur after a compact is finished) along with it.  The MCC will have to spend some of that money, instead, to extricate themselves from unfinished projects like the airport terminal construction site.

Mali’s coup is an unwelcome reminder of how quickly events can change even in well-governed developing countries and the huge impact political upheaval has on a country’s economy. The U.S. government is rightly pushing for a quick restoration of constitutional civilian rule in Mali. Democracy prevailed in Senegal  (whose MCA compact is underway) and Malawi (whose MCA compact is currently suspended). Let’s hope things turn around in Mali, too (I need to ask my colleague Todd Moss how his prescient draft novel about a coup in Mali and the diplomatic efforts to reverse it ends).

He has conceded to me the argument over what the economic studies say. So his main thesis now is:

I think David, in his faith in the power of numbers, has strayed too far from the path of Common Sense. Example: If, as David concludes, there is value to be found in the creation of an industry that has corrected—for many millions of people—the “market failure” of financial systems in Developing Countries, is it logical to then conclude that the impact on global poverty of this gargantuan investment has been zero?

Are we to believe that joining millions of people who earn their living each day in the informal sector, and who previously had no access to credit or savings services, and now do, are not materially better off as a result of it? I think these two conclusions marry up like drunkards in a Las Vegas wedding. Let’s get them across the border to Mexico for an annulment.

He then tells a couple of good stories of encounters he has had with borrowers, the first being of a borrower who says she climbed from poverty thanks to microcredit. He concludes:

David would say “No causality!” Rupert would say “Hey, listen to the woman! Maybe she’s not making it up!”

I’d respectfully correct a few things Rupert says about me:

• The thrust of my analysis of the econometric impact literature is actually about my lack of faith in the power of numbers—about why I doubt most microfinance impact studies. That said, Rupert’s description is fair to the extent that I do think the results of the new, experimental studies need to be taken seriously, and that in my judgment they trump small, ad hoc assemblages of stories.
• I don’t hate stories. Chapter 3, 4, and 7 include lots of them, and not for the purpose of disparagement. I know that people understand and explain the world through stories. I do think though that in talking of microfinance, in particular of whether it should receive charity, we should tell stories that are representative. And we should judge representativeness through rigorous analysis of evidence of both the narrative and numerical kinds. I don’t see Rupert doing that. I would say “Listen to that woman, and to a bunch of other women who are selected in some transparent and representative way.”
• I do not conclude that the impact of Rupert’s career on poverty has been zero. I conclude, rather, that zero is the best estimate given limited rigorous evidence of the impact of microcredit on the income and spending levels of clients. For more see Why Celebrate Institutions that Deliver a Service that Doesn’t Reduce Poverty. Yes, that is a pretty serious challenge to Rupert’s work, especially since I think it should affect funding for microfinance. But I’d like to avoid exaggerations of my certainty that imply sloppy reasoning.
• Just for the record, I don’t describe microfinance as reducing “market failure.” The term is jargon and it elevates an unrealistic economic abstraction to the level of an ideal.

A last point: as lovely as Rupert’s metaphor is, the one about drunkards in matrimony, I don’t think it applies. It doesn’t violate common sense to suggest that a massive ramp-up in a lending industry could hurt people.

Budget concerns will almost certainly put downward pressure on federal spending across a host of government programs for a number of years.  Although some think it is almost heretical to point out the obvious, the international affairs budget will not be immune from this dynamic. In fact, international spending could take a disproportionate hit compared to domestic spending – despite the fact that discretionary international spending is a very small part of the overall budget puzzle.

International affairs, and more specifically foreign assistance, have rarely been popular budget items among the public or on Capitol Hill – despite consistently comprising only about 1 percent of the total federal budget.  Even so, foreign aid and international engagement make good political targets for elected officials out on the stump. It is far easier to demonize foreign aid than to explain how relatively modest programs to improve living standards in the developing world have consistently proven to be in the national interest over the long-term.

The central question then becomes how do we maintain U.S. global leadership in development and improve the effectiveness and efficiency of aid programs at a time when the international affairs budget is surrounded by so much uncertainty?

Last fall, we set up a bipartisan working group to think through this question and look at how to reorient the international affairs budget during this current period of austerity. The resulting report, Engagement Amid Austerity, is now available.

This report outlines four big ideas as a framework for reorienting the foreign affairs budget:

• Be more selective and focused on what types of economic and security assistance are provided to which countries.
• Put PEPFAR programs in upper middle income countries on an increased cost-sharing trajectory.
• Reform U.S. food assistance programs by eliminating monetization, cargo preference, and allowing more local and regional purchase of emergency food aid.
• Create an International Affairs Realignment Commission to examine and redesign programs and architecture.

While we are not advocating for cuts to the foreign affairs budget, it is abundantly clear that the United States is spread far too thinly in its assistance programs. The United States currently provides economic assistance to 103 countries and security assistance to 143 countries.  U.S. assistance programs are trying to do too many things in too many places without clear objectives.  In addition, the United States continues to provide aid to far too many countries that are simply poor partners. If a country’s leadership is unwilling to embrace reform, democracy, and more open markets, there is little reason to think that U.S. aid programs will make much of a difference over the long haul.

We believe that programs can be better focused for greater impact.  In short, we should be directing more resources into fewer countries. Such a footprint would be far easier to manage, entail fewer operational costs, and help shape countries into partners that no longer require U.S. assistance five to ten years from now. Based on a data-informed process assessing a country’s need, capacity, governance, and commitment to development, as well as subjective judgments, our report rates every single one of the 146 countries receiving U.S. assistance as to the likelihood that U.S. aid will be effective. We recommend focusing economic assistance in 53 countries, and focusing security aid in 72 countries.

Others may reach alternative conclusions using this same data, and we have provided as much information to readers as possible so that they can do so.  We do not expect universal agreement about our conclusions. But amid all the debates that will take place over the next year on how much we spend on international affairs programs, it is equally vital to engage in an overdue conversation on how we spend it.

This strong and long overdue pitch to bring back the ‘P’ word into the environmental debate is most welcome.  In recent decades, international attention has shifted from rapid population growth to other urgent issues, such as the HIV/AIDS epidemic, humanitarian crises, climate change, and good governance.  But reproductive health and voluntary family planning programs are still very much needed, especially in high fertility countries, and they require political leadership and long-term financial commitment.  Broader access to family planning services will be needed to accelerate the decline of high fertility rates, particularly in countries where unmet needs for contraception are high.

However, as the report highlights, policies to address population and the environment must go well beyond family planning.  They should stress, first and foremost, the importance of inclusive development.  Today, 1.3 billion people still live with only US$ 1.25 per day.  The international community needs to lift them out of absolute poverty, which will require focused efforts in economic development, education, and health including family planning.  For their part, developed and emerging countries must first stabilize and thereafter reduce their levels of material consumption.  This can be achieved through greater efficiency in the use of resources as well as an array of practical measures to reduce waste, invest in sustainable resources, technologies, and infrastructures, and systematically decouple economic activity from its environmental impact.

Other policy levers should be explored as well.  One should harness the potential for urbanization to reduce material consumption, remove barriers to achieve high-quality education for all at both primary and secondary levels, implement comprehensive wealth measures (i.e., reform the system of national accounts and improve natural assets accounting), and develop new socio-economic systems that are not dependent on continued material consumption growth.  More research is also needed into the interactions between demographic change, consumption, and environmental impact.

I hope the ideas presented in People and the Planet will usher an entirely new way of thinking about population issues and sustainable development and that it will be taken into account at the upcoming Rio+20 Conference.  Population policies and programs should no longer be viewed as necessary and relevant in their own right.  On the contrary, such policies should be integrated with broader and comprehensive interventions that address economic equality, health and education needs, and technological advances.  The ultimate goal should be to improve the life of all human beings as a necessary condition to protect our environment and safeguard the sustainability of our way of living.

This month, John releases a new book, World Population Policies, that examines the links between population and reproductive health policies and economic development efforts. Of the book, demography great John Bongaarts writes: “After more than a decade of neglect, population trends and their adverse social, economic, health, environmental, and political effects have returned to the global policy agenda. Government interventions aimed at minimizing unfavorable demographic developments have often been contentious. This timely and comprehensive book provides a wealth of valuable insight and thoughtful commentary on controversies and policy options.”

CGD has a history of working at the intersection of population and development, as demographic changes affect economic growth and poverty reduction in many parts of the world. CGD’s past work on this topic includes the Demographics and Development in the 21st Century initiative and the Population and Development working group, among others.

We are pleased to welcome John to our team—look for more from him here on our Global Health Policy Blog.

One of the most prominent issues underlying all these discussions was the growing disenchantment in Europe with general budget support programs. The main political argument being leveled against budget support is that it can’t demonstrate performance, and the basic response has been to move toward project-specific aid. If European aid agencies move in this direction it will be a shame – throwing out the baby with the bathwater. My hope is that they will see that they can preserve the good aspects of budget support – working through country systems and giving recipients greater ownership – by agreeing to disburse flexible funds in relation to progress on a few key high-level indicators, such as educational attainment, reduced child mortality, better security, less deforestation or cleaner energy.

The discussion of Results-Based Aid is also extremely useful for uncovering the dynamics of foreign aid politics. For example, a key advantage of results-based mechanisms is that they can reduce transaction costs associated with tracking inputs. Nevertheless, we heard several cases in which the measurement of outcome indicators was simply added on top of existing spending control mechanisms. In addition, results-based aid from one government to another should be an opportunity to keep attention on broad high-level goals and leave the recipient with flexibility on how they respond. Instead, it is extremely tempting for the discussion to fall into using the payments to get “them” to do what “we” think they should.

The entire conception of foreign aid is changing – with new actors, new constraints, and new ideas. I think the process of working out these new ideas in practice will show if the system can really be reformed or whether it will be increasingly marginalized.

Today Debrework Zewdie, the Deputy General Manager of the Global Fund, spoke on “The Global Fund at Ten Years: Reflecting on its Impact and Looking Forward to Challenges Ahead”, followed by extensive Q&A at the Council for Foreign Relations. Her brief remarks summarized the immense challenges of undertaking the Consolidated Transformation Plan (see Jaramillo’s slides here for more details). In particular, I am very encouraged about the prospect of two changes: (1) the creation of a new Division called ‘Strategic Investment and Impact Evaluation’ which will shape the optimal portfolio of investments by country and disease (rather than its prior ad-hoc, or more kindly, its one-size-fits-all, approach), and (2) the creation of new committees for each disease (AIDS, Tuberculosis, and Malaria) that will meet monthly to discuss the bigger picture of the ‘disease war’, and not individual grant/country ‘battles’. Both of these have great importance for achieving better ‘value for money’.

The new Division and committees will both help ensure that the Fund’s portfolio obtains greater health gains for the funding invested. Whereas in the past there had been consternation among observers that the Technical Review Panel failed to consider the costs of interventions proposed by a country and did not optimize disease control strategies relative to spending in its set of countries, these two changes will help gain better ‘value for money’. Nevertheless, care should be taken to not mechanically allocate funding by disease burden, as it may be optimal to control diseases based on the population at risk or other factors in disease spread (in the case of malaria or tuberculosis for example). Furthermore, getting the mix of interventions funded right is as important as getting the right level of funding by country. Most encouraging is that the positions in this new Division will be staffed by those who manage grants and are in the grant-making team, and not external reviewers. We look forward to seeing that this new Division has the needed leverage in staff to optimize their current portfolio.

Zewdie also noted an increased focus on impact evaluation of Global Fund grants. This could also be a potentially huge improvement for getting better value for money. Zewdie mentioned that while the Global Fund had done some impact evaluation in the past, it had not been systemized and integrated with grants nor had external experts been sought (it would be nice to know more about these past impact evaluations). I asked Zewdie on the potential uses of impact evaluation, in particular for learning general lessons on delivery strategies which may be applicable to multiple countries and whether impact evaluation might be integrated with the Global Fund’s current ‘performance’-based financing (PBF) model. One new delivery strategy that Zewdie mentioned, for example, was better coordination and collaboration of the Global Fund with PEPFAR, starting in Nigeria and 1 other country. An impact evaluation of this strategy would greatly benefit other countries as well. Learning into better delivery strategies and approaches, if looped back into grant management, will give more value for money.

As for ‘performance’-based financing (PBF) and impact evaluation, although the Fund pioneered PBF, the current PBF approach has many limitations; most indicators are related to ‘outputs’ e.g. number of bed nets distributed (not a true measure of performance in health and vulnerable to unintended consequences) and decisions on disbursements have not necessarily related to performance. But exactly how impact evaluation might be integrated with PBF will need careful thinking, so that such evaluation helps countries to learn and rewards them with successful, while avoiding bad incentives through harsh punishment (which is true for PBF in general). For the Global Fund to make appropriate connections between impact evaluation and PBF would be an important, if not forward-thinking, step among global health donors. Work at the Center for Global Development on Cash on Delivery, with its approach on minimal indicators, independent verification, and public dissemination of indicators, may be useful as the Fund moves forward. By further creating incentives for countries to reach certain levels of performance, the Fund can also gain better value for money.

The challenge of conducting impact evaluation will require able staff with expertise in a wide array of tools for impact evaluation – which are not necessarily onerous with high costs. The methodologies of impact evaluation, largely from the fields of economics, political science, statistics and perhaps to a lesser extent epidemiology, have advanced, such that an array of quasi-experimental tools (e.g. staggered phase-in) can be employed without the burden of ‘randomization’ or exclusive surveys per se. Moreover, this work on impact evaluation may be used to build national health information systems which have been suffering from neglect in many countries in recent years despite the growth of global health funding.

I’m very encouraged and hopeful that the Global Fund is on the right track. I look forward to seeing these new changes and we hope our Value for Money working group will give useful recommendations to the Global Fund and other global health funding agencies to save more lives for the money.

(Thanks to Amanda Glassman, Denizhan Duran, and Jenny Ottenhoff for their very helpful comments.)

I also thank my colleagues who participated in the Non-Commission and encourage readers to visit them at their sites:

• New Rules for Global Finance Coalition: http://www.new-rules.org/ (Jo Marie Griesgraber)
• G-24: http://www.g24.org/ (Amar Bhattacharya)
• Brookings Program on Global Economy and Development: http://www.brookings.edu/global.aspx (Colin Bradford)
• Ted Truman: http://www.iie.com/staff/author_bio.cfm?author_id=122
• Ralph Bryant: http://www.brookings.edu/experts/b/bryantr.aspx
• Domenico Lombardi: http://www.brookings.edu/experts/lombardid.aspx

Building on the recommendations of the Zedillo report, I suggest four priorities below for reformat the World Bank. I then add one worry about an issue the report failed to grapple with.

1. IBRD should go 50/50 (voting shares), with developing countries rising not from 44 to 47 percent (as recently endorsed by the G-20) but to 50 percent. The 50/50 allocation should be tied to the planned capital increase. It’s the borrowers that want a bigger bank. Many have surplus reserves and could put in more paid-in capital; perhaps non-borrowers with big fiscal problems like the U.S. could increase their contributions in the form of callable capital (which would not require budget appropriations). Need the paid-in ratio be the same for everyone? By the way, the Inter American Development Bank (IDB) has been a 50/50 bank since 1994. The effect there has been more than symbolic (greater legitimacy and ownership have affected the agenda and the budget) but by no means has constituted a revolution — for good or ill.
2. IDA governance. IDA should have a separate governance structure from IBRD. The Commission recommends a voting arrangement for IDA more tied to recent contributions, but is not explicit about separating IDA from IBRD governance. Here lies a possible bargain with the UK and other Europeans: they keep more votes and chairs at IDA but give up some chairs at IBRD (where they have 8 of 24 now). IDA should also be 50/50, including only donors and IDA recipients. Middle-income countries like China and Brazil could join the donors by contributing to IDA directly.
3. Presidential selection. The G-20 and the Zedillo Commission have spoken: it is time to end the grip that the U.S. and Europe have on selecting the heads of the World Bank and the IMF respectively. On this issue, I advocate double majority voting – a majority of weighted votes, and a majority of countries. This method would enhance institutional legitimacy since small, poor countries could, if they formed a coalition, be able to block a candidate they didn’t like – just as the U.S. can (and could) do all by itself. That way an elected president would have serious and broad support, thus enhancing his or her legitimacy – and therefore the effectiveness – of the institutions. Meanwhile the U.S. Congress still needs to be reassured that the U.S. would retain a de facto veto on any candidate.
4. Global public goods. As I argue in a recent speech, a new wing at the World Bank should be a huge priority. Its governance should be shaped by three principles. First, periodic member contributions should be related positively to per capita income and to emissions per capita. Second, developing countries as a group should have influence equal to that of developed countries, whether through 50 percent of weighted votes or other voting rules. Finally, in legal and operational terms, the new entity should be as distinct or even more distinct from the main lending arm of the bank as the Multilateral Investment Guarantee Agency (MIGA) and the International Finance Corporation (IFC). And why not put the new wing in Beijing or New Delhi? Developing countries have $3 trillion in reserves. Surely they would welcome an opportunity to lend to or capitalize a substantial GPG Trust Fund. Better for them and for the global economy that some of those reserves be intermediated through a legitimate global institution. The Climate Investment Funds are already governed under a 50/50 arrangement and a GPG or Climate Fund should be similarly set up. To start, it should be at least $3 billion with at least $1.5 billion from advanced developing countries.

Finally, I have one deep worry: The Commission calls for greater flexibility and speed and for reduction of the “hassle factor” in Bank operations. But at the same time it wants increased attention to environmental, social, anti-corruption safeguards, both internal and in all lending operations that have piled up like barnacles on World Bank operations. How to reconcile these two fundamentally conflicting demands? The Commission should have pushed for more emphasis on independent, periodic ex post audits of ongoing programs and projects, and for the Bank to exit (projects and countries) that fail audits.

In general, the Commission failed to deal with the Bank’s obsession to lend (ignoring the problem of procyclicality), its focus on disbursements, the lack of ability to exit once the Bank is engaged in a country, the limited incentives to innovate, and the willful pretense that all risks – fiduciary, waste and corruption, environmental and so on – can be managed ex ante.