A randomized controlled trial conducted among some of the poorest residents in Nairobi included threatening the disconnection of water and sanitation services if landlords didn’t pay outstanding debts.
The study aimed to understand how to enforce payment for water and sanitation services and resulted in 97 of the 299 compounds selected for the enforcement intervention seeing their water and sewage services cut off — some for up to nine months. This sparked a Twitter-fueled backlash over concerns that marginalized communities lost access to water for the sake of research.
As the authors scrambled to clarify that their study did not increase anyone’s risk of disconnection, the debate resurrected concerns about how to ethically conduct and present research on vulnerable communities, particularly when it involves access to essential services.
“This is not an RCT [randomized controlled trial] to improve people’s lives. This is an RCT to assess whether or not there is a political backlash.”Raul Pacheco-Vega, political scientist and geographer
Consequences of research
The working paper, published by the U.S. National Bureau of Economic Research in late July, was drafted by a team that includes researchers from the World Bank, the University of Maryland, and the University of California, Berkeley. It emerged from a partnership with the Nairobi City Water and Sewerage Company to improve access to water and sanitation services within informal settlements. That partnership has boosted service uptake by 90%, the World Bank’s Aidan Coville — one of the authors — told Devex, but 70 % of the accounts in the research area quickly fell into arrears.
“This made it more difficult for the utility to expand services to new households that were not yet connected,” Coville said, prompting the team’s decision to research two different strategies for encouraging payments from delinquent compound landlords without drawing political consequences from tenants. Those consequences were measured in participation in political activities, like joining community meetings or reaching out to community leaders.
A softer intervention saw NCWSC authorities visit households and share information on how to pay had little effect. The hard threat of disconnection, resulting in the actual disconnection of some compounds, on the other hand, “increases payment and the financial position of the utility without incurring political costs,” according to the July working paper, which emerged from this research.
The paper publication fueled the backlash from social scientists and members of the WASH community, aghast that a randomized controlled trial might have caused unknowing participants to lose access to water. They were particularly alarmed by a footnote in the working paper that confirmed that, despite having a disconnection policy in place, NCWSC rarely employed it in poor communities that were likely unable to pay.
“This is not an RCT to improve people’s lives,” said Raul Pacheco-Vega, a political scientist and geographer, who researches the factors that contribute to natural resource governance. “This is an RCT to assess whether or not there is a political backlash. If there is no backlash, it’s not because people were happy, it’s because they don’t have enough political capital to feel safe being politically active.”
In response to the outrage, the authors issued two ethics comments clarifying that NCWSC had already made the decision to reverse its haphazard nonenforcement policy and move forward more systematically to disconnect compounds where landlords had not paid. They said the study only affected compounds that would have received notices anyway and actually reduced the overall number, since some households that would have faced disconnection received the softer intervention instead.
They also reframed the research question, writing that the research was asking whether the “standard policy of service disconnection is a good policy in order to make the provision of water and sanitation to the poor-sustainable or are the costs of the disconnection too high.” The focus on political costs was diminished.
Their responses have not resolved all of the outstanding questions, though, including whether the participants understood they were involved in a randomized controlled trial, what relationship they had with the landlords — who were ultimately responsible for the overdue bills, and what steps were taken to protect the health of the participants — particularly those that lost access to water.
On the latter question, the working paper only said that the authors monitored whether the oldest child under 5 years old had diarrhea or fever over a two-week period, but Kawango Agot, director of the Kenyan NGO Impact Research and Development Organization, dismissed that as “a rudimentary way of looking at health.”
“Issues regarding risks, benefits, and informed consent were all considered extensively in the planning and implementation of the work,” Coville told Devex, emphasizing that a revised paper will offer a fuller explanation.
Chris Prottas, motivated by his own work trying to improve the sustainability of water points as the executive director of The Water Trust, wrote a detailed analysis of concerns that emerged from the initial working paper and the revisions offered by the ethics comments.
One of the key unanswered questions for him is what the research accomplishes. “When I look at the findings, it seems like there’s a marginal increase in spending, but that in no way demonstrates it’s adequate to improve functionality or service quality.”
The reaction to the working paper has engendered its own backlash, as economists and others have accused critics of assuming the authors operated in bad faith. But Pacheco-Vega said the scrutiny is warranted, because the working paper raised legitimate concerns about its effect on people’s health, but also because its initial conclusion might encourage other utilities to take a similar approach.
He argued that a more ethical approach to research around essential services would not have arrived at a conclusion that might be interpreted to justify cutting people’s access and would have worked more closely with local researchers to better understand the causes of nonpayment — including tenant-landlord relationships — and alternatives for overcoming them.
A local researcher might also bring an additional sensitivity to the particular risks of working in marginalized communities, Agot said, or to recommend against it entirely. In this instance, she believes, “It was not ethical to conduct such a study in such a population. They didn’t need to locate it where they located it to answer their question.”
While none of the authors are Kenyan, they did collaborate with sociologists working in the community, Coville said. And Kwame Owino, CEO of the Institute for Economic Affairs, a Kenyan think tank, said it was also important to remember that a Kenyan utility was involved.
“That they chose those methods and asked for those methods to be tested is not something to blame the researchers for,” he said. “That’s the nature of research. Sometimes people study difficult things.”
At the very least, Prottas said, there could be more clarity around the steps that are being taken to ensure that difficult research is not actually harmful and that the researchers have satisfactorily grappled with the ethical implications.
He called for greater transparency from the institutional review boards that approve potentially fraught studies so there is at least an opportunity to understand exactly what debates have already taken place and what protections are in place.
The water supply study received institutional review board approval from both a Kenyan university and from Innovations for Poverty Action, an organization using research to identify policies to assist the impoverished. A representative with IPA’s institutional review board confirmed to Devex that the organization will soon be releasing a statement authored by the entire board about the trial.