The Attribution Moloch

Introduction

There’s something that I’m worried about that is fairly close to the core of EA. I don’t know to which degree I should be worried, and I don’t know how pervasive it is, but nonprofit insiders have written about it and other insiders have told me about it with regard to specific cause areas, and I’m afraid prioritization that ignores it may exacerbate it.

Many thanks to Lukas Gloor, Melanie Joy, Sara Nowak, Jacy Reese, and Anne Wissemann for proofreading the draft of this article and adding many valuable thoughts!

Scarcity Breeds Competition

I don’t want to imply that any specific space suffers from this problem, so here’s one that I made up.

Everyone agrees that paperclips are not treated with the dignity that their charming smiles deserve. Few people, unfortunately, do something about it. Not so with us. Five years ago I founded Clippy Guards International (CGI). I found a small number of donors that are passionate about paperclips and donate to CGI so that I could employ two of CGI’s most diligent volunteers full time, Erin and Pat.

Recently acquaintances of ours founded a second organization, the No Staples Association (NSA), with the express goal of instituting a staple tax and banning staple ads – a promising but risky idea we had considered as well that is sure to increase demand for paperclips if it succeeds. If we had more resources, we could afford to take more risks, but as it is, we rather decided to rely on more incremental interventions so we don’t lose donors if a risky campaign fails.

The new organization is prominent in the social media circles that our donors read, and I’ve heard talk that some of our donors are interested in it. The week after NSA published its strategic plan for 2016, two of our donors cancelled their recurring donation orders to us.

Today Alice, the executive director of the No Staples Association, emailed us. NSA’s campaign is at an impasse, but Alice heard through the grapevine that we’re in touch with several key politicians. If I were to put Alice in touch with them, she’d have a shot at convincing them. I know Alice and know that she’s competent and won’t ruin our rapport with the politicians, I think that NSA’s campaign is highly promising, and I know that all of us only want the best for all the Clippies of the world.

But if NSA succeeds, more of our donors will reconsider their donations. On our tight budget that means that one of us will have to agree to leave our organization or work part-time. Erin is the one many of the politicians know and trust. Pat is the public face of the organization for our donors; if he left, we could lose even more donors. If I, the founder, were to leave, surely the organization would founder and donors would consider that I may not believe in it anymore.

I’m really sorry for NSA, and I wish there were another way, but it’s either us or them, and our campaigns are the less risky ones, so I vote for us. I deleted Alice’s email from our shared account before Erin and Pat could see it. I’m so sorry, Alice!

This is what I call the Attribution Moloch – in reference to Scott Alexander’s Moloch, a personification of defection in the prisoner’s dilemma–like situations that makes everyone worse off.¹

The basic idea of the prisoner’s dilemma is the following (adapted from Douglas Hofstadter):

Two people meet [without being able to identify each other] and exchange closed bags, with the understanding that one of them contains money, and the other contains a purchase. [The seller desires the money more than the product, and the buyer desires the product more than the money.] Either player can choose to honor the deal by putting into his or her bag what he or she agreed [can cooperate], or he or she can defect by handing over an empty bag. [Emphasis added.]

The best situation for both, together, is to realize their mutual gains from trade by cooperating, but it is rational for either to defect, since (1) they may get the money and the purchase, and (2) it is rational for the other to defect, so that each expects to lose money and purchase if they cooperate.

Aggravating Factors

Perverse Incentives from Partitioning Impact

I think resource scarcity is not the only problem facing our two paperclip charities. Another problem is lacking coordination² and a third is attribution.

To explain the problem with attribution – crediting agents for impact – I need to reference the concept of the INUS condition that John Mackie introduced in Causes and Conditions.

INUS Conditions

According to Mackie, what we typically think of as causes of an effect are really clusters of factors, each of which is necessary but insufficient to bring about the effect. Together this cluster of factors is sufficient but not necessary for the effect since other such clusters may have had the same effect.³ One typical example:

So to say that short circuits cause house fires is to say that the short circuit is an INUS condition for house fires. It is an insufficient part because it cannot cause the fire on its own (other conditions such as oxygen, inflammable material, etc. should be present). It is, nonetheless, a [necessary] part because, without it, the rest of the conditions are not sufficient for the fire. It is just a part, and not the whole, of a sufficient condition (which includes oxygen, the presence of inflammable material, etc.), but this whole sufficient condition is not necessary, since some other cluster of conditions, for example, an arsonist with gasoline, can produce the fire.

What donors do all too often when extrapolating from average cost-effectiveness to marginal cost-effectiveness is that they partition impact: They see some of the factors that were necessary for the impact and credit these factors with some fraction of the impact so that all fractions add up to one. But none of the impact would have manifested if even a single one of the factors had been missing, so there is no valid partition of the impact.⁴

School-based mass deworming programs, for example, are only as effective as they are because there are already schools and teachers in place. If there were enough teachers, no worms, and no schools, building schools would be more effective (or advocating for it on government level).⁵

The Ultra Poor Graduation Program is an explicit attempt at providing several factors following the realization of their necessity but individual insufficiency:

The Ultra Poor Graduation program is designed to graduate ultra poor households out of extreme poverty to a more stable state. This 24-month program provides beneficiaries with a holistic set of services including: livelihood trainings, productive asset transfers, consumption support, savings plans, and healthcare. By investing in this multifaceted approach, the program strives to eliminate the need for long-term safety net services.

Perverse Incentives

But when we partition impact as if one of the factors were necessary and sufficient, then we greatly overestimate the difference in overall cost-effectiveness between the final factor and the factors that paved the way for it for three reasons:

1. We assign the full impact to it when the impact was really shared among all factors.
2. Seeming causation is often attributed to the final factor to join the cluster, upon whose addition the impact emerged. It would not have been the last one to join if it had been obvious, so it was likely preceded by a number of failed attempts by others that informed it – attempts that did not yield the desired impact because they did not address the critical bottleneck.⁶
3. During the time until the final factor was added, the other factors had time to build considerable capacity, which the final factor free-rides on, to put it at little cynically.

When we’re looking for cost-effective interventions, we’re looking for the currently worst bottlenecks to creating impact and the ones that can be reduced most efficiently. We’re not looking for an intervention that can – by some artificial algorithm or other – be attributed with the greatest impact.

When we, as funders, make that mistake, we encourage charities to game the algorithm and hog credit rather than to bring about the best version of the future, and I know of no algorithm that could be used in practice to attribute impact in a way that would not create perverse incentives, that is, an algorithm that grants greater credit to an agent when the agent chooses actions that increase the overall impact compared to a counterfactual case where the agent chose a worse action. Hence (absent such an algorithm) we should abstain from attributing impact and think of top charities not as the charities that should be credited most highly (as the word “best” may imply) but as the charities that address the currently most compelling (i.e., important, tractable, and neglected) bottlenecks to achieving impact.⁷

Examples

As an example, consider an algorithm that attributes impact according to the staff time and budget invested by the agents: Two charities agree to fund half of the nets of a universal coverage campaign of long-lasting insecticide-treated mosquito nets. Universal coverage means that every bed is protected, which comes with additional valuable community-wide effects. The first charity buys the nets and ships them to the country. The other charity goes to some length to find the closest supplier to save shipping costs and makes a contract with the supplier that also covers a future distribution to get a greater bulk discount, resulting in 50% savings. The number and quality of the nets that the charities provide is identical, yet the second charity gets awarded only 33% of the impact. That’s not a good incentive structure.

Or consider an algorithm that attributes impact according to the capacities of the agents: In the case of school-based deworming mentioned earlier, schools, teachers, and students are in place, so a charity only needs to provide the pills. Great impact will be generated and the charity will get attributed a share of it. But if the charity heavily invests into building new schools right next to the old ones, hiring teachers for those schools, and luring over all the students from the old schools, it implements many of the factors that cause the impact itself. Hence it will get attributed more of the impact even though, compared to the counterfactual, it merely achieved the same impact at a greater cost.

Funding ratios are a silly metric for other reasons already, but they can get even more silly in situations that involve a misattribution of impact. Rohin Shah:

For example, perhaps Alice learns about EA from a local EA group, goes to a CFAR workshop, starts a company because of 80,000 Hours and takes the Founder’s Pledge and GWWC pledge. We now have five different organizations that can each claim credit for Alice’s impact, not to mention Alice herself. … However, the total impact caused would then be smaller than the sum of the impacts each organization thinks they had.

Imagine that Alice will now have an additional $2,000 of impact, and each organization spent $1,000 to accomplish this. Then each organization would (correctly) claim a leverage ratio of 2:1, but the aggregate outcome is that we spent $5,000 to get $2,000 of benefit, which is clearly suboptimal. These numbers are completely made up for pedagogical purposes and not meant to be actual estimates. In reality, even in this scenario I suspect that the ratio would be better than 1:1, though it would be smaller than the ratio each organization would compute for itself.

The only attribution algorithm that I can think of that could avoid these problems is one that compares the actual outcome to the optimal outcome, determines the agent whose decision led to the degradation of the actual outcome, if any, and penalizes it proportionately. This approach is rendered more complex as soon as several agents make less than optimal decisions, but even apart from that, the practical impossibility of knowing the optimal outcome and determining the decision that prevented it renders it purely hypothetical.

Who, for example, is ever going to find out about the email that the founder of CGI deleted in the introductory tale? CGI has impact-minded donors who donate to the best value-aligned charity they can find, but it’s opaque to them whether the “best charity” holds that position in the ranking fairly because of the impact it achieves or whether it hold the position unfairly because of the impact it prevents its “competitors” from achieving, thereby decreasing the overall impact. Such is the nature of rankings, which I am subsequently to critique.

Perverse Incentives from Prioritization

Prioritization that does not heed the above problem and does not go to some lengths to avoid it, will exacerbate it.

GiveWell need not worry though. Its top charities live in a postscarcity world thanks to Good Ventures and they have no reason to compete for funding. They also have fairly different programs and thus inhabit fairly different cause areas, reducing any potential for competition (and maybe also the potential for cooperation). But as we create more impact-minded donors and thus incentivize more charities to prioritize impact, comparative research into charities and the number of funding gaps that are compared will increase. And that’s awesome!

But resource scarcity will tend to increase among the majority of charities that are not considered top charities and thus lose some donors to the top charity group. Hence, naive prioritization that does not incentivize cooperation sufficiently can instead incentivize many kinds of defections among potential top charities, which can be very inconspicuous especially when they consist of omissions rather than actions. Such inconspicuous defection may include the following:

1. Organizations may systematically fail to warn one another of a potential public relations blunder – with direct negative effects on the affected organization and additional negative effects for the whole sector.
2. Organizations may systematically exaggerate their funding gaps. The staff of each organization may convince themselves that they’re more cost-effective than another organization so that their marginal cost-effectiveness will drop to the same level later, so they can legitimately hold funding for longer.
3. Organizations may systematically avoid mentioning other organizations in its materials or to donors.
4. Organizations may systematically fail to inform one another when they have access to information or contacts that would be crucial for another organization.
5. Organizations may systematically underinvest into preparatory work that would only allow others to plug the “low-hanging fruit,” the final factor that makes the cluster sufficient to cause the effect. As Lukas Gloor put it, “A universal focus on plucking low-hanging fruit in charity means that important preparatory work won’t get done.”

The total impact will be much curtailed, but no one will know the counterfactual.

What’s worse, the organizations that defect so clandestinely will emerge as seemingly more effective than those they defect against, which will be rewarded through increased donation flows. Those that are defected against will see diminishing donation flows and, when taken to the extreme, will be discontinued until the whole space is one of organizations that defect against each other at every opportunity – the worst case for everyone involved since really everyone cares about the agent-neutral impact.

Where to Go from Here

These are open questions with preliminary ideas for answers. I would be happy to start a discussion on these questions.

How Bad is it Really?

1. The for-profit world is motivated almost completely by agent-relative incentives so the “god’s eye” view that counterbalances the Attribution Moloch is almost absent. Still it’s not a complete catastrophe. Effort is duplicated over and over, but there are tangible improvements anyway. Does that mean that even the worst case is not terribly terrible? Or is the agent-relative profit motive so much stronger, more homogeneous, and more pervasive than any of the many agent-neutral motivations there are that the two are incomparable?
2. In my introduction I had to go to some lengths to set up a situation that was sufficiently resource constrained not to let the narrator seem unrealistically evil. That’s maybe an indicator that in reality scarcity is usually insufficiently bad to incur these problems often.

What Can be Done?

1. Competition could be rendered otiose if the Open Philanthropy Project endowed the ACE top and standout charities with a yearly grant along the lines of its grants to the GiveWell top and standout charities – proportional to the funding gap while avoiding fungibility.
2. Organizations can also pay it forward whenever feasible to establish or strengthen a climate of cooperation and reciprocation that may be highly tractable because of how ingrained reciprocation already is in our culture. Animal Charity Evaluators, for example, is doing this by providing educational materials and academic resources.
3. We can reward cooperation where we see it to incentivize it. In particular prioritization organizations can make an organization’s willingness to cooperate a criterion in their prioritization process. Animal Charity Evaluators told me that it’s already using this criterion. A technical solution for measuring willingness may be something along the lines of Whuffies – awarded and awardable only between nonprofits.⁸
4. Value aligned organizations with compatible marketing strategies can merge or share departments they would otherwise duplicate or form an umbrella organization as leviathan.

Footnotes