Some futures are much richer than others, and most people would also assign radically different probabilities to different futures. Mechanisms such as a “windfall clause” help distribute riches within particular futures. But for a windfall clause to be useful, several conjunctive assumptions have to be true:
- The technology (such as transformative AI) has to be benign enough to not remove our civilization or all biological life from the picture entirely,
- yet powerful enough to prevent other similar but less benign systems from doing the same,
- yet again not so powerful as to just fix all problems and make the windfall redundant, and
- it needs to be early enough to be able to exercise this power.
Hence we didn’t previously think that windfall clauses were important.
But we now think that an impact market could allow us today to borrow against the windfalls in these improbable futures. The impact market could simultaneously pull the windfall back into a time when it can still make a greater difference (today) and help make the windfall itself more likely, thereby increasing its expected value. This can be applied to other kinds of windfalls too, such as exits from successful startups or last wills.
For a more comprehensive explanation of impact markets, see Toward Impact Markets.
In short, an altruistic retroactive funder announces that they will pay for impact (or “outcomes”) they like. It resembles a prize competition in this way. But (1) they’ll pay proportional to how much they like the idea and not only the top n submissions; (2) the impact remains resellable by default; and (3) seed investors offer to pay the people who are vying for the prizes or provide them with anything else they need and receive in return rights to the impact and thus prize money.
It is analogous to the startup ecosystem: Big companies like Google want to acquire (or “ac-hire”) small companies with great staff or a great product. Founders try to start these small companies but often can’t do so (as quickly) without the seed funding and network of venture capital firms. When the exit happens (if it happens), the founders may not even any longer own the majority of the company because they’ve sold so much of it to the investors.
The benefits are particularly strong for high-impact charities and hits-based funders:
- If a hits-based funder usually funds projects that have a 1 in 10 chance of success and switches to retroactive funding, they save:
- the money from 9 in 10 of the grants and
- the time from 9 in 10 of the due diligence processes.
- Investors can thus speculate on making around 20x return on their successful investments, and they can further increase their expected returns: 3. by specializing in a narrow area (such as AI safety) to make excellent predictions about which project will succeed, 4. by providing founders with their networks in those areas, 5. by buying resources at a bulk discount that founders need (such as compute credits) and providing them in return for rights to impact.
- Charities can attract top talent and align incentives with top talent who may not be fully sold on the charity’s mission: 6. by promising them a share in all impact sold, 7. by locking that share up in a vesting contract, 8. by (possibly) sharing rights to the impact with another company that is the current employer of the talent so that they don’t need to quit and can draw on the infrastructure of the other company. (In fact, somewhat value-aligned companies may be interested in becoming investors themselves if they want to retain talent who want to work on prosocial applications of their knowledge.)
A typical product that is suitable for impact markets is a scientific paper. Papers, like many other projects, have the property that they often get stuck in the ideation phase, sometimes have to be abandoned (for other reasons than being an interesting negative result) during the research, sometimes don’t make it past the reviewers, and sometimes turn out to have been a bad idea only decades later.
When an investor wants to invest into a paper that has not been written, but which they are highly optimistic about, they may see these futures:
The x axis is the time (in years), the y axis is the Attributed Impact (proportional to dollars), blue lines are possible futures, and the red line is the median future.
There are two big clusters: all the futures in which the paper gets written, published, and read, versus and all the futures in which it either never gets finished or gets read by too few people.
One to three years into the process, it becomes clear in which cluster a given future falls, particularly if it falls into the upper cluster. (Otherwise there’s a bit of a halting problem because it might still take off.) Maybe the paper has been published on arXiv and is making rounds among other researchers in the field.
After 10 years, the majority of the impact has become clear and the remaining uncertainty over the value of the Attributed Impact of the paper is low.
After 15 years, we’re asymptotically approaching something that looks like a ceiling on the Attributed Impact of the paper. Experts have hardly updated on its value anymore in years, so their confidence increases that they’ve homed in on its “true” value. (“True” in the intersubjective sense of Attributed Impact, not in any objectivist sense.)
This is a vastly idealized example. In practice it may be that a published paper that used to be held in high regard suddenly turns out to have been wrong, an infohazard, plagiarized, etc. Or it may be that it’s suddenly noticed that a decade-old forgotten-about paper (that had high ambitions at the time but seemed to fall short) contains key answers to an important new problem.
If an investor is a specialist in some small field and profits from economies of scale in the field (e.g., the compute credits bought in bulk that we mention above), then they may expect to make a 10× profit from each retro funding that they receive. That’s the difference between the size of the retro funding at which the retro funder breaks even (ignoring interest) and the cost to the investor. We assume for simplicity that monetary and time costs (grants and due diligence) are the same. So, 2 · 10i − 2 · 5i = 10i, where i is the average seed investment. (We’re using the parameters from above where retro funders save 10× from making fewer grants and 10× from saving time spent on due diligence. We also assume that a patient, well-networked, specialized investor has twice the hit rate of the generalist funder.)
If, counterfactually, they would’ve invested this money at 30% APY, the impact market ceases to be interesting for them if they expect the retro funding to take longer than 8–9 (≈ 10.6 ≈ 1.39) years: 2 · (1 / ratefunder) − 2 · (1 / rateinvestor) = (1 + apy)years.
Here we’re comparing an investor at different hit rates to a retro funder who would otherwise have a 10% hit rate under four counterfactual market scenarios. The impact market is profitable for any number of years less than the break-even point.
If the retro funder wants to save money, they can pay out less, but will need to do so earlier. For simplicity, the following chart is only for the scenario with a counterfactual 30% APY: 2 · (1 / ratefunder) · (1 − savings) − 2 · (1 / rateinvestor) = 130%years.
A retro funder needs to take this into account when deciding how much certainty they want to buy from the investors. More added certainty comes at a higher price. They can regulate this through the size of their retro funding or through the timing. Depending on the impact in question there are usually certain sweet spots that they can aim for, and do so transparently so that investors know what time horizons to speculate on.
When it comes to our example above, it seems fairly clear whether the paper was a success (was written, published, and read by some people) after about 2–3 years. So one sweet spot may be to wait for the moment of publication (as a draft or after peer review) or after the initial public reception can be gauged. The second is interesting because investors may be well-positioned to help with the promotion.
But there are other options – less profitable options much later.
We can imagine a chain of retro funders from a particular set of futures into the present: Someone makes a binding commitment that if they are successful in making a lot of money – if they receive a windfall – they will use the money or a fraction of it to buy back impact that has previously been bought by a certain set of existing retro funders who the person trusts. They can continually add new ones to this set.
This can also be formulated as a prize contest: If I receive the windfall, I’ll use that budget to buy impact from my favorite retro funders at a reasonable bid price. If 1 in 5 projects still fail between the time when the retro funder bought them and the time when the windfall happens, the windfall retro funder may buy them at 120% of the price that the previous retro funder paid.
Under this framing there is no qualitative difference anymore between investors and earlier retro funders. They’re all just different investors with different attitudes toward risk. (Some of them may choose to burn their certificates, though, to signal that they’ll never resell them.) There may even be investors who choose to invest into “whatever project person X will do next,” so earlier than the abovementioned seed investors.
A startup may be interested in making such a commitment because they have the choice to either do the research in-house or at least pay for it immediately or to pay for it later and only if they are successful. Since startup success is typically Pareto distributed, they’ll have vastly more money in the futures where they are successful than they have now or in unsuccessful futures. So this deal should be interesting for most startups.
For investors it’s a question of whether they want to expose themselves more to the field or to a particular team. If they’re excited about the team behind the startup and trust that team to do well regardless of what field they go into, they’ll want to invest directly into the startup. But if they’re more agnostic about all the teams in a field but are very excited about the field, they may prefer investing into the research projects to bet on the retro funding.
The AI safety example:
- Many AI companies realize that to make AI useful, it needs to be safe and controllable.
- They commit to windfall clauses. Specifically they commit to use some of the windfall to buy AI safety impact from the Open Philanthropy Project, the FTX Future Fund, and other retro funders they trust.
- Some investors make seed investments into AI safety research projects (papers, books, experiments, etc.) either to sell them to the existing retro funders; to hold them until new and richer retro funders join the market; or even to appeal to the companies directly who have the windfall clauses to be added to their set of trusted retro funders.
- The funding from seed investors and retro funders and the benefits for incentive alignment in organizations all allow AI safety projects to launch much faster.
- Many projects won’t be bought by the existing retro funders.
- Many retro funders will fail to sell some of the impact they did buy.
- Eventually, the now more likely future might happen (or maybe it will happen on some Everett branches), and the enormous budget from the windfall clause will likely be enough to buy all the impact from the retro funders.
There could be perverse incentives here if some of the retro funders prefer to be rich over living in a moneyless utopia. In such a case the already narrow target of okay outcomes could narrow further to just okay outcomes that don’t obviate the need for financial markets.
The same approach can also be applied to other fields where companies are dependent on research that might as well be public:
- Cultured meat (or “cell/clean/c meat”) startups may require a lot more research to be done on how to scale their production and make it cost-competitive. But they don’t yet have the money to do all of that research in-house.
- They commit to “windfall clauses” in the sense that they’ll invest a large portion of the money they’ll make from an exit into buying impact. Specifically they hash out particular terms with an organization like Founders Pledge that stipulate what impact related to cultured meat research they will buy from which retro funders.
- As above, the promise of great potential future riches boosts funding and opens up hiring pools.
- Eventually, the now more likely future might happen, and the large budget from the exits serves to buy most of the impact from the retro funders.
We’ve received a grant via the Future Fund Regranting Program to work on this. If you’d like to join our discussions, please join our Discord.
Thanks to my cofounder Dony for reviewing the draft of this post! He gets 1% of the impact of it; I claim the rest.