Owen Cotton-Barratt
This is part of a series of posts on how we should prioritise research and similar activities. In a previous post, we investigated methodology for estimating the expected returns from working on a problem of unknown difficulty. We argued that ex ante the expected benefits should often scale approximately logarithmically with the resources invested (in fact that if B are the benefits and R are the resources invested, then B is proportional to log(R) + a constant).
In this post we turn our attention to the case where rather than a single problem, there is a wide range of problems in a single area. The paradigmatic example is a research field, but we could also think of more diverse areas such as lobbying for a range of related policy changes, or producing successive innovative improvements in an industry. The individual problems in the area should be of a one-off nature, and not too similar to one another.
Central claim
This post explores the following claim:
Law of logarithmic returns
In areas of endeavour with many disparate problems, the returns in that area will tend to vary logarithmically with the resources invested (over a reasonable range).
More precisely, we will argue that under some assumptions about the distribution of problems in the area, we can expect to find approximately logarithmic returns to resources ex post as well as ex ante. Of course, it is a ‘law’ only in a sense like Moore’s law: it is a good first-order approximation to the true behaviour.
This post will be concerned with the consequences of the claim and how its predictions match up with real-world data; in a companion post we explore the theoretical justifications for this claim.
Previous work
This claim is not entirely new. It is a mainstream (if not universally accepted) position that research has diminishing marginal returns as the low-hanging fruit are taken. And in some contexts people have argued for a law of logarithmic returns.
In particular Nicholas Rescher has claimed a law of logarithmic returns to work in scientific fields [Scientific Progress, 1978]. He makes his deduction in the opposite direction: starting with the observation that progress (counted in terms of the number of “first rate” discoveries) goes linearly with time while resources increase exponentially, he deduces that the underlying behaviour is a logarithmic return to resources. Rescher also gives one theoretical model as a possible explanation for this behaviour: arguing that it is based on the limits of instrumentation. This model is only applicable in reasonably restricted domains.
In contrast with Rescher, our work here began with the theoretical model, which made a prediction largely validated by empirical data. Indeed I was unaware of Rescher’s work until reaching the prediction that we would see logarithmic returns gave me better search terms. An advantage to having a general theoretical model is that we can understand which features are key to generating the behaviour, and therefore we can reasonably predict when the behaviour will extend to fields where we do not yet have empirical support.
Also note that previous work has mostly had a descriptive flavour. In contrast our interest in understanding these questions arises from a priority-setting perspective. In order to make decisions about when to invest resources in different fields, we need predictive models about the effect of such investment.
Consequences
A key corollary of the law of logarithmic returns deals with the case of exponentially growing inputs:
Linear outputs (corollary):
If a field has exponentially increasing resources with time, and logarithmic returns with resources, the returns will be linear with time.
This corollary can more directly be checked empirically, and was the starting point for Rescher. Because the economy as a whole is growing roughly exponentially, if a fixed fraction of its resources are devoted to a field, then the conditions for the corollary to apply will hold. Even if it is not a fixed fraction of the economy, often the growth rate of a field as a fraction of the entire economy may look exponential over a period of decades, which would mean exponentially increasing resources for the field. That said, it won’t always hold; for example the share going to a field cannot increase exponentially indefinitely.
Examples
Rescher’s work is a good source of examples in the domain of science. I’ll give a few more examples where the prediction of the law of logarithmic returns seems to line up with observed data.
Drug approvals
The number of newly approved drugs in the US has been approximately linear with time, despite exponentially increasing resources. A recent paper discusses possible reasons for this phenomenon, and it is unclear that the effect is entirely to do with the picking of the low-hanging fruit. Nonetheless note that the law of logarithmic returns alone predicts this qualitative behaviour.
Experience curves
For production of a fixed good or service, as volume of production goes up, costs fall. In fact, a doubling in production tends to lead to a fixed percentage fall in costs (although this percentage varies with different goods).
If we assume that:
-
there is a ‘learning by doing’ process occurring, where we can assume that the resources going towards learning how to be more efficient are proportional to the production;
-
knowledge gained about how-to-be-efficient produces a decrease in costs in such a way that independent breakthroughs tend to have a multiplicative effect, so for example two breakthroughs such that each on its own would halve the cost together cut the cost by a factor of four;
then the law of logarithmic returns predicts the experience curve effect. Each of these assumptions is doing some work but each also seems quite plausible.
With an additional assumption that semiconductor production is increasing exponentially, we can also see Moore’s law as an example of an experience curve. So we posit that the underlying mechanism behind Moore’s law is essentially the same as that underlying the exponentially increasing resources needed for new drug approvals. This position contrasts with the default assumption that there are very different kind of forces at work producing diametrically opposed effects.
Quality of life
The idea of logarithmic returns can offer insights in quite different domains. For example, a question of concern to welfare economists is how quality of life varies with consumption level. Empirical data suggests a roughly logarithmic behaviour.
We can provide a theoretical justification for this behaviour. Assume the opportunities to improve one’s life are distributed in cost over several orders of magnitude, and that the benefits of these opportunities are also distributed, but not quite so broadly. This seems plausible: think of buying clean drinking water and of upgrading your house to a mansion. Then we have an area of endeavour satisfying the assumptions in the technical companion post, so we should expect that the best/cheapest opportunities are taken up first. So we’d predict you’d see logarithmic returns to consumption (at least in a region away from the top and bottom of the scales).
Discussion
We’ve presented various examples here. I’m keen to hear of other examples people are aware of. I’d also like to know of places where it seems like the law should break. The theoretical justification breaks down at the very start of a field and the end of a field (see companion post; roughly speaking early in a field the returns may be higher than this, and late in a field they may be lower). This fits with some empirical work: one paper by Roland Wagner-Döbler counts the first rate contributors in a field compared to total contributors. It concludes a logarithmic behaviour for mature fields, and a square-root behaviour for nascent fields. It’s unclear how good a proxy the number of such contributors is for progress in the field, however.
Having a theoretical model behind the prediction of logarithmic returns improves our confidence in applying the rule to new contexts. And the fit of the empirical data with the model lends support to the theoretical model which dealt with one-off problems.
Application: estimating the cost-effectiveness of research
Having a model for the returns from investment into an area of endeavour such as a research area is a key component of estimating the cost-effectiveness of further investment in that area. This is because it’s not enough to consider the average value of the research — this is the question we’d face if any research we didn’t do now could never be done. But in practice it is likely that if it isn’t done now it will be done later. To estimate this counterfactual value of research, we need to have an idea of how much extra progress we’ll have made at each future point on account of an investment now. An assumption of logarithmic returns lets us do just that.
If the resources flowing into a field have been increasing exponentially, logarithmic returns to investment would lead to linear returns with time. We can use the observed rate of progress to estimate what log-returns mean quantitatively in the case of interest. For example, if investment in medical research has been going up by 3% p.a., and the research has been generating an increase in healthy life-expectancy by around 6 months every decade, then we can observe that every 3% increase in total resources to the field of medical research gives us an extra 0.6 months in healthy life expectancy. Together with a model of how future funding to the area will progress (and a simple model says it will continue to increase exponentially), this lets us estimate the total future effect of extra research today, relative to a ‘business as usual’ baseline.