Claim identification & assessment

“pivot penalty,” where the impact of new research steeply declines the further a researcher moves from their prior work.

Authors claim that biomedical scientists, who are the experts relied on in NIH peer review, have a preference for “dissensus” among reviewers’ scores (i.e., higher variance). They also claim that a tighter budget will decrease the dissensus tolerance among scientists. Both claims are well substantiated in the experimental findings shown in Tables 2 and 5. It should be noted that the effect of the overall average score is 2-9 times larger than that of score variance on participants’ decision of which studies to fund. They also claim decision makers at science funding institutions would benefit from taking the variance of peer reviewer scores into account, rather than considering the average alone. It is argued that higher score variance is correlated with higher risk research with greater potential to have positive impacts for the public. This claim is crucial to connecting the findings of the experiments to policy recommendations, however it lacks direct evidence from the studies conducted, “Our study evaluates the value researchers place on alternative peer review aggregation methods taking this conjecture [that greater dissensus in project scores can identify more radical projects] as given.” (p. 3). Results in Table A8 show that the risk-averse subgroup has a stronger preference for score variance than average, undermining the assumption that the variance preference is a risk preference.

Reviewers have a preference for grants with increased variance in review scores, and that there is an asymmetry in preference for high-variance applications in response to funding shocks in the positive vs. negative direction

The authors estimate that biophysical conditions can support natural regeneration in tropical forests over 215 million hectares globally, and five countries (Brazil, Indonesia, China, Mexico and Colombia) account for 52% of this estimated potential (source: abstract and last paragraph in p. 132). The estimation is based on spatially explicit machine learning predictions of natural regeneration potential. The claim is important due to ambitious forest restoration targets requiring science-based guidance on spatial targeting.

"We estimate that an area of 215 million hectares... has potential for natural forest regeneration..." (Williams et al. 2024, Abstract & Results section). They further estimate this area holds an "above-ground carbon sequestration potential of 23.4 Gt C (range, 21.1–25.7 Gt) over 30 years." This claim is highly important because it provides the first spatially explicit, high-resolution estimate of the global biophysical potential for natural regeneration across the tropics. It quantifies a major, often overlooked, cost-effective pathway for achieving large-scale forest restoration, which is critical for meeting international climate change mitigation and biodiversity conservation targets (like the Bonn Challenge and the Global Biodiversity Framework). By mapping this potential, the research aims to guide policy, investment, and planning towards leveraging natural processes for restoration, complementing or potentially replacing more expensive active tree-planting efforts in suitable areas. It provides a crucial (though, as discussed in the review, potentially overestimated and uncertain) baseline figure for understanding the scale of this opportunity.

The most important factual claim is that one-time unconditional cash transfers of $500 or $2,000 to poor people in the US increases personal spending (Figure 5), but not self-reported well-being (financial, psychological, cognitive, and physical)—and may even decrease self-reported wellbeing (Table 2). The evidence comes from comparing bank data and survey responses between the randomly assigned treatment groups among the sample of 5,243 individuals. This claim is important in that it updates our estimates of the causal effects of one-time unconditional cash transfers on self-reported wellbeing. Given the cost of administering and studying these programs, and likely heterogeneous effects across study populations and treatments, a well-powered RCT with multiple treatment levels and longitudinal outcomes measurement moves the ball forward in our understanding.

There are no positive effects of small unconditional cash transfers.

The primary claim is that exposure to additional years of deworming treatment in primary school results in lower child mortality of the children of those exposed. The evidence provided is from analysis of an intervention randomized at the school level that compares the mortality of the children of those exposed earlier vs later via the intervention. The outcome data is derived from long-term follow up survey data of these experiment participants. I discuss the importance of this claim in detail in my report.

For these reasons, US officials could raise these issues at international negotiations and advocate for higher prices than presently set in high-income ROW 35 countries. A multi-country agreement in this direction would represent a serious effort to support improved world health.

People do not hold the neutrality and procreation asymmetry intuitions. They believe that adding a happy person is good and is as good as adding an equally intense unhappy person is bad. This ‘non-neutral symmetry’ result is observed in studies 2a and 2b of the paper. If people expect future populations have net positive welfare levels (that are at least as high as past and present generations), the symmetry view entails that policymakers should prioritize existential risk reduction above suffering reduction.

Section 6 of the paper provides the most crucial remarks for policy designers and practitioners. I would further highlight three points. First, the papers evidence on substitution versus complementarity is vital: a one-standard-deviation increase in university patents reduces corporate patenting by 51%, underscoring the need to carefully calibrate technology-transfer and licensing policies. Second, embodied human capital in the form of PhD graduates plays a powerful complementary role—each one-standard deviation rise in relevant doctorates yields a 53% boost in firm patents—highlighting the high leverage of targeted doctoral- training initiatives. Third, regional innovation strategies and university–industry partnership programs must balance the benefits of widespread technology diffusion against the risk of competitive displacement among incumbent firms.

The stagnation in science is correlated with the rise of bibliometrics and other citation measurements

The most impactful claim in Bruers (2023) is the following: the median estimate of the animal welfare cost of chicken meat is 10 euro/kg, whereas its mean estimate is several orders of magnitude higher. The evidence underlying this comes from a survey of Belgian consumers conducted in 2022, who are responding to a hypothetical valuation question that involves how much money they would be willing to accept for getting the experience of chickens from a conventional farm. This claim is the most important because it 1) provides further quantification of animal welfare externalities, 2) suggests a high degree of variability, and 3) indicates that some consumers have valuations that are very large by most measures.

Increased exposure to public invention decreases a firm’s private R&D expenditure.

Increased exposure to public invention decreases a firm’s private R&D expenditure.

The manuscript's main claim is that macroeconomic damages from climate change are six times larger than previously thought (around 12% instead of 2% reduction in world GDP following 1 degree Celsius of warming). This is because previous work does not account for (or indeed misses the effect of) global shifts in warming.

“the macroeconomic impacts of climate change are six times larger than previously documented.” (p1) see review text for details

The most important and impactful claim that this research makes is that a majority of AGI governance experts exhibit consensus in supporting 49 out of 50 specific safety-related policies and practices. This is summarized in this statement: ‘For every practice but one, the majority of respondents somewhat or strongly agreed that it should be implemented. Furthermore, for the average practice on our list, 85.2% somewhat or strongly agreed it should be implemented.’

Document a substantial greening up of Beijing by the MMP programme, specifically NDVI growth to accelerate after 2012, the year of the project implementation Quantify the impact of urban forests on downwind air quality improvement using a quasi-experimental research design i.e. increased vegetation growth by MMP reduces average PM2.5 concentration at city population hubs by 4.2 percent and led to a 7.4 percent increase in pollen exposure.

Urban tree planting provides net health benefits by reducing air pollution (4.2%) despite meaningful increases in pollen exposure (7.4%). Overall, the health value of Beijing’s Million Mu Project (55 billion CNY over the past decade) will likely exceed its cost (75 billion CNY) in the next decade. This is important due to the popularity of urban greening in the face of climate change and urbanization. The empirical evidence on pollen exposure and its comparison to air pollution is especially novel.