The Importance of (Exponentially More) Computing Power: A Quantitative Analysis

1. Introduction & Overview

This paper investigates the fundamental relationship between increases in computing power and improvements in real-world outcomes. Moving beyond abstract economic measures like IT spending, it provides direct, quantitative evidence by analyzing five specific domains. The core finding is that computing power explains 49% to 94% of performance gains, but these gains follow a counter-intuitive pattern: exponential increases in computing power are required to achieve linear improvements in performance. This clarifies the critical, non-linear role of Moore's Law in driving progress and highlights the economic challenges posed by its slowdown.

2. Methodology & Domain Selection

The study selects five domains to construct "production functions" linking compute (FLOPS) to performance metrics. The domains are split into two categories:

2.1. Computing Bellwethers: Chess & Go

These are classic AI benchmarks with clear performance metrics (Elo rating) and well-documented compute histories. They serve as controlled environments to isolate the compute-performance relationship.

2.2. Economically Critical Applications

• Weather Prediction: Measured by forecast skill (e.g., Anomaly Correlation Coefficient).
• Protein Folding: Measured by accuracy in CASP competitions.
• Oil Exploration: Measured by the resolution and accuracy of seismic imaging.

These represent areas where improvements have significant economic and scientific value.

3. Quantitative Results & Analysis

The analysis reveals a powerful and consistent relationship across all five domains.

3.1. Performance Attribution to Compute

3.2. The Exponential-Linear Relationship

The most significant finding is the shape of the production function. Contrary to standard economic assumptions of power-law relationships, the data best fits a model where:

Performance Improvement ∝ log(Computing Power)

Or, rearranged: Computing Power ∝ exp(Performance Improvement). This means to get a linear unit of better performance (e.g., +100 Elo points, +1% forecast accuracy), you need to multiply the underlying computing power by a constant factor—an exponential requirement.

4. Technical Framework & Mathematical Model

The core analysis involves fitting production functions. The standard Cobb-Douglas form is $Y = A \cdot L^{\alpha} \cdot K^{\beta}$, where $Y$ is output, $L$ is labor, $K$ is capital, and $A$ is total factor productivity. This paper treats computing power ($C$) as a distinct, primary capital input. The tested relationship is:

$P = a + b \cdot \log(C)$

Where $P$ is the performance metric (Elo, forecast skill, etc.) and $C$ is computing power in FLOPS. The logarithmic fit outperformed linear and power-law ($P = a \cdot C^{b}$) models, confirming the exponential-linear relationship. The coefficient $b$ represents the marginal return per log-unit of compute, which was positive and significant across all domains.

5. Results, Charts & Interpretation

Chart Description: The seminal chart of this paper would plot Performance (Y-axis) against Computing Power in FLOPS (X-axis, logarithmic scale) for all five domains. Each domain would show a series of historical data points (e.g., Deep Blue, Stockfish, AlphaGo, AlphaZero for Go; various supercomputers for weather models). The key visual result is that all trend lines are approximately linear when compute is on a log scale. This visually proves the $P \propto \log(C)$ relationship. The slopes of the lines differ, indicating varying "compute efficiency" across domains (Chess has the steepest slope, Protein Folding shallower).

Interpretation: The linear-log plot means moving one unit right on the log-scale X-axis (a 10x increase in compute) yields a constant linear improvement on the Y-axis. This exponential cost of linear progress was sustainable when Moore's Law delivered exponential growth for free. As Moore's Law wanes, sustaining the same rate of performance improvement requires conscious, costly investment in scaling compute, making progress more expensive and potentially slowing it down.

6. Analytical Framework: Case Example

Case: From AlphaGo to AlphaGo Zero & AlphaZero

Framework Application: This case perfectly illustrates the exponential-compute-for-linear-gain principle.

1. AlphaGo (2015): Defeated Lee Sedol. Used 176 GPUs for training and 48 TPUs for inference. Estimated compute: ~10 petaflop/s-days.
2. AlphaGo Zero (2017): Surpassed AlphaGo's performance. Trained solely by self-play. Used 4 TPUs. Key insight: Better algorithms improved compute efficiency, but massive scale was still essential.
3. AlphaZero (2017): Generalized algorithm mastering Chess, Shogi, and Go. Used 5,000 first-gen TPUs for training.

Analysis: The performance jump from AlphaGo to AlphaZero represented a massive linear gain in Elo rating and generality. This was achieved not by a linear increase in hardware, but by a combination of algorithmic innovation (a shift in the production function) and a massive, order-of-magnitude increase in training compute. The paper's model would attribute a large portion of the Elo gain to the log of this increased computational budget.

Non-Code Insight: The framework asks: For a given performance target, what is the required $\log(C)$? If a company wants a weather model 10% more accurate, the historical data provides the $b$ coefficient to calculate the necessary multiplicative increase in supercomputing power. This shifts planning from "we need faster computers" to "we need computers that are X times faster."

7. Future Applications & Research Directions

• Beyond Moore's Law: The search for new computational paradigms (quantum, neuromorphic, optical computing) is no longer a niche pursuit but an economic imperative to maintain the slope of progress in critical fields.
• Algorithmic Efficiency as a Counterweight: Research into more compute-efficient algorithms (like the evolution from AlphaGo to AlphaZero) becomes exponentially more valuable. The ROI on algorithmic research increases as hardware scaling gets harder.
• Strategic Allocation of Compute: Organizations must prioritize compute allocation to domains with the highest marginal return (steeper $b$ coefficient). This paper provides a methodology to calculate those returns.
• New Domains for Analysis: This framework should be applied to Large Language Model (LLM) scaling (following the work of Kaplan et al., "Scaling Laws for Neural Language Models"), drug discovery, and material science to validate and generalize the exponential-linear law.
• Policy Implications: National investments in computing infrastructure (exascale computing, AI research clouds) are directly linked to future productivity growth. The slowdown of Moore's Law may require policy interventions to avoid a broad slowdown in innovation.

8. References

1. Solow, R. M. (1957). Technical change and the aggregate production function. The Review of Economics and Statistics.
2. Brynjolfsson, E., & Hitt, L. M. (2003). Computing productivity: Firm-level evidence. Review of Economics and Statistics.
3. Jorgenson, D. W., & Stiroh, K. J. (2000). Raising the speed limit: U.S. economic growth in the information age. Brookings Papers on Economic Activity.
4. Kaplan, J., et al. (2020). Scaling Laws for Neural Language Models. arXiv:2001.08361.
5. OpenAI. (2023). GPT-4 Technical Report. arXiv:2303.08774.
6. Thompson, N. C., et al. (2020). The Computational Limits of Deep Learning. arXiv:2007.05558.
7. International Technology Roadmap for Semiconductors (ITRS) Reports.
8. Top500 Supercomputer Site (historical data).

9. Industry Analyst's Perspective