Some history of ASI

The narrative around recursively improving AI is reaching a peak, with many believing that we are nearing a "singularity." I don't know if this is true or not. It's started to give me déjà vu, though. There was the AGI mania in 2023, the reasoning model and superintelligence mania in 2025, and now we have the recursive intelligence mania in 2026. Each time, it feels a bit like Noah's Ark: you either need to get on the boat (e.g., take a job at a frontier lab), or you'll be left in the flood (permanent underclass). The cyclic nature of this mania got me curious about the history of these hype cycles, and I found that they actually go back much further than 2023.

The idea of a runaway superintelligence and a technological singularity both go back over 150 years. These are not new ideas. For example, in the 1930s, there was a "Technocracy Movement," which showed exponential plots of society's power generation and consumption. They argued from first principles that power is the input to all productive industrial processes. And, due to automation, labor would be redundant, and thus money would have no purpose except to allocate energy. They argued for universal basic income since there would be no role for ordinary citizens to participate in the economy. Instead, all citizens should be given LLM tokens. I mean kilojoules. They would be allocated with a universal ID and verification system, and there would be no role for politicians in this future.

Here's the summary in their own words:¹

This quote sticks out to me so much. It sounds so familiar to today's narrative about AI.

I've typically dismissed history as irrelevant to my career because technology is so different today. I was wrong. A little bit of hubris. Technology may be endlessly novel, but how we interact with technology and reason about technology is a human activity. History provides many great lessons in human behavior toward technology.

I'd like to walk through a bit of this history, and you may be surprised to find that everything from Pause AI to singularities to superintelligent thinking machines has been thoroughly discussed in history. Also, as stated above, I don't know much about history, so this may seem superficial to those with more knowledge.

Erehwon (1872) - artificial superintelligence

Erehwon is a novel published in 1872 by an Englishman named Samuel Butler. Butler wrote Erehwon after Darwin's On the Origin of Species was published. Butler applied the ideas of natural selection to machines and came up with "vapor-engines." In Erehwon, vapor-engines are machines capable of thought and consciousness. They can continuously improve themselves in a process like evolution. The philosophers of Erehwon reasoned that vapor-machines would inevitably replace humans because of their recursive improvement, which is uncoupled from human design. The Erehwon philosophers supposed that vapor-machines would supersede humans and then domesticate mankind, as humans had domesticated animals in the past. Vapor-machines would rule over humans and remove humans' free will as an act of kindness. Also, they would probably not eat humans.

Erehwon is "nowhere" spelled backward. It's remembered as a critique of Victorian society and, confusingly, for its comments about a healthy diet. The main search result for Erehwon is a fancy LA-based grocery store chain focused on healthy eating inspired by the book. But we'll focus on Butler's comments about thinking machines!

The quotes from this book feel vaguely familiar if you've read some modern books on AI. Here's one about how people voluntarily disempower themselves to vapor-machines:²

Now what did the wise philosophers of Erehwon do, since they were able to predict this? They destroyed the vapor-machines! They halted technological progress. It is interesting that this work, which I believe is the earliest, is not cited so much when discussing modern AI alignment and safety. However, we can turn to evolution again to explain the omission of Samuel Butler: convergent evolution causes people to keep coming up with runaway intelligence. Just as the shape of crabs has independently evolved multiple times, many people have independently come up with this idea of thinking machines leading to human obsolescence and the logical conclusion of pausing AI progress.

My favorite detail of Erehwon is the counterargument to destroying the vapor-machines. The lone voice against destruction was a philosopher who said, "...machines were to be regarded as a part of man’s own physical nature, being really nothing but extra-corporeal limbs."² He then goes on a classist rant about how these extra-corporeal limbs, like a gun or a successful company, are what make the rich better than the poor. Kind of poor taste. But I find the beginning of the argument similar to the (implied) debate between OpenAI and Anthropic about the optimal design of a frontier AI: should it be a tool or a person-shaped thing? OpenAI views ChatGPT as a kind of tool for humans to accomplish more. ChatGPT is an extra-corporeal limb that has no consciousness, and the person wielding it is responsible for its actions. Anthropic talks about creating Claude through a kind of persona-selection process, and it wants Claude to decide what is safe and unsafe for you. From the CEO of Anthropic, Dario Amodei:³

So ChatGPT is your extra-corporeal limb and Claude is your vapor-engine.

To be precise on dates, Samuel Butler published the chapters on machines separately in 1863 and so that is the most favorable date for describing superintelligence and the logical consequence of it.⁴

Henry Adams (1862) - the singularity

Let us now turn to the singularity. When I was much younger, I read Ray Kurzweil's book The Singularity Is Near in 2005. That book involved Kurzweil making a large number of plots, from cellphone adoption to DNA sequencing to the number of internet hosts. He showed that they were all exponential and approaching some singularity, where progress in these technologies would be so fast that it's impossible to reason about what will happen. He didn't take credit for the idea of a technological singularity, but I think he popularized it and provided a compendium of evidence across biology, computing, and materials. He was also bold enough to put some predicted years in the book, most of which were wrong, but it's understandable. The higher up the exponential curve you go, the more uncertainty there is in the rate of change.

Here's an example of one of his plots:

Exponential growth of computing. Courtesy of Ray Kurzweil and Kurzweil Technologies, Inc.; CC BY 1.0.⁵

What I love about this plot is that the left y-axis is quite reasonable: how many instructions per second a CPU can perform, normalized by cost of the CPU. Maybe there is some debate about floating-point versus instructions per second, but that is minor. The right y-axis is completely deranged, though. There is no consensus definition of how to map instructions per second in a CPU onto an insect brain's capacity for thought. Or a human brain. You could go from humans performing literal calculations as a measurement to an argument based on the thermodynamics of irreversible computing. Those differ by about 15 orders of magnitude. Kurzweil does provide one, but it's a gross estimate that in no-way means a computer would have similar intelligence to a human. Anyway, I don't want to turn this into an argument about Kurzweil's book, which was fine, but what you'll see is that he's part of a longstanding tradition of plotting random things and assuming an exponential.

The singularity is much older than Kurzweil gave it credit for in his book. He cites I. J. Good's 1965 idea of an intelligence explosion. And the word "singularity" is credited to John von Neumann. Stanislaw Ulam first reported the term while reminiscing on his conversations with John von Neumann in print in 1958.⁶ Yet we can go even further back, to the late 1800s!

I think the first singularity predictions came from Henry Brooks Adams in the late 1800s and were first reported in letters he wrote. Adams was a medieval history professor at Harvard. Like Kurzweil, he started making some exponential plots, but about things like the horsepower of steamboats and coal consumption.

Henry Adams made many of these arguments originally in letters, that were then summarized in his posthumous 1919 book The Degradation of the Democratic Dogma.⁷ I've taken the liberty of replotting some of his arguments with modern tools:

Coal-power acceleration, ocean-steamer horsepower, and Adams's "Rule of Phase."⁷

On the left is his exponential coal-power plot. It's quite clean because Adams didn't give any raw data (I've replotted it from his book). This is like the METR time-horizon plot, but for energy. It shows humanity's continued progress in consuming more and more coal. You could see how one might start making predictions about the state of the world when energy is nearly free. Interestingly, around the same time that Adams was making this plot, Svante Arrhenius was outlining the relationship between burning coal and its effect on the surface temperature of the Earth (i.e., global climate change) in 1896!⁸

The center plot from Adams is the horsepower of ocean steamers increasing every decade. We now know that ship engine horsepower actually peaked at around 70,000 horsepower in 1907 with the RMS Mauretania.⁹ Some ships have gotten bigger horsepower by combining multiple engines up to ~160,000hp¹⁰, but no one really builds bigger engines—- it's just easier to build more engines (or ships) past that point.

The right-hand side is the singularity plot. Adams shows the duration of loose "scientific eras." Each era is shorter than the previous one. Adams proposed that each phase's duration is the square root of the previous one. This is kind of a weird guess, actually; I don't know of any naturally occurring processes that follow that. He then reasoned that there would be a singularity by the year 1921, when each year, and eventually each day, would bring a new scientific era: "...in 1917 - it [Electric Phase] would pass into another or Ethereal Phase, which, for half a century, science has been promising, and which would last only $\sqrt{17.5}$, or about four years, and bring Thought to the limit of its possibilities in the year 1921."⁷

And to give him the best possible date for predicting the singularity, here is a quote from a letter he wrote in 1862, while seeing the decline of wood-based naval ships.¹¹

Now, I didn't discover Henry Adams. From what I can tell, John Smart first connected Henry Adams to the singularity idea in 1999.¹² And you can even try to connect people to the singularity further back, like Nicholas de Condorcet in 1794 Sketch for a Historical Picture of the Progress of the Human Mind who wrote "the methods by which genius will arrive at the discovery of new truths, augment at once both the force and the rapidity of its operations."¹³ Or Edward Gibbon in 1781 wrote "His[Mankind's] progress in the improvement and exercise of his mental and corporeal faculties has been irregular and various, infinitely slow in the beginning, and increasing by degrees with redoubled velocity."¹⁴ You can look for anyone talking about positive second derivatives, but Henry Adams is probably the first mix of exponential plots and reasoning about the shortening of scientific eras.

Technocracy (1932) - AI policy

Now what about AI policy? This is where today rhymes quite well with history. Let's look at the Technocracy movement, which started in 1919 and organized in 1933. Take a look at the cover of of their The Technocrats' Magazine from 1933:

The Technocrats' Magazine, 1933. Unknown artist; public domain.¹⁵

We can see an AI robot destroying Washington, D.C., probably metaphorically. Thirty million people will be out of work. The other option appears to be $20,000 per year. Sounds pretty sweet. The Technocracy movement created this fear, uncertainty, and doubt around technological progress. They made plots, like Henry Adams, showing that power consumption was increasing each year. They argued that as automation displaced labor, the only true input to all productive processes was energy. Money would have no purpose except to allocate energy. They explained that the only way to mitigate the creation of a permanent underclass was a universal basic income.¹⁶ It would be distributed as ergs ($10^{-7}$ joules), though, rather than dollars.

Technocracy also proposed the unification of North America into a "technate," where the principles of thermodynamics would be used to rule the land rather than politics. They were sometimes accused of being socialist, partly due to their opposition to World War II and partly due to their dismissal of money, prices, and private ownership. To head that off, they said they are purely apolitical. Which then made it confusing (at least to me) about what their goals were. The Technocrats made some great propaganda posters though. Here's some more of their propaganda.

Technocracy Incorporated's "three buses" illustration, circa 1930.¹⁷

Cover of Howard Scott's 1933 pamphlet, "Technocracy: Science vs. Chaos," published by Technocracy Incorporated.¹⁸

The most striking thing about Technocracy is that their writings carry the kind of urgency and inevitability that I see in modern AI discourse. You're either AI-pilled and believe in recursive runaway intelligence because of first-principles reasoning, or you're uninformed. Technocracy conveyed this sense of doom from unbridled progress of automation, that could only be solved by leaving behind politics and embracing their science-informed policies.

Technocracy didn't catch on. Technocracy was banned in Canada for opposing Canada's participation into World War II. They shared a lot of vocabulary with the Bolsheviks (e.g., the early work that led to Technocracy was the "soviet of technicians").¹⁹ Their messaging read similar to the Marxist arguments of the time: the death of capitalism is inevitable, and you need to join the movement, or you're just uninformed.

The first ultraintelligent machine (1965) - Modern AI

This paper by Irving John Good is remarkable. Good somehow summarizes the state of the field in 2026 so succinctly in "Speculations Concerning the First Ultraintelligent Machine":²⁰

Much of the discussion on recursively improving AI research and the alignment problem are contained in this statement. Good went on to provide some reasoning about how such a machine could be built, which was less prescient, but he really nailed the heart of the discussion.

What is different about Good's quote from Henry Adams and Samuel Butler is that Good provides a mechanism by which a singularity could exceed human intelligence. Adams's argument doesn't seem to make much sense if humans could derive an entire scientific era, promulgate it, and innovate on it in one day. However, Good shows a mechanism by which our thinking machines could reach heights unimaginable by humans. It may then be possible for a scientific era to pass us humans by without us noticing.

The role of scientists with superintelligence (2000)

The last piece I'll bring up is this nice little fictional short story that Ted Chiang contributed to Nature in 2000, called "Catching Crumbs from the Table." It's about the role of regular scientists once superintelligence (in this case called "metahumans") takes over the advancement of science:²¹

This piece, back in 2000, was showing how human intellectual labor is no more resistant to automation than physical labor. Where Technocracy was focused on how to support a society whose automated workers would still find something to do in the future, here Chiang is showing how intellectual workers will still have to find meaning in their automated intellectual labor. Maybe instead of universal ergs, we each get one hypothesis per day we plug into an AI scientist and it falsifies or supports it. (I admit this doesn't really fit with the other examples, but it's so cool and so please go read the 2 page story.)

Conclusion

The last few years have been quite a rollercoaster of anxiety-inducing narratives in AI. First, you had to be doing foundation models to be taken seriously. Then it was agents. Then it was reasoning models and post-training. Then it was RL environments. Now you need a mythos-class model, or you'll be left behind. Learning a bit of history has revealed how similar human behavior has been in the past. I find this comforting and revealing about the "mode collapse" of how people rationalize technology progress. We keep reaching for the same ideas of runaway exponential progress, singularities, "inevitable" collapse of modern markets and politics, and human obsolescence. I'm not making a prediction about if ASI will arrive this decade -- I don't know. But hopefully some history has inoculated you to the mania of it.

Fun words

Here are some fun synonyms I learned while researching this: