Joshua B. Plotkin✓

This piece assumes much of the reader, but gives much more in return. Sadly I don’t imagine the AI overlords will even understand this eulogy.

RT @pastramimachine: We are searching for a new colleague in our Department of Biology at University of Oregon! This is a super special opportunity-- an endowed chair position in Genetics/Genomics. Come join us! I'm happy to answer any questions folks might have

RT @NITMB_Chicago: Applications are open for the NITMB Fellows program! NITMB is seeking early-career researchers who have an interest in transforming biological research and inspiring new mathematical discoveries Apply by November 15, 2026:

Opportunity: Penn MindCORE Fellowship! MindCORE seeks to recruit outstanding postdoctoral researchers for our Research Fellowship for Postdoctoral Scholars. Housed within the School of Arts & Sciences of the University of Pennsylvania, MindCORE is an interdisciplinary effort to understand human intelligence and behavior. Applications due December 1, 2026.

It saved some for you. (One shot!)

Credits: lyrics by me and Claude · music and vocals by Suno · video written in code by Claude on @_mexicat's pdoom engine

An evolutionary origin of collective decision making in humans and machines. New paper, with Guocheng Wang and Qi Su: Human institutions and machine-learning algorithms both decide by layered, non-linear voting. Engineers train networks with back-propagation. Institutions have no engineer. We find that rewarding individuals only when their vote is pivotal, at every layer, lets collective intelligence evolve by peer imitation alone. That reward scheme plays the role of back-propagation in ML. This result gives a naturalistic, evolutionary account of decision making, by human institutions and machines alike.

1/3 Steve, I agree with your main point: a ranking against a uniform field of strategies and the outcome of evolutionary dynamics are different objects. But there is more to say, both about what Wolfram's post leaves out and about what the field has established since Nowak & Sigmund. The second part matters more, because much of it is proved rather than simulated. I also ran the experiment you propose on Wolfram's own finite-state machines; results at the end. I. What Wolfram's post leaves out 1/ Finite automata as strategies for the repeated Prisoner's Dilemma are a forty-year literature: Rubinstein, Binmore & Samuelson and others started this in the 1980s. Linster (1992) studied the evolutionary dynamics of all 26 one- and two-state finite-state (Moore) machines, which is exactly Wolfram's 2-state space. 2/ Axelrod ran Wolfram's adaptive-evolution experiment in 1987, with a genetic algorithm (in Davis, Genetic Algorithms and Simulated Annealing). Evolving against a fixed field of eight strategies from his second tournament, 11 of 40 runs produced rules that scored substantially higher than TFT. Those rules defected on the first move and exploited one member of the field. When the rules evolved against each other instead, reciprocating rules eventually spread. 3/ The experiment you propose, iterating the tournament, has also been run on Wolfram's whole class of strategies. van Veelen (2012) evolved populations drawn from every strategy that can be encoded by a finite-state automaton. The populations showed perpetual oscillation between cooperation and defection, through neutral mutants that allow other mutants to invade. 4/ Without errors, 20 of Wolfram's 22 two-state machines are deterministic memory-one strategies. TFT, Grim and WSLS are three such machines. His 2-state ranking is therefore a ranking of corners of the memory-one strategy space, the four-dimensional cube of cooperation probabilities that Nowak & Sigmund studied in full. 5/ Wolfram's play has no errors, or trembling hand. Without errors, Grim never enters permanent defection against an opponent that does not defect first. Errors are the reason WSLS and generous strategies are favored in the Nowak–Sigmund work, and errors really change the outcome in Wolfram's setting too (see below) 6/ Wolfram writes that it will be difficult to "prove a theorem" about which strategy is best. Theorems about which strategies resist invasion do in fact exist, and they are what I would most add to your thread.

For my thoughts, criticisms, and suggestions re Wolfram's work, as cited by @nntaleb below, see:

2/3 II. What has been understood since Nowak & Sigmund 7/ Your point that no strategy simply "wins" is a theorem. Boyd & Lorberbaum (1987) showed that no pure strategy is evolutionarily stable in the repeated PD. van Veelen (2016) proved that no strategy is evolutionarily stable in repeated games with discounting. They also proved that, for a large enough continuation probability, no strategy is even robust against indirect invasions: a neutral mutant can increase in frequency by drift and then give a second mutant a selective advantage. (Errors change this, and the cleanest theorems are about finite automata with errors are due to economists like Fudenberg & Maskin in the 1990s.) 8/ Press & Dyson (PNAS 2012) discovered zero-determinant strategies. A memory-one player using one can unilaterally enforce a linear relation between her own long-run payoff and her opponent's, extortion included. 9/ Press & Dyson also derived a lemma about memory. Against a memory-one player X, any strategy Y with longer memory yields exactly the payoffs of some memory-one strategy, obtained by averaging Y's responses over the histories that occur in play against X. The memory-1 equivalent strategy of Y thus depends on X. The upshot is that within one pairing, longer memory gives no advantage against a memory-one opponent. 10/ The opponent-dependence in the Press–Dyson lemma explains Wolfram's finite-state machine with 10 states, which scores the maximum against every 2-state machine in matching pennies. Against each 2-state opponent its payoff equals that of some memory-one strategy, but a different memory-one strategy for each opponent. Wolfram reports the same thing directly: the machine uses a different subset of its states against each opponent. 11/ Stewart and Plotkin (PNAS 2013) asked which memory-one strategies are evolutionarily robust. Robust means that, in a population of size N, no single mutant fixes with probability above the neutral value 1/N. For the donation game under weak selection we classified all robust memory-strategies, against invaders of anuy memory length. 12/ We also proved, for strong selection and any 2×2 game, that a robust memory-one strategy must be one of four types: • it cooperates with itself; • it defects against itself; • it alternates against itself; • it is an equalizer, which fixes its opponent's payoff. We gave exact robustness conditions for each type. 13/ The ZD construction extends well beyond two actions. McAvoy & Hauert extended ZD strategies to arbitrary action spaces with discounting. Stewart and I extended it to arbitrary numbers of discrete actions, arbitrary memory length, and n-player games, etc. 14/ Your point about whether Axelrod's properties describe successful strategies now has a mathematical answer. Axelrod identified these properties by examining which strategies scored highest in his tournaments and simulations. We have derived them mathematically (2013, 2014) as necceasry and sufficient conditions for cooperative strategy to resist invasion: • nice: a robust ZD strategy cooperates with copies of itself; • retaliatory: a robust cooperative memory-one strategy must defect, with positive probability, after its co-player defects against its cooperation; • forgiving, within limits: a robust ZD strategy is generous, so a co-player who departs from cooperation earns more than it does, but the co-player's advantage is less than half of the generous strategy's own shortfall below R.

3/3 III. Your proposed experiment on Wolfram's finite-state machines 15/ I retrieved Wolfram's own code from the Wolfram Function Repository and reproduced his 2-state results, TFT ranks 8 of 22. I then iterated his tournament: replicator dynamics from the uniform start, and a finite population of N = 100 with rare mutation. I did this with and without 1% execution errors. 16/ For 2-state machines without errors, the machine ranked first in the uniform field remains the most common machine. And the population converges to nice machines only, with full cooperation: Grim 0.67, TFT 0.13, WSLS 0.12, and one other nice machine 0.07. Grim and TFT are the only robust machines. Linster reported the same qualitative outcome in 1992. 17/ For 3-state machines without errors, your prediction holds: nine of the ten top-ranked machines defect on the first move, and the top one exploits an unconditional cooperator forever. Under repeated generations, howerver, these defecting strategies are replaced and the population quickly consists only of nice machines that are fully cooperative. 18/ With 1% errors, the 2-state ranking barely changes: Grim ties ALLD for first. But the evolutionary dynamics change completely. • Among 2-state machines, cooperation in the replicator dynamics falls to the error rate, and no machine is robust. • Among 3-state machines, cooperation is about 0.94 under both dynamics. Under strong selection the only robust machine, up to its start state, is one of Wolfram's 3-state machines, known in the literature as AON₂. 19/ The reason AON₂ is favored at Wolfram's payoffs is exact arithmetic. ALLD earns (T+P)/2 against WSLS, which equals R when 2R = T + P, so WSLS cannot resist ALLD. In the donation game this payoff is b/2. ALLD earns (T+2P)/3 against AON₂, which is less than R. So at Wolfram's payoffs, provably no deterministic 2-state machine sustains cooperation with errors, and a 3-state machine does. (The machine that selection favors in his own 3-state space is the strategy Hilbe et al. derived from axioms in 2017.) 20/ In short: Wolfram's recent experiment was first run in 1987, your proposed extension soon after, and in the decades since the field has produced theorems of the kind Wolfram says might be hard to find, plus genuine surprises like Bill Press and Freeman Dyson's discovery of zero-determinant strategies. It's an exciting an ongoing field, something that maybe Taleb might enjoy learning about.

Recommended. A clear explanation of new mathematical result, unsolved since Erdos. Gives me faith in the value of humans. Note the most important parts of the video is when the human explains the intuition (about injectivity), without writing on the blackboard at all.

And now there is also a great blackboard talk by David Wood presenting the entire proof ( 3/

RT @AlexKontorovich: This is *exactly* why I say we need *more* human mathematicians now than ever, not fewer! I watched the video, start to finish. I'm not a combinatorialist, but I understood every word, and could have understood every word as an undergrad. (The idea of counting things and showing that the count is positive, and therefore concluding that the things actually exist, is very familiar to number theorists!!..) I'm embarrassed I never even heard of this problem, despite my great Rutgers colleagues Szemeredi and Komlos and others (the Hungarian mafia) having worked on it. It's a beautiful problem, and beautiful solution. "From the BOOK", as a certain Hungarian might say :) If AI keeps coming up with amazing arguments like these, we, again, need many more mathematicians to go through them, make sense of them, incorporate the ideas in new research and textbooks, etc etc. So much work to be done; all hands on deck!

Happy to share a paper led by Anzhi Sheng. We find that allocation policies that promote cooperation tend to concentrate benefits among the well-connected. And so structured societies must weigh collective success against inequality it generates.

RT @PNASNews: Explore peer-reviewed applied mathematics research from PNAS, including machine learning, stochastic modeling, quantum computation, biological systems analysis, and more. Discover articles that advance understanding across disciplines:

Very fun.

Mount Zaō, Yamagata

Social learning with complex contagion — new work by Hiroaki Chiba-Okabe. We introduce a framework that combines payoff-based imitation with conformity or anti-conformity biases, and we study long-term behavioral outcomes in a population.

Q343 low light stress test.