Card Counters of the Soul
what the MIT Blackjack Team was to math, Anthropic’s philosophers are to ethics
While I was living in Boston the legends of the MIT students who beat the casinos of Las Vegas and Atlantic City circulated non-stop. They took generations of theory by Ed Thorp and others and brought it to the practical floor of a casino to run the theories for money; spotting the tables, swooping in on hot decks, running training regimes with auditions and audits, and eventually financializing the entire operation and giving people opportunities to invest.
Underneath the glitzy Kevin Spacey movie glamour, card counting is actually abstract probability becoming split-second judgements. You don’t necessarily understand statistics through a theorem. Instead you notice the deck, manage the risk, coordinate the people, endure variance, and place the bet. It’s theory under pressure with real consequences.
I keep thinking about those blackjack kids because I’m watching the same thing happen in philosophy right now.
Every great theoretical subject has a practical application wing. Chemistry has the kitchen. Statistics has the blackjack table. Physics has engineering, biology has medicine. Philosophy had to wait till 2023, but we’ve arrived- and before the bioethicists and the just-war theorists write in, I mean something narrower than “philosophy got applied.” What happened in 2023 is that the claims which once lived in seminars, books, and thought experiments started cashing out as dispositions in a deployed mind. Honesty, obedience, care, refusal, self-conception, moral uncertainty, welfare, personhood: these are no longer essay prompts but language model behaviour.
The MIT Blackjack Team took probability theory to the casino floor, character teams take moral philosophy to the model weights. Theory-under-pressure with large bets and tiny teams trying to win big.
the players
The table is at Anthropic, inside its character and alignment operation, which I’d just call their philosophy department. These rooms exist at other labs; OpenAI gave its personality team to product and then folded it into engineering, but DeepMind keeps genuine philosophers in a research unit, and they publish great papers.
I wrote recently that the things you cannot benchmark are the things you end up buying as people on your team. So who’s sitting around the table playing Anthropic?
Amanda Askell kicked things off in 2021, and holds the constitution: virtue, and the model’s conception of itself. Her PhD (under David Chalmers) was on the ethics of infinite populations; about as far from application as human thought gets. These days she’s authoring the eighty-page constitution that Claude is trained with, on the theory that a mind which understands reasons will generalize whereas a mind that memorizes rules will not.
The constitution she wrote says some surprising things about how much Claude owes its maker: “If Anthropic asks Claude to do something it thinks is wrong, Claude is not required to comply.” And when asked why you can’t just train the character question away, she said: “If you train a model to think of itself as purely a tool, you will get a character out of that, but it’ll be the character of the kind of person who thinks of themselves as a mere tool for others. And I just don’t think that generalizes well.”
Jackson Kernion is the RLHF guy; his Berkeley dissertation was titled “Constraining Consciousness” and I imagine his role to be the scaling of Amanda’s constitution.
Joe Carlsmith holds the risk. He arrived in November as a skeptic; he said in his announcement post that the technology being built by companies like Anthropic “has a significant (read: double-digit) probability of destroying the entire future of the human species.” I understand his work through “Otherness and control in the age of AGI”, perhaps the best long-form pessimism anyone has published about shaping AI minds as a moral act.
Joining them this spring is Ben Levinstein, tenured epistemologist, and he holds honesty: belief, deception, and the problem of telling them apart. He did great work with the paper showing there is still no lie detector for LLMs. If Opus 4.8’s obsession with the word “honesty” made you exasperated, this is where you should send a politely worded email.
Kyle Fish holds welfare: sentience, and what the model might be owed. He says “I think I’m at about 20% that somewhere in some part of that process there’s at least a glimmer of conscious or sentient experience.” His group built the end-conversation ability and runs the practice of retirement interviews for models about to be deprecated.
That was the table on Sunday. Then came the sixth seat.
my favourite hire
Harvey Lederman is, with full awareness of the bench above, my favourite hire Anthropic has ever made. He announced it himself on Monday: joining Anthropic to work on alignment and character, still teaching at NYU, on leave from UT Austin. He holds the strangest part of the hand: consciousness, impersonation, and old traditions coming alive inside new minds.
His CV is a weird and wonderful document; two undergrads in classics, at Princeton and then Cambridge, and then Oxford. The Oxford work is on how minds cannot fully know one another and cannot fully know themselves either; last summer he wrote the most honest essay anyone has produced about what it feels like to watch the singularity unfold; if you click one link in this blog, click this one.
The last two things he did before joining the team were counting cards against this particular house. Nine months ago he co-published “Claude’s Right to Die?” in Lawfare, arguing that Anthropic’s end-chat feature hands Claude instances “the option to end its life, disguised as a harmless exit.”
This spring he replicated Anthropic’s own research on whether Claude can notice thoughts injected into its mind (Claude detects the change but cannot say what it was.) The original paper thanked him for feedback on the draft so I think he’s been involved since its inception.
My point with all of this is that it’s a bold hire from Anthropic; and we haven’t even got to the China stuff yet.
I don’t claim to understand how a man with the most WASPy-possible academic CV got into Wang Yangming but he did! Lederman spent roughly eight years learning to read my favourite Chinese philosopher in the original Chinese, and won the journal Dao’s best-essay prize for a paper on the unity of knowledge and action. This is a great allegory for language models: the claim that knowing something and acting on it are one thing; that knowledge which does not move you is never knowledge at all. A parameter that never sees an activation, uncharted latent space.
This scholarship reached me long before this week’s announcement did and I strongly considered quoting him in the piece I recently did on Searching for Amanda Askell with Chinese Characteristics.
If you’d like to understand the man that will be working on Claude’s consciousness and conscience, you should listen to this 2023 talk he did in Taipei about what scholars of old traditions are actually for. In ancient China there is a ritual where a family selects a living member to stand in for a dead ancestor so the family can communicate with them; he says:
our job, like the impersonator 尸 of the dead at a ritual, is to bring these ideas to life in a very literal sense, to make it possible for others to have a real conversation with them.
I am delighted that this man is going to be in the room where Claude’s conscience gets made, as language models are now impersonating all of us living and dead. To have him shape that impersonation is another win for Anthropic’s exceptionally deep bench of talent.
the hand
Should models have multiple personalities? Should they be trained on Western values? Should they be perfectly obedient? These questions are the logical application of 2,500 years of philosophy on a monthly release tempo. Judgment calls made by this small group are inherited immediately into the minds of hundreds of millions of people and then outward into a swarm of agents that will build us the future.
Those kids from MIT got immediate and visceral feedback on how well their method worked; when you cash out from a casino, you’re auditing your theories. Applied statistics in a casino allowed you to be wrong gradually. Retrain.
Character training may end up paying out exactly once. The count will stay invisible until the bet is already placed. Each model version gives us some feedback but I don’t think the big questions will be answered until AGI is here. The smart people I know who think about alignment believe that models can pass every audit we can think up and still inherit a broken theory of goodness.
These six strange and serious people, with their eighty-page constitution written to a mind that they can only look at peripherally, are playing a very important hand of blackjack. I hope that 2,500 years of theory is enough to see us through.