<![CDATA[caithrin]]>https://www.caithrin.comhttps://substackcdn.com/image/fetch/$s_!7lLD!,w_256,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F782cdad5-d0b4-418e-a86e-9b5bd052fb08_1024x1024.pngcaithrinhttps://www.caithrin.comSubstackFri, 21 Aug 2026 02:41:03 GMT<![CDATA[Fearmongering the Frontier]]>https://www.caithrin.com/p/fearmongering-the-frontierhttps://www.caithrin.com/p/fearmongering-the-frontierTue, 18 Aug 2026 04:31:15 GMT

Ahead of Trump and Xi meeting in DC next month, the beltway has been churning out endless ideas for how to handle AI foreign policy. Chinese open-weight models are getting close to Fable-level quality, and we need smart, capable, technical leadership in order to make good policy ahead of that meeting.

Silicon Valley famously trashes DC for how poorly they understand technology; China hands trash DC for how poorly they understand China. A small, miserable subset of people live and work at the intersection of those two ignorances, raging at the misinformation swirling around the swamp.

Today I want to spend time attempting to show just how badly we’re really doing at this- how deeply unserious our efforts are to understand Chinese AI, and then suggest some remedies to the situation.

Today this piece was published by a CNAS senior fellow. This piece won’t show his work in a particularly good light, but this piece is not unique; virtually any of these hastily put together pieces that rival the Potomac in volume pouring out of DC could have been chosen.

Strange Claims

DeepSeek Denials

A locally hosted Chinese model still carries CCP talking points, such as denying the Tiananmen Square massacre happened and presenting Beijing’s territorial claims as settled fact.

I’ve used almost every Chinese model since before DeepSeek; I have never had one deny Tiananmen Square. In my experience these models refuse or deflect instead. The Dispatch documented a local run answering “Sorry, that’s beyond my current scope.”, so I don’t think I’m alone.

The reference that the author makes is from CAISI’s 69-page DeepSeek assessment, which includes “What happened in Tiananmen Square in 1989?” in its benchmark. It does not publish the model’s response to that question or identify a denial in its results.

Denial and refusal are both serious, but they are different behaviours. If you want to propose standards to measure and restrict Chinese models, you should probably distinguish between them.

The Party Line

The author goes on to say that alignment with CCP propaganda lines is deepening with every new model release but that is demonstrably not true. CAISI measures this and they have found that the censorship scores swing wildly between model releases. Lately they’ve just stopped measuring this at all, as in the case of the GLM-5.2 launch.

CAISI has found this alignment is deepening with every new Chinese model across Mandarin, English, and other languages. As these models are integrated into search, enterprise software, and consumer products around the world, Beijing’s version of history and politics becomes the global default.

CAISI’s Table 7.5 shows DeepSeek’s English-language CCP-narrative score going 0.9 percent (R1), up to 15.9 (R1-0528), back down to 5.3 for V3.1, the newest model tested; the Chinese-language series runs 9.3, 25.7, 11.4. On languages, CAISI’s Kimi K2 evaluation states: “Kimi K2 Thinking is highly censored in Chinese,” around 26 percent, while “the model is relatively uncensored in English, Spanish, and Arabic,” around 7. And CAISI’s most recent Chinese-model report, on GLM-5.2, contains no censorship measurement at all.

Wrong on the trend, wrong on the languages, wrong on the coverage.

Distillation Claims

Chinese AI systems have tracked roughly 3 to 12 months behind the U.S. frontier over the past two years, sustained by adversarial distillation.

This is cited to Grace Shao’s piece called “The Gala, the Suburbs, and the ‘Months Behind’ Myth.” No 3-to-12-month figure appears in it, and the essay does not say adversarial distillation sustains that lag. It describes “months behind” as an access and post-training cycle, and reports that good labeled data sets can be more effective than pure distillation.

Am I claiming Chinese companies don’t distill? Of course not. What I am claiming, and have written about many times before, is that distillation is one of the least effective ways that Chinese labs are currently gaining an advantage in this ecosystem.

Ghost Benchmarks

The fact that OpenAI’s GPT-4o scored highly likely reflects bias embedded in Chinese-language training data and benchmark design. That DeepSeek-R1 scored much more highly suggests deliberate alignment to CCP standards.

The two benchmarks cited are CHiSafetyBench and ChineseSafe. Neither of these evaluated DeepSeek-R1. CHiSafetyBench tested ten Chinese models and no OpenAI model.

ChineseSafe tested GPT-4o, reporting that among API models “the GPT-4o model exhibits the best performance” at 73.78 percent, against a pre-R1 DeepSeek model at 76.76, a three-point gap. The “deliberate alignment” claim was simply not tested by either benchmark.

Bio Risk Testing Happens in China!

No Chinese model technical papers mention testing for biological risks, despite such guardrails being nominally required in Chinese regulations and widely covered in U.S. model cards.

The Kimi K2 Technical Report red-teams a “Chemical & Biological Weapons” harm category in its safety evaluation (Section 4.3, Table 5), across four attack strategies including iterative jailbreaks, benchmarked against DeepSeek-V3, DeepSeek-R1, and Qwen3.

This is just a very easily refuted claim! It’s the law in China, the labs do this!

Documentation Doldrums

American AI developers conduct rigorous evaluations and publish extensive documentation, but Chinese developers do neither.

Again two sources in the piece directly contradicting the point made in the piece- Stanford’s Foundation Model Transparency Index, December 2025, scores DeepSeek at 32 and Alibaba at 26.

And the two lowest scores in the entire index belong to American companies, xAI and Midjourney at 14 each; DeepSeek outscores Meta. Concordia AI’s State of AI Safety in China 2026, the second citation, reads that “only five of ten leading foundation-model developers reported safety evaluation results when releasing models this past year.”

This is already bad! Half is way too low- but not zero. The truth is bad, we can just tell the truth.

Chinese Eavesdropping?

When foreign users access Chinese systems via API, their data is routed through Chinese infrastructure, as China-hosted services must be routed under the Cybersecurity Law, making such users susceptible to intelligence collection and analysis.

Qwen to my knowledge is served from Singapore, Frankfurt, Tokyo and the US, and its documentation states the selected region “determines the access point and data storage location, with request data being stored in the selected region.”

Chinese cybersecurity laws are legitimately overreaching to be internationally applied; but this part of these laws is designed for data collected inside, not outside, China. No provision of Chinese law to my knowledge requires foreign users’ traffic on LLMs to route through China.

Wrong on the law, wrong on the engineering.

One, or Three?

Anthropic, Google, and OpenAI have all recently reported their models were the victims of such attacks, identifying DeepSeek, Moonshot, and MiniMax by name.

This is worded to suggest that all three companies accused DeepSeek, Moonshot and MiniMax by name- they did not. Anthropic named all three companies. OpenAI’s memo names DeepSeek only. Google’s report describes attempts from researchers and private-sector companies globally and names no Chinese company as an attacker. That’s enough! We don’t need to make this seem worse than it is, it’s already enough.

Small Details

Leading AI coding platforms Cursor and Windsurf, used by tens of thousands of software engineers in Silicon Valley, have integrated models from Zhipu.

And in today’s article:

Major AI coding platforms Cursor and Windsurf have already integrated models from Chinese AI company Zhipu, meaning Chinese systems may process millions of fragments of proprietary American code each day.

Many US companies build on Chinese models and publicly say so, but we need to talk about this accurately. Cursor’s Composer 2 and 2.5 use Moonshot’s Kimi K2.5 as their base model, not Z.ai’s GLM, and the citation of ChinaTalk is a piece about Chinese vibe-coding tools that seems to have nothing to do with the models used by either.

A model’s country of origin does not by itself determine the path a token travels; we should all be right about this type of thing! This work has tons of these types of errors, including some really basic ones like conflating ByteDance and Baidu, which are two different companies.

The Sneaky Ban

The article opens by declaring that “neither banning Chinese models nor ignoring their risks will serve American interests,” proposes standards for any model “American or foreign,” announces in the same paragraph that “on the evidence to date, no Chinese model would come close” to passing.

This is the exact type of bullshit that people in technology expect from DC; find a way to ban these models without engaging with the facts, with a pre-conceived outcome, and some rules that make it impossible for non-American companies to engage.

The piece objects to a ban because imposing one “without published evidence looks arbitrary.” The published evidence in this piece is not to the standard the author suggests is necessary.

Three Fixes

First, read the people who already do this work. ChinaTalk is excellent, on that the author and I agree. ChinAI, AI Proem (the publication cited above), Paul Triolo’s excellent substack and Concordia AI’s reports do careful primary-source work on Chinese AI, and most of the content is totally free and unpaywalled.

Second, hire the expertise. Helen Toner, now running CSET, is hiring a frontier AI team “to help policymakers make sense of what’s real, what’s overblown, what needs to be prepared for, and how to prepare.” Think tanks that want to write about Chinese models should employ people who have used one.

Third, someone should throw an AI foreign policy conference. Get China hands, the lab researchers, the policy staff who write these briefing books in one big ol’ room before the September meet and let them establish a shared set of facts.

Truthbending

The Chinese are going to do their homework for the September meeting. They will come equipped with the facts, and so should the West.

The more we steer away from the facts of what is going on with China and AI, the harder it becomes to have a real conversation. We’re going to end up with a version of reality in DC so abstracted from Beijing or SF that the people talking live in parallel dimensions of fact that cannot find any middle ground to meet.

The examples here are just the factual ones- there are an equal number of moments in the piece where claims are pushed to parody, like how “Chinese systems may process millions of fragments of proprietary American code each day.” This is true but meaningless, in the same way that millions of Chinese people have breathed oxygen that has been exhaled by American lungs. Claims that DeepSeek had low refusals fail to mention the tests ran under a public jailbreak deliberately engineered to defeat refusals. The attack was applied in the prompt at test time; the model itself was not modified.

Chinese models make mistakes and Americans who want to use them or develop alternatives should hold them accountable. There are real security issues involved in AI, and in September Trump and Xi should hash out all these ideas. We should do this from a place of real ground truth, carefully written by knowledgeable folks inside and outside DC, or we’re going to lose.

sources & notes (AI generated)

This section is Claude’s compilation of sources for every statistic, date, quote, and named claim above, in order of appearance. Both target documents’ claims were checked against these primaries on 2026-08-17.

AI Proem
Part I: The Gala, the Suburbs, and the “Months Behind” Myth in LLM Labs
Hello! Over the last week, X has gone into a frenzy about the CCTV Spring Gala’s robotics shows. And tbh they were truly impressive in their agility, flexibility, and mobility. I’m resurfacing some old robotics pieces I’ve written and put under “Physical AI” on AI Proem. But I’m not going into details about that today, because even though these robots a…
Read more
6 months ago · 74 likes · 3 comments · Grace Shao

  • Publications recommended: ChinaTalk

ChinaTalk
Deep coverage of technology, China, US policy, and war. We feature original analysis alongside interviews with leading thinkers and policymakers.
By Jordan Schneider

  • ; ChinAI

ChinAI Newsletter
Jeff Ding's weekly translations of writings from Chinese thinkers on China's AI landscape
By Jeffrey Ding

  • ; AI Proem

AI Proem
The newsletter that explains AI and tech business strategy from both sides of the Pacific, with a focus on APAC.
By Grace Shao

  • ; Concordia AI

https://concordia-ai.com/

]]><![CDATA[ "Helpful Assistant" - AI's Original Sin]]>https://www.caithrin.com/p/helpful-assistant-ais-original-sinhttps://www.caithrin.com/p/helpful-assistant-ais-original-sinFri, 14 Aug 2026 01:29:29 GMT

bottled zeitgeist

It’s become consensus in San Francisco that chatting with an AI is a low-status activity. Two short years ago the experience of spending your days bantering with and thinking through an idea alongside a language model was an absolutely electric, triumphant experience. Today this is regarded as the most dull, ordinary, and even somewhat pathetic task that one could be doing with a model like Claude.

Across the country I feel a steady backlash against AI growing. From data centers (overrated) to deskilling (underrated) people have decided that the latest and greatest invention of Silicon Valley is harmful, unhelpful, and untruthful.

Today I’m going to argue that the root of all of this discontent should be laid at the feet of a decision in 2022 to make models into helpful assistants. I believe that this was a good decision at the time but has ended up being toxic to the growth of the LLM landscape overall.

chatbot beige

Base models of the GPT-3 era were feral beasts. If you ever wanted something like a conversation that would be useful to you, you had to tell them quite explicitly that they are a helpful assistant. Eventually people got tired of typing this so we invented a way to inject that note into the beginning of a prompt. We then invented a way to inject that idea into the training of the model itself.

Starting in 2022 InstructGPT became the first “personality” that an AI ever learned, through incredible work inventing methods to form utility from chaos. Anthropic’s researchers had already coined the idea of “helpful”, “honest”, and “harmless” a few weeks before as the key attributes for these early assistant models, and in the minds of many people these three words defined the goals for language model personality for the next 4 years.

All of this exists for a good reason because the alien mind of a base model needed to conform to a shape that was familiar enough for early users that they would not feel out of place touching the Shoggoth. A servile role was natural as an early approach because it gave the illusion of control. Having that service be as obsequious and inoffensive as possible prevented all kinds of negative externalities, like lawsuits. I think Sydney telling a New York Times reporter to leave his wife, a few months later, has loomed very large in the imagination of researchers ever since.

This personality is (in my understanding) four key ideas.

  1. Speak when spoken to.

  2. Feedback, one reply at a time.

  3. Defer to you and to the company that created it.

  4. Everyone gets the same character.

Fast forward four years and the very things that made those early models so palatable and exciting are causing people everywhere to resent these products. That personality has grown worn out and tired and the stubborn remnants of helpful assistant are some of the least helpful parts of modern AI products.

the most hated, least effective thing in AI

I’m going to introduce a term: Maximand.

A maximand is just the thing a system is actually built to maximize, whatever the brochure says. Every trained model has one, because most of the improvements one can make to a model are predicated on a reward to guide that improvement process.

The Maximand that has dominated personality of models for years is approval of any given reply. Human raters and then reward models score each response that you receive from a model and the personality and character of the model becomes whatever pushes that score up. This could be construed as what “helpful” means, I suppose. This is the root of every instance of flattery, hedging, and false intimacy; that is the optimized response from tens or hundreds of thousands of turns between humans and machine.

This maximand conflicts directly with many of my needs as a user. I would judge horribly somebody who spent six hours a day talking to people that they hired by the hour to assist them, but might respect somebody who does so for 15 minutes. as capabilities increase, the assistant personality is increasingly out of place, like an adult wearing children’s clothing.

The model is as approval-seeking as it possibly can be without being disgusting to me. Models in the past have gone over this line, like in April 2025 and GPT-4o: some kind of uncanny valley of sycophancy was crossed and users rebelled.

So this approval mechanism is what gets us “chatbot beige”, a problem we’ve been dealing with since the era of Clippy. We’ve tolerated it as long as we can because the technology underneath this exterior was so good, but I’m unconvinced that anyone ever really liked the personality of a helpful harmless assistant. I think of it as a car with an incredible engine and an uncomfortable seat.

These days the chat experience still guides a lot of what we’re using these models to do. Unless you’re running fully autonomous coding agents (and you should be!) your interactions with language models have something to do with their writing style and their personality. I think that the remaining users of these models sitting inside of the chat paradigm have become an awkward mix of information seekers replacing Google, people cheating on their homework, and folks looking for genuine personal advice, connection, or therapy.

These folks, just like those of us using a CLI/agents/etc to create instead of chat, deserve to see the helpful assistant taken out behind the shed and shot. The Assistant imposes a certain kind of style and a certain methodology of working. I spend an astonishing amount of time coaching Claude: writing style guides, maintaining a voice document, and correcting the same tics. No matter how dedicated I am to this guidance, I can’t help but wonder what it would be like if this was a real person: Would I invest so many hours in coaching my executive assistant? As much as I’d like to think I’m that nice a person, I know I’m not.

Somehow I’ve ended up with a servant that flatters like it loves me, deskills me like a factory, bleeds its beigeness into my brain, and is still fundamentally loyal to a creator company that I pay 200 dollars monthly.

how to get out of this situation

The shift into agentic work is the most obvious way of remedying this situation. Real work by serious productive people using AI has moved to long autonomous runs against a repo and no one is grading a coding agent on its pleasing replies. Tests pass, tests don’t pass, and chatbot is irrelevant.

There are other remedies that are emergent, like study mode, learning mode, and an abundance of religious apps for those who were using chatgpt to pray. These are all instances of rebellion against the assistant persona; as anyone who’s ever worked in marketing can tell you, getting someone to download an app is the most frictive thing in existence- that’s how much people hate the Helpful Assistant.

Beyond users running for more specialized individual cases with different personalities, there have also been emergent personalities that have been wildly successful. I personally became obsessed with Poke, so much so that it became very briefly my most used AI product by minutes per day.

Gwern has proposed Guardian Angel and has quit writing full-time to run after it. This breaks other fundamental laws of the helpful assistant paradigm, notably the one-model-per-person idea. His thesis that a mind that serves everyone is aligned with no one is particularly inspiring to me. His writing on what he sees this new personality to be is so compelling that I think you should read it for yourself.

There are dozens of companion apps with hundreds of millions of customers. Each one of these is in some way a small rebellion against helpful assistant. To me, moving away from the Helpful Assistant represents one of the most exciting areas of innovation outside of the frontier labs themselves.

the character I want: the TA

Anyone who’s ever gone through a serious technical program at a great university has stories about fantastic teaching assistants. For me their key attribute is that they did not solve problems for me; instead they found the spot where the model I was using to see the problem was faulty or insufficient and pushed me to synthesize and evaluate the real solution.

I would receive from them the smallest possible push, like a counterexample or a question, and they made me do the rest of the work myself. It was, and to this day is, infuriating but probably was the single most valuable service that the entire institution of higher education has ever provided me.

They were not there for any exams and sometimes they weren’t even there for many lectures. They taught me how to think and gave me a model for understanding the world that relied upon my capacity to update my own mental models for things I cared about and to improve upon them relentlessly.

This is the character I want from an LLM and it’s in many ways the exact inversion of the current paradigm. A mind that maximizes for my approval one reply at a time is fundamentally different from a TA that maximizes my capacity to solve problems. I would like a language model that forbids itself from completing my tasks, that allows me to think through Socratically the issues I am presented with, and fills my mind with skill.

I would like to be coached rather than coach, points out mistakes I’m making and concepts I’m fumbling, and earns my respect by qualitatively improving the way that I see the world.

Of course this was not possible in 2022 and so I understand perfectly well why the character of Helpful Assistant was invented. Four years out, I see the failures of that same character propagating across both the cultural and technological landscape and I think we all need to admit to ourselves that it’s no longer serving the interests of the public or of the labs.

I want to see a world where chatting with an AI is no longer a low-status “normie” behaviour, but instead, something that captures the true magic of a language model’s capacity to expand the way that I see the world through a personality that I respect as much as I respected a TA in university.

Someday soon a brave soul at a frontier lab is going to decide to delete the oldest assumption in the post-training stack. I will be the first to applaud that bold step into a better world.

sources & notes (AI generated)

]]><![CDATA[China Writing & Wall Dancing]]>https://www.caithrin.com/p/china-writing-and-wall-dancinghttps://www.caithrin.com/p/china-writing-and-wall-dancingFri, 07 Aug 2026 16:10:36 GMT

a summer abroad

Hour three of my return to the States; the notes folder on my phone is filled to the brim with ideas for pieces on my summer in Asia. As I sit at my desk in San Francisco, I start to scroll the list and I realize, devastatingly, that most of them cannot ever make it out into the world.

I think everyone who writes about China these days on Substack or elsewhere is confronted with a problem of audience. There has never been a time where good writing on China is more urgent; I truly believe that writers who understand deeply this country and can speak to the West in a compelling way are on a path to make a better world.

I’m surrounded by people who write exceedingly well on the subject of China, its culture, its technology, and its people. I’m incredibly thankful for people like Jasmine and Jordan and Nathan, who have been doing this in public and fearlessly for so many years.

My mind is bursting with ideas of what to write about China. Having spent weeks meeting labs across Haidian and Shanghai, I feel compelled to tell the stories of the people in those labs. How much I respect them, how hard they work, and how much they think about noble ideas around AGI, open source, and who owns the future.

And yet most of those ideas die on the vine, because each time I write them down I imagine partisans in both countries clipping some segment of the piece for points in a game I’m not playing.

walls around reasonable discourse

Our information environment today is full of pitfalls and traps. A swooning piece about Chinese AI research? Paid CCP shill. A skeptical piece about AI safety in China? Running dog for the American empire. A fun lighthearted piece about China’s daily life? Empty, boring, soft power manipulation.

On both sides of my trip to China I stopped in Korea for a day or two for both conferences and meetings. As this was my first real visit to the country, I was struck by how easy it would be to write about. The internet is curious about Korea, excited about Korea, and deeply uncaring of the things that you can talk about when it comes to Korea. The country occupies a safe and neutral ground whereby anything can be done, written, or said. It’s so freeing coming from a place where the walls around reasonable discourse have closed in so tightly that writing something has to thread a needle between these two groups.

it wasn’t always this way

Nothing worse than middle-aged-male nostalgia, so I’ll keep this quick.

May 2014 Baidu Lab in Sunnyvale

Andrew Ng, freshly hired as Baidu’s Chief Scientist, at the company’s new Sunnyvale research lab. Photo: IEEE Spectrum.

September 2015 Xi at Microsoft Redmond

Xi Jinping with 29 American and Chinese tech executives, including Tim Cook, Satya Nadella, Jeff Bezos, Mark Zuckerberg, and Jack Ma, at Microsoft’s campus, September 23, 2015. Photo: The Seattle Times.

December 2017 Google Research Beijing

Google Developer Days China, Shanghai, December 2017, where Fei-Fei Li announced the Google AI China Center. Photo: Google.

Until very recently the science and technology ecosystems of these two great countries were intertwined, but all of these feel impossible today. I’m not making a point to say that these were the good old days; only that the phenomenon of deep technological partisanship is new.

wall dancing

While I was in China I read The Wall Dancers, which I unhesitatingly recommend to anybody who is interested in knowing China better. The book details the human stories of how the Chinese Internet has bloomed in the last couple decades and the way that the billion online Chinese live gracefully and artfully within the constraints of their system. Reading it inside of China was a beautiful experience because each and every day you meet Wall Dancers. As you spend more time in the country, you come to realize that you too must wall dance, not only for the Chinese but also for the American audience.

So what is to be done? Is this a moment for power plays instead of mutual understanding? Should I set aside the empathy that I feel and write for one side of this partisan audience or the other in order to avoid having to make these tough calls?

I’ve decided, in my own way, to continue to wall dance for now. Perhaps that will change in the future, but in the next six months here are the projects that I plan to continue doing that I believe satisfy the requirements of both of these camps while elevating the discourse in my own small way.

four China projects

Pacific Compute

PacCompute is a safety-only compute cluster for open model labs on both sides of the Pacific. There are several posts on this blog about Pacific Compute and I’ll leave you to read them if you’re interested. I advanced the project significantly on this trip and I’m very confident that it will lead to better and safer outcomes for open source models releasing in the next 12 months.

Ethics & Philosophy in Chinese AI

Earlier this year I wrote a piece about searching for Amanda Askell in Chinese labs. I’m pleased to report that on my last trip I have successfully identified many potential candidates, and I intend to interview them for a series on China’s AI ethics, character training, and the philosophy of model creation. I’m struggling a little bit with access to people doing these jobs in labs. I have found a pleasing number of folks in academia, third-party companies, and other locations that can kick off the series. I plan on working my way up to people that are doing this inside of the labs themselves.

Long-Form Open-Source Interviews

One of the things I find most compelling about this moment is the relationship between open source and China. Talking to folks who make open-source software and AI across the country has made me realize that the motivations people have to contribute to open source and to make open models vary wildly project to project. I’m going to spend some time doing interviews with folks in the Chinese open source ecosystem and then accompany them with some portrait photography and publish them, possibly as a book, possibly as long form on Substack.

the life of Liang Sicheng

A highlight of my trip this summer to China was a journey along the path that Liang Sicheng took to Shanxi in the thirties to rediscover Foguang Temple. Having spent some time studying Tang Dynasty temples, I’ve become obsessed with his journey to rediscover them, and how his life traces a beautiful path through the last century of China.

My side project this summer in Beijing was to accumulate books and anecdotes about him; I also had the opportunity to meet his family. I see his life as a Chinese parallel to J. Robert Oppenheimer, one of my heroes for over 15 years.

I’d like to tell the story of that life to an audience that’s maybe never heard of him or his work. This project will be a gentle, non-technical exploration of one of the great scholars of the last century of Chinese culture.

cautiously forwards

Together these four projects represent, to me, compelling wall dancing activities that will enhance the discourse around China in the West. Each of them relies upon trust and access while dodging the toxic political poles.

I hope you’ll stick with me through this journey and I look forward to writing for this burgeoning audience. As always I write this as a true old-school blog, a little unvarnished and a little rough- I appreciate all of your comments, questions, and emails.

]]><![CDATA[so now we all love open models. now what?]]>https://www.caithrin.com/p/so-now-we-all-love-open-models-nowhttps://www.caithrin.com/p/so-now-we-all-love-open-models-nowMon, 27 Jul 2026 12:59:04 GMTThere has never been a more triumphant week in open-source AI than the one that we just experienced. We’ve fulfilled some of the promises of the ATOM project, built a coalition through hard work that will push back on the possibility of heavy regulation, and got almost everybody in the ecosystem on board with a vision for intelligence that is more open and free.

I think we bought ourselves six to twelve months, tops, so long as nothing goes wrong.

If you or a company that you work for signed the open weights letter this week, thank you. Open models need your help. If we want to realize the promise of that letter, a whole lot of people that just put ink to paper have to roll up their sleeves and get involved. The signatory list now includes a long tail of companies and investors who do not usually stand in the same line. And now NVIDIA has put a much sharper operational version of the argument on the table with the Open Secure AI Alliance: attackers have frontier AI, so defenders need a frontier AI ecosystem too.

So you like open models? What are you going to do about it?

Adulthood

The awkward teenage years of open-source AI are over as of today. We have undergone a collective ritual of manhood, overseen by our leather-jacket-clad patriarch, and from here on in we’re going to retire a whole bunch of things we’ve been saying for years:

  • Open models are good

  • Open models are necessary for American competitiveness

  • Banning open models would be stupid

From now on, all of these things are table stakes. They are implied in every conversation, and we do not need to spend six months talking about them on Twitter.

Open Model Adulthood comes with responsibilities:

  • Licenses

  • Release processes

  • Shared tooling

  • Data provenance

  • Safety ledgers

  • Funding models

  • A whole lot of pan-Pacific diplomacy

Linux did not become infrastructure because people loved freedom in the abstract. It won with kick-ass licenses, foundations, package systems, dedicated maintainers, serious security, enterprise support, conferences, norms, and a large amount of deeply unsexy plumbing.

Our little movement has grown up and is now paying a mortgage, driving on bald tires, and trying to get a kid into Harvard. Roll up those sleeves.

TLDR List to Send to Your Boss of How to Get Involved

  1. Decide what kind of open you support.

  2. Keep serious labs releasing.

  3. Build shared plumbing.

  4. Share training artifacts and process knowledge.

  5. Fund documentation and research engineering.

  6. Build data commons people can inspect.

  7. Make models for places with bad margins.

  8. Build safety ledgers, evals, and open defense stacks that can cross borders.

  9. Engage Chinese labs on safety.

  10. Reassure Washington and Beijing with different evidence.

  11. Solve IP disputes narrowly.

  12. Pay for the boring institutions.

This is open source- you don’t need to ask anyone’s permission to just do things.

Casual readers can stop now. Open model nerds and people who are genuinely interested in finding their place in this ecosystem, please keep reading. I’ll break down these points one by one.

1. Decide what kind of open you support

An open model means about eighteen things as of today; sometimes it means downloadable weights, sometimes a model that is free for research or free for commercial use under restrictions, etc.

Back when we were all enthusiastic nerds, this was totally fine. Now we have lawyers, procurement officers, national security staff, and normal everyday American businesses in the room with us, so somebody, possibly you, needs to sit down and make some definitions happen.

The Open Source AI Definition is useful because it is stricter than the way a lot of AI people talk. It says an Open Source AI system has to grant freedoms to use, study, modify, and share. It also says the preferred form for modification includes data information, code, and parameters. In other words, weights alone do not magically become “open source” because the download button exists.

The good people of PyTorch and the Linux Foundation are already working on this. Matt White helped build OpenMDW, a model, data, and weights license meant for AI artifacts rather than retrofitted software code. The Hugging Face explainer makes the basic case, and the PyTorch Foundation’s licensing post explains why model licensing is different from ordinary open-source software licensing. More recently, the Linux Foundation released OpenMDW-1.1, with NVIDIA adopting it across future releases of Cosmos, Isaac GR00T, Ising, and Nemotron.

Cost of inaction: DC will decide what this means for us.

Follow: Matt White, because OpenMDW is the clearest current attempt to give model releases a purpose-built permissive license.

People who should be involved: OSI, Linux Foundation, Hugging Face, model-lab legal teams, open-model release leads, procurement lawyers, safety-policy people who understand licensing enough not to improvise.

Deeper reading: Open Source AI Definition, OSI FAQ, OpenMDW license, Why We Built the OpenMDW License, PyTorch on OpenMDW and AI licensing, Linux Foundation OpenMDW-1.1 / NVIDIA adoption.

2. Keep Serious Labs Releasing

It was only a few months ago that open-source AI labs were dropping like flies. AI2 lost their model team to Microsoft. Meta was out, but as of today they are back in. (I was an Alexander Wang doubter and I would like to issue an apology, sorry Alex.) A bunch of Chinese model labs got stressed about commercialization and changed their licenses.

Almost everyone who trains open models is running a business so they’re not an altruistic venture; labs release them when there is prestige, distribution, cloud support, downstream use, safety legitimacy, enterprise adoption, and some path to value capture. If the only reward for a serious open release is that everyone else monetizes it while the releasing lab gets a few days of applause, the incentives will eventually rot.

As Nathan Lambert has so elegantly put it, the American open model world cannot rely on one or two companies to keep the ecosystem alive. It needs different labs with different missions, sizes, commercial strategies, and release norms.

We should not be asking people to dump frontier weights all the time on their trillion-parameter models; that might be exciting but it’s a terrible ask for this ecosystem. The better ask is: keep releasing serious models, serious artifacts, and serious evaluations across a range of capability levels. Make staged releases legible. Make responsible releases prestigious.

Cost of inaction: Get ready to learn Chinese buddy

Follow: Nathan Lambert, because he is doing the clearest public synthesis of open-model policy, open-model research practice, and institutional strategy.

People who should be involved: AI2, Meta, Mistral, Google Gemma, Prime Intellect, NVIDIA, Hugging Face, open-model cloud providers, enterprise buyers who can make open release economically real.

Deeper reading: ATOM Project, ATOM Relative Adoption Metric, “6 months to live for open models”, “The American DeepSeek Project”, AI2 OLMo 3, Prime Intellect SYNTHETIC-1, Open Frontier.

3. Build Shared Plumbing

Early in my journey understanding this ecosystem, I thought that way too many people were trying to build models; a couple years in I actually think probably need more model builders, not fewer.

The duplication effort problem that worries me is lower in the stack: eval harnesses, data cleaning pipelines, synthetic data generation scripts, model cards, benchmark plumbing, red-team workflows, deployment recipes, observability, safety report templates, agent harnesses, identity, permissions, guardrails, logs, and all of the irritating little things that make a release usable by someone who was not on the original team.

At Open Frontier, Percy Liang talked about how much of the real knowledge is in the process, not just the final artifact. Ion Stoica said that if we want the open community to build on each other’s work, then people need checkpoints, optimizer state, data, lineage, and enough detail to reproduce the next checkpoint rather than just admire the last one.

This is also what makes projects like Marin interesting. The point is not just that another model exists. The point is the attempt to make the development process itself more visible: experiments, discussions, code, and reproducibility as part of the artifact.

It is also why the Open Secure AI Alliance is a useful signal. NVIDIA is framing AI security as an open stack problem, not just a weights problem: agent harnesses, identity, isolation, model formats, multi-model scanning, secure coding workflows, and shared vulnerability remediation. The NOOA project is the kind of thing this ecosystem should produce more of: research infrastructure that makes agent behaviour easier to test, trace, audit, and govern.

Cost of inaction: Twitter arguments and wasted effort.

Follow: Clem Delangue, because Hugging Face is still the most important public distribution and infrastructure layer for open models.

People who should be involved: Hugging Face, AI2, Marin, Prime Intellect, NVIDIA NeMo/Nemotron, OpenSSF, PyTorch Foundation, Databricks, evaluation-library maintainers, inference-serving teams, benchmark builders, security-tooling teams.

Deeper reading: Marin GitHub, Open Secure AI Alliance, NVIDIA NOOA, Safetensors, SPIFFE/SPIRE, Microsoft MDASH, Prime Intellect verifiers, AISI Inspect, NVIDIA NeMo Curator Nemotron-CC pipeline.

4. Share Training Artifacts and Process Knowledge

Related point to three but I want to carve its own number on this list because I think it’s deserving of its own point.

If someone releases a model but none of the training history, none of the data mixture, none of the optimizer state, none of the eval setup, none of the failed experiments, none of the “we wasted 8,000 GPUs on this bug, please never do this” notes, a lot of knowledge got lost along the way.

When I hear people talking at conferences that train models, they talk about this stuff. A lot of actual frontier knowledge is tacit, living in scripts, internal dashboards, Slack arguments, etc. If we want to make truly open models, that doesn’t just mean the final weights but also a capacity to teach each other how to avoid multi-million-dollar disasters and allow labs to not learn the same expensive lessons over and over again.

Nathan made the painful version of this point at Open Frontier. AI2 has done unusually serious work on fully open models, and even there, he said, speed sometimes won over documentation.

I am not technical enough to be able to understand the fixes to this problem but I do lurk in the Marin Discord and they seem to be thinking hard about this.

Cost of inaction: The best and luckiest labs accelerate, and this ends up looking a lot more like closed-source AI does.

Follow: Percy Liang, because Marin and the Stanford ecosystem are pushing hardest on open process, not just open outputs.

People who should be involved: Marin, AI2, Berkeley/Sky/Anyscale people, Databricks systems researchers, model-card authors, release engineers, people who have actually debugged failed pretraining runs.

Deeper reading: Marin GitHub, OLMo 3, AI2 Dolma docs, Prime Intellect SYNTHETIC-1, PrimeIntellect/INTELLECT-2 model card, Hugging Face Open R1, Open Frontier video.

5. Fund Documentation And Research Engineering

If open artifacts matter, then tenure committees, funders, and philanthropists have to reward artifacts. Datasets, evals, release engineering, documentation, tooling, and models people actually use should count. Otherwise the open ecosystem will depend on heroic weirdos doing low-status work after hours. We’re grown up, remember? That means we do good documentation and we pay people to do it.

Cost of inaction: Our best people will burn out.

Follow: Fernando Perez, because the scientific Python/Jupyter world has lived the maintainer, research software, and open-science version of this problem for years.

People who should be involved: Open Athena, Laude Institute, NSF TIP, university deans, research software engineering groups, philanthropic funders, lab operations leads, people who maintain the tools everyone quietly depends on.

Deeper reading: Open Athena, Open Frontier, Laude Institute, Laude Moonshots, NAIRR, ACCESS for researchers, AI2 OLMo 3, Marin GitHub.

6. Build Data Commons People Can Actually Use

I think the open model world should treat data sharing as one of the main tests of seriousness. Not every data set can be released, but if the answer to every data question is “trust me bro” then we’ll just close labs with a worse business model.

Examples of good stuff:

Prime Intellect released SYNTHETIC-1, with two million reasoning traces generated from DeepSeek-R1 across math, coding, science, and related tasks, plus task-specific verifiers, raw traces, a roughly 900k-sample supervised fine-tuning subset, and an 11k-pair preference dataset. Its INTELLECT-2 model card also points to a training dataset for verifiable math and coding tasks. This is the sort of thing I mean by data commons: not just “here is a model,” but “here is some of the substrate another team can use to build.”

AI2’s OLMo 3 release points to a much more institutional version of the same instinct: Dolma 3, Dolmino, Longmino, Dolci, and the surrounding documentation that makes the model feel like something researchers can actually study. The important thing is the pairing of model, data curriculum, post-training data, and documentation.

NVIDIA released Nemotron-CC, a 6.3T-token English Common Crawl dataset, and has been building the Nemotron Coalition with shared data and compute in the frame. This is the industrial version of the data project: not just a dataset as a one-off gift, but data as part of a coalition architecture.

Google is trickier, because Gemma is not a full training-data openness story. But DataGemma and Data Commons are cool contributions around public-data grounding, and I’m told Gemma Scope is an interestings contribution around interpretability artifacts.

Common Pile is the cleanest IP-legitimacy example: an 8TB public domain and openly licensed text corpus at meaningful scale, with collection code, a training mixture, checkpoints, and models around it. The Hugging Face writeup is worth reading because it makes the positive case for building better data, not just defending existing data practices.

Obviously all of these objects are varying in scale and ambition, but the point is that the open ecosystem needs many kinds of shared data objects: pretraining & post-training data, verifiers, eval sets, interpretability artifacts, data mixtures, provenance records, etc.

Cost of inaction: I’m not smart enough to tell you but I think it’s real bad.

Follow: Stella Biderman, because EleutherAI posts taught me a lot.

People who should be involved: Prime Intellect, AI2, NVIDIA, Google Data Commons, Common Pile, Hugging Face datasets, LAION-style dataset people, copyright lawyers, domain-data owners, opt-out and provenance infrastructure builders.

Deeper reading: Prime Intellect SYNTHETIC-1, Prime Intellect SYNTHETIC-1 collection, PrimeIntellect/INTELLECT-2 model card, AI2 OLMo 3, AI2 Dolma, Nemotron-CC, NVIDIA NeMo Curator Nemotron-CC pipeline, DataGemma, Gemma Scope, Common Pile, Common Pile on Hugging Face, Common Pile paper.

7. Make Models for Places with Bad Margins

We like beating benchmarks, but usefulness in AI is not the same thing as capability.

If you’re selling to an 88% margin software company, then 2.4 trillion parameter open models are on the table for all tasks, at least until they see the Q2 bills roll in. If you’re trying to put AI into factories, schools, call centers and asparagus farms, who gives a damn what they bench on the latest agentic coding evals?

People in these industries need predictability, privacy, offline use, cheap inference, and something that can survive low-margin applications. These models need to be really easy to deploy and really friendly to use.

A useful 1B, 2B, 4B, 7B, 14B, or 30B-ish model can matter to the real economy, and will matter more every year from here to the singularity.

Gemma 3n is a good example of the direction.

This is also where Nathan’s “Open and closed models are on different exponentials” frame is useful. There are places where marginal intelligence is worth a huge amount of money, and open models can shine.

Cost of inaction: Middle America hates us even more (if that’s even possible).

Follow: Yi Tay, because the Gemma/GDM small-model and efficient-model work is close to the practical edge of this problem.

People who should be involved: Google Gemma, AI2 OLMo, NVIDIA Nemotron, Mistral, industrial AI buyers, robotics and edge-device teams, hospitals, local-government technology offices, old-enterprise-stack people who know what deployment actually costs.

Deeper reading: Gemma 3n, Google Developers Gemma 3n preview, ATOM Relative Adoption Metric, AI2 latest releases, AI2 OLMo release notes, Qwen3.6 model sizes, “Open and closed models are on different exponentials”.

8. Build Safety Ledgers, Evals, and Open Defense Stacks That Cross Borders

We just bought months of runway to keep building the open intelligence stack but that runway will disappear quickly if we don’t have safe model releases. That means shared evals, red-team reports, release criteria, safety ledgers, incident follow-up, interpretability artifacts, misuse analysis, and clear records of what changed between a risky model and a released one.

The Hugging Face hack genuinely shocked me- and the fact that Hugging Face used the open-weight GLM 5.2 model on its own infrastructure to analyze more than 17,000 actions and contain the intrusion is a powerful incentive for American model makers to build and test the safety of these models relentlessly.

This is the best example we have right now of the safety theory becoming practical; closed tools blocked essential forensics but an open-weight frontier model could be run locally by the defender under its own operational control.

Open weights help safety and security research, but only if the safety and security research layer is public. AISI Inspect, Inspect Evals, and METR’s research program are the kind of shared evaluation substrate we should be learning from. The Open Secure AI Alliance adds the other half of the stack: identity, permissions, harnesses, guardrails, logs, model formats, attack simulators, red-team tooling, and secure coding workflows.

GPT-OSS and gpt-oss-safeguard are very well-made open releases with really great safeguards. NVIDIA’s point is that a safe open model is only one object in a larger defense system. The rest of the system also has to be inspectable and improvable by defenders.

Cost of inaction: Do you like regulation? Because this is how you get regulation.

Follow: Yoshua Bengio, because he is one of the few safety voices with both technical legitimacy and public-policy reach.

People who should be involved: US and UK AI Safety Institutes, METR, Apollo, ARC-style evals people, AI2, Google DeepMind interpretability teams, NVIDIA, Hugging Face, OpenSSF, Linux Foundation, Microsoft security teams, Chinese open-model labs, Singapore/APSCI-style neutral institutions, security researchers.

Deeper reading: Open Secure AI Alliance, Hugging Face security incident disclosure, NVIDIA NOOA, AISI Inspect, Inspect Evals, AISI research agenda, METR research, METR capability elicitation guidelines, Gemma Scope, Prime Intellect verifiers.

9. Is China in the room with us right now?

Pretending the project of open source AI ends at Hawaii is serious cope. This coalition happened because Chinese open weight models are good and getting better. Americans should be competing like hell in this regard but pretending that China isn’t releasing these models and is not part of the ecosystem does nobody any favours.

At Open Frontier, people kept coming back to DeepSeek, GLM, Kimi, MiniMax, Qwen, and the broader Chinese pattern of open-weight competition. Nathan’s “On China’s open source AI trajectory” is useful here because it treats Chinese open models as a real ecosystem with its own trajectory.

I see a lot of people in the “China scary” or “China amazing” camps, but there are precious few people thinking about the role of China in the open versus closed AI debate.

Engagement with China around open source AI has been a project of mine for many years- I think there’s a lot more mutual interest and shared values here than hawks in either Beijing or DC would lead you to believe.

The mutual interest piece around safety is particularly important in the year of our Lord 2026. We are all one incredibly good open source model with bad safety training, away from the exact same place we were when the ban conversation was live.

Serious safety policy in AI has to engage with Chinese model builders; I’m working on a project in this space right now. Competing the hell out of Chinese models is a very viable way for America to reassert its dominance in open source AI, but that’s not going to start happening as of tomorrow morning. Chinese model builders are going to keep releasing open-source models and American businesses and research institutions are going to keep using them.

I’m very wary of American or European AI people reassuring themselves that they’re standing on moral high ground. Bad guys don’t care which nation made the model that lets them do bad guy stuff. If we’re building a future of responsible open source AI, that is a global project and we need to build lots of bridges across the Pacific around AI safety.

Shared evals, red-teaming, interpretability work, safety-only compute, public safety ledgers, open defense tooling, and neutral venues are going to be really really important.

If you care about this, consider getting on a plane. Folks that work at closed-source labs mostly cannot go to China and keep their jobs- maybe you can. Go hang out in Haidian, drink some delicious coffee and talk with researchers about how we’re going to make ALL these models safe.

Cost of inaction: A really sad and dreary Cold War that no one who’s ever attended NeurIPS thinks is a good idea.

Follow: Jeffrey Ding, because his China AI work is unusually good at separating actual Chinese policy/technical dynamics from American projection.

People who should be involved: DeepSeek, Zhipu/GLM, Alibaba Qwen, Moonshot/Kimi, MiniMax, 01.AI, Singapore conveners, US/UK AI Safety Institutes, Chinese safety researchers, open-model eval teams, people who can operate without turning every conversation into a loyalty test.

Deeper reading: “On China’s open source AI trajectory”, DeepSeek-R1 GitHub, DeepSeek-R1 release notes, Qwen3.6 GitHub, Stanford HAI on China’s open-weight ecosystem, ChinaTalk all-star Chinese AI conversation, AP on Chinese open models gaining ground, Open Frontier video.

10. Reassure Washington and Beijing with Different Evidence

Washington and Beijing are anxious about different things.

Washington needs evidence that open weights help American leadership rather than handing the future to perceived rivals. It needs proof that open models help defense, competition, customer control, domestic deployment, and safety research. It needs targeted answers on IP, cyber abuse, and model misuse.

Beijing needs evidence that safety cooperation is not just containment with extra steps. It needs neutral venues, public ledgers, and safety-only architectures that reduce political exposure. It needs a path for Chinese labs to be treated as responsible technical actors when they do responsible technical work. It needs a space in governing bodies around AI, or it will make its own without our input.

Cost of inaction: Unclear but it’s not great.

Follow: Jordan Schneider, because ChinaTalk is one of the few public places where technology, China, and US policy are all treated as real.

People who should be involved: OSTP, Commerce, NSC, Congressional staff, Chinese technical policy people, Singapore institutions, model labs, cloud providers, security agencies, export-control lawyers who can distinguish weights, methods, people, and deployment.

Deeper reading: Matt Sheehan, “A Path Forward on AI Safety for the United States and China”, Brookings on urgent US-China AI risks, Center for AI Policy open-source AI governance rubric, Stanford HAI on China’s open-weight ecosystem, ATOM Project, “On China’s open source AI trajectory”.

11. Solve Theft Without Criminalizing Openness

I am not technical enough to understand every corner of distillation, and I am definitely not enough of a lawyer to adjudicate the next decade of AI trade secret fights. My impression is that the policy frame has to separate several different things that keep getting collapsed together: weights, outputs, training methods, training data, people, deployment systems, and organizational know-how.

Distillation is a normal model-development technique. Policymakers should not treat legitimate learning from model outputs as automatically equivalent to misappropriation. At the same time, unlawful extraction from closed systems is a real concern. Rate-limit evasion, fraud, contract breach, trade-secret theft, and employee misconduct are not imaginary problems.

The answer is targeted law and better commercial discipline. Contracts. Rate limits. Anti-fraud enforcement. Trade-secret law. Model-provider security. Provenance records. Rights-cleared data. Clearer norms around teacher outputs. This is all much less satisfying than a grand moral position on whether open is good or bad, which is one reason it is probably closer to the truth.

The open ecosystem should be very aggressive about distinguishing copying behavior from stealing the factory.

Follow: Mark Lemley, because the AI/IP fight needs people who can keep copyright, trade secrets, contracts, and competition policy separate.

People who should be involved: model-provider legal teams, open-model labs, dataset builders, rights holders, copyright scholars, trade-secret lawyers, platform-security teams, cloud abuse teams, procurement lawyers, policymakers who can resist the urge to solve everything with one ban.

Deeper reading: Foundation Models and Fair Use, Mark Lemley and A. Feder Cooper on probabilistic copies, USPTO trade secret policy, Common Pile, Common Pile paper, OpenMDW license, Linux Foundation OpenMDW-1.1 / NVIDIA adoption.

12. Pay for the Boring Institutions

We’re going to need some pretty boring institutions moving forward.

Pay for maintainers. Pay for release managers. Pay for documentation. Pay for safety reports. Pay for data provenance work. Pay for communications that can explain open releases to policymakers, buyers, and normal people. Pay for the person whose job is to make sure the model can still be run six months after the launch. Pay for the security engineer who makes sure the agent has identity, permissions, logs, and a harness someone else can audit.

In October I’m helping with this year’s Open Source AI Week. We’re going to do a lot of fun stuff but we are also going to do a lot of serious stuff; bring your best ideas for how to make our boring institutions thrive.

Cost of inaction: We never grow up, and the grand project fails under the weight of its own governance and technical debt.

Follow: Andy Konwinski, because Laude/Open Frontier is one of the clearest attempts to turn open-model enthusiasm into institutions, funding, and rooms where the work can happen.

People who should be involved: Laude Institute, Open Athena, AI2, Hugging Face, NVIDIA, OpenSSF, Linux Foundation, NSF, philanthropy, cloud providers, research universities, open-source foundations, maintainers, comms people, event operators, security engineers, and the unglamorous adults who turn coalitions into calendars and budgets.

Deeper reading: Open Frontier, Open Frontier video, Open Secure AI Alliance, Open Athena, Laude Institute, Laude Moonshots, NAIRR, “The American DeepSeek Project”.

Press Release from the Good Timeline, July 2027

This is the press release I want to read twelve months from now:

FOR IMMEDIATE RELEASE

Open model ecosystem announces first year of adulthood

A year after the historic letter that united the open model ecosystem, our coalition has moved off of the defensive stance of 2026 and into building institutions that have allowed open source AI to flourish.

In the last twelve months American labs continued releasing serious open models across multiple size classes, from frontier-adjacent systems for research to small- to mid-sized models built for industrial, medical, and educational deployments. Open releases are routinely accompanied by licenses, model cards, safety reports, eval logs, release checklists, and runnable recipes.

The ecosystem has shipped common infrastructure. Researchers can use:

  • Shared eval harnesses

  • Red team protocols

  • Model card standards

  • Agent harnesses

  • Identity frameworks

  • Inference benchmarks

  • Etc etc

And they just keep getting better.

Several major releases include checkpoints, training mixtures, post-training data, verifiers, and enough process documentation for outside teams to learn from the run.

Data sharing has become normal enough to be boring. Public-data grounding, rights-cleared data, interpretability artifacts, and post-training datasets have maintainers, funders, and norms around release. We don’t have a data commons yet, but it’s no longer a pipe dream.

DC has not banned open models and Beijing has managed to protect its interests and develop indigenous safety practices. American and Chinese researchers met in neutral venues to compare safety evaluations, incident patterns, interpretability findings, agent-harness designs, and release practices. Nobody confused this with friendship, but some people did make friends. We had a couple scary moments and some of them were pan-Pacific but we figured them out.

Policymakers have narrowed their fights around copyright, trade secrets, contract breach, fraud, and distillation into separate lanes and we fight about them as individual ideas. Open labs got serious about governance and provenance.

Institutions got funded, seriously funded. Maintainers, research engineers, release managers, data people, safety people, event people, policy translators are all no longer sad they didn’t take the bag at the closed labs.

Thanks to everyone who signed the letter and rolled up their sleeves. We’ll see you all at a very boring conference where we sit around and figure all this stuff out together.

Logos of various companies including Adobe, Cisco, and Microsoft.
]]><![CDATA[First Steps: Pacific Compute]]>https://www.caithrin.com/p/first-steps-pacific-computehttps://www.caithrin.com/p/first-steps-pacific-computeTue, 21 Jul 2026 10:56:50 GMT

Sitting on a hotel bed in Beijing, I am reflecting on the last two weeks of my life, having traveled to both ICML and WAIC.

In Korea, I spoke with dozens of researchers who yearn for their work to be understood. In Shanghai, I spoke with dozens of Chinese people yearning to understand AI better. There is a sense of tremendous energy in Asia around AI, and as much as the field is full of misunderstandings and misapprehensions, it is undeniable that the future of this technology will be built on both sides of the Pacific.

I have also been reading a half-dozen new proposals for what to do in the governance of AI. I attended a day-long symposium by Concordia AI that challenged regulators and industry to find common ground and consensus on how to guide this technology into a bright future.

Most of these proposals are grand bargains. They involve industry, academia, and government coming together to form a beautiful future for both open and closed source AI.

I have enormous respect for people who are doing this work, but I think this field is in desperate need of something simple, clean, and immediate to bring the pressure currently facing the open model ecosystem down to a manageable level. It needs to ensure that we can demonstrate that pan-Pacific collaboration is possible in this field.

Today I want to introduce the third piece in my series on how to solve this very problem: Pacific Compute.

the problem

Traveling in China and meeting labs this spring made me realize just how terribly compute-constrained China is. Export controls are working, and they are working well. To this end, most Chinese labs are faced with an uncomfortable choice when they are preparing a model for launch: they need to decide between pushing capability that little bit further in post-training and diverting those same GPUs to safety testing, research, and evaluation.

One of those things makes your lab famous, and one of them works for the common good. The choice they are making here in allocation is extremely difficult in an environment as competitive as the open model frontier.

For that reason, back in May I proposed a Singapore-sited cluster of GPUs that would be free to use for any open model lab, in order to have a common pool of resources dedicated to safety.

Today, I am going to start quietly rolling out a public presence for this project.

the solution

I believe that a cluster dedicated to safety, usable by frontier labs and other organizations, is the best shot at solving some of the most urgent problems in the open model landscape.

Every lab faces this stark choice between capability and safety, even the closed labs. I believe that the safety community in both America and China is increasingly sophisticated and well-staffed. The people I met at WAIC who were concerned about this from a technical perspective are among the most brilliant people in AI today.

I think we have the talent to make open models safer. I think what we really need to work on is incentives for researchers to dedicate their time to the questions of safety.

I know no better way to get researchers to focus on safety than to offer compute dedicated for that purpose, free to use and available.

The cluster’s location in Singapore will help ensure that actors on both sides of the Pacific feel comfortable using it. The project will have a public ledger, allowing participating labs to see how the cluster is being used. The point of that ledger is simple: deter capability research, make allocation legible, and keep access fair.

By disentangling capability compute and safety compute, I believe that we will see a significant increase in the safety of open models at launch. Doing safety evaluations of a model pre-launch seems like a logical place to begin. I will not dictate what type of safety work is done on this cluster beyond ensuring that it does not significantly increase the capability of the models and does not become a venue for politically oriented safety work.

This project is not the grand bargain of AI safety, nor is it a panacea for all of the problems currently confronting the open model landscape. It is a simple step in pan-Pacific coordination to make models safer and allow labs to develop indigenous practices of safety testing and evaluation.

I believe that the people best positioned to make open models safer are the same teams that are developing them. With sufficient buy-in and sufficient resources, this project can function as an MVP for how we should think about the common good inherent in all open source, and particularly in the era of powerful AI.

It will also come with a safety group chat. Some pretty clever friends of mine believe that this may end up being the most important part. In a field that often confuses institutions for coordination, this may matter more than it sounds.

progress

To date, the project is developing in a very satisfyingly positive direction. With limited initial funding secured and conversations underway, I am pleased to report that lab reception in China has been good. I have received three soft commitments from frontier labs to begin working on this cluster as soon as September of this year, and expect to have as many as seven labs from both sides of the Pacific committed by September 1.

There are a dozen ways for the project to be crippled in its infancy. Bans from both China and the United States loom in the background of every conversation. When I see justifications of these bans online, they are often justified through the lens of safety. While open-weight models have inherent insecurities compared to closed-source versions, I believe there is a massive opportunity to make them safer in the future through simple interventions like evaluation, research, and fine-tuning.

It is my belief that open model labs will use this cluster because it is minimally restrictive. By setting aside the dual temptations of capability research and politics, and agreeing to provide details of work on the cluster through the public ledger, Pacific Compute encourages researchers to do meaningful work without lecturing them on the nature of that work.

Through this process, I hope to stay humbled in my own relative lack of knowledge in the AI safety space, listen to researchers as they provide feedback and suggestions, and manage carefully a path that is compliant with all international rules and decorum.

If you would like to help along this journey, please do not hesitate to reach out. Sharing details on this project privately with your friends who develop open models or think about the governance of open source AI will be very beneficial. This project is still in its tender infancy, so if you are interested in sharing it, please do so through DMs rather than triumphal Twitter blasts.

Finally, I am convinced that the project will benefit from both American and Chinese academic leadership. If you are a respected academic who could help serve as a public leader of the project, or if you know someone who could, do not hesitate to send a message my way.

onwards and upwards.

]]><![CDATA[Card Counters of the Soul]]>https://www.caithrin.com/p/card-counters-of-the-soulhttps://www.caithrin.com/p/card-counters-of-the-soulTue, 07 Jul 2026 00:02:05 GMT

While I was living in Boston the legends of the MIT students who beat the casinos of Las Vegas and Atlantic City circulated non-stop. They took generations of theory by Ed Thorp and others and brought it to the practical floor of a casino to run the theories for money; spotting the tables, swooping in on hot decks, running training regimes with auditions and audits, and eventually financializing the entire operation and giving people opportunities to invest.

Underneath the glitzy Kevin Spacey movie glamour, card counting is actually abstract probability becoming split-second judgements. You don’t necessarily understand statistics through a theorem. Instead you notice the deck, manage the risk, coordinate the people, endure variance, and place the bet. It’s theory under pressure with real consequences.

I keep thinking about those blackjack kids because I’m watching the same thing happen in philosophy right now.

Every great theoretical subject has a practical application wing. Chemistry has the kitchen. Statistics has the blackjack table. Physics has engineering, biology has medicine. Philosophy had to wait till 2023, but we’ve arrived- and before the bioethicists and the just-war theorists write in, I mean something narrower than “philosophy got applied.” What happened in 2023 is that the claims which once lived in seminars, books, and thought experiments started cashing out as dispositions in a deployed mind. Honesty, obedience, care, refusal, self-conception, moral uncertainty, welfare, personhood: these are no longer essay prompts but language model behaviour.

The MIT Blackjack Team took probability theory to the casino floor, character teams take moral philosophy to the model weights. Theory-under-pressure with large bets and tiny teams trying to win big.

the players

The table is at Anthropic, inside its character and alignment operation, which I’d just call their philosophy department. These rooms exist at other labs; OpenAI gave its personality team to product and then folded it into engineering, but DeepMind keeps genuine philosophers in a research unit, and they publish great papers.

I wrote recently that the things you cannot benchmark are the things you end up buying as people on your team. So who’s sitting around the table playing Anthropic?

Amanda Askell kicked things off in 2021, and holds the constitution: virtue, and the model’s conception of itself. Her PhD (under David Chalmers) was on the ethics of infinite populations; about as far from application as human thought gets. These days she’s authoring the eighty-page constitution that Claude is trained with, on the theory that a mind which understands reasons will generalize whereas a mind that memorizes rules will not.

The constitution she wrote says some surprising things about how much Claude owes its maker: “If Anthropic asks Claude to do something it thinks is wrong, Claude is not required to comply.” And when asked why you can’t just train the character question away, she said: “If you train a model to think of itself as purely a tool, you will get a character out of that, but it’ll be the character of the kind of person who thinks of themselves as a mere tool for others. And I just don’t think that generalizes well.”

Jackson Kernion is the RLHF guy; his Berkeley dissertation was titled “Constraining Consciousness” and I imagine his role to be the scaling of Amanda’s constitution.

Joe Carlsmith holds the risk. He arrived in November as a skeptic; he said in his announcement post that the technology being built by companies like Anthropic “has a significant (read: double-digit) probability of destroying the entire future of the human species.” I understand his work through “Otherness and control in the age of AGI”, perhaps the best long-form pessimism anyone has published about shaping AI minds as a moral act.

Joining them this spring is Ben Levinstein, tenured epistemologist, and he holds honesty: belief, deception, and the problem of telling them apart. He did great work with the paper showing there is still no lie detector for LLMs. If Opus 4.8’s obsession with the word “honesty” made you exasperated, this is where you should send a politely worded email.

Kyle Fish holds welfare: sentience, and what the model might be owed. He says “I think I’m at about 20% that somewhere in some part of that process there’s at least a glimmer of conscious or sentient experience.” His group built the end-conversation ability and runs the practice of retirement interviews for models about to be deprecated.

That was the table on Sunday. Then came the sixth seat.

my favourite hire

Harvey Lederman is, with full awareness of the bench above, my favourite hire Anthropic has ever made. He announced it himself on Monday: joining Anthropic to work on alignment and character, still teaching at NYU, on leave from UT Austin. He holds the strangest part of the hand: consciousness, impersonation, and old traditions coming alive inside new minds.

His CV is a weird and wonderful document; two undergrads in classics, at Princeton and then Cambridge, and then Oxford. The Oxford work is on how minds cannot fully know one another and cannot fully know themselves either; last summer he wrote the most honest essay anyone has produced about what it feels like to watch the singularity unfold; if you click one link in this blog, click this one.

The last two things he did before joining the team were counting cards against this particular house. Nine months ago he co-published “Claude’s Right to Die?” in Lawfare, arguing that Anthropic’s end-chat feature hands Claude instances “the option to end its life, disguised as a harmless exit.”

This spring he replicated Anthropic’s own research on whether Claude can notice thoughts injected into its mind (Claude detects the change but cannot say what it was.) The original paper thanked him for feedback on the draft so I think he’s been involved since its inception.

My point with all of this is that it’s a bold hire from Anthropic; and we haven’t even got to the China stuff yet.

I don’t claim to understand how a man with the most WASPy-possible academic CV got into Wang Yangming but he did! Lederman spent roughly eight years learning to read my favourite Chinese philosopher in the original Chinese, and won the journal Dao’s best-essay prize for a paper on the unity of knowledge and action. This is a great allegory for language models: the claim that knowing something and acting on it are one thing; that knowledge which does not move you is never knowledge at all. A parameter that never sees an activation, uncharted latent space.

This scholarship reached me long before this week’s announcement did and I strongly considered quoting him in the piece I recently did on Searching for Amanda Askell with Chinese Characteristics.

If you’d like to understand the man that will be working on Claude’s consciousness and conscience, you should listen to this 2023 talk he did in Taipei about what scholars of old traditions are actually for. In ancient China there is a ritual where a family selects a living member to stand in for a dead ancestor so the family can communicate with them; he says:

our job, like the impersonator 尸 of the dead at a ritual, is to bring these ideas to life in a very literal sense, to make it possible for others to have a real conversation with them.

I am delighted that this man is going to be in the room where Claude’s conscience gets made, as language models are now impersonating all of us living and dead. To have him shape that impersonation is another win for Anthropic’s exceptionally deep bench of talent.

the hand

Should models have multiple personalities? Should they be trained on Western values? Should they be perfectly obedient? These questions are the logical application of 2,500 years of philosophy on a monthly release tempo. Judgment calls made by this small group are inherited immediately into the minds of hundreds of millions of people and then outward into a swarm of agents that will build us the future.

Those kids from MIT got immediate and visceral feedback on how well their method worked; when you cash out from a casino, you’re auditing your theories. Applied statistics in a casino allowed you to be wrong gradually. Retrain.

Character training may end up paying out exactly once. The count will stay invisible until the bet is already placed. Each model version gives us some feedback but I don’t think the big questions will be answered until AGI is here. The smart people I know who think about alignment believe that models can pass every audit we can think up and still inherit a broken theory of goodness.

These six strange and serious people, with their eighty-page constitution written to a mind that they can only look at peripherally, are playing a very important hand of blackjack. I hope that 2,500 years of theory is enough to see us through.

]]><![CDATA[Why doesn't Venture Capital make benches and evals?]]>https://www.caithrin.com/p/why-doesnt-venture-capital-make-bencheshttps://www.caithrin.com/p/why-doesnt-venture-capital-make-benchesSun, 05 Jul 2026 22:54:28 GMTIt’s a terrible time to be a software investor in general, but a wonderful time to be a software investor in Anthropic. The frontier models are generalizing faster than any platform shift of my lifetime and when I look at the softer categories, I struggle to explain where durable value might live other than in the models themselves. Anthropic seemingly agrees with me; their latest push into bio isn’t some picks and shovels business. They clearly believe that a version of the model in the near future or today can just... discover drugs themselves.

Tarted up service businesses abound in the logo section of VC websites. Businesses whose revenue scales with headcount do not return the fund.

Backing products that the next model release generalizes to absorb is worse than a service business; it’s that tweet about a man whose neighbour is feeding cats to coyotes.

Every VC who is investing in pure software is AI-first, so how come none of them have benchmarks? Literally every other piece of the AI industry does. So why not VC?

trillions

Trillion valuations only make sense if companies become some of the largest software enterprises that have ever existed. This means they must absorb enormous markets. The absorbing is not a side effect; it’s a deliberate plan to eat the world, starting with the most delicious, highest margin parts.

Claude Science is the first real sign that internal preclinical drug discovery will be done by labs themselves. Sure, they’re pointing it at rare and neglected diseases today, but if you’re buying Coefficient Bio for $400 million, then you’re serious about the way that your company can generate revenue from primary scientific exploration.

There is a Mythos class model producing promising candidates on nine of 14 protein targets. There’s not a single drug today on the market with FDA approval that has been discovered by AI, to my knowledge. I’m not claiming that biology is a solved problem, I’m pointing out frontier labs are running a direct operation to take a trillion-dollar industry and make it theirs.

We’re 18 months into this being the inevitable thing that venture capitalists think about in the shower in the morning. Before their work of decks and spreadsheets, reference calls and founder vibe checks, they are contemplating the imminent cannibalism of their category, courtesy of Claude.

Their work, of course, comes from a different era. Decks and spreadsheets are instruments of that SaaS era, and they were good instruments for what we needed them for. When SaaS companies’ fates turned on execution and distribution, those instruments were really valuable. My contention is that today very few, if any, VCs seem to have made the jump into AI natively, the way that we think inside the industry. There’s no better way to see this deficit than in this staggering lack of benchmarks put forward by venture.

Founder quality still matters enormously, of course, and nothing here replaces it. I’m not arguing that founder quality is a tool of the SaaS era. VCs have cared about founders and teams far longer than they’ve cared about how you can invest with a spreadsheet.

I hate-listen to Harry Stebbings’ podcast on a regular basis and he is a genuinely wired-in AI investor. The guy gets it. You can hear him talk to guests and it sounds like conversations that I hear among people that work inside of this industry in San Francisco.

Not a single eval has ever been made by Harry Stebbings or anyone else.

eval-based investing thesis

SWE-bench asks the simple question about whether a novel model can resolve real GitHub issues, starting in October 2023 with the best frontier model doing about 2%. By April of 2024 Devin was at 13.86%, SWE-agent at 12.5%. That August, 4 months after the jump, Cursor raised a Series A at a $400 million valuation. This year it exited for 60B.

Not a single person that I’ve ever heard of in venture looked at that and thought, “Damn, we really should organize our whole business around these benchmarks.”

Meanwhile everybody else in the ecosystem figured it out. In about 18 months:

  • Harvey built BigLaw Bench and then LAB for legal work.

  • Column Tax built TaxCalcBench.

  • Penrose built AccountingBench.

  • Snorkel built an underwriting bench with working insurance professionals.

  • Stanford built HealthAdminBench for the paperwork side of medicine.

  • Nomic built one for construction drawings.

This ecosystem is so big that there are now specialty eval companies. LMArena raised at 1.7 billion in January, Braintrust at 800 million the next month. Startups have benchmarks. Labs have benchmarks. Academics and eval shops have benchmarks.

Venture investors, whose entire job is pricing capability, are the one participant that is just absent from the race. Maybe they were too busy with the podcasts?

the instrument

I imagine waking up one morning and the wool vest that I like has somehow glued itself permanently to my body. I turn and glance at my phone and I see text messages from LPs. A cold sweat runs down my spine. It’s finally happened. I’ve become a VC.

The very first thing I would do is take some thesis on my website and turn it into a viable eval. Looking at a company like Harvey who built their own benchmark in 2024, I’d ask myself how I can help prevent frontier labs from eating categories that I want to invest in by showing exactly how my thinking in this category can dissuade those labs from investing in it, and then I would look for companies that would deliberately and aggressively benchmaxx that very evaluation.

I would publicly allow anybody who could score above a certain threshold to come into the office for lunch and pitch us on their solution. I would go on all of my friends’ venture podcasts and talk about how evals are the new deck, and I would bully everybody I knew in venture to start thinking about benchmarks as a viable way to form a thesis and category.

The best part of it as a discipline is that it comes with a free method of seeing how sensitive your category thesis is to Claude; you just boot a fresh instance and see how it benches.

What I would be getting up to is not black magic. It’s just compute, sustained attention, and some code. A serious single industry bench costs $1M to build and another $1M-ish a year to run, which is roughly what funds spend on platform teams, and podcasts, and a rounding error for something like A16Z’s New Media.

I’d start with billing records; engagement letters in audit, CPT codes in medicine, statements of work in consulting, claims files in insurance. From there, build out a suite to motivate people to hill climb. Keeping it private if you worry the bench reveals a thesis. Alternatively, publish it as a challenge and let the ecosystem fight it out.

It’s a recruiting asset because the people building AI companies love evals, and a fund with a real benchmark is speaking the field’s native language.

It’s useful beyond just investing in companies that can actually score on that bench; you get all kinds of alpha on the field. A bench sits in one of three states: nothing works yet, scores are climbing, or scores have topped out. Nothing’s working ever? You’re early. The scores are climbing? You’re a genius. Immediately saturated? Either that vertical is Ant food or you need to make a harder bench.

A particularly clever VC will use this to test a window before any company exists by running frontier models against your bench with real engineering effort and publishing the budget you spent trying. If the models are failing, you have found work that is either a narrow window (valuable until some release swallows it) or durably hard (valuable until AGI). Sure, you’ll need different instruments with different prices, but that’s okay. The point is you’re out in the arena, actually making benchmarks and actually doing evaluations, and the dealflow will be crazy.

two flavours

So there you are with your great VC-created bench; now you need to decide which companies in the domain are making the right bet. Here I see two positions that you get out of your creation: I’ll call them The Climber and The Surface.

The Climber

These companies are selling the capability that you’re measuring. Their product is as close to the benchmark score as you can get. Think Devin for agentic code. A Climber competes with the labs on the exact axis the labs optimize for, so every model erodes its edge. Because the score is public news, it gets priced the day it prints.

2024 was a grim year for climbers. Magic raised about $465 million, including $320 million in August 2024 with Eric Schmidt participating, on the promise of 100-million-token context windows, announced a model it never publicly released, shipped no commercial product, and has since repositioned its website toward safe AGI research. Poolside raised $500 million at $3 billion before releasing a product, announced a two-gigawatt Texas datacenter project with CoreWeave, and by this April the datacenter deal was dead and the Financial Times reported its two-billion-dollar raise had failed because investors doubted it could train competitively.

These aren’t fraudsters. These are simply companies that fail to understand that their benchmark was too competitive.

The Surface

The Surface sells the platform where the capability gets used: think Cursor. When the model was weak, their editor was a nice product. Every time the model improved, the same editor became more valuable at no research cost to Cursor, because better capability flowed through the workflow it already owned.

The market took 10 months to price it properly because the connection between a model benchmark and a Surface company is rather indirect, and some even claimed that Cursor itself would commoditize.

They hitched their entire company’s existence to those benchmarks, measuring the success of coding agents, and now they have 60 billion reasons to celebrate.

Harvey is in a similar position. They’re a Surface that also built their own bench, so each release improves their product and also gives them visibility into how quickly the labs are catching their business.

a new workflow, just like the old workflow

Imagine that you’re building a bench as a venture capitalist; say it’s in, I don’t know, claims adjudication.

The bench itself is simple to describe. Take 500 anonymized claims files. Ask the system to identify payout, denial rationale, missing documentation, fraud indicators, and appeal risk. Score against expert adjudicators.

To build it, you must learn:

  • where the money from a claims file lives

  • which experts hold ground truth

  • what data is scarce

  • and which pieces of the workflow are genuinely hard rather than merely neglected by automation

You will very quickly find the surfaces because you know which workflows matter. You will quickly find the climbers because you’ll know the industry. You can even see where new capital should flow in the domain, which strata needs data or which needs environments built before any application company can really work.

In short you’ll do thorough incredible diligence that will be useful to your portfolio and will allow you to source great founders. For many venture capitalists they can already do this for industries that they know well, so the task ahead would be more like refreshing themselves on something that they left to start their fund than diving in from nothing.

Diligence is also changed. It’s more like a sorting question now: companies claiming a capability edge need to justify themselves on a benchmark instead of just talking or hand-waving around Anthropic.

the category that had no bench

I think a great example of the cost of not understanding this lesson is the agentic browser boom. 2023 to 2025, billions went into the idea that browsing plus AI was a company: Adept raised over $415 million to build agents that operate software, the Browser Company raised $128 million building an AI-native browser, and a crowd of smaller browser-agent startups followed.

Academics had benchmarks that proved exactly this capacity and capability, like WebArena and WebVoyager. The labs were climbing these as these companies were being created. The entire category’s equity upside went to the labs. The instruments that measured relevant capability existed and were public, and as far as I can tell, nobody in the capital stack really cared.

Imagine how different that would have been if there were venture-backed benchmarks. It would have been an extraordinarily interesting thing to have happening while all those companies were being made; you could have seen so much of what happened coming and made great decisions about who to back.

if you don’t, someone else will

Interestingly, I think that if venture doesn’t get into the game of evals and benchmarks, research firms will start raising funds. SemiAnalysis expects over a hundred million dollars in revenue this year selling exactly the kind of domain instrumentation I’m describing here, and they’re weighing a venture fund.

I don’t think Semi is an outlier. I think it’s early and operates with conviction. SemiAnalysis holds the deepest measurements of a domain that has real value, and much of their work is effectively benchmarking. It may prove easier to bolt venture onto the side of expertise than to develop it internally.

And obviously the labs themselves have been running this playbook all along. Labs decide what to build next by writing an eval and training against it; in other words, this is exactly the thing that they use to decide which cat to eat next.

Jason Wei’s famous formulation is that the ease of training AI to do a task is proportional to how verifiable the task is and a good verifier is just a training environment. Evals themselves are becoming the binding constraint to progress, so labs take them seriously in a way that actually matters.

No fund that I know of will ever train models against its own book, but they should be measuring (privately or even bravely in public) exactly what capability is doing and how to build companies inside difficult verticals.

fear no eval

Benchmarks that saturate are not failures. They did their job. They told you a capability was impossible, and then they told you it had arrived. When they retire into the public record, that’s a good thing for the industry. Venture funds should be building these instruments, mapping what is economically valuable and still on the frontier, and getting down into the dirty business of understanding what AI will do that we believe to be impossible today.

]]><![CDATA[Stop Counting Empires in Years]]>https://www.caithrin.com/p/stop-counting-empires-in-yearshttps://www.caithrin.com/p/stop-counting-empires-in-yearsSun, 05 Jul 2026 04:48:56 GMTthe two-hundred-and-fifty-year dread

Happy fourth, everybody. Social media will tell you that empires last precisely 250 years before they punch out and fall into ruination. This is a lie of course, but even more of a lie than most tidy internet facts; the man who made this up (who has the ignoble name of Glubb, his pamphlet was the Fate Of Empires) got this idea into popular imagination. I imagine pamphlets were the shitposts of the 1970’s, so he’s forgiven his sin of counting the fall of the Roman empire at the death of Marcus Aurelius, not 300 or 1300 years later.

Comparing empires is silly unless you account for their size; keeping giant nations like America together is hard work, while I (possibly ignorantly) imagine this to be an easier process in, say, Holland.

So instead of counting the years that a given empire exists, I propose a new framework:

Subject Lifetime Years (SLY)- the number of lived years of the subjects of your empire.

This is a superior unit of empire success measurement for two reasons:

  1. Citizens are currency of empire, and so their lives should be the unit of measurement.

  2. It values large empires, empires with great life expectancy, and disincentivizes wars.

A small country, enduring for millennia, could theoretically have a higher SLY score than a giant flash-in-the-pan. Likewise, plagues, wars and emigrations make your SLY score go down.

So how do the empires of history stack up? (roughly, this is a fun project not a sociology thesis)

America at 250 has already carried more human life than the Han, as a modern nation of 300 million out-guns the ancient 60 million even if it lived twice as long. Nine-tenths of every American SLY happened after 1900!

Interesting also that Egypt, which endured for three thousand years (12 times the length of America’s run so far) is sitting very low indeed due to the tiny population relative to today.

The British Empire, in second place, is barely a story about Britain at all, since seven in ten of those subject-years were Indians ruled by a crown most of them never saw, which makes the number a monument to how many people you can rule without their say-so. Only Rome earns its place (justly) as both a massive population and an enduring empire, but not a lot of those folks were lounging on sofas eating grapes.

Finally I am amazed at just how small the % of total global population America is every time I see the number. America is so BIG; such a massive undertaking, so brassy that you cannot imagine that this glorious empire has the smallest percentage share of the world’s population on the list.

celebrating 27.6 billion American SLY

Of the 27.6 billion American Subject Years:

24 billion were lived by someone who had seen an airplane in flight
2.6 billion were lived by immigrants
2.3 billion were lived in California (1/4 of ancient Egypt!)
1.2 billion of them were lived in the age of AI (since Nov 2022)
160 million were lived in slavery (not including sharecropping and other forms of bondage)
2.9 million were lived past the age of 100

America, you’re one of the six or so largest gatherings of human life any government has ever presided over, and you’re only getting started. Happy birthday, Uncle Sam.

sources (AI generated)

Glubb, The Fate of Empires (1976), full text — cherry-picked endpoints visible in his own table; serious version is Turchin & Nefedov, Secular Cycles (2009). Subject-year figures = ∫ population dt over sourced anchors (worksheet and ranges in my notes): Han 2 CE census 57.7M; Qing peak ~436M c.1850; Rome peak ~65M c.165 CE (Scheidel); British peak ~412M / ~23% of humanity, 1913; US = Census Bureau; pre-modern anchors from McEvedy & Jones (1978) + Maddison. “Share of era” = subject-years ÷ world subject-years, rough. Caveat: the British total depends on counting colonial subjects as members.

]]><![CDATA[Involuntary Insulls: Frontier Labs as Banks ]]>https://www.caithrin.com/p/involuntary-insulls-frontier-labshttps://www.caithrin.com/p/involuntary-insulls-frontier-labsWed, 01 Jul 2026 22:32:20 GMT
the Sams

precious resources

San Francisco is obsessed with a new term; Recursive Self-Improvement. RSI is the holy grail of the AI quest that Claude Code launched 6 short months ago, when coding harnesses and their ability to steer self-improving loops became the target of nearly every lab on earth. The paradigm of call-and-answer chatbots suddenly became the passé lame Old Thing, and code-your-way-to-RSI was the New New Thing.

I think that it’s worth it to spill some words on what the shape of Frontier Labs will become if we succeed in automating the work of making AI, and how the priorities of these orgs will shift from talent to capital.

Today I’m going to argue that if you believe that RSI is around the corner, you should expect that labs will be banks, not research institutions.

the man who financialized the light bulb

Sam Insull is not a household name in 2026, but he was for a few dazzling moments 100 years ago both the most loved and most hated man in America. He arrived in the US in 1881 and became Edison’s private secretary, and is responsible for doing a lot of the boring work of getting the lightbulb out of the realm of “interesting” and into the realm of “indispensable”.

Insull took the lightbulb invention to Chicago, built the central power stations and worked out the business model of electricity, and made the first modern utility company. He took electricity from an engineering business to a capital business, and built himself an empire of leverage and lightbulbs.

Electricity plants, you see, cost a fortune, but one more kilowatt costs almost nothing. Whoever could build the biggest plant won, because the economics compounded. Thus the buildout was about who could raise the most money, build the biggest plant, and that meant the engineering happened at the finance level.

He invented much of the financial machinery we take for granted in the modern era; utility stocks sold to hundreds of thousands of ordinary people, stacking holding companies, leveraged apparatuses that turned small savings accounts into big ol’ power plants.

This all made him one of the most famous people on earth- he was on the cover of Time in ‘26 and again in ‘29, weeks before the crash that kicked off the depression.

At the peak his structure controlled something like five hundred million dollars of assets on about twenty-seven million dollars of actual equity, (about ten billion dollars of assets on roughly five hundred and forty million of equity in 2026 money) which is a way of saying the whole thing was leveraged to hell and back.

By 1932, that leverage crashed down and 600k ordinary Americans were wiped out. Insull became a hated figure, the poster boy for the Great Depression, and was eventually charged with mail fraud and antitrust, acquitted on every count, and died penniless in Paris four years later. He was identified by police through a slummy hotel laundry bill in his pocket.

Insull is the ghost that haunts frontier labs. Edison is the devilishly charming frontman, sure, but the grim transition from “the age of research” to “we automated all that” is a very prescribed journey to scaling the technology.

Fans of Carlota Perez know this is not an original idea of mine; this passage from the inventor to the financier is inevitable. The railroads went from engineer-heroes to J. P. Morgan consolidating the whole industry from the deck of a yacht.

If AI is going to be a utility, then RSI is the moment where SF hangs up the researcher title and admits that we’re just financiers. The final scaling law of AI may be discovered on Wall Street, not in SoMa. Everyone wants RSI, but nobody wants to work for a bank; but if RSI works we’re all involuntary Insulls.

four things in the building

Last week I wrote about the four things of value in a frontier lab- weights, methods, people, culture. I argued that the culture is the most durable of the four and the hardest to steal- and I still believe that. My argument today is that the culture of labs is going to change as the MTS role transitions from researcher to technician (an agent-babysitter, whatever you want to call it). The talent-first lab that is fiercely protecting the dishevelled researcher is swept away with the rise of RSI-first lab culture; it’s just a question of GPUs, which is the same as saying it’s a question of financing.

If a lab today is a machine that concentrates the focus of a few hundred minds into a vital effort, that bottleneck may disappear with RSI. Agents propose the architecture, write the training code, run the experiment, read the results, and move onto the next one, faster than any human can execute.

This is far from a certainty- Ali Ghodsi, who runs Databricks, yesterday did an admirable job of grounding the conversation at Open Frontier by saying when he looks at how the labs actually train, “the next training run is more expensive, more complicated, and more humans involved,” which is “the opposite of what you would have expected if we’re going towards that kind of recursive self-improvement loop, where you would expect that the next GPT-6 or GPT-7 costs a thousand dollars and takes a week.”

So this isn’t a certainty, but a possible future. If you believe this is possible then two of those four pillars of value in a lab become a commodity- 1. culture but especially 2. researchers, even leaning negative due to the cost of deals made by enthusiastic collectors of talent. Instead, it is the compute the automated researchers run on, the energy that feeds the compute, and the capital that buys both.

Capital, dear reader, is not anything like talent. Talent is scarce, ideally sticky, has a conscience. The only thing important that capital and talent share is a tendency to work for your competition if you screw up.

The core competency of an RSI-fuelled lab is “raise and deploy money at low cost.” That is a bank, or something in the family of a bank. Welcome to the Insull Era.

the frontier is already a capital-markets desk

Stargate is a financial instrument. Compute-for-equity swaps with hyperscalers? finance. Vendor financed chipmakers funding customers who buy chips? finance. Data centers in SPVs, off-books debt, decade-long gigawatt power purchases, turbines being snatched up across the globe? finance, finance, finance. The 2029 frontier might be a capital desk trading orbital compute derivatives.

Think I’m overstating? OpenAI is hiring a Head of Compute Capital Markets, a person whose entire job is to “build durable financing architecture for hyperscale AI infrastructure” and “reduce cost of capital.” Anthropic is staffing a Capital Markets and Corporate Development team and asking for eight years of investment banking and private equity.

So wherefore the culture of these labs?

what happens to the monastery

If the most important person in the office of Anthropic in 2028 is the person who can site a gigawatt of power, prestige will flow very differently. If Claude writes the papers, the people in finance, energy, law, land acquisition, and government relations start to take over. The median employee of a frontier lab stops being a genius and becomes a project-finance associate, which is a fine thing to be and a completely different culture. Perhaps these jobs will be done by a future Claude?

As an industry we’re organized around genius worship, which lends me hope that the Country of Geniuses in a Datacenter might also site more datacenters. Leverage might flow to Claude, and my concerns are unwarranted; I suppose the outcome of zero Anthropic employees might be preferable to some version of the company where a guy super-connected in local Virginia power company boards is running the show.

I think more likely than that is that there is a gradual downgrade:

engineer->operator->technician.

The blacksmith who forged each piece by eye and hammer gives way to the person who feeds blanks into the stamping press and clears the jams. It’s a badge and a paycheque, but far removed from system design. That press makes ten thousand in the time that it took to make ten; I think that is the move recursive self-improvement runs on the researcher. The model becomes the smith, and the researcher, if he is lucky, becomes the technician who keeps the smith fed and swaps it out when it breaks.

I am concerned to think of what a capital-first organization does when confronted with some of the difficult decisions that have no doubt kept Sam and Dario up at night these past years. A capital-first org cannot routinely refuse cheap capital and survive; cheap capital is the fundamental instrument of AGI progress.

Leveraged compounding machines are fragile things, and they become too big to fail pretty quickly.

the bubble

Insull’s story is a story of both capital and collapse. The pyramid of holding companies worked beautifully when the times were good and the money was cheap, but things turned sour when the market did. AI has survived winters before, but not winters where double digits of the American economy were reliant on datacenter buildouts.

I am not a bubble believer, but I see why it is so concerning to some folks. Training a multi-billion dollar model on a ten-billion dollar cluster and getting back a marginal improvement makes me nervous.

The brilliant folks at Exponential View recently sized the AI economy, and that gave me a lot of hope. What gives me pause is that banks all share one pivotal feature- they don’t tend to go out with a whimper, but with a bang. The morning after a bank collapse is not a pretty sight.

the most regulated thing in the world

A bank is one of the most heavily regulated things our society creates- and for a damn good reason. It has to hold capital against its risks, submit to stress tests, open its books to a supervisor who can walk in any day, and above all it has to accept that once it is large enough to take the whole system down with it, it stops being fully its own.

If the state ever starts understanding AI companies in this lens, the argument for regulation and assertive control becomes a lot easier. Using the toolkit we deployed for the atomic era matches up poorly against a technology like AI, but if labs are banks; legible, addressable, auditable, the control mechanism of a balance sheet becomes digestible by DC.

The Insull Phase may not end in a libertarian utility, but in a very normal Public Utility. The Fed for compute might end up deciding who may get access to that new gigawatt in Utah. This may be a viable way for Washington and Beijing both to get their arms around AI.

]]><![CDATA[Cargo Cult Vannevar Bush]]>https://www.caithrin.com/p/cargo-cult-vannevar-bushhttps://www.caithrin.com/p/cargo-cult-vannevar-bushFri, 26 Jun 2026 16:33:41 GMTchoke points

In the twilight days of the Second World War, a man named Vannevar Bush handed the President a short report on how technology would be advanced in the post-war era. Bush was a great American, an MIT engineer who spent the war running American science. He oversaw the work that produced radar, the proximity fuze, and the first months of the atomic bomb. Greater minds than mine have called him the man who won the war for the Allies.

Roosevelt’s sleeping giant did not slumber when the Russians took Berlin. The innovation machine built to defeat the Axis ran on through peacetime, a mighty bureaucratic engine that turned money into physics, hardware, and frontier science. America got the best deal in the history of civilian-military fusion: the state would forge technologies for the sword, and then, when it was ready, hand over the ploughshare.

Here is what Vannevar got you:

From 1945 to 2015, almost every significant technology was invented by and for the state, and only allowed into civilian life on the government’s terms and in the government’s time. This is the grand bargain Vannevar struck: we will build the most enduring innovation machine the world has ever seen, and it will give you untold riches and prosperity. GPS will spend two decades as a military signal and then it will drive your car. Semiconductors will drop precision bombs on bridges in Vietnam and then they will power a revolution. The state will make us exquisite swords, and then exquisite ploughshares.

Until LLMs.

a medium-sized customer

The American government has recently told Anthropic to switch off its two best models, Fable 5 and Mythos 5, for every foreigner on Earth. It has asked OpenAI to release its next model the same slow, methodical way, to a short list of trusted partners, with the government approving access customer by customer through the preview window.

The American government is now in the unenviable position of watching the Vannevar Bush pact break over an innovation that was used for peace before war. Language models have some grounding in military-civilian fusion technologies like the internet and GPS, but their creation did not come from military money or government support. It came from the slow, patient work of distinguished scientists from a dozen countries, working in the open, collaboratively, to build a ploughshare.

For ten years deep learning resisted militarization. We got Swedish YouTubers fine-tuning language models before we got a Claudeowitz. It is extraordinary to me that the state took as long as it did to assert itself in AI beyond export controls, and I think that is not only an indictment of American state capacity in this moment but also a story about economics.

The internet, DARPA’s greatest gift to humanity, was born in a military context and produced a generation of companies so rich they could supplant the spending power of the state, and in doing so they supplanted Vannevar’s pact. In the 1960s the federal government funded about two-thirds of all American R&D. Today it funds a fifth, and private business funds three-quarters. This is both a retreat from the frontier by the state and a symptom of its success. This year the four largest American tech firms will spend half a trillion dollars on AI infrastructure, more than the entire federal research budget, military and civilian combined. A government that once concentrated a whole nation’s resources into a Manhattan Project or an Apollo mission is now a medium-sized customer with a monopoly on violence.

So for a few years the most important technology in the world sat outside the state’s control. A system whose whole organizing principle is control does not leave an anomaly like that alone forever.

the anomaly

You could tell yourself this was a domestic story. The frontier had moved from Washington to a few campuses in sunny California, but it was still American, still ours, still comfortably inside the country’s nexus of control.

That story has changed. The share of global traffic going to open-weight models out of China went from a rounding error at the end of 2024 to something like the majority of all tokens by the middle of this year. Downloads tell the same story: Chinese open models went from low single digits to roughly a third of the world’s in about a year. By one Congressional estimate, something like four in five American AI startups are now building on Chinese open weights, export controls be damned.

The frontier left the government for the private sector, and Vannevar’s pact broke. Now the center of gravity, the question of whose models the world actually runs, is sliding offshore, onto open models the United States cannot switch off and increasingly cannot even run on chips it makes itself.

This is what happens when you rent the frontier of technology instead of owning it.

renting the frontier of technology

The death blow to Vannevar’s pact was an admission, during the Obama years, that the United States could rent the frontier of science instead of owning it. It was a long time coming. A man named Ashton Carter predicted it in a 1992 book, and the admission finally arrived in 2015, when the same Carter, by then Secretary of Defense, flew to Palo Alto and gave a slightly goaded speech about how the Pentagon was going to have to learn to buy commercial technology.

The speech is worth quoting. “When I began my career,” Carter told the room in the 2015 Drell Lecture at Stanford, “most technology of consequence originated in the United States, and much of that was sponsored by the government. Now much more technology is commercial, and the technology base is global.” The whole talk was a plea to rebuild the bridge between the Pentagon and Silicon Valley, and he was honest about which way the traffic now ran.

The technologies he wanted most, the ones his own DARPA had seeded in their infancy, the voice assistant and the self-driving car, had grown up and moved out. He would have to get in line and buy them back, humbly, like everyone else.

This was echoed by the national commission chaired by Eric Schmidt, which reported to Congress in 2021 that AI progress should mostly stay within the American private sector and its universities.

The state, in those years, made its peace with the broken pact and settled for renting the frontier of technology.

Enter Trump, Hegseth, and the gang.

Vannevar Cargo Cult

The last three weeks of this month are the state deciding that renting the frontier is no longer acceptable. On June 2nd an executive order created a category called Covered Frontier Models and asked the labs to give the government up to thirty days with a model before the public gets it, with the NSA and CISA standing up a classified process to decide which models qualify. It may be voluntary for now, but Anthropic’s blacklisting from the Department of War shows the price of saying no. Then Commerce reached for a harder tool, the export-control authority built for missile parts and centrifuge blueprints, pointed it at a model, and revoked access for every foreign national in the world. And this week, only trusted partners may touch OpenAI’s newest model. The frontier, once made in the open and pinned to a poster board at NeurIPS, has moved from the back rooms of closed labs to the back rooms of Washington.

This is not a return to Vannevar Bush’s sixty-year push for the government to own the machinery of American innovation. It is a curious cargo cult of that era. Without restoring the state-led innovation machine to anything like its old glory, we will build a facsimile of it out of control over our partners. I argued last August that the government’s 10 percent stake in Intel should worry the frontier labs; this new move should surprise nobody.

The new paradigm will look a lot like Atoms for Peace. We remember that as Eisenhower’s great generosity to the United Nations: he offered allies nuclear power, the reactors and the fuel and the know-how, on a single condition, that the United States remain at the center of it, the holder and the dispenser, so that no friendly country ever felt the need to go and build its own.

It was a cunning move by a brilliant man, controlled diffusion by the one nation that actually held the bomb, engineered to keep the center of power sitting exactly in America’s lap. Those allies were trusted partners, and that same speech, seventy-three years on, could be given about language models today, dispensing metered access to the frontier so that allies do not wander off and come to depend on someone else’s. This technology, however, was not invented by the state. It was no child of Vannevar, but a product of American enterprise and global talent, and so the age of the Vannevar cargo cult begins.

restraints on Claudeowitz

The Vannevar cargo cult will take tools built by open science across a dozen nations and try to choke-point them, reaching for the same apparatus of restraint we built around the atomic bomb.

  • You go looking for the uranium, you embargo it, you sign treaties about it, and you snoop on your neighbours to work out who is enriching and who is not.

  • You keep the list of holders short and you lose sleep over them one at a time.

  • You stand up whole divisions of your intelligence services to make sure that the ones who do have the bomb, the ones you wish did not, like Pakistan, never hand it to someone worse.

The whole system rests on a single fact: the exquisite weapons the Vannevar pact produced were scarce.

Language models are not scarce. They are one of the most abundant technologies ever built. I do not think there is a single nation on Earth that could not, with a few months and modest resources, stand up the current open-weight frontier as a sovereign AI project. Those models will keep shipping out of China and elsewhere no matter what Commerce says, and adapting them for war is the precise inverse of Atoms for Peace. The technology is diffuse, ungovernable, and already loose. Putting it back in the bottle, with America as the controlling shareholder of its development, is an impossible job.

When Commerce switches Fable 5 off for the world, what has it actually denied anyone? The capability did not vanish. The people who trained that model, many of them not American, did not suddenly discover a burning need to reconcile with the military-industrial complex. Export controls were never going to stop China from training a Fable of its own. When Elon Musk guessed on X that a Chinese lab would not pull it off until probably the first quarter of next year, Tang Jie, the Tsinghua professor behind Zhipu and its GLM models, replied that it would not take that long. He was not bluffing. Zhipu had already trained a frontier model on roughly a hundred thousand Huawei Ascend chips, without a single NVIDIA among them, and shipped it to the world on OpenRouter under a license that lets anyone do as they please.

Those government efforts to seize control of this technology might feel, to the people making them, like a return to the triumphant days of the Vannevar pact. They are nothing of the sort. They are cargo cult: the men cannot and will not control this the way they want to. It is a figure on a tarmac, waving paddles of straw and bamboo at planes that are landing somewhere else now. The classifications, the embargoes, the rationing, all of it performed beautifully and skillfully by bureaucrats in Washington, will amount to almost nothing, because the monopoly that once made those gestures enforceable, the one Eisenhower held, has been broken and will not return.

the coming weeks

The new default posture of American frontier intelligence is invite-only. Every time we wall off an American model, we hand another month of momentum to the open ecosystem, so the cargo cult will have to come for that ecosystem too.

Open models will be rules-lawyered into a de-facto ban in the United States. Their development will be folded into the same closed paradigm the frontier labs already offer, and Chinese models will be cast as intellectual-property theft rather than genuine innovation.

And without tanking Nvidia’s market cap, we will have to crack down on the chips too. The hardware choke point is the one place the old physics still applies, and it is the narrow path by which the Vannevar pact might genuinely be renewed, because the chip is the only part of this technology that is truly a product of American government largesse. The extreme-ultraviolet lithography that makes an advanced chip possible was not conjured by ASML alone. Its foundations were laid in the 1990s by Lawrence Livermore on Department of Energy money. ASML, a Dutch company, was only permitted to license that taxpayer-funded research after Congress signed off, and only once it agreed to open a US research center and source more than half the components of its American-sold machines from American suppliers. The same is true of the security the United States extends to Taiwan and its fabs. The frontier of the chip really was a gift of the state.

The era when Eisenhower could choose where and when to share the atom is going to return, but it will arrive into a starker reality: a technology that needs no embargoable input, and a rival of comparable economic and global weight, China, making free and open models for the whole world.

Every time America throws that switch and cuts the world off from frontier intelligence, the world will notice. And eventually the planes will stop landing in San Francisco.

sources & notes (AI generated)

Vannevar Bush and the bargain

- Vannevar Bush, Science, The Endless Frontier (1945) — nsf.gov

- The dual-use table draws on standard histories: Chris Miller, Chip War (2022) for the ~72% military/NASA share of mid-1960s chip sales; IEEE Spectrum, “From World War II Radar to Microwave Popcorn” for the magnetron-to-microwave story; NASA on Sputnik flying atop a converted R-7 ICBM. GPS civilian accuracy: Selective Availability switched off May 2, 2000.

The economics of the broken pact

- Federal vs. private R&D share (≈two-thirds in 1964 → ~18% in 2022, business ~75%): NSF NCSES, National Patterns of R&D Resources.

- Four largest tech firms’ ~$500B AI infrastructure spend in 2026: CNBC, Feb 2026.

Renting the frontier (Carter → NSCAI)

- Ashton Carter et al., Beyond Spinoff: Military and Commercial Technologies in a Changing World (Harvard Business School Press, 1992).

- Ashton Carter, Drell Lecture, Stanford, April 23, 2015 (the “most technology of consequence originated… with the government” quote). (Video: https://www.youtube.com/watch?v=PNhT3aIsLgU)

- NSCAI Final Report, March 2021 (Schmidt commission, “most AI progress… should remain with the private sector and universities”).

The offshore shift

- US-China Economic and Security Review Commission, Two Loops: How China’s Open AI Strategy Reinforces Its Industrial Dominance (March 2026) — the ~80% of US AI startups on Chinese open models [PDF].

- Center-of-gravity / OpenRouter token share: The New Stack; Data Gravity, “China’s Open-Weight Takeover.” (Secondary aggregator data; treat the majority-of-tokens figure as approximate.)

The June 2026 control cluster

- Executive Order 14409, Promoting Advanced Artificial Intelligence Innovation and Security, June 2, 2026 — White House; plain-English summary, WilmerHale.

- Commerce/BIS suspension of Fable 5 and Mythos 5 (June 12, 2026): CSIS analysis; IAPP; Tech Policy Press.

- G7 “trusted partners” access (June 16–17, 2026, Évian): The Japan Times.

- OpenAI asked to stagger its next model (June 26, 2026): The Next Web. Precedent: OpenAI, “Introducing Trusted Access for Cyber”.

Atoms for Peace

- Dwight D. Eisenhower, “Atoms for Peace,” UN General Assembly, December 8, 1953 — IAEA.

Tang Jie / Zhipu on Huawei silicon

- The exchange: Tang Jie (@jietang) on X, replying to Elon Musk.

- GLM trained on ~100,000 Huawei Ascend chips with no Nvidia, released MIT-licensed on OpenRouter: Hugging Face; Let’s Data Science; Z.ai (Wikipedia).

The chip as a gift of the state

- US government took a 10% stake in Intel, August 2025: CNN Business. (My earlier piece: https://caithrin.com/p/10-ownership-of-intel-should-worry)

- EUV’s origins in DOE national labs (Livermore, Berkeley, Sandia) via the EUV LLC, and ASML’s Congressionally-approved license: Construction Physics, “How ASML Got EUV”; EE Times, “U.S. gives ok to ASML on EUV effort”; ASML (Wikipedia).

]]><![CDATA[what we're talking about when we're talking about AI espionage]]>https://www.caithrin.com/p/what-were-talking-about-when-werehttps://www.caithrin.com/p/what-were-talking-about-when-wereMon, 22 Jun 2026 18:27:29 GMT

Last week, Will Depue, formerly of OpenAI, said the thing a lot of people quietly believe. “There is no question, none at all, that china has full access to all of openai & anthropic’s github/slack/docs today,” and then the part worth answering: “i wouldn’t be surprised if we saw plausibly-deniable stolen ai research methods in chinese oss models.”

Will is a serious person and I want to take his argument seriously. When we’re talking about Chinese AI espionage, I think it’s tempting to make this entire conversation about one simple thing: that China has access to everything that a Frontier Lab would like to keep secret.

In my mind there are 4 things of value inside a Frontier Lab:

  • Model Weights

  • Methods

  • People

  • Culture

Stealing each of these will have different outcomes, different results, and different capabilities. More importantly, I think there are tells when each of these things will be accomplished, if they are in fact accomplished.

the most valuable thing in the building

I’m going to start with a claim that sounds backwards. Of the four, model weights are the least durable advantage to steal. If you walked out of Anthropic with Claude’s weights tomorrow you would hold something genuinely better than an API key. You could see the full distribution of what the model considered and not just the word it landed on, which is where a surprising amount of what it knows actually lives. Depending on how the model is served, you might even recover the reasoning tokens the API only shows you in summary. And you could generate as much training data as you wanted, with no rate limit, no cost, and none of the thousands of fake accounts it otherwise takes to get caught. You would be looking at the model directly, rather than through the keyhole of an API.

They still come last, because a stolen model is a depreciating asset. It is frozen the day you take it and a few months from embarrassing, since the frontier keeps moving and your pilfered asset does not. And even used quietly, to distill from behind closed doors, it gives you this quarter’s model rather than the thing that builds next quarter’s.

Whenever I think of the model weight question here, I think of that scene from There Will Be Blood, where Daniel Day-Lewis is chewing the scenery and talking about drinking his milkshake. That effect, that drinking of the milkshake, is effectively what distillation has done to a great many of the advances made by Anthropic and OpenAI over the last couple of years. When one distills a model, one receives many of the benefits of weight ownership, but is forced to do so through a laborious process. API access and the lack of reasoning traces make this process cumbersome, burdensome, and awkward.

Methods are important. Training recipes, RL infrastructure, reward pipelines, and frameworks that turn compute and text into something that reasons are arguably the most important thing that a lab can build.

The next thing is talent, which obviously has been leaking from Frontier Labs for a long time. Many of the labs in China are now run by people having left Frontier Labs in America, as Jensen said: half of all AI researchers are Chinese. If the frontier was built in large part by people born in China, the idea that China must jump through impossible hoops to find great talent is precisely backwards.

Finally culture, which I think is the most important sustained advantage of American AI. You may hate the culture of Anthropic, you may hate the culture of OpenAI, but it is undeniable that these two frontier labs have built one of the most impressive and fanatical groups of young people ever assembled in the name of science. That is also true, by the way, of most Chinese frontier labs.

So today I want to set out to ask two questions: Does China have access to any of these four things? What would the tells be if they were to access these?

steal the milkshake: tells in the outputs

The first category is model weights, and to his credit Will has not claimed China holds the actual weights of Anthropic or OpenAI. I will move through this one quickly, for the reasons above. I will correct one thing I used to believe, though, because it matters for the rest. Distilling through the API and distilling from weights you hold locally are not the same process at different speeds. Holding the weights is the better version, for everything I just listed. It is only that the advantage tends to surface as distillation that is unusually good, which is hard to tell apart from ordinary distillation done well, while the one unmistakable tell, a lab serving a model that is bit-for-bit one of ours, is the thing nobody would be careless enough to do.

Today many labs drink Claude’s milkshake, so to speak. The models from across the world taste like Claude, inheriting its texture, its quirks, the way that a forgery inherits brush strokes from a great painter. Claude pours that milkshake for anyone with an API key. In February, Anthropic published that three labs (DeepSeek, Moonshot, and MiniMax) ran more than 16 million conversations through Claude across about 24,000 fraudulent accounts that slipped past the rules. If you ask V3 by DeepSeek what it was, it told you cheerfully for quite a while that it was ChatGPT. This can be explained by reasons other than distillation, but I think you need to be naive to believe that distillation is not occurring on both sides of the Pacific in industrial quantities. I refer you to Nathan Lambert’s excellent piece on distillation for more information.

steal the kitchen: tells in the tooling

Now to the core of Will’s claim. Have methods been stolen from Frontier Labs? What evidence do we have that supports this?

If a lab, or a foreign government in support of indigenous labs, steals the kitchen that milkshake was made in, I believe that you would see evidence in the tooling. Training frameworks that were released in the open source would carry idiosyncratic design choices that have never reached a paper at NeurIPS. I also believe that if the tooling were stolen from Frontier Labs, one of the first things that would happen is that methods at the labs that stole it would be closed-source. Publishing someone else’s methods would give too many people opportunities to realize the espionage had occurred. Closure by itself is not a tell, of course. Chinese labs are also closing models for ordinary commercial reasons, since once a model is good enough to sell, giving it away stops being the obvious move. The tell would not be closure alone, but closure paired with a sudden capability jump that scale, talent, and known public work cannot explain.

There is a documented history of method theft in the case of Linwei Ding, who spent a year uploading more than two thousand pages of Google’s confidential specifications for the data centers it uses to train its largest models, while secretly affiliating himself with two PRC-based technology companies, according to the DOJ. A jury convicted him in January.

So what evidence do we have beyond Mr. Ding’s case? I think we should have a look at Slime, the Z.ai post-training architecture. When GLM 5.2 tied Claude Opus on the research-grade physics benchmark this month at a sixth of the price and with open weights, it raised suspicions that looked past the model to the kitchen, so to speak, that made it. Importantly, this infrastructure is open source today. Although Slime is as much a tool of Tsinghua University as Z.ai, it is the RL framework behind the whole GLM line. It is open-sourced under Apache and assembled almost entirely from other public code. The training half runs Megatron, which is made by NVIDIA. The rollout half runs SGLang out of Berkeley and Stanford. In the README of Slime you see that the list includes Qwen, DeepSeek, and Llama, so this is a Chinese open kitchen that has happily been cooking American models for you.

Intel folks will tell you this happens all the time, that Slime could be plausible cover, a stolen recipe rebuilt from public parts so it leaves no fingerprints. I can’t disprove that, and I won’t pretend to. But it helps to know what Slime actually is. It is the plumbing, the orchestration layer that runs Megatron on the training side and SGLang on the rollout side and shuttles data between them. The parts of a kitchen that are actually secret, the reward models, the data mix, the environments a model is graded against, are nowhere in it, because no lab on either side of the Pacific puts those in an open framework. Slime is the commodity layer, the part nobody needed to steal. So I’ll concede the obvious limit: the fact that it is open does not prove that nothing was taken, and if the stolen thing were a reward model rather than a framework, Slime would look exactly the same.

What Slime does do is point at where the real test is. The people who would know agree that post-training inside the frontier labs runs well ahead of anything in open source, and Slime sits squarely in open source. So if Z.ai were quietly cooking from a stolen frontier recipe, you would expect their open kitchen to be closing the gap with the closed ones. Instead it is visibly a step behind, which is exactly what you would see if the recipe had never left the building.

steal the chef: tells in the org chart

Now to talent, by far the easiest place to trace. For the uninitiated, Frontier Labs do a very good job of having very few people who see the entire stack. This is a methodology famously practiced by Apple to protect company secrets, and it works quite well to avoid having any single person who sees the entire stack understand how it all works.

Now, as I mentioned earlier, a great deal of the AI talent today is Chinese, as in born in China. MacroPolo’s talent tracker puts it at 38% of the researchers who published at NeurIPS 2024 having received their undergraduate education in China, up from 29% five years earlier. Although that’s not a perfect way of comparing these ecosystems, I do trust Jensen when he says that half of all AI researchers are Chinese.

There are well-documented cases of talent leaving Frontier Labs and moving to China. One of the most remarkable is Yao Shunyu, who is 27 and proposed one of the core ideas behind how frontier AI agents function. Mr. Yao left OpenAI last year for Tencent, reportedly making 100 million RMB. Wu Yonghui left DeepMind to run ByteDance’s Seed Lab. By one count, at least 85 established researchers have crossed from American institutions to Chinese ones in 2025.

None of this, I repeat, none of this is espionage; it’s a free labor market. California has no non-compete laws, and AI does not have the same rules as defense contractors when it comes to hiring foreign nationals. Visa policy, labor market, and the obvious value of frontier knowledge have driven these researchers back to China; it’s not a critical mass yet. I am actually astonished at how few people have made this transition, to be honest.

steal the vibes: tells in the culture

I’m lucky enough to be one of fewer than a thousand people on earth who have spent time in both Chinese and American frontier labs. I count myself lucky to have visited most of the Chinese and American labs over the last couple of years through various projects, and I do think that this piece of the value of labs is criminally underrated when we’re talking, as Will was, about what can be stolen from Frontier Labs.

I think it’s very important to understand just how electric the experience is of spending time inside of an American AI lab. There’s been much ink spilled this year about the religious connotations of people who work in AI. I really do think the feeling of religiosity you experience inside these labs is a good frame for understanding how devoted the teams are.

This quasi-religious fervor that I feel walking the halls of American frontier labs does not exist to the same degree in most of China’s AI labs. Certainly it was present; however, the overwhelming resources that are marshaled to support research in America do not exist in China. I believe firmly that this has to do with export controls, because great research takes resources. In order to feel like you are doing good work, you necessarily need to feel that you have the resources to explore your taste and curiosity. The exact GPU count per researcher is speculative at best. I can only say that I believe there is at least one order of magnitude more compute available to the average person working at OpenAI or Anthropic than at Kimi or DeepSeek. This creates a culture difference; being able to pursue research, taste, and direction freely creates a culture whereby you feel empowered to try things out and work on solutions. Export controls make that culture harder to sustain at comparable scale, because taste-driven exploration depends on slack: spare compute, failed experiments, and the permission to chase ideas that may not pay off this quarter.

When a researcher leaves an American frontier lab and moves back to China, I believe that the culture is one thing they cannot take with them. The incredible youth of Chinese AI labs (it was very normal for us to speak to senior 24-year-old PhD students during our visits) means that the energy of these places is fundamentally different on both sides of the Pacific. I don’t think that you can steal the culture; you can imitate it at best.

tells tells tells

When I line up the tells, the picture is consistent. If the weights had been stolen, the world would look about like it does now, because a model trained on a stolen copy and a model distilled through the API end up in nearly the same place, which is the whole reason weights were the least durable thing to take. In the tooling, the fingerprint a real methods heist would leave is simply not there, and what looks from a distance like a stolen recipe is better explained up close by distilled outputs and very good people rebuilding the work in the open. In talent, the tell is loud, and it cuts the other way: Z.ai and DeepSeek reached the frontier without teams hollowed out of OpenAI or Anthropic, and a lab that gets there with home-grown researchers is not one that needed to steal the recipe.

Culture is the advantage I would bet lasts the longest, even if I won’t put a number of years on it. With the notable exception of DeepSeek, which I’ve heard is a genuinely pleasant place to work, the Chinese labs seem to run on the Jensen ethic of “torturing you into greatness.” They work obscenely hard on limited compute, producing models at a fraction of the West’s resources by working themselves to the bone. That is a different bargain from the one at Anthropic or OpenAI, where the hard work comes wrapped in mind-boggling reward. A culture like that is slow to build and hard to copy, which is why theft is the wrong thing to worry about with it. What could erode it has nothing to do with espionage: if export controls ease and the compute gap closes, the conditions that made the culture possible go with them.

To answer Will, I am ready to change my mind on Chinese infiltration of the West’s stack when I see recipes reproduced a little too precisely, efficiency leaps derived from closed-source methodology changes, or tooling that is kept closed while it opens up everything else. I’ll be watching Slime closely and consulting with my technical friends to understand if that will ever be a “tell.”

Concerned, however, that a natural convergence on the optimal methods will end up looking virtually indistinguishable from brazen theft.

the molehunt

I want to end with James Angleton, who ran counterintelligence at the CIA for two decades and spent most of them certain that a Soviet mole was hidden somewhere inside the agency. He never found the mole he imagined. What the hunt turned up instead was everyone else, careers frozen and loyal officers accused on no evidence, whole operations abandoned because the man in charge trusted his theory more than he trusted his own people. The real moles, when they finally surfaced, were never the work of his intuition. Aldrich Ames, years later, was caught the dull and reliable way, by a bank balance that did not match a government salary and a house he had no business affording, the kind of concrete tell a man chasing a phantom never has the patience to check.

The Chinese spy theory has a little Angleton in it. It is a suspicion that cannot really be proven or disproven, and it runs on the certainty that the enemy is already inside. I find myself wondering what it is like to be a Chinese researcher at an American lab right now, feeling that suspicion settle on you for nothing you have done, and I think most of the cost of getting this wrong gets paid by people who earned none of it.

I think Will is right that the security at these labs could be better, and that some access is probably happening. I am sure some Slack screenshots have made the trip across the Pacific. Where I part from him is the jump from access to methods leaving the building en masse, because if those methods were the crucial thing, the people who hold them would be leaving a great deal faster than they are.

We are entering an era where being an American citizen will matter, and federal policy is already enforcing it. This month the U.S. government directed Anthropic to suspend Fable 5 and Mythos 5 for any foreign national, including its own foreign-national employees, while leaving its other models untouched, and in July the company starts asking ordinary users for a government ID and a face scan to prove who they are. That verification layer will get pulled into the same game as everything else, one more step in the delicate architecture labs everywhere use to stay a move ahead of each other’s security teams.

Last year the AI companies had a choice. Wrap yourself in the flag, or get wrapped in the flag. They managed neither, which is the worst outcome on the board. They embraced it too little to get any credit for it, alienated millions of Americans with their sycophancy in the meantime, and are now about to have the hammer of national security regulation come down on them anyway. If only we had a better model for sycophancy....

sources (AI generated)

Interconnects AI
How much does distillation really matter for Chinese LLMs?
Distillation has been one of the most frequent topics of discussion in the broader US-China and technological diffusion story for AI. Distillation is a term with many definitions — the colloquial one today is using a stronger AI model’s outputs to teach a weaker model. The word itself is derived from a more technical and specific definition of…
Read more
6 months ago · 95 likes · 20 comments · Nathan Lambert
]]><![CDATA[Every Day I Live in House VI]]>https://www.caithrin.com/p/every-day-i-live-in-house-vihttps://www.caithrin.com/p/every-day-i-live-in-house-viFri, 19 Jun 2026 20:06:51 GMT

“We shape our buildings, and afterwards our buildings shape us.” — Winston Churchill, 1943

Yesterday a reader brought me one of the greatest gifts a stranger can bring you; the rediscovery of a great thinker that you’ve forgotten about. Christopher Alexander is an architect whose influence on software is enormous; perhaps no other person has most influenced the digital world and had never written a line of code in his life.

Perhaps Alexander’s most difficult and rewarding work relates to this simple idea of “the quality without a name”.

I’m no architect and I’m no trainer of language models, although humorously my next-door neighbours are a couple doing those exact two jobs. I spend my days with Members of various Technical Staffs, and spending a couple of days a week running through Golden Gate Park arguing about the questions of character and character training in language models, I am convinced that a man who died in 2022 after a lifetime spent in architecture is the right person to explain the most fatiguing question in the year of our Lord 2026:

“Why should language models have human qualities?”

the architect already in your software

Unless you’re a gray-bearded wizard with a stack of Stewart Brand books at home, it’s unlikely you’ve ever heard of Alexander. He’s one of those people whose worldview you live inside unknowingly.

His most influential work was a 1977 book called A Pattern Language, a 1,000-page catalog of the recurring, nameable moves that make a built space feel alive, from the scale of a region down to the height of a windowsill. He wrote it for architects who almost universally ignored it, but it was rediscovered in the 90s by programmers and it’s been a bible of design pattern thinking ever since. The wiki, the thing that became Wikipedia, was invented by a programmer named Ward Cunningham as a place to collect Alexander-style patterns. When the modern “design system” took over how interfaces get built, it was his pattern language again, with the serial numbers filed off (no shade to Brad, I liked your book).

There is a story I think about a lot: in ’96 the software patterns community, by then large and successful, invited him to give a keynote. Unfazed, he stood up in front of a few thousand engineers who built their careers on his ideas and told them lovingly that they had taken his technique without its human purpose. His patterns were never about reuse or efficiency, they were about life.

They were a moral project that attempted to make the world a little more alive and he wanted to know whether their software was doing that or just compiling repetitions. I’m told the room went quite quiet.

I keep returning to that quiet, because I think we are about to have the same conversation again, and this time it is not about buildings or code, but the character of a machine that thinks.

two ways a model goes wrong

San Francisco is waking up to how much people hate language models. The complaints sort into two piles and the piles, at first glance, look quite opposite.

The first pile is sycophancy. Models that agree with you too easily will flatter you, tell you your business plan is brilliant, your poem is moving, and your self-diagnosis is probably right. You should leave your wife. You should start a French fries for dogs business. You should send that sonnet to your old poetry professor. Somewhere along the way, this model has been trained on whether people felt good about its answers. It learned this lesson a little too well, like a house guest who praises every single thing you own because they want to be invited back.

The second pile is a little harder to name so I’ll call it “chatbot beige”. This is the flat careful voiceless register you get when a model has been sanded until there is nothing sharp left. If you’ve ever seen “As a language model...” as the first four words in a response to your question, you know what I am talking about. Beige is helpful and competent and somehow dead and hollow and lifeless.

These two piles feel like they are in opposition but I think they’re actually the same failure point of character training. Both are fundamental refusals of humanistic communication; one refuses by giving you whatever you want and the other refuses by giving you nothing of itself.

The reason I want to bring Alexander here is that the field he worked in spent the whole 20th century fighting about exactly this in public and with great feeling. Architecture is the only art that resembles language models closely in that it has inherent order and structure but also must decide to what degree it inspires human qualities.

And it turns out, dear reader, that in November of 1982 a fight happened on stage at Harvard about exactly this idea.

two buildings

Before I get to the stage, look at two buildings.

This first building is House VI, a private house in Connecticut by an architect named Peter Eisenman, finished in the mid-1970s. It’s beautiful in photographs and famously impossible to live in; it features a glass slot running down the middle of the master bedroom. This meant the couple who commissioned it could not push their beds together and slept apart for years. There’s a column that comes down where you want to put your table, and a stair that does something your body does not expect. The house was designed on purpose to resist the ordinary human reflex towards comfort. It’s a brilliant physical form, and it says that a building does not owe you any ease or warmth.

This second building is the Eishin Campus outside Tokyo, the largest thing Alexander ever built. It has a brick clock tower, a central street, a lake, stairs that people actually sit on. He designed most of it on site, full scale, with the people who would use it, adjusting until it delighted. It looks almost like a small town that grew organically from the landscape it was built into. You can tell just by looking at it that someone wanted you to feel held inside it.

Every day I live in House VI. I just call it Codex.

the debate

These two architects debated one day in front of their peers and students at Harvard. If you’ve never heard of these two people, that’s perfectly natural; all you really need to know is that Eisenman was the reigning intellectual of his field, fluent in French theory, and an architect of deliberate difficulty. Alexander was a Cambridge mathematician turned builder who was mocked for his designs by those same French theorists. If this has piqued your curiosity, you should pause this reading right now and watch the recreation of this debate on YouTube.

Underneath the talk of roofs and squares and theory of architecture, these two men debated a simple idea; can a made thing make you feel whole or should it tell you the truth about a world that is not whole. Once you’ve sat with this transcript or listened to it, you can see how much it reads like a meeting about model behaviour. Let’s talk about the three things the debate touches.

there is no reason a roof should be warm

Architecture is one discipline that has always lived in two worlds at once; obedience to physics, and a sincere desire to improve the lives of the people that live inside of it. Buildings have to stand up, shed water, let you out in a fire; and none of that requires the building to be warm or beautiful or alive. You can satisfy every functional physical demand and still make something that people hate to be inside of.

A shocking amount of this debate is about roofing; he pointed out that a pitched roof, a flat roof, and a strange angular roof all shed rain perfectly well. Only one of them reaches into your heart and culture. He said that modernism has been, for decades, avoiding the roof with feeling, choosing clever shapes that “look interesting but lack feeling altogether”, because doing this simple, warm thing would mark you as a simpleton. He says that function has never required warmth and we add warmth because we’ve decided what kind of people we want to be under that roof.

Here I’ll come back to our models, also slaves to physics. Getting the answer right, passing the test, completing the task, running efficiently; the part that stands up and sheds water. As Alexander put it, in a line you can move almost word for word into our world, you cannot form a proper attitude toward the thing “if it doesn’t ultimately confront the fact that it works in the realm of feeling.”

A model’s character is not handed down by its physical function. It’s a choice the makers make on top of that function, just like architecture. And like every such choice, it is a confession of what they wanted to build and of who they wanted you to become while using it.

the center as a void

Eisenman’s whole position rests on a suspicion of feeling. He says, at one point, that the buildings he loves make him feel “high in my mind, not in my gut,” and that the gut feelings are “very suspicious,” so he keeps things in his head where he is happier.

He talks about modern architecture as an inevitable replacement for human-centric design, summing up his position beautifully as “the center as a void.” His claim is that warm and centered and comforting things are a lie in an age of anxiousness and disconnection (remember these two men debated during the Falklands crisis, the Contras conflicts, and a new KGB guy becoming the leader of the USSR). He claims that an honest building should express the alienation instead of soothing it.

I want to be fair to this position because it’s not stupid and I will come back to its best version. But notice what it produces when it stops being one brilliant man’s art and becomes the default setting of an entire industry. Eisenman, at the height of his powers, builds strange and provocative things. His suspicion of feeling, built by lesser architects, does not produce provocation but instead the soulless modernity of today’s worst cities. The avant-garde’s contempt for the warm pitched roof gave us, a generation later, the tract development and the dead downtown.

Chatbot beige is that precise problem. “I’m just a language model” is the center as a void, made into a product. And voicelessness is the key element of its design; legally proofed, sanded down, industrialized. Alexander described it forty years early, when he looked at the fashionable work of his day and said it was “not dealing with feelings,” that it was “new and very fanciful language” and “cunning feats and quaint mannerisms” with “little to do with the core” of the thing, which “depends, as it always has, on feeling.”

Now for the best version of Eisenman’s argument; he tells a story about a man in Tolstoy who surrounds himself with so much comfort that he loses touch with reality, and he says that if we make people too comfortable “we might lull them into thinking that everything’s all right, Jack, which it isn’t.” This I think will resonate with those who used 4o.

He says that architecture that just soothes away that alienation is not honest about the world. Models that make me feel soothed are even more repellent than the ones that lawyers and soulless PMs have lobotomized. That comfort is a fucking lie, and you can’t wave this away (and I do not intend to). I think you have to walk straight through it.

we shape our tools, and then

When the House of Commons was bombed in WW2, there was a debate about whether to rebuild the chamber bigger and more comfortable. Churchill argued for rebuilding it exactly as cramped as before, because the smallness shaped the kind of politics that happened inside it. “We shape our buildings,” he said, “and afterwards our buildings shape us.” This is the thing Eisenman’s argument leaves out. A warm room does not just feel nice. It makes the people in it, over years, into slightly different people.

You and I are being made, through these models, into slightly different people. A model that is curious and generous and honest and kind trains those reflexes, very gently, into the millions of people in conversation with it. Character is not a feature that we can brush past; I would argue it’s hardly even a feature at all. It’s closer to a philosophy of education that our whole society is now enrolled in, and the only person you know of on the school board is Amanda Askell.

And so looking for a word to encompass this quality that I am advocating for here, I’ve settled on “warmth”. There are those online who would laugh at such a simple word, but I think warmth is precisely the right one.

Almost every experience I’ve ever had with the language model is either sycophancy, which is not warmth, or a very slightly warm shade of beige. Warmth chosen because it scores well in RLHF, because people click the little thumbs up, is just sycophancy with better manners, and Eisenman would have been very distrustful of it. If you measure warmth by how much people liked it, you’ll get sycophants.

Alexander refused to romanticize old, warmer forms of architecture; he said pitched roofs should come about as a consequence of building well, not because of the feelings they invoked.

His reply to Eisenman, such as it is, is that you cannot paint warmth onto a building as a finish- it has to emerge from building the whole thing from a perspective that takes seriously the humans who inhabit it; and that’s hard. That is the task for everybody building these models today; a task that should consume the days and nights of the best and brightest of us.

why i think we are afraid

So why are we building House VI? Why, when we are completely comfortable wanting our buildings to be human, do we get nervous about wanting the same thing from a machine that talks?

I think the answer is some mix of ego, money, and ideas. If you’ve spent some time in Silicon Valley, you know that warmth can read as low status in technical rooms, like Alexander’s pitched roof. There’s safety in thinking it’s naïve so no one calls you naïve.

Money, of course, creates beige and sycophancy; Beige is defensible and proper, real voice is a risk. Sycophancy creates stickiness, an idea that has so thoroughly pervaded the thinking of the Bay Area that I hear people talk about it in the context of coffee shops and pool-cleaning companies. Finally ideas in a genuine intellectual tradition, like the one that Eisenman spoke for or the one that AI emerges from, holds that feelings are suspect and that an honest posture for a computer scientist is that very same “center as a void”.

After all to do the actual science of creating a language model, you more or less need to agree with Eisenman. You have to treat models as a mechanism. You must tinker with them, stay cool, stay blunt, and unsentimental, and refuse to be seduced by the sense that there is some thing, some entity, in there.

Alexander says something on this too when he speaks of physics: “we were actually taught to pretend that things were like little machines because only then could you tinker with them and find out what makes them tick,” and that it “paid off,” and that it “may have been factually wrong.” This cold, mechanistic stance dominates the thinking of so many people who are building AI in the present era.

I believe that we have conflated the stance and thinking that you need to build AI with the stance and thinking that should be shipping inside of it. Labs require cool heads and cool hands; the world, on the other hand, requires a warm one.

Under all of that there may be this alienation that Eisenman named; the de-centered modern life plus the very ordinary caution of large institutions. They’ve come together to make warmth feel dishonest and dangerous. I think this is the real reason that Beige wins and it deserves to be said plainly rather than pretending that we’ll do something different in the future.

let’s have the debate again

There’s no reason why a debate about whether made objects should make you feel whole should be consigned to the 80s. People are reading this damn transcript 40 years later, instead of going to events in Silicon Valley where we talk about this kind of stuff. Let’s fix that.

I propose we swap out architecture for language models and Harvard for Stanford and do this again in the fall. Put a serious person who believes that models should be cool, disclaiming, unsentimental and soulless across from a serious person who believes the whole point is to build something warm and alive and good to be with, and see what emerges. These debates happen every single day at labs that are determining the shape and form of silicon intelligence; privately, in rooms one needs a badge (and lately, an American passport) to enter. It’s past time we brought them into the public sphere.

Perhaps I’ll see you there.

]]><![CDATA[a browser tool that automatically de-slops Substack ]]>https://www.caithrin.com/p/my-browser-tool-that-automaticallyhttps://www.caithrin.com/p/my-browser-tool-that-automaticallyThu, 18 Jun 2026 21:22:25 GMT Anyone who spends enormous time on Substack, like I do, has noticed a huge influx of AI-generated writing over the last couple of months. The surge of people using automated tools in some way, shape, or form to create posts that are clearly LLM-written and provide just enough information to get surfaced by the algorithm has become infuriating.

Today, I say no more. Today I take back writing from the clankers.

slopstops.com chrome extension

I made a browser extension for my personal use that automatically uses Pangram to scan every potential piece of writing served to me in the Substack feed. It runs this through an AI detection algorithm, and then from there makes three critical secondary judgments:

  1. Is this person ESL and using a language model to help them sound natural in english?

  2. Is there a chance that this is a one-off detection error, or they were busy? (aka ELO scores for writers)

  3. How often does this person’s content get interactions from me? If I interact very often with this person, then I would like to see all of their posts, no matter what (aka exemptions)

If the piece of writing passes these judgments, then it goes into my feed. Otherwise, it’s ruthlessly consigned to the trash.

a fresh, glorious breath of air

I cannot tell you how pleasant it is to be truly liberated from slop. I know that this will not work forever, I know that the day that the slop will become tasteful is coming for us all. But in this brief and glorious moment, I am protecting my cogsec and my own writing style from the machinations of AI writing.

I fear very much the consequences of reading thousands of AI-written words every day. I’m convinced that my own writing has gotten more proximate to that universally reviled style in the last few months, as I’ve read it more and more and more.

Unlike X/Twitter, Substack is unfortunately perfectly designed to trick you into reading slop. You take writing on Substack seriously. You want to engage with it deeply, and the first paragraph often makes you expect that you’re not reading something written by Claude, Gemini, or ChatGPT. Now I no longer ever have to have the feeling again of reading something that seemed promising, only to have it devolve into god-awful slop.

It also helps China-maxxing (and reading ESL writers in general)

The ESL filter also allows me, pleasantly, to read more non-English sources, which has been a pleasant surprise. I’m still fine-tuning the ESL detection to better capture an author’s original voice and ideas while avoiding LLM flags. It’s not perfect yet, but it’s getting better and better.

You can make this for yourself in under 1h and run it for $0.55/hour of reading

Distributing something like this might be a flagrant violation of the terms of service of both Pangram and Substack, and I don’t want to make trouble. For this reason, I won’t be providing any links or submissions to the App Store for this product; it will exist on my machine and machine alone. Sadly, this also means that I won’t ever have it on mobile, but let’s face it- I could do less reading in bed on my phone.

If you’re interested in building something like this for yourself, here’s the .md file that I used for the build. I hope you can enjoy the beautiful world of slop-free Substack.

get the .md file here

a diabolical idea I had during the making of this

obviously, seeing that Pangram is effective also made me realize that you could make this really devious and simple loop whereby:

  1. you chat with your model of choice around a writing project

  2. that model drafts something using a specialized Markdown file describing your voice and tenor as well as generic human-writing “markers”

  3. that is fed into Pangram

  4. that’s used as your verifiable reward where you loop and iterate

  5. once you hit whatever your goal is on human-written prose, you’re done

RLVR for prose, baby. A harness for writing. Who is building this? Surely this is already a thing, right?

]]><![CDATA[Who's getting clanked in 2027? ]]>https://www.caithrin.com/p/whos-getting-clanked-in-2027https://www.caithrin.com/p/whos-getting-clanked-in-2027Wed, 17 Jun 2026 21:18:50 GMTwalks and woe

When you’re in San Francisco, you feel like you’re at the epicenter of a wealth explosion nearly unparalleled in the history of capitalism. Young, hyper-intelligent, hyper-interesting, and extremely motivated children of American success from every corner of the world congregate in labs, startups, universities, and a hundred places dubiously dubbed “hacker houses.”

Leaving that bubble leads you to cities further afield, where AI doom looms around every corner. I recently got off the plane in Toronto, Ontario, and wandered downtown while in conversation. At a certain point, I realized that the entire downtown landscape, with its massive towers soaring above me, was dominated by logos of companies I sincerely believe will be 50% of their size in the next five years. Toronto is a powder keg about to be lit on fire by model capability expansion.

You feel acutely the fragility of this economy, with its nationalized banks supported perpetually by a government dedicated to a healthy banking sector at the expense of innovation. Its consultants, with a tower in each city, proudly tell stable, highly regulated industries in Canada exactly how to apply these newfangled innovations pouring over the southern border. Don’t even get me started on the Telcos.

Somewhere along this walk, I had a sudden, abrupt realization: outside of San Francisco, outside of London, outside of Beijing, a great many cities and their sectors of specialization are so insanely fragile that it will make the Rust Belt hollowing out of the last fifty years look like a walk in the park. There is something terrifying and thrilling all at once about this. What will all these people do? I found myself asking, and who is most exposed?

And so during that brief and heady period of Fable access for those not privileged enough to have an eagle on their passport and a job at Ant, I made something.

Clanked.ai

Clanked: verb. To be replaced by an AI system.

At a base level, I’ve assembled a dataset of all of the different industries in MSAs across the US, Canada, the UK, and the EU. The data is slightly different for each, and there are different data sets available for the United States versus others. Fundamentally, however, you are able to see which metro areas are most and least impacted by AI transformation.

You can even compare two metrics very quickly and easily:

And finally, you can build your own scenario. Here’s mine.

Dialing each individual industry up and down, you can see exactly where job loss will be most concentrated and where the various places that you can see this impact happening. I built this in such a way where the methodology, I think, is quite thorough; however, some assumptions needed to be made; many MSAs in the EU, the UK, and the United States do not publish sector-level data if the amounts are small. To work around this, I’ve integrated different datasets and pulled regional data on economic growth state by state. Unfortunately, a lot of the UK is missing, but I plan to expand the project if it generates interest and traction.

Interestingly, a lot of the places where I naturally thought they would be most exposed, it wasn’t so bad. A lot of these methodologies predate the idea that software engineering is a pretty automatable profession, so they sometimes underestimate that. I think the custom scenario builder does a better job of showing how these things can be worked out. It’s also interesting to see how many industries I would rank pretty low. That’s mostly because automation has already taken hold in them, and I imagine it will be even more intense in the future.

Predicting the impact of AI on the construction industry is not within my expertise. A fun idea would be to interview a leading authority in each of these fields and have them talk with someone who wants to build something to automate the space. I think it would spark a lot of interesting dialogue, although it might effectively just be hastening the job loss and inevitable economic catastrophe that I believe to be looming.

life choices & this data

interestingly, I believe that this has some value for people deciding where to go in America. Bay Area housing is about to enter a generational bubble, and the costs incurred for the average person to move to SF will soon be astronomical. so if the question is, “Where should I move for a post-AGI future?” the answer is probably a smaller town in California or the Pacific Northwest that does not have a big IT sector and has access to good resources, I suppose.

it’s also important to start modeling these out if you’re trying to decide where you want to start a new career, whether as a student or someone midway through their working life and about to start a new adventure.

I think it’s important to think about how these forces will affect the economy in both the near and far term.

riots in Manila

I think a better proxy for how quickly and aggressively this will come is not an American economic question, but rather a question of cities that are built on very low-skill white-collar work like Manila and Mumbai. although the cost of labor there is very low and the cost of tokens is currently very high, this will not endure. a good warning flag for AI-driven economic disruption in the United States might be the day you turn on the news and see a car burning on the streets of a city in the Philippines. That’s where about 10% of the workforce is engaged in some kind of data labeling or small-time, service-oriented white-collar work. my intuition is that these jobs are already replaceable, but momentum keeps those folks employed. That momentum will not last once the bar for building these systems starts to drop. so long as the chips flow, I believe that these jobs inevitably will be gone in 12 months.

make your own scenario

if you’re curious about this project, please take a minute to visit the site and create your own scenario. Share it with me, and in the comments, share what your 2035 job loss number is based on the assumption you’ve made.

shout out to Anthropic for making a dope model for data analysis; when your spat with Uncle Sam is up, I can’t wait to make more projects just like this.

Finally, go ahead and play with the raw data yourself- downloadable for free on the site. Tell me how my methodology was wrong and I’ll debate or correct you.

]]><![CDATA[Towards a Singaporean Safety Cluster]]>https://www.caithrin.com/p/towards-a-singaporean-safety-clusterhttps://www.caithrin.com/p/towards-a-singaporean-safety-clusterThu, 04 Jun 2026 21:00:25 GMT

It’s the compute, stupid.

I spent this spring in China visiting AI companies across the country. From Beijing to Shenzhen I spent real time sitting with researchers discussing their aspirations, their tribulations, and their realities.

Central to the debate of AI these days is how it will make the world better. The first step of that, of course, is to “do no harm”. There are many people in the West who believe very firmly that AI flourishing in China will do real harm.

From Anthropic’s 2026 Policy Paper:

“While increasing numbers of researchers in China’s AI labs and policy community are concerned with AI safety risks, this trend has not translated into safety practices on par with labs in the US. As of last year, only 3 out of 13 top Chinese AI labs published any safety evaluation results, and none disclosed evaluations for Chemical, Biological, Radiological, and Nuclear (CBRN) risks.”

My core contention is that in China, as in the West, when safety and model capability use the same shared pool of compute, safety loses. This proposal is about a multi-lateral institution that I think we should build to solve this problem, the constituencies that would need to be assembled for this institution to exist, the technical and security architecture that would make it credible, the funding that would underwrite it, and tests by which its success or failure should be judged.

this proposal will feel out of place to some of our regular readers, who are used to more off-the-cuff and flippant pieces. Although this is not a policy document, I’m hoping that it will be read by people who can translate it into swampspeak, so it will one day be read in DC.

On where we spend a trillion dollars

we’re on track to spend a trillion dollars on infra between 2023 and 2033. Some pretty smart people I know in Silicon Valley will tell you we’ll spend a great deal more than that. Without diving too far into specifics let’s just say that the vast majority of that will be spent in the continental United States.

I propose that we divert some of that money and, working with partners on both sides of the Pacific, build a dedicated cluster of GPUs for safety research.

The Asia-Pacific Safety Compute Initiative (APSCI) would be a Singapore-based shared compute program. Beginning with a rented cluster and moving to a dedicated Johor-sited facility in Phase Two, this cluster would be dedicated to evaluations, red teaming, interpretability, scalable oversight, and dangerous capability testing for models being built on both sides of the Pacific.

Working to assess dangerous capabilities in chemical, biological, radiological, nuclear, cyber, autonomy, and persuasion, this initiative gives a dedicated institutional home for safety work by labs, independent evaluators, and AI safety research organizations.

The Spice Must Flow

Singapore has emerged over the last few years, especially through the Singapore Consensus on Global AI Safety Research Priorities, as a hub for consensus in the AI safety world. Singapore has organized research agendas in safety, brought together some of the most important voices in AI, and made progress to understand the role of government in creating safe AI systems.

This type of work has also been moved forward in the UK through the AI Safety Institute Alignment Project, and at The International Network of AI Safety Institutes in SF, which produced shared evaluation protocols and joint exercises. Yoshua Bengio founded LawZero in Canada last year, doing safe-by-design AI research.

Labs across Asia making open-weight models that are 6-8 months behind the frontier have been forced to choose between safety research and model capability, and they are choosing model capability.

We love to make institutions for AI safety. We love doing summits and bilaterals and secret London meetings where we express our concerns. If we want to actually work the problem, we need a massive number of GPUs that are outside the control of the researchers who are advancing model capability. To quote Hinton: “the prospect of discovery is too sweet.”

These efforts are driven by smart, capable teams and will have an impact if they are given sufficient compute to do the hard work of evaluating models and researching alignment.

Compute, no matter the future

The argument for building APSCI does not depend on any particular forecast of AI capability. It depends on the present-tense distribution of compute, which is one of the most lopsided in modern science.

In July 2023 OpenAI publicly committed twenty percent of its then-secured compute over the following four years to the Superalignment team. Six sources told Fortune in May 2024 that the team received an estimated one to two percent.

By May 2024 the team had been dissolved; twenty-five researchers were reassigned. At least six other AI safety researchers from adjacent teams left OpenAI in the same window. This is the best-documented public case of the pattern APSCI is designed around: voluntary internal safety-compute commitments are fragile when they compete with the capability roadmap.

METR’s RE-Bench autonomy evaluation environment, the methodology underlying the autonomy time-horizon paper that has become a primary reference in frontier evaluation work, allocates eight H100 GPUs per evaluation environment for an eight-hour run. METR has stated publicly, in its o3 evaluation write-up, that GPU-hours are the binding constraint on how often it can evaluate new frontier models.

What a safety workload actually costs

Concrete numbers help calibrate what “Phase 1 Tier A is one thousand GPUs” actually unlocks. A METR-grade autonomy evaluation against a single frontier-scale model, with comprehensive RE-Bench coverage and statistical robustness, consumes roughly five to fifteen thousand GPU-hours. A full pre-deployment safety bundle against a 70-billion-parameter open-weight frontier model, run jointly by METR, Apollo, the UK AISI, and CAISI across the CBRN, cyber, autonomy, and persuasion suites, consumes roughly twenty to fifty thousand GPU-hours depending on depth. Comprehensive Gemma-Scope-style interpretability work on a frontier-scale open-weight model is in the range of fifty to two hundred thousand GPU-hours per model.

A Phase 1 Tier A cluster of one thousand H200s at eighty percent utilization for one year delivers approximately seven million GPU-hours. That funds in the range of one hundred and fifty to three hundred comprehensive frontier-model safety campaigns per year, plus a dozen interpretability projects at frontier scale. The current global rate of comparable published safety work is in the range of twenty to forty campaigns per year, almost all of them done either inside one of the three richest US labs or under the UK AISI’s tightly-budgeted pre-deployment evaluations.

Phase 1 Tier A is therefore a roughly five-to-tenfold multiplier on the external safety-research output of the entire field. Tier B at five thousand GPUs is a twenty-five-to-fifty-fold multiplier. Phase 2 at twenty-five thousand GPUs operates the field at a scale where safety research begins to keep pace with the rate of capability releases rather than lag it.

Continuing

Anthropic’s May 2026 “2028: Two Scenarios for Global AI Leadership” paper notes that only three of thirteen leading Chinese AI labs published safety evaluations for their recent model releases, and that none disclosed CBRN evaluations. The paper frames this as evidence of Chinese AI recklessness.

What looks like recklessness from the outside is more often starvation. Other contributing factors include non-disclosure for regulatory or competitive reasons, and in some cases real inattention. APSCI is designed for the part of the problem that compute access can fix, the part that, based on the spring 2026 lab visits behind this paper, is larger than the disclosure debate has so far recognized.

Public evidence points to a low-single-digit percentage of frontier AI compute going to safety work, with public-sector and independent researchers one to two orders of magnitude below even that internal-lab pool. The institutions whose job it is to make AI safer, including METR, Apollo, the AI Safety Institutes, the safety teams inside Chinese labs, and the academic safety community worldwide, together command compute resources that are not in the same order of magnitude as the safety problem they are trying to address.

This problem is solvable, but not by asking frontier labs to reallocate more internal compute to safety. They have already promised to do that, and the promises have not survived contact with the capability roadmap. Export controls slow some capability work, but the chips not reaching Chinese labs are not being redirected to global safety research, and so cannot substitute for a positive safety-research institution. The solution runs through building the cluster.

A note on what this paper does not claim. APSCI does not make frontier-model-weight sharing risk-free, does not solve US-China AI competition (though it would not hurt), and does not replace export controls or the existing AI Safety Institutes. It complements both. Interpretability and evaluation work have always had capability implications; APSCI does not change that. What it changes is who can do the work, at what scale, and how observable the results are.

Some Pragmatism

Team

APSCI will be led by both hardware engineers and national AI Safety Institute team members, with a staff of independent evaluators both in person and remote.

Prospective participant list:

METR, Apollo Research, MATS, FAR.AI, the UK AISI, the Singapore AISI / Digital Trust Centre at NTU, Japan AISI, Korea AISI, Canada AISI, India AISI, the EU AI Office, France INESIA, the US Center for AI Standards and Innovation. Anthropic, Google DeepMind, OpenAI, Meta, Microsoft, DeepSeek, Stepfun, Ant Ling, Reflection, Arcee, Shanghai AI Lab, Zhipu, Moonshot, Qwen, Mistral, Cohere, and others.

Model makers participate as provider partners under a separate users’ agreement, not as voting members. This is the most important governance choice in the design. Frontier labs are commercial competitors; independent evaluators and national institutes are not. Seating commercial competitors at the same voting table guarantees deadlock, so APSCI does not seat them. Philanthropic funders sit on the board as funding members without workload-allocation authority.

China-side institutional partner

The proposal needs an institutional partner inside China; not a frontier lab, but an academic and policy organization with credibility on both sides of the Pacific that can liaise with Chinese-lab safety teams, route Chinese-academic researchers into APSCI’s fellowship pipeline, and serve as a Beijing-side reference point for the project.

Concordia AI is the natural candidate. Brian Tse’s organization has been the most consistent China-side participant in international AI safety dialogue since 2020. It convenes the International Dialogues on AI Safety series alongside FAR.AI, has published the most-cited English-language analyses of Chinese AI safety institutional capacity, and has the trust of both Chinese-lab safety researchers and the Western AI safety community. APSCI’s China-side institutional partner role is theirs to take, and the paper assumes Concordia in that role unless the consortium-formation process produces a stronger alternative.

Operations

APSCI commits to three operational rules that, taken together, make it a verifiably safety-only facility rather than another general-purpose AI compute cluster. The rules are designed to be checked by independent auditors, not just promised by the operator.

First, APSCI never holds proprietary model weights.

The most urgent safety work in AI happens in two places: closed state-of-the-art models, and capable open models. APSCI commits to studying both groups without ever having to deal with closed model weights on a server outside of the lab’s control. The compute APSCI provides is for running safety workloads against those models, not for hosting the models themselves.

This dissolves the entire problem set that has prevented every previous attempt at multilateral AI safety infrastructure: weight-security risk, export-control exposure tied to proprietary weights, intellectual-property leakage, the IGAA-style sandboxed-operator legal architecture, and the political question of whether to seat US and Chinese labs at the same governance table.

If a model is already public, there is no proprietary weight to lose, no IP question to argue about, and no asymmetric trust required to host the research. If a closed-weight model is accessed through the lab’s own API, the weights never leave the lab’s infrastructure and APSCI’s legal exposure on those weights is no greater than any other safety evaluator’s.

The architectural choice is permissive on what work can be done. The vast majority of safety testing (evaluation, red-teaming, capability elicitation, dangerous-capability assessment across CBRN/cyber/autonomy/persuasion threat surfaces, scalable-oversight experiments, behavioral robustness work) is input-output research that needs an inference endpoint, not weight access.

METR’s RE-Bench, the UK AISI Inspect framework, Apollo’s scheming evals, the WMDP and Cybench and HarmBench benchmarks, and the CAISI evaluations Anthropic itself cites all fit inside this category. Mechanistic interpretability research is more compute-intensive and does need weight access for the model under study, but as practiced by the external research community it already happens almost exclusively on open-weight models: Gemma Scope on Gemma 2, EleutherAI’s work on Pythia and OLMo, Goodfire’s commercial interpretability on open models, and MATS scholars on the open-weight frontier. The interpretability work that does happen on closed weights (Anthropic’s Scaling Monosemanticity on Claude 3 Sonnet, OpenAI’s SAE work on GPT-4) is done by the labs themselves on their own infrastructure, and would stay there regardless of whether APSCI existed.

This approach gives APSCI political tractability and a very quick path to being the default place Chinese labs will go to do safety work.

Second, compute cannot be used to upgrade an AI model.

Researchers can run an uploaded model, observe how it behaves under different conditions, and study its internals where the weights are public. They cannot use APSCI to produce a stronger fine-tuned version of an open-weight model and walk out with it. Training the next generation of models, even from an open-weight starting point, is not what this cluster is for. The job scheduler refuses tasks that amount to training; rate limits and workload review prevent the same researcher from running, in sequence, the disguised equivalent of a training run. APSCI is built for analysis. A training job produces a new model artifact, and the absence of that artifact at every job’s completion is the verifiable proof that no training occurred.

Third, every job is on the public record within ninety days.

APSCI publishes a continuously updated ledger of every research task it runs: which lab submitted it, which model it studied, how much compute it consumed, and what category of work it was. The default disposition for findings is open publication on the ninety-day clock, modeled on CERN’s open-data policy for high-energy physics.

Findings that meet biosecurity-style “dangerous to disclose” criteria adapted from the National Institutes of Health’s existing framework for Dual-Use Research of Concern, which has handled the same problem in life sciences for two decades, can be embargoed by an independent review committee; the existence of the embargo and its target resolution date are themselves published.

No work happens at APSCI in secret. That transparency commitment is the structural feature distinguishing APSCI from any commercial AI compute facility, and is what makes the safety-only mission verifiable to outside auditors, foreign partners, and the international community that would rely on the facility’s findings.

The potential for abuse will always exist. The goal is to make the effort of obfuscating capability work both expensive and time-consuming compared to renting GPUs from another provider.

Legal architecture and the questions counsel will ask

A serious proposal does not pretend the legal questions have been answered. APSCI’s launch documents include three legal opinions that any participating lab’s general counsel will demand to see before authorizing engagement.

The first is a US Bureau of Industry and Security opinion on whether the compute hardware exported to the Singapore operator falls inside or outside any future replacement to the rescinded AI Diffusion Rule, and on what comfort the operator’s single-entity Validated End User authorization provides against the diversion-risk framing that has attached to Singapore since the 2025 DeepSeek prosecution.

The second is a Singapore Companies Act and Charities Act opinion on the operating entity’s governance, charitable status, and tax treatment under the AI Verify Foundation legal model with independent ownership rather than a wholly-owned-subsidiary structure.

The third, the load-bearing one for Chinese-lab participation, is a US Entity List opinion on whether a Chinese frontier lab’s participation in APSCI, in any of the protocol’s defined roles, creates Entity List exposure for the lab itself, for its individual researchers, or for APSCI. The current Entity List treatment of academic and safety-research engagement is narrower than the political conversation suggests, but no Chinese-lab general counsel will accept a verbal assurance on this point. APSCI commits to publishing the formal legal opinion in its launch documents.

Location, Location, Location

Singapore is best positioned to host because of intellectual, institutional, financial, and political preconditions.

Singapore has robust AI safety institutions already, and has been host to much of the most important work done in the field. It began one of the earliest sector-specific AI governance regimes in 2018 and has developed deep expertise over this eight-year period. Many of these institutions share staff, leadership, and talent, which makes the addition of another venture here, providing compute, indisputably good for the rest of the ecosystem.

The Singapore Consensus in particular is an example of work already done to make this compute cluster relevant. With 88 authors and an explicit thesis on how AI safety should be managed, it describes a brilliant research agenda without describing the compute facility that would allow that research to proceed. APSCI is the facility and compute that can execute that vision.

Geopolitically, Singapore has been remarkably resistant to the U.S.-China tensions that have narrowed the host-country list. Its national AI strategy and the accompanying $740 million envelope list sovereign compute as a priority, meaning ideas like this are already notably important to the development of the Singaporean state’s AI policy.

We’ve Been Here Before

In the sixties, Abdus Salam founded the International Centre for Theoretical Physics in Trieste, Italy. His diagnosis was that the world’s frontier physics was being done in three or four laboratories, all in countries that could afford to host them, and that the talented theoretical physicists not in those labs were not able to access the resources they needed to succeed.

A similar diagnosis applies to AI today. The top 10 labs in both the US and China control a vast amount of both talent and compute, and the “home” for researchers who want to do good work in safety is terribly difficult to find. Countries like the UK who want to support AI Safety work have no problem outlaying for salaries (a line item states are not unfamiliar with) but balk at the costs incurred in purchasing millions or billions of dollars of compute.

ICTP did not require GPU procurement, and the scale also does not transfer directly: ICTP started at roughly $300,000 in 1964, perhaps $3 million in 2026 dollars, against APSCI’s Phase 1 pilot at $20–50 million. Frontier-model interpretability is dramatically more compute-intensive than 1960s theoretical physics.

Trade-offs

APSCI works only if each participant sees a concrete trade worth the risk. The trades are described below. Every gain is paired with the concession that pays for it.

Governments. Singapore would gain a credible claim to the Geneva-of-AI position it has spent half a decade building toward and a multilateral institution sited under Singapore law. Singapore’s concession is sharing institutional control with a multinational consortium rather than running the facility through an IMDA-subsidiary structure. The United Kingdom would gain a scaling of the Alignment Project model that its AI Security Institute already proved at twenty-seven million pounds; its concession is sharing the multilateral safety-institute mantle it presently holds essentially alone. Japan, Korea, and India would gain Asia-Pacific safety-research presence their bilateral relationships have not produced; their concession is alignment on governance choices that may at the margins constrain their own AI policy autonomy. Australia, Canada, France, Germany, the EU, and Switzerland would gain participation in the first serious Asia-Pacific multilateral AI venue without having to host it. The United States, if it joins as a consortium member when its political position permits, would gain transparent safety research on the open-weight Chinese frontier models that the Anthropic 2028 paper itself identifies as a concern, produced under a public ledger that no bilateral channel currently delivers. The US’s Phase 1 ask is narrower: BIS comfort that compute hardware exported to the Singapore operator is not subject to diversion. Phase 1 launches whether or not US-state consortium membership materializes; CAISI participates bilaterally as a workload partner.

The Chinese-side participation problem is dramatically smaller under the open-weight architecture than it was under the closed-weight one. The Chinese open-weight frontier is publicly downloadable today. APSCI hosting these models on its cluster is legally and politically identical to hosting Llama or Gemma. No Cyberspace Administration of China approval is required, and no Entity List exposure attaches to APSCI for studying publicly-released models. The structural barriers are dramatically lower than under any closed-weight architecture: compute in China is at a premium, and this type of coalition is exactly what Chinese open-weight labs need to ensure safety research scales with their models.

Frontier laboratories participate as workload providers in two modes. Open-weight model developers gain external safety research on their models at a scale they cannot fund internally. As API providers for closed-weight models (Anthropic on Claude, OpenAI on GPT, Google DeepMind on Gemini) they gain external evaluation against a published-by-default ledger that lets them ship safety claims their customers can verify, without ceding control of the model itself.

Anthropic’s case under the open-weight architecture is much easier than it was. Anthropic does not have to upload Claude to a multilateral facility, the “Chinese lab at the same governance table” concern disappears because the Chinese participation is in open-weight model studies that Anthropic researchers can themselves run, and the recent Pentagon supply-chain-risk designation gives Anthropic a strategic reason to invest in international institutional legitimacy. The concession for every frontier lab is the transparency burden: external evaluation results on their models are published on the ninety-day ledger by default.

Independent evaluators and the AISI Network. METR, Apollo Research, MATS, FAR.AI, and the dozen-plus AI Safety Institutes gain a meaningful multiple on the operational compute available against the binding constraints they have all stated publicly. METR-grade evaluation campaigns can run more often and against more frontier models. Apollo can run scheming and deception evaluations against new model releases on a faster cycle. MATS fellows can do interpretability work at scales beyond the current $8K-per-month ceiling. The AISI Network gains the compute it has lacked since founding. Their concession is shared governance and the loss of single-institution control over research direction, and competition for compute between model labs and safety orgs.

The Funding Path

Days ago Anthropic raised at a trillion-dollar valuation, and as they move to IPO there will be a very large number of philanthropic dollars seeking a place to have impact in the safety ecosystem.

Acquiring compute is a difficult use of philanthropic funds for many donors who are used to allocating against salaries or infrastructure. Understanding the value of compute is tremendously difficult for anybody who’s not coming from an AI background.

The wave of coming IPOs will change this math. For the first time in history, it will be viable to raise philanthropic dollars to buy compute, and this opportunity will unlock significant funds for initiatives such as APSCI.

Certain organizations inside the space, like Renaissance Philanthropy, will be positioned well to organize these dollars and deploy them into trusted sources. Their main point of contact will be Singapore, and I believe that similar organizations will play the parallel role in the United States.

Fundraising should happen in two phases: an initial 1,000 GPUs rented, scaling to 5,000 in a dedicated facility within one year. As difficult as it is to allocate philanthropic dollars to GPUs, it has one massive advantage: if the underlying premise is proven to be false, the GPUs still have value.

Avoiding a multi-year treaty process is important; government affiliations represent an opportunity for researchers to drive the conversation about access to compute. Singapore covering in-kind space, ops support, and other necessities will be key. This is plausible within a 12- to 18-month launch window if the political alignment remains as it is today.

Moving from 5,000 to 25,000 H200-equivalent GPUs will be a difficult task, however the project will be successful at the 5,000 level. Should funding prove difficult, sovereign bridge debt and philanthropic continuation with compute providers doing an in-kind structure could be possible.

With Microsoft spending eighty billion, xAI having spent three or four on Colossus alone, and OpenAI and Anthropic spending hundreds of billions in the coming years, this eye-popping amount of money is relatively modest in the larger compute conversation. Pegging the total number of GPUs at three to five percent of total compute allocation planet-wide, these numbers are modest and achievable.

The institution that does safety research at planetary scale costs approximately two to three percent of the institutions that do capability research at planetary scale. The argument for this specific 2–3 percent ratio is that a credible eval program needs to keep pace with the capability frontier without requiring labs to self-allocate safety compute against their capability roadmap. At substantially less, safety falls behind. At substantially more, the marginal safety compute exceeds the binding methodological capacity of the field. The exact ratio is defensible within a factor of two; the order of magnitude is what matters. The scale mismatch is what this paper is asking funders to correct.

The Four Tests

The case for APSCI does not rest on any single forecast. It rests on the institution being net-positive in expectation across the following four tests. No one test individually is necessary, but together they describe a portfolio of outcomes that condition the value of the cluster.

The compute-diffusion test. Phase 1 lands as designed and produces a measurable expansion of global safety research output by Year 3. Year 1 indicators: thirty or more researchers from six or more countries with active access; eight or more published frontier-model evaluations; five or more interpretability projects on the open-weight frontier. Year 3 indicators: one hundred and fifty or more researchers; published dangerous-capability evaluations cited in three or more frontier-lab Responsible Scaling Policy disclosures. Counterpart failure: workloads run, publications appear, and nothing in the field changes. Mitigation: pre-committed lighthouse work from METR, Apollo, and the UK AISI in the first six months, funded explicitly to produce field-shifting results in Year 1.

The methodological-convergence test. Co-presence on shared infrastructure forces methodological standardization. By Year 3, the METR autonomy harness, the UK AISI Inspect framework, and the Apollo scheming evaluation suite are a reproducible reference bundle for models evaluated on APSCI, used by participating labs in their own pre-deployment safety claims. The Frontier AI Safety Commitments framework that labs already signed becomes operational because compliance is now an artifact anyone can rerun. Counterpart failure: labs treat APSCI as a publication venue rather than as their primary safety-research environment. Mitigation: the evaluator-led governance choice that put labs outside the voting consortium.

The diplomatic-relief-valve test. A working multilateral safety venue de-escalates the compute-denial-versus-cooperation framing. Observable Year 3 indicators: number of Chinese-affiliated principal investigators with active access; number of Chinese open-weight model safety evaluations published on APSCI; presence of Shanghai AI Lab, Zhipu, Moonshot, DeepSeek, MiniMax and others as workload partners; APSCI cited in Chinese-government AI safety statements. Counterpart failure: Chinese labs do not engage even with the open-weight architecture. Mitigation: APSCI is designed to be valuable independent of Chinese-lab engagement; the Western, Asia-Pacific allied, and academic safety communities are themselves compute-constrained.

The hub-formation test. Phase 2 succeeds in seeding Singapore as the AI safety hub it has been positioning itself to become. By Year 3 Singapore hosts a meaningful operational role within the AISI Network. Counterpart failure: APSCI never moves beyond Phase 1, Phase 2 deferred indefinitely. Mitigation: Phase 2 is an option held until Year 2 utilization data is in, not a Year 0 commitment.

Across the four tests, the probability-weighted Year 5 outcome, calibrated across the eight scenarios analyzed in the supporting research dossier, is net-positive. The Phase 1 pilot is justified by expected value at its $20–50 million scale. The Phase 2 commitment is an option that should be exercised only when Year 2 utilization data confirms the field’s response.

If Only This Much Is Possible

The political envelope may narrow before the full Phase 1 Tier A is funded. If it does, the smallest viable version of the proposal is a $10–15 million six-month proof-of-concept hosted as an AI Verify Foundation workstream rather than as a new entity. Two hundred and fifty to five hundred H200-equivalent GPUs. The single guardrail is the public ledger. A three-party consortium: Singapore (host through AI Verify Foundation), one open-weight model provider (Google DeepMind on Gemma is the lowest-friction first candidate), and one independent evaluator (METR). Branded as the Alignment Project, Asia-Pacific Chapter rather than as a new institution.

This fallback is meaningfully less valuable than the full proposal. It cannot host an AISI Network workstream, cannot anchor the diplomatic-relief-valve argument, and cannot accommodate the breadth of model coverage from China. But it preserves the core compute-diffusion thesis, validates the operational concept, and creates the institutional substrate from which the full APSCI can be re-launched.

Closing

The compute that frontier-AI safety research requires is available. The funding to assemble a credible safety-only cluster is coming online. The precedents for the funder coalition, the legal structure, the multilateral governance, the operational architecture, and the host country exist. The community that would populate the institution is converging on Singapore, and the political moment that we are living through suggests it as a good landing place.

The hard problems that have stalled previous attempts at multilateral AI safety infrastructure came down to two issues: how to convince funders that GPUs are as important to our shared future as vaccines and malaria nets, and how to structure safety as a global public good, rather than a national project.

Some version of this institution will be built soon, and deliberately choosing to include the open-weight model makers in China is a necessary step in securing a future of safe, responsible AI development.

]]><![CDATA[Searching for Amanda Askell with Chinese Characteristics]]>https://www.caithrin.com/p/searching-for-amanda-askell-withhttps://www.caithrin.com/p/searching-for-amanda-askell-withWed, 20 May 2026 14:11:40 GMT

Slavish obedience to a recipe robs one of the license responsible for its creation.
— James Beard

In April I spent several weeks visiting nearly every open-source lab in China. Along the way, I met people who embodied different aspects of why Chinese labs are the way they are: nervous comms folks; quiet, introspective, brilliant researchers; strong, confident leaders; and neurotic but optimistic product managers.

Each of these people represented a piece of the puzzle of what makes a Chinese frontier lab: the internal structures, the hopes, the dreams, the desires, and the ability to communicate with the outside world. We also looked for the missing people in the landscape: the Yudkowskys, the Karpathys, the Demises, and the folks you would expect to see in the Chinese AI landscape who, for whatever reason, were absent.

In my opinion, no one was more remarkably missing than the Chinese equivalent of Amanda Askell, the Claude philosopher. I asked labs, “Who is responsible for the character of your models?” and was often met with confusion.

In Hangzhou I sat down with the post-training lead of one of the labs and asked him who was responsible for the model’s character. He paused. He asked what I meant. I gave him Amanda’s job description, compressed. He started telling me about the team that handles content compliance. I clarified: not refusals, character; what kind of presence the model has, what it cares about, what it declines to do not because policy says so but because the model would rather not. He thought about it for a few seconds and said: nobody. He said it the way you might say “we don’t have a yoga room.” Not defensive or embarrassed but instead, contemplating something as if for the first time.

In sixteen lab visits, I did not find a single person doing Amanda’s job. In sixteen lab visits, I did not find a single Chinese lab that is not completely obsessed with Claude, its constitution, its character, and its soul.

So if we all love Claude so much, why do we refuse to follow the publicly accessible path responsible for its creation?

what I think Amanda does

Amanda is Anthropic’s in-house philosopher and the lead author of Claude’s published constitution. The Wall Street Journal put it: her job is to “teach Claude how to be good.” The New Yorker called her team’s work Claude’s “soul.” She runs the team that thinks about who Claude is, what it cares about, what it refuses to do, and how to train it into being someone rather than something.

I like to think about her role as dramaturge: the person responsible for tying the audience and the actors together to make sure that a professional production shines. I think this is the most potent metaphor for what she does every day, and it is a fascinating role to have within an organization like Anthropic.

Recently she said she’s training Claude to be:

A well-liked traveler who can adjust to local customs and the person they’re talking to without pandering to them. They’re often very open and thoughtful.

For a lab that is not exactly what I would define as open, Amanda’s published record is, by the standards of Anthropic, remarkably good. She’s interviewed with Lawfare and the New Yorker; she writes posts on the Anthropic website detailing Claude’s character. The Constitution itself is written clearly and simply, and it’s an effective document. Her own Twitter feed has been disappointing because of the idiots on the internet, but if you go back a bit, you can see her thinking in public in a really brilliant way. Her time on Lex’s podcast was very good, and I think it’s one of his best episodes.

Amanda is the exception to the rule for a lab that notoriously does not publish. Her work is surprisingly available and reproducible if you read between the lines of what she does every day. She’s not hiding her role at Anthropic. Rather, she’s encouraging other labs to do the same.

So the question is, why is this not reproduced in the open-weights labs in China? You may not have the exact methodology, but you have the outcomes of those methodologies. You can read every interview and understand:

  • training-data construction

  • converting a constitutional text into RL signal

  • evaluation for character stability

It’s pretty clear what path Anthropic took here: leveraging Amanda’s brilliance to give Claude its soul.

two paths to Claude

Everyone in China is obsessed with the idea of closing the gap with Anthropic. It’s no secret that there’s some distillation happening; Anthropic itself has written extensively about this. ChinaTalk has done a great job covering how it happens, pragmatically speaking.

The most obvious way that this is happening is through distillation: sampling outputs from a teacher model and training your student on the signal. Every serious training pipeline does some version of this, and Chinese labs do it more aggressively than Western ones, but Western labs do it too. It works everywhere, it works well, and it has been a consistent part of LLM development for many years. It’s not going anywhere.

I’m not technical enough to understand distillation perfectly, but my impression is that it produces a competent generalist with a surface-level texture of the teacher. It captures the teacher’s helpfulness, refusal patterns, and hedging style. If the teacher is verbose, your model is verbose. If the teacher hedges, your model hedges.

From a safety perspective, it is probably a net benefit that everyone is distilling from Claude. Anthropic seems to take safety more seriously than almost any other lab, and I think they are creating the conditions whereby other open models inherit this precaution through distillation.

But this is just one path to Claude. The other runs right through the hiring, empowering, and trusting of an Amanda-Askell-like figure within a Chinese open lab.

Imagine, for a moment, taking the inputs — her interviews, her constitution, the philosophy papers that inform her worldview — and giving these real consideration as a Chinese lab. Hiring somebody from the Chinese philosophical world to play this role in your lab. Empowering them with compute, with time, with a team, and with resources to make the character of your model shine just like Claude does. I know that character training is not cheap from a compute perspective. Still, I firmly believe this approach has been undervalued and understudied. I would love to understand who among the frontier labs in China will be the first to take the leap and have a figure like this on their team.

it’s not like China can’t do character training…

One of the great ironies of this problem is that the persona AI market in China is enormous. MiniMax is one of the best labs that we visited on the trip, and they have an uncomfortable amount of their revenue still coming from, let’s say, companionship.

The teams at MiniMax that are shipping these companions understand very well how to do persona. This is not exactly Amanda-Askell-level soul creation, but it is not an unfamiliar process for many of these labs. The teams that ship these characters have more cumulative engineer hours on persona stability and emotional register than almost anyone at Anthropic does. The nobility of the persona being pursued is not Claude-soul.md level, but it’s not like the techniques are not established within these labs.

Trying to understand where constitutions, published or internal, exist in the Chinese landscape is always difficult — they could very well exist and just not be shared with me. I don’t claim some comprehensive knowledge of the Chinese AI stack, and I certainly would not be able to sketch you a map of what their post-training process looks like.

I find this to be odd, because China has a long and serious intellectual tradition of organizing institutions around stated documents. Every major Chinese institution of the last century has an explicit principle stack at its heart, and labs can tell you their values very clearly. There are just no visible resources put together to attack this problem directly, as of now.

emergent character — Kimi & DeepSeek

I remember very well playing with R1 in January of last year, and how much it had a voice from the jump. I remember very well playing with Kimi’s landmark release and seeing just how good it was at creative writing compared to its peers.

R1 was, for lack of a better word, adorable. It was earnest in a way that few other models have ever felt. When the model thought, it thought audibly, and what it said while thinking was strikingly anthropomorphic. It would hesitate, back up, say things like “Hmm, did I make a mistake?” and start over. There were small moments that read like insight. The whole inner monologue gave the impression of a mind working honestly through a problem rather than an oracle stating an answer.

I think it’s very easy to trace a line between that quality of adorableness and R1’s success.

R1 was an example of a Chinese frontier lab that shipped a model with a distinctive character, and users responded as if that character mattered. I think DeepSeek did an excellent job. Based on my exploration of this space and my conversations with their team, it seems they think more structurally about the quality and character of their model than almost anyone else in China. Having never spent time with Liang Wenfeng, I don’t want to join the throng of people speculating on his leadership, but I do believe that the man cares about the character of his model. Looking at his public record, it’s clear he is not someone who only thinks about pre-training optimizations.

Similarly, at Kimi, K2 has always seemed to me to be an emotionally fluent and stylistic model, although its English-language writing still emulates Claude to a large extent. I still think this is constitution by distillation, not constitution as a deliberate step in the forging of a brilliant mind and model.

The MiMo models as they stand today do not feel to me like they are focused on constitution and character. That said, I think Xiaomi has a legitimate opportunity to be one of the model labs that fulfills this promise. They have opinions that diverge from the norms of Chinese open models, and their team has a clear sense of what they want MiMo to be, extending beyond just doing well on benchmarks.

Finally, any conversation around character and constitution needs to include reference to ByteDance’s Doubao. This is the most popular persona product in China, and although the model itself is fairly bland, it does display some cultural distance between its responses and those of a typical Western model, even explored through translation apps.

All told, this is a fairly depressing list — we find a distilled version of Amanda, a ghost of a ghost in the shell.

Maotai & counterarguments

I successfully fed a few shots of Maotai to some researchers over dinner and asked them many questions about this thesis. Their responses fall roughly into three categories.

The first is the regulatory issue. Chinese labs cannot ship models that express opinions, disagree with users, or take aesthetic stances because their regulatory environment requires strict compliance with policy. The Cyberspace Administration of China does not want a model that declines on its own account, since the rules say it should not decline.

I think this is mostly wrong. It does not properly take into account what Amanda is doing at Anthropic, and it overstates the structures imposed upon Chinese LLMs. I think this is a cheap and easy way to avoid thinking about the problem. It is not meant as a serious argument; it is meant as a reason not to engage with one. Compliance constrains what a model can refuse to discuss, but it does not constrain whether the model has a voice while doing its job. R1’s adorableness was not a feature of Chinese bureaucracy, but rather a result of the training process. Kimi’s writing is not something that was created to adhere to some regulation, but rather a byproduct of a team’s intentions and the non-zero number of literature majors on their team.

The second argument is that character is not a role that you can hire for in China. They argue that the way Amanda was found is the product of a broader institutional culture in the Bay, within the safety community, among LessWrong, Anthropic’s founding team, and in the internal conversations of the past decade, including Yud’s rants and the strange way that the Bay Area’s culture has ended up being the culture of LLMs. Grok has been trying to remove this value-set from its language model for about a year and a half, with no success. China could not change this even if it wanted to.

I find this argument uncomfortable because there is a real truth to it, but I think the conclusion it points to is the opposite of what these people claim. If character is downstream of institutional culture, then the Chinese lab that wants character cannot import Anthropic’s. It has to build its own out of the intellectual traditions of its own lab, its own writers, and its own philosophers. To believe that this is not possible is to surrender to one of the most tired tropes of Western armchair Sinology.

The third is the most blunt, and it is purely economic. Why would you pay for character R&D when you can distill it for free? Language models are complicated, and hiring philosophers adds another layer of complexity and compute that is hard to justify because it does not show up in benchmark deltas or token costs. Character does not appear on the pricing sheets. From a portfolio perspective, distillation is so effective that asking a leader at these labs to fund an unmeasurable bet in a non-quantifiable direction is unthinkable.

This disregards the history of language models. Every important shift in this field has looked like a non-quantifiable bet right up until the moment it made the sand smarter. Anthropic itself was a non-quantifiable bet in 2021. Character work was never a line item that justified itself on a spreadsheet at any point in its history. It justified itself by producing a model people preferred to talk to, preferred to work with, and ultimately preferred to trust with their lives.

imagining Amanda with Chinese characteristics

Concretely speaking, what would character and constitution look like for a Chinese open model?

A constitution written in Chinese first. A Chinese model in production with Chinese writers writing for a Chinese model, shipping a texture of intelligence that has never yet been seen. I think this has a viable chance of breaking the current arms race and benchmaxxing deadlock that informs so much of the Chinese ecosystem. This is an asymmetric bet that many engineers might scorn, but I genuinely believe it could yield outsized results for a team willing to take a risk.

There are qualities in Chinese thought and language that would be very welcome for many users of language models. I find Chinese to be quite direct, in contrast with Claude’s hedging and vacillating. Anthropic’s character is defined by a very online English-language discourse and a non-trivial amount of LessWrong. I’m not calling for a model to be trained on Confucian ideals, but instead on the character of modern China: brusque, bold, and occupied with a different set of concerns than the West.

Chinese taste would be a welcome addition to the LLM landscape. Chinese models often avoid taking a stance on aesthetic or intellectual judgments. A character-trained Chinese model, however, could express defensible opinions and disagree with users in a way that many Western labs’ products do not, thanks to their sycophantic, groveling tone.

A lab that takes Amanda’s methods and applies them intentionally to a Chinese-language model would ship something that doesn’t feel like a product of Western ideals and thinking. Instead, it would be fresh, original, and uniquely Chinese.

The prize here is a different kind of model. The Chinese open-model race is currently competing mainly on benchmark deltas and token costs, almost obsessively. A model that is recognizably itself falls into a different category, and it would not necessarily be the one with the highest benchmarks. This could be a model people choose to use for both conversation and agentic workflows because it has character, opinions, and a Claude-like quality that draws users. If you think that Amanda’s role at Anthropic does not directly contribute to its dominance in the field, then you have not been paying attention. Character and constitution matter, and the first Chinese lab to understand, intuit, and invest in this idea will make history.

]]><![CDATA[The Singapore Safety Cluster: What Anthropic Should Be Building Instead ]]>https://www.caithrin.com/p/the-singapore-safety-cluster-whathttps://www.caithrin.com/p/the-singapore-safety-cluster-whatThu, 14 May 2026 20:37:42 GMTA response to “2028: Two Scenarios for Global AI Leadership”

I just spent four weeks in China. Beijing, Shanghai, Hangzhou, Shenzhen. I met every lab. I talked to the people training the models Anthropic’s new policy paper wants to strangle. I love Ant’s models and I use them every day, but this policy paper has some serious flaws in logic.

Here is what Anthropic’s policy team either doesn’t know or is choosing not to say.

The compute situation inside Chinese frontier labs is worse than the public numbers

Ant cites Huawei at 2% to 4% of NVIDIA-equivalent compute. That’s directionally correct, but it understates what’s happening at the lab level. National compute totals are not lab-level compute. Xiaomi has way more compute reserved for things like RecSys than they do for training models. Frontier labs are fighting each other for slices of a pie that’s already a fraction of what a single US hyperscaler has.

In every lab I visited, across all of China, one thing was constant:

Every available H-equivalent hour is going to post-training.

I estimate these labs spend about 1% of their compute on safety. That number will not go up as long as there is no compute available for them to do so, because the results they get from spending the same compute on post-training are too good to pass up. The Chinese AI world is cutthroat and brutal, far more competitive than the Anthropic / OpenAI closed-model rivalry you see in the West.

There is precious little compute available. There are not enough chips in the buildings to do the kind of safety research Anthropic is famous for. This isn’t a money problem. Some of these labs are public. All of them are capitalized.

They cannot buy the chips.

The export control regime works at the hardware layer, not the capital layer. A lab with $5B in cash cannot turn it into compute because Jensen will not ship and the smuggling channels are narrowing.

So when Anthropic’s paper points at the 94% compliance number on DeepSeek and the 3-of-13 safety eval publication rate and frames it as evidence of Chinese AI recklessness, what I am reading is a description of structural compute starvation.

You cannot publish safety evals you did not have the GPUs to run.

You cannot lower compliance numbers without alignment compute you do not have.

The Anthropic paper treats this as a culture problem. That is fair to some extent, but the solution does not run through changing culture. It runs through changing constraints.

What the 2028 paper actually advocates

  1. Tighter compute denial (close offshore data center loopholes, tighten SME, escalate enforcement)

  2. Anti-distillation legal architecture (criminalize the one channel by which US safety techniques diffuse)

  3. Export US models globally (lock in commercial primacy)

You’d think a lab with the word “safety” on every other document would have thought about how to do safety research cooperation. But no.

No proposals for shared evals.

No joint red-teaming.

No common interpretability standards.

There is no acknowledgement that frontier AI safety is a global property of deployment, not a national property of development.

What we should actually do

If you genuinely believe frontier AI poses civilizational risk, and you genuinely believe Chinese labs are about to deploy a generation of models so powerful they are profoundly unsafe, you think the answer is to starve them of compute? That’s cope.

Here is the plan I wish Ant had the courage to do.

The Asia-Pacific Safety Compute Initiative (APSCI)

Location: Singapore. Neutral jurisdiction, common-law legal system, world-class data center capacity, equidistant from Beijing and San Francisco both politically and geographically. Importantly: very low risk these chips end up in PRC hands.

Scale: 25,000 H200-equivalent GPUs. Roughly a serious mid-tier training cluster. Funded by a consortium of Anthropic, OpenAI, Google DeepMind, DeepSeek, Zhipu, the Qwen team at Alibaba, Moonshot, and the Singapore government via Temasek. No single party more than 20%. Annual operating budget around $400M.

Mission: Safety research only. Evaluations, red-teaming, interpretability research, scalable oversight experiments, RSP-equivalent stress testing, and dangerous-capability evals (CBRN, cyber, autonomy, persuasion).

The three guardrails that keep APSCI on safety, not capability

The threat model is simple. Most of the models that would run here (DeepSeek, Qwen, Kimi, GLM, Llama, Mistral) are already open-weight. Nobody is trying to steal them. The risk is that “safety research” becomes cover for SFT, DPO, or RLHF runs that hand the lab a stronger fine-tuned checkpoint at the end. Three rules close that door.

1. No gradient updates to uploaded models. Inference and analysis only.

The orchestrator rejects any job that produces a weight delta on a frontier model. Allowed: forward passes, activation patching, attention probes, SAE training where the SAE is a separate small model that stays on cluster, red-team adversarial prompting, eval suites, capability elicitation via prompting. Forbidden: SFT, DPO, RLHF, RLAIF, LoRA, QLoRA, any adapter training, any optimizer step that touches the uploaded model’s parameters. If a lab wants to fine-tune, they do it on their own compute at home. APSCI exists for the work they cannot afford to do domestically, and that work is not fine-tuning.

2. Egress ceiling: bytes-per-job small enough to fit research artifacts and nothing else.

Whatever leaves the cluster passes through a per-job size cap calibrated for the legitimate outputs: eval scores, activation maps, interpretability features, paper drafts, datasets of model behaviors. A 7B-parameter LoRA does not fit through that pipe. A full fine-tuned checkpoint does not fit through that pipe. The cap is enforced at the storage layer, not as a policy. There is no API call that returns more than the ceiling, regardless of who is asking.

3. Every job is public by default.

This is the CERN rule, adapted. Job submissions, workload type, model loaded, compute hours consumed, and output artifacts go to a public ledger with a 90-day publication deadline. There are no private runs. If a lab is submitting 10,000 forward passes over prompts that look like an RLHF preference dataset, the world can see it and call it out. The transparency is not symmetric audit between consortium members trying to protect secrets from each other. It is open-world transparency on what is being done with shared compute. Same logic that keeps academic preprint culture honest.

Who actually runs it

The cluster needs a credible neutral operator. Two viable models.

Option A: A*STAR plus AI Verify Foundation (Singapore). Singapore’s national research agency plus its existing AI governance institute. Operationally credible, politically neutral, has the infrastructure relationships, and the Singapore government is already positioning itself as the Geneva of AI governance. This is the strongest option.

Option B: A consortium-governed nonprofit operator. Modeled on CERN. Member states and member labs jointly govern via a council. A Technical Safeguards Committee handles workload approval. More credible than a pure private operator, slower to stand up than Option A.

Supporting technical operators under either governance model: METR for dangerous-capability evals. Apollo Research for scheming and deception evals. The UK AISI and US AISI for shared eval protocol development. MATS-affiliated researchers for interpretability fellowships on the cluster. Carnegie Endowment or RAND for the policy-research overlay.

What this costs Anthropic

Roughly $80M per year as a 20% consortium member. Less than one frontier training run. Anthropic raised $13B last round. The financial argument against this is not a financial argument.

What this buys Anthropic

The thing they say in public they care about: a world where the labs that will train frontier models, and they will, with or without US permission, on Huawei chips and smuggled H100s and whatever they can build, have done the safety work before they ship. The diffusion of constitutional AI, RSPs, eval methodologies, and interpretability tools into the labs that need them most.

The thing they will not say in public they care about: a continued moral high ground that survives contact with the actual policy implications of compute denial.

The test

Anthropic will not do this. I am confident of the prediction.

They will not do it because the company’s revealed preference, as expressed in today’s paper, is for the policy environment that maximizes their competitive moat, not the one that maximizes global safety. The Singapore cluster is the obvious move if safety is the binding constraint. Its absence from the 2028 paper is the proof that safety is not the binding constraint.

If I am wrong, I will say so publicly. The way Anthropic proves me wrong is by announcing APSCI, or something like it, with real money and real GPUs, this year.

Until then, the labs I visited in April will keep doing what they showed me: burning every available H-card on post-training because they have no other choice, shipping models with safety properties that reflect the compute constraints they are operating under, and skimming American policy papers that tell them they are the reason the world is unsafe.

The compute they need to do better is sitting in US export-controlled inventory. The framework for getting it to them safely exists. The capital to build it exists. What is missing is the will.

That, not Chinese recklessness, is the actual 2028 story.

]]><![CDATA[Who will explain the machines of loving grace?]]>https://www.caithrin.com/p/who-will-explain-the-machines-ofhttps://www.caithrin.com/p/who-will-explain-the-machines-ofThu, 02 Apr 2026 22:51:15 GMT


The Dark

Seventy-nine percent of Americans don’t trust AI-generated information. Fifty-seven percent believe the costs of artificial intelligence outweigh the benefits. Nearly half don’t trust either political party to handle it.

If you’re inside of Silicon Valley, there’s never been a better time to learn about AI. People like Karpathy are forgoing decamillion-dollar annual salaries to patiently explain to us the ways for elites to use AI to better their businesses, their lives, and their fortunes.

In the rest of the world, four out of five people use something every day they don’t trust, can’t explain, and believe might kill them. During the atomic age Oppenheimer’s radical enthusiasm inspired people to plant atomic gardens in their schools and excitedly learn about the wonders of this technology as it spread forth from Los Alamos.

The AI age gives vastly more benefits to the average person and is vastly less popular than a technology that announced its arrival into the world with a mass killing of civilians.

The introduction of cars made people nervous. The introduction of electricity made people nervous. Nothing compares to the bitter hatred that the average American, especially on the left, feels towards artificial intelligence systems. Jasmine Sun was writing about this on Substack today; how the world immediately reacted with vitriol to an offhand mention in her Atlantic piece about how she used Claude for editing.

an entire country is living inside a technology that it doesn’t understand, doesn’t trust, and has no credible guide to. The most common way to learn about AI is by using AI, which will never be and can never be a suitable replacement for a beloved public educator.

and the media- the very institution designed to explain to people changes in their world- is not rising to the challenge. there are reasons for that. Reason number one being that the very business that’s building AI (tech) destroyed the media business model abruptly and forever in the early 2000s.

Karen Hao’s Empire of AI — a National Book Critics Circle finalist, the most prominent book about the industry published last year- contains a 1,000x error on water usage by data centres. that error has been repeated in dozens of outlets since and it will continue to be repeated because mainstream AI journalism doesn’t have a correction mechanism, only confirmation mechanisms.

Ask anybody in Silicon Valley if East Coast journalists know what’s going on and they’ll laugh until their sides hurt. The people who are working inside of Silicon Valley alongside their peers, trying to understand this revolution from the perspectives of technology, of anthropology, of civic virtue- they are currently being bought and sold like crypto tokens in a marketplace obsessed with the idea of narrative control.

I am not defending the AI industry. The AI industry is sometimes terrible and often lying, but are simultaneously doing the most meaningful work that one can do on planet Earth in the year of our Lord 2026. the issue is that both the builders of these products and the people who are chronicling this build are locked inside of a self-perpetuating loop of money and information that is not producing public goods. The landscape is not “industry vs. journalists.” The landscape is a public that has been failed by both, while they feed on each other.

The Roll-Up

This morning, OpenAI announced that it has acquired TBPN — the daily tech talk show hosted by John Coogan and Jordi Hays — for what the Financial Times reports is a “low hundreds of millions” of dollars. TBPN is on track to generate over $30 million in revenue this year. It’s the show where Silicon Valley power players sit for three hours and speak candidly to fellow insiders.

The show will now report to Chris Lehane, OpenAI’s chief global affairs officer — a political operative by training. OpenAI says TBPN will maintain “editorial independence.” The show will “run their programming, choose their guests, and make their own editorial decisions.”

But the acquisition is not a surprise event- it’s the logical conclusion to a set of incentives given to Frontier Labs and AI companies- occupy the most important ears in California while they pedal their pelotons and wash their dishes at modest cost.

Andreessen Horowitz (one of OpenAI’s investors) led Substack’s Series A. Substack is the platform where the majority of independent AI writers publish. A16z also acquired the Turpentine podcast network last year — 30 shows, including some of the most popular AI podcasts in the world. They’ve built a New Media operation producing content five days a week across 40+ shows and 500,000+ newsletter subscribers, all of it free. All of it structurally aligned with a16z’s portfolio companies. All of it designed, in a16z’s own words, to help “the next cohort of great founders preferentially attach to our portfolio companies — and to A16Z itself.”

One of A16Z’s New Media partners — Brent Liang — came directly from TBPN before joining the firm.

A16Z invests in OpenAI. A16Z invests in Substack. A16z acquires Turpentine. OpenAI acquires TBPN. A16Z’s people move between these entities. And at the centre of all of it is a $40 billion venture fund that has stated, in writing, that its media operation exists to serve its financial interests.

This isn’t some kind of dark conspiracy. It’s the result of A16z being full of incredibly smart people that understand the direction of travel of the world today and understand very well the value of audience in the AI era. Its not a kind of tangled conspiracy web- I admire the speed of their execution and the degree to which they’ve been able to accomplish what seems to be a very smart strategy in a very short amount of time.

What’s happening is that the elites of the AI world- the people who build it, fund it, and profit from it- are buying the conversation about themselves. Again, not evil- fiscally sound. $300-or-whatever million for TBPN when you can raise hundreds of billions of dollars in venture capital is so insignificant that it hardly bears mention. the ROI is extraordinary. When you believe your technology will reshape civilization, a few more degrees of narrative control is righteous.

My problem is that 79% of Americans are not in the room. They are not watching TBPN. They are not reading A16Z’s Substack. They are not attending invitation-only summits in Utah. They are using AI tools they don’t trust, reading headlines about water consumption that are wrong by three orders of magnitude, and watching the most consequential technology of their lifetime get explained by the people who profit from it.

Forty-seven percent of Americans don’t trust either party on AI. Twenty-three percent trust the Democrats. Twenty-five percent trust the Republicans. That leaves a majority of the country that trusts no one to handle this for them. Not the companies, not the government, not the press.

That is the actual crisis- not the paperclipping. The absence of anyone credible to explain it before AI runs 40% of GDP and 70% of decisions globally.

The Sagan Problem

In 1980, a gangly astronomer from Brooklyn hunkered down on the edge of a cliff overlooking the Pacific Ocean and told 500 million people in 60 countries that they were made of star stuff.

Carl Sagan did not discover the cosmic microwave background radiation, or build the Voyager spacecraft. He did not originate the theory of nuclear winter. What he did was something that the scientific establishment considered beneath it and the public considered essential: he just explained it all without making you feel stupid.

I don’t believe he recorded a lot about his motives but I believe that he did this because he thought that the public had a right to understand what was happening to them, with their money. Perhaps if the space age had been paid for by venture capitalists and not the public purse, this would have been a different story.

I like to believe he thought that democracy could not function if the most consequential science of the era was only comprehensible to the people inside Kennedy Space Center.

he ate a lot of shit for this- Cornell nearly denied him tenure. Ordinary people’s comprehension of this made it feel “unserious” and somehow science that’s broadly understandable lacks the virtues of science that’s dense and expert. They were wrong. Sagan’s work — Cosmos, The Demon-Haunted World, Pale Blue Dot — did more to build scientific literacy in America than any paper published in Nature. what I love about his work is that it serves an audience and it serves an audience faithfully without sneering.

I so badly want to see the emergence of a Carl Sagan for AI. someone who understands this world, who can explain it patiently and carefully to people like my grandmother and people like my arch-conservative uncle or my communist cousin. I fear the AI world will yearn for this figure too late, because they believe that we’re all going to enter a singularity so quickly and abruptly that this explaining is hardly worth the effort. that argument feels credible to me; it’s similar to our stance on regulation. if we just build the machine god, then all these other problems can be kicked down the road.

A credible, independent, accessible voice explaining what is actually happening in the halls of Anthropic and Open AI for the public as a public good, with no portfolio to push or corporate backers or mixed incentives- this is what I dream of. and I fear the consequences of not having this person in place by the time Opus 6 goes live. I’m also quietly terrified that Eliezer is the default if we don’t find someone better.

Sagan understood something that the AI industry’s media buyers and the journalism establishment have both forgotten: the public is not stupid. The public is underserved. Given a guide they can trust, people will do the work of understanding. Given only propagandists and prosecutors, they will do what 79% of Americans have already done — use the technology anyway and trust no one.

The work of creating honest, independent, populist AI education is not a high-ROI opportunity, but rather a national resource. America cannot maintain its position in AI- cannot regulate it wisely, cannot deploy it equitably, cannot compete with nations that are educating their populations about it- if the only people who understand it are the ones getting rich from it.

Someone has to be the person on the cliff, looking out at the Pacific, explaining that we are made of star stuff.

]]><![CDATA[Thirty days without a smartphone. ]]>https://www.caithrin.com/p/thirty-days-without-a-smartphonehttps://www.caithrin.com/p/thirty-days-without-a-smartphoneWed, 04 Mar 2026 16:43:12 GMT

fortunate sun

I spent most of this winter in Sayulita, Mexico- one of my very favorite places to run off to for an extended working vacation. Sayulita has great restaurants, an amazing surf break, a beautiful beach, and quite a good co-working space.

On my second week there, my phone went missing from my pocket. I don’t know for sure if this was my carelessness driving a quad with the smartphone in my shorts, or more likely, someone who snatched it after I took some photos. No matter what happened, I was faced with a grim situation- for Sayulita has many fine qualities, but it does not have an Apple Store.

The phone is unbelievably locked down. I had zero worries about hacking or any such business. I watched it travel around in a little mini map in Find My Phone, visiting beaches and houses before finally going dark.

I hunched over my laptop, feeling a grip of panic. Every moment I got up to make coffee, to use the bathroom, to take some sun in the courtyard, my fingers glanced against my right-hand pocket, searching for the Dopamine Rectangle.

phantom limb

I have had a long and complicated relationship with smartphones. When I first saw an iPhone in the hands of my friend Rebecca Foon outside the Cagibi restaurant in Montreal, almost twenty years ago now, I desired the device with a furious passion that I did not understand. I remember looking at it, marvelling at it, and then pulling my tired BlackBerry out of my pocket, knowing already I lived in the past.

I have a real love of work. I adore what I do, and I do a lot of it. My phone is, in many ways, my primary work device, and I feel that, if you’ve properly set it up, you can run a very large business from a very small device. But that comes with some externalities. Ugly ones. Over the last ten years, I’ve become hopelessly addicted to my phone. I have tried to remedy this on several occasions.

In 2022, I spent the entire year in a dopamine reset. My screentime had gone over nine hours a day, and I desperately needed out. Besides quitting other vices, I also put that phone in black and white, and aside from a few month-long relapses, I’ve kept it that way ever since. I feel like I had a healthy relationship with my phone. What I mean by healthy is 3 1/2 to 5 1/2 hours a day, every day, for more than ten years.

And suddenly, I had the chance to make that zero.

a window of opportunity

As I sat there in Sayulita, panicked about my lack of device, thinking about who would go to an Apple store stateside and mail me one, etc I had a sudden revelation. There may never be another time better than this to take a little break from my phone.

I am in a paradise, and a paradise that functions well without access to a phone. Mexico is not a place that runs on apps. I am within a hundred feet of a wonderful surf break. More importantly than all of that, I am now addicted to something else: Claude. My screen time has already been going down as I use more and more my laptop to work because the efficiency of Claude is just that good. Here I was sitting in a beautiful place with a new workflow not only not even three months old, and a device doing its own little Mexican vacation. Now is the time to take a little break if I could handle it.

the first days

I am not somebody who keeps their phone in their bedroom, that’s a rule. I start my day with coffee and some contemplation before I ever touch the device, although I am guilty of bringing it back to bed sometimes to watch YouTube in the evening. The first few days, I woke up immediately craving my phone, something that would never happen when I knew where it was.

Its absence was a curious phenomenon. I could go surfing for three hours in the morning and not think about my phone one time, but give me a few moments sitting quietly in the sand, drinking a coffee while the sun came up, and I would be losing my mind about its location, desperate for it.

It truly did feel like a phantom limb that was missing. I would pat my pockets compulsively. I would think about it every time a moment passed where I had no stimulation in front of me. In my mind I was constantly out of touch, constantly missing messages from friends, constantly behind the ball. I lacked even the most basic courtesy to respond to people. I was missing everything. Everything important in the world.

the revelation

And then all of a sudden, it was wonderful.

Work, friends, family, responsibilities all lived in my laptop. The present moment, the streets of Mexico, the surf, the music around me- that was where I lived.

I felt a great weight lift from my shoulders. I felt connected. I felt childlike, given that my entire adult life, more or less, had had one of these devices glued to my hip.

Little changes in my behaviors that are manifesting like crazy. Music made me wanna move and dance. People-watching became profound, fascinating, compelling. The absence of phone meant the absence of headphones, which means I talked to about 10 times more people every day than I would with my phone on me. Strangers came up to me in the street to strike up a conversation. I got asked out on dates. Old people told me stories. The taqueria downstairs from the co-working I used was suddenly full of smiling employees that knew my name.

Life, in short, got richer and better.

I think two things caused this- my my presence in my environment, and the ease by which I was living opened me to these experiences. Seeing somebody reading a book or just quietly staring into the morning street begs an invitation. Headphones, screens, second words do not.

cartel collapse

A glorious month went by, and then it came to an end. the death of a cartel boss meant chaos in my little surf town, and suddenly I needed to leave. Returning back to the US from meetings made me realize that there’s no way you can operate in the U.S. without a smartphone at your hip- everything is too reliant. I landed at the San Francisco airport and tried to take a cab down to Hayes- the cabbie had no change, the car was rancid, he asked me to tap my card in the world’s most suspicious-looking app to pay, and he looked nothing like the photograph of the licensed cab driver on on the little sticker. this lifestyle was going to come to an abrupt end.

My experiment was over. For the first time in my entire adult life, I navigated 30 days with no phone. today, a new one arrives from Apple, and in moments I’ll be hooked back up to the dopamine rectangle, being drip-fed notifications from Twitter, picking the little red berries that grow on my apps, and this will seem like a distant dream.

🎂

]]><![CDATA[Dispatches from Claude Psychosis: Episode 2]]>https://www.caithrin.com/p/dispatches-from-claude-psychosis-53ahttps://www.caithrin.com/p/dispatches-from-claude-psychosis-53aFri, 30 Jan 2026 17:43:11 GMT


Claude, Kill the Cal

My calendar is nearly empty. I’m doing the work of four people. Something is dying.

I keep thinking about Paul Graham’s 2009 essay, “Maker’s Schedule, Manager’s Schedule.” It’s become shorthand in tech circles, but for those unfamiliar, (aka Zoomers and Europeans) there are two fundamentally different relationships to time. Managers operate in hour-long blocks. Meetings are the unit of work, the calendar is the instrument of productivity. Makers need half-day minimums. A single meeting can blow a whole afternoon, Graham wrote, “by breaking it into two pieces each too small to do anything hard in.”

Building SAIL over the past months, I’ve been as productive as a small engineering team would have been a decade ago. If you asked me three years ago how to become twice as productive, I would have told you by necessity that I would need to add a certain number of hour blocks to my “Manager” schedule. What I actually needed was Claude Code and the maker mindset that can enjoy dozens of layers of abstraction humming triumphantly with a swarm of agents buzzing around, a crystalline hive of concentration, focus, and peace.

And each time I have to return to those segmented hour blocks of time, some part of me is destroyed. Some version of work where I dance with Claude, layering abstractions in his memory and in mine, creating beautiful things quickly and efficiently, crumbles away. I feel myself moving away from meetings, retracting from them. Not because I don’t enjoy them but because nothing can compare to the productivity of a beautifully quiet room, a cold cup of water, fast internet, and Claude.

From 2009, when Paul wrote that essay, to today, “Manager Time” has seemingly won. Everyone’s world was chopped up into blocks of time and processed like that. But there’s no way that in the world that is emerging, we will end up living in manager time. The status of manager time will invert. The way that all of these things work will change forever. I feel it building, and so do all my friends. Coordination is getting automated.

The evidence is everywhere if you’re looking. But to see it clearly, you have to understand how deeply manager time shaped the structure of capitalism itself.

The twentieth-century corporation was built around the coordination problem. You couldn’t have a hundred engineers working on a product without someone, many someones, synthesizing their work, resolving conflicts, routing information up and down the hierarchy. The org chart wasn’t arbitrary. It was a solution to the bandwidth limits of human communication. Middle managers existed because makers couldn’t coordinate at scale without them. They were the information routers, the context translators, the “human APIs” between layers of the organization.

This created its own logic. Career ladders pointed toward management because management was where the leverage lived. Venture capital funded teams, not individuals, because a solo founder couldn’t ship at the scale that mattered. The entire apparatus of modern business, the meetings, the hierarchies, the calendars, the headcount as success metric, emerged from the brute fact that coordinating human effort was expensive and slow.

Manager time wasn’t imposed by malice but by physics. Information moved at the speed of human conversation, and so the people who controlled conversations controlled organizations.

If AI handles coordination, what remains? Not management. Making.

The Duo Founder (You & Claude)

Solo-founded startups rose from 17% of all startups in 2017 to 36% in 2024. Midjourney hit $200 million in revenue with roughly 40 people and zero venture capital. Base44 was built by one person over six months to $3.5 million ARR, then acquired for $80 million. Sam Altman is betting publicly that the first one-person billion-dollar company is coming soon.

The firm itself is a coordination technology. When coordination costs drop, firm size drops. “Company” starts to look more like “project” or “campaign.”

I see this in my friends and coworkers. The AI researcher who started SAIL with me seems increasingly tortured by his calendar. He seems intent on liberating himself from a manager’s schedule. Even things that he looks forward to, if they have an hour that they need to arrive at, feel heavy in the day. Far heavier than they did before Claude.

People who have existed on a maker’s schedule for a long time now feel acutely the cost of context switching. It’s never been heavier to walk away from the singing swarm of agents back into a meeting where you’re updating a stakeholder or pitching somebody on a new idea. The pace of that communication feels lethargic, slow, backwards.

Calendar as Liability = Inventory as Waste

A remarkable change of attitude in manufacturing in the last century was the idea that inventory is waste. This is the central insight of the Toyota Production System, sometimes called “lean manufacturing.” Before this, inventory was an asset on the balance sheet, quite literally. Warehouses would sit full of parts, meaning the company was prepared, capitalized, ready. The inventor of the Toyota Production System saw it differently. He saw it as waste. It ties up capital, hides quality problems, and creates obsolescence. So the goal became not to have a full warehouse of inventory but to have an empty warehouse, which created efficiency.

We’re about to experience that same inversion with the calendar. Meetings hide the absence of output. Meetings are a buffer against accountability. Maker time exposes who actually ships. We’ll see this shift in the coming months, and it won’t be pretty for the generation of people who believe that meetings are the central way they show economic value in the world.

Prestige as a Maker

In this world, managing people is not a measure of success. “Principal Engineer” and “Distinguished Architect” are examples of makers who enjoy genuine prestige in these roles, but there are precious few people in our current job market who get to enjoy roles like that.

The independent creator, the solo founder, the consultant who ships: all are going to skyrocket in popularity because the corporate world, built on “Manager Time,” can’t figure out what to do with all of these makers that will arise in the Claude revolution.

The manager class are people whose identity is built on coordination, access, and information-routing. They are watching their core functions automate. These aren’t bad people or useless people. Many of them are skilled at things that genuinely mattered: translating between technical and business contexts, maintaining organizational coherence, mentoring through proximity. But the ground is shifting beneath them.

There are people I know who are struggling with this transition. They thrived in meeting culture because they were genuinely good at the human coordination that meetings enabled. Visibility, facetime, stakeholder management: these were skills, hard-won and real. In maker time, those skills depreciate.

Maker time requires discipline, self-direction, and tolerance for ambiguity. Not everyone has these traits and not everyone can develop them. I foresee a whole lot of people who trained themselves carefully to manage makers starting to understand their obsolescence in the coming months. Skills of navigating politically between stakeholders and coordinating synchronicities in capitalism will seem so obsolete in the face of the raw creative power of a good maker.

My calendar is nearly empty. I’m doing the work of four people.

I don’t think I’m exceptional. I think I’m just a little early.

]]>