The Road to Serfdom (again)
LF221 | As consumers we gave Big Tech our digital lives. Now as employees we're doing it again. Perhaps its the time to take federated intelligence seriously.
In addition to the weekly Four For Fridays, Looking Forward publishes occasional pieces on topics that need more room to breathe. Today’s subject is one of those - how organisations are becoming AI operating systems. And how workers need to know this, and know their value.
AI-first companies are turning knowledge workers into cognitive manual labourers - throwing coal into the furnace of a corporate brain they won’t own, or benefit from in the long-run. Tim Berners-Lee’s dream of personal data pods, almost twenty years late, is one of the ideas worth investigating before it’s too late.
It took most of us two decades to realise that the price of free search, free maps and free social media was us. Our hopes, our fears, our late-night searches, locations, lovers and secrets. All of it harvested, modelled and sold to advertisers as targeted persuasion. Yanis Varoufakis calls this technofeudalism. Law courts and policymakers are belatedly waking up to the harms done to our children and collective mental health, but it’s hard to turn this ship.
The first round of the deal sold our digital exhaust as consumers. The second round, getting underway, sells us as employees.
Hayek warned that central planning would erode our freedom. The AI-first corporation may be the unlikely route to the same destination, via the boardroom rather than the politburo.
If you’re also down the AI rabbit hole your LinkedIn and Substack feeds, like mine, will be bursting with posts urging CEOs to make the move to be an “AI-first” organisation, building a second brain for the company. We’ve heard of cognitive offloading - this is cognitive uploading, the company superbrain gets the contents of its employees’ brains.
It’s still early days for this and new ideas are emerging every day. For those obsessed by this idea, we jump on the latest head-spinning tweet by Karpathy, or Git release by Y Combinator CEO Garry Tan, point our Claude Code at it and say build this. Every few days there are new approaches, and Google’s just announced Open Knowledge Format, to make this kind of thing easier. Results will vary, and if I’m honest it’s a work in progress.
Capital loves an AI operating system
The idea that a company can become a continuously-learning organism, a ‘world model’ of itself and its customers. Every email, every meeting transcript, every Slack thread, every decision is hoovered into a central brain that gets steadily smarter at running the business. Someone connected his Whoop to his calendar to figure out which colleagues stressed him out; this could soon be table stakes, as the borg optimizes all interactions.
Although there are teething problems, expect this to rapidly become a shiny new thing - an experience that will hoover up calls, data and insights into the Brain. Many tech companies are building versions of this internally.
Jack Dorsey, who knows a thing or two about building companies - he was CEO of two public ones at the same time, Twitter and Block - is one of the most articulate advocates of the model. Y Combinator is another. Block’s recently announced a major restructure - cutting 4,000 of its 10,000 employees, in favor of the AI way. It replaces traditional middle management with three roles: deep-specialist individual contributors, temporary cross-functional DRIs focused on outcomes, and player-coaches who mentor while still building. AI world models do the coordination, the prioritisation, the information-flow plumbing. Humans get judgement, creativity, ethics and the messy reality at the customer edge. To me, this org chart he described felt like a glowing super ovum surrounded by a swarm of furiously hard-working (and mostly unrewarded) sperm, each armed with Claude Code and a Notion subscription.
The problem: humans are not meat machines
Marvin Minsky’s quip that humans are ‘meat machines’ is a sentiment held by many of the billionaires whose hands are on the levers of society today. Humans are just machines that clank less loudly. This reductionist approach ends up missing a lot of what makes life worth living and humans so interesting.
It doesn’t reward human-to-human collaboration. The Roman-Legion organisational chart - top-down, hierarchical, designed to filter information through humans because humans were the only context-routing infrastructure we had - was always a poor fit for genuinely creative work. Replacing the human router with an AI router improves the routing, but it does little for dense, lateral, often unstructured human-to-human work that actually generates new ideas and strategy.
It risks skewing to visible metrics. Directly accountable individuals being given metrics mean they’ll obsess about that metric. Goodhart’s Law tells us that’s likely to drive optimisation for the metric, not necessarily the broader outcome. Connecting metrics to strategy and mission is, of course, possible but requires dynamism and tension, which may be missing from a simple application of this idea.
It transfers cognitive ownership upward. This is the bigger one. Every meeting transcribed, every decision logged, every model fine-tuned by an employee’s tacit knowledge is now an asset that lives inside the corporate brain. When the employee leaves - and most will, eventually - the brain stays. The person who contributed the most has the least portable asset on their way out the door.
Another point, related to specifically Block’s decimation of half its workforce - why limit the ambition to today’s strategy. If AI is already giving us new superpowers, surely the new Block strategy could be 5-10x what it is today with the same workforce?
A different approach - federated personal data
Tim Berners-Lee saw something like this coming twenty years ago and proposed Solid - personal data pods, federated across services, with the user controlling who saw what. Patrick Deegan’s been working on an AI-driven version of this too. These entrepreneurs have been fighting a valiant battle against the ad-driven business model of the open web. Instead, we got silos, hate-fuelled massacres and a mental health crisis. Personal health data tells the same story: few people would disagree that individuals should own their own health data, yet there are five trillion reasons why this is hard, and a few plucky startups looking to do things differently haven’t thrived.
It’s not too late to federate employee brains
Unlike with consumer data, the second-brain era is, by contrast, brand new. The defaults haven’t hardened yet. There is a brief, narrow window - a year or two, tops - to course correct.
I was at a Claude Code impact hackathon a couple of weeks ago and ended up in a long conversation with Ben Flint, who shares my concerns. And he’s doing something about it. He’s building Alive Context, a tool that fits inside Claude Code and lets an individual build, structure and carry their own knowledge base - aiming to be a portable second brain, shareable with an employer on terms the individual sets. The example Ben uses: drafting a reply to a client who mentioned, three emails ago, that they have a dog. A federated system can surface that detail from your context, not the company’s. Your knowledge of your work is yours; you choose what the company sees.
The key to federation - starting with a conversation
This isn’t about forming a union and standing outside company headquarters with placards - and anyway, why do we still have company headquarters? - it’s about employees, individually, owning the asset they create as they create it, and choosing how to mesh it into whichever systems they happen to work inside. As Naval Ravikant puts it, we’re shifting back from factories and corporate headquarters to the era of hunter gatherers - running our own portfolio careers - and we now need our own tools.
A federated model, in which individuals and organisations mesh their knowledge bases and each is rewarded for the quality of its own constellation, is, probably, more resilient than one in which a central brain accumulates without limit.
The better questions for us to tackle aren’t just about data privacy, they’re about cognitive ownership.
And as Coase notes, once transactions are frictionless, we don’t need the organisation in its current form. Instead we can build lightweight, mission-driven organisations that coalesce talent to address today’s polycrisis.
Looking forward, I’ll be tracking the early builders of the personal second brain - ALIVE Context and others - and the standards they may, between them, manage to set. If you’re building in this space, or thinking about it, I’d like to hear from you.