Nobody Asked You to Consent to the Joins

I spent six months building an assistant that needed to know everything about my working life to be any use, and every individual thing I gave it was harmless. A calendar slot. A note about a client’s mood on a call. Which train I usually catch on a Tuesday. Ask me to defend any single one of those and I couldn’t tell you what the fuss would even be about.

It isn’t any single secret. It’s the joins.

The fuss starts once they’re all in one place, cross-referenced against each other. Alone, a Tuesday train time is nothing. Next to six months of calendar entries, a health note and a record of every strained relationship in my network, it becomes a fairly complete model of a person, assembled by something that isn’t me and doesn’t answer to me. Data protection has spent thirty years arguing about which individual facts are sensitive. That was always the wrong unit of analysis. The value, and the exposure, live in what a system can infer once enough of the pieces are sitting next to each other.

Every consent mechanism you’ve ever clicked through, a cookie banner or a permissions screen, asks for your agreement one fact at a time. Nobody asks whether you consent to the assembled picture: your location history sitting next to who you call late at night. That combination is never on any form, because the form was built around the field, not the join. Decline every individual request politely, and you can still end up fully profiled, if enough separate systems happen to hold pieces that fit together.

We guard what we’re accountable for

I found the sharpest version of this doing precisely the opposite of what I now think is sensible. For my consulting clients I keep confidential material behind an encrypted store on my own machine, queried by a model that never leaves the building, so nothing sensitive of theirs ever reaches a cloud model. I built a proper professional standard, and applied none of it to myself. My own life went straight into the commercial model I was using to save time: the joined-up version, unencrypted, on someone else’s terms.

That’s not really a personal failing. It’s a pattern. We’re rigorous about the data we’re accountable for and careless about the data that’s merely ours. A company will audit its supplier contracts for exactly this kind of risk and let its own staff’s health information sit in a spreadsheet nobody’s checked properly. The standard you’d never let a client fall below is very often the one you quietly don’t hold yourself to, because the penalty for getting it wrong lands on someone else.

So think in aggregates, not fields

I don’t think the fix is refusing to hand anything over. Most of what any of us shares is genuinely low-stakes on its own, and pretending otherwise just means nobody uses anything useful. The fix is asking a different question before you consent to anything: not “is this one fact sensitive”, but “what does this look like next to everything else I’ve already given away”. That’s a harder question, and it’s the only one that actually maps onto how these systems create value and risk. A single field is never the unit that matters. The join is.

I’m not sure regulation catches up to this in my lifetime. Consent law is still drafted field by field, and I don’t know what a workable aggregate-consent regime would even look like in practice. What I do know is which side of that asymmetry I want to be sitting on, and it isn’t the one I was on for most of this year.

Companion essay to I gave an AI my whole life for efficiency — the AI Sabbatical, post 5.

Public Interest