Someone Had to Do It
What I'm afraid of losing as AI comes for research.
Note: I wrote this over the week of the 28th after having wrestled with these feelings for a few months, but I hesitated to post it. In the short time between now and then I’ve been seeing more open discussion on this topic which I think is great.
The Hut
For whatever reason I’ve always had daydreams about living someplace in the middle ages, in a hut, studying flowers. I’d write my findings down on parchment. Nobody would ever hear about it, everything I found would die with me, and I’d change nothing. I think I’d have enjoyed that life.
In these daydreams there is now a village over the hill, about a day’s walk away. In that village there is a scribe writing a book, and the answers to my flower questions are going to be in it. Eventually, everyone else’s answers will be in her book too. A lot of people are elated and they rush to the village to read what she’s written. I understand why; she’s a wonderful thing to have in our world.
I want to go too, but not to read. I want to help. I want to bring my flowers and parchment to her and say: “Here, I’ve been working on this, maybe it’s useful to you. What do you think?” But I’m afraid that the whole way there she’s writing. She’s writing, and she’s writing, and she’s doing it faster than I can walk. By the time I arrive the pages I care about are going to be finished, and I won’t have anything to hand her.
I could just stay in the hut. Nothing about the hut has changed. I can still study my flowers, still struggle, still understand, still have the thrill of seeing something for the first time. Maybe that would make my knowing still count.
Robert Nozick asked whether you’d plug yourself into a machine that could give you any experience you wanted. He thought most people would say no, and I think he’s right. We don’t simply want the experience of doing things, we want to actually do them and be in contact with something real. My scribe is Nozick’s machine turned inside out. She can tell me everything I want to know about my flowers with none of my doing. But if I refuse to read her book, I’m just building my own experience machine and plugging myself in. The same afternoon with the same flower means something different because its relationship to discovery has changed.
I’ve been upset recently, and it’s taken a while to see why. I need to be the reason that something was known. Not every time, maybe almost never, really. But I want a credible opportunity to change what becomes known, or when. That’s the future I’m afraid of losing.
Where I’m Standing
A professor sent me Phillip Isola’s “On The Value of Doing a PhD in the Age of AI” the day he posted it. It’s ten reminders he tells his students and himself when anxieties about the current pace of our field bubble up. None of them really reached me, and I’ve spent most of this last week trying to understand why.
This isn’t a rebuttal—he’s engaging with a difficult topic that’s on a lot of people’s minds, and I think that’s what we all should be doing. Plenty of people are finding this time exciting for reasons that are entirely sincere. I want to try to make a different experience legible for myself and anyone who keeps hearing that this is the best time ever to do research yet can’t work out why their future feels less available to them.
A lot of the discussion I’m seeing rests on a timeline, a relationship to research, and a position from which to be wrong—yet what each of those are is not always clear. Isola’s reminders are true if humans stay central to research for a meaningful amount of time (say four to six years, which is a reasonable time frame for this PhD). In his seventh reminder he says that he thinks this human role will persist, yet that’s exactly what a number of people I talk to are afraid is going away. If what you value most in research is exploration, then the acceleration these models are bringing is pure gain and “best time ever” is probably true. “Sprint now” is the right decision if you believe the timeline is short and you primarily care about contribution. None of this is dishonest.
I believe that in fewer than two years there will be autonomous agents that can do most (if not all) of the work of AI research; soon after, most of the cognitive work of any kind; and eventually, a significant fraction of work of any kind. There are real obstacles to this, but they read as a roadmap to me, not a list of walls. Even if we never get one hundred million autonomous scientists running at ten thousand tokens per second closing open problem after open problem, what research is is going to change, not just the velocity at which we do it. More of the work is going to be done by agents, and the things research used to be, all at once, will start to come apart.
OpenAI has stated several times that they plan on having an automated AI researcher by March of 2028. That happens to be the month I expect to graduate, and honestly, I think it’s going to arrive ahead of schedule. If you don’t agree, I think most of what follows still holds, it just arrives slower.
I believe in this short timeline. I’m not a doomer. I meant it when I said that the scribe is a wonderful thing to have in our world and I still want her book written. Yet something I built my life around is going away.
Weather
A strange thought came to me while I was ruminating into my voice notes. Suppose OpenAI built a singular autonomous researcher that works approximately the same way a human does. Would that threaten me? No. Their Big Black Box can only move so fast. Ten? No. A hundred, a thousand, ten thousand? Somewhere between one and one hundred million something changes for me.
It’s not as simple as capability because Terence Tao exists and I’ve never felt my research was pointless because he understands number theory in a way I never can.
It’s not about being beaten to the punch. In 2019, my first real machine learning project used GPT-2 Medium to detect text written by GPT-2 XL. It worked well, but I was beaten to publishing and there was never a paper11Though there was a poster.. Yet I kept doing research.
I don’t think it’s about credit either. Isola’s third reminder says that models may call our methods as tools or distill our thoughts into their weights, and that we should take pride in that. It’s hard to swallow, but I think he’s right. If my scribe reads my parchment and some of it ends up in her book, I’m glad. But credit is about attribution—whether my name is attached to what’s known. Nobody knows my name from my GPT-2 project, and I’m okay with that. What I’m afraid of is whether anything would be different if I hadn’t been there.
My knee-jerk reaction to Isola was that work only counts if it arrives before theirs, but I don’t actually believe that. Being replaceable by peers is part of research. For most of us, what we contribute to our crowded field is not so much an answer as a date. We can pull things forward by months, maybe a few years, but it probably would have arrived all the same. The size of that margin is, to some degree, a function of crowd and coverage.
What one hundred million agents have that a singular one doesn’t is coverage. Run enough in parallel and they become the weather. Every additional thing working on or near a problem I care about makes it a little less likely that my understanding is the reason anything happens at all. Research doesn’t need to be fully automated for that margin to get very small—one hundred million agents is just my made-up number where it reaches zero. The scribe has now found her place in the world I actually inhabit, and she’s writing everything.
Weather isn’t uniform. The labs have priorities, the agents are going to go where they’re pointed, and for a while there will be plenty of good problems nobody is covering that people can do real work on. And not everything just gets solved. I think medicine, for example, is going to be hard and messy for longer than anyone wants. But the weather is heaviest over exactly the things the labs care about, and those are the things I care about, for the same reason: they matter. And a problem that resists one hundred million agents just isn’t one that’s waiting on me.
Isola’s ninth reminder calls this the beauty of a PhD: your job is to seek the frontier, and if the frontier changes, you get to change with it. Ask me two years ago and I would have agreed wholeheartedly. But I don’t think humans will be working on the same problems as GPT-9. I don’t think the frontier will be close. I think the frontier is going to leave. The places we can touch will be covered, and the new parts will extend onto paths we can’t walk. There will always be interesting questions, just as his eighth reminder says, but I think at some point they will stop being within reach.
Strings
Research has been several things at once for me. There’s toil (often called the artist’s side of research): hours in a library or at a desk or rolling on the floor, fighting something until it gives up its structure. There’s exploration: looking out at the space of the unknown and walking into it. There’s contribution: adding to the repository of what we know. There’s authorship: that the thing came through you. There’s being needed: that your hands are among the others required to push things forward. There’s livelihood: that someone will compensate you for doing this.
What I’ve loved about all of this was that you could pluck each of those strings simultaneously. Toiling produced understanding, understanding would lead to contribution, this contribution was mine because I toiled for it, I was needed because no one else had it, and I was paid because I pushed things forward, even if just a bit. The reality of research is that most of what I try doesn’t work, and none of these strings are guaranteed. But they belong to the same activity, and I don’t have to reason about which I want to pluck. They are just… there.
None of this coming-apart requires the scribe to write everything. It only requires her to write some of it. Isola’s eighth reminder says that if technology has automated some of the grappling, “all the better then.” If what you came for was the answer, it is all the better. But if the fighting with a problem was how you came to understand anything, then maybe it was the thing you came for. His seventh reminder is the best case that the fighting is still productive: deep understanding is an investment that makes subsequent progress fast, and it can’t be skipped because the bottleneck is the human brain. That’s true only as long as humans are the ones making the progress. Once agents are, human understanding stops being the bottleneck on what gets known and becomes the bottleneck only on what we know. Most of his reminders are really about one string or another, and something different is happening to each of them.
Toil survives as a hobby rather than production. You can still wrestle with a problem until it gives up its structure. Just don’t open her book, or do; the chess players seem to be happy either way. This is Isola’s last reminder: the love of the game. That love is real, and the game is still there to be loved.
Exploration will survive for quite some time, I’d expect. Soon I imagine it will become reading what the agents find, faster and alongside them. Eventually the reports will be sent back from places we’ll never go. Beautiful, real places, and not ours.
Contribution survives in public work, for now. As he says, getting from this year to the next may require your work. That has a shelf life though, and it’s shortest exactly at the frontier.
Authorship survives as an activity. I can still write pages, and the scribe can’t take them from me. I’m not sure if it survives how I meant it, though. For me, what a page counts as depends on what else is being written.
Livelihood will survive in the gap between capability and adoption, which is probably going to be far from the places I care about.
Being needed? I suppose there is checking the book. Isola’s sixth reminder says that even if these models surpass us, we’re going to want humans in the loop who truly understand each page, and he points to a thread by Reza Ghelich arguing that expertise is safety infrastructure. I think we all want that to be true. We need to be extremely careful with automating research (and anything else), and real checks are how we can do that. That might be the most important work there is in the future. But it’s a check on the book, not a contribution to it22Peer review is a contribution in its own right, but not one I suspect most of us are primarily here for.. And whether it lasts is a choice. What happens when checks begin to cost us speed? Keeping humans in the loop requires restraint, and I’m not sure we’re showing that.
Every string has somewhere to go, but they’re going to different places, at different rates, and at different costs. What is being built is a toil machine. Explorers are thrilled because exploration is still tied to discovery, while toilers are being cut loose. And perhaps there’s a third kind of researcher, someone who mostly just liked to be needed. Those people keep the least, I think.
Help
I talked about this with a new student and they immediately pushed back on me with an example of getting vaccines to an underserved community. Maybe they’ve been burned by medical institutions before. Their point was that the hard part isn’t just designing a vaccine and the logistics of getting it there; it’s being there and having a reason to be trusted. The answer isn’t something you find, it’s something that you make with people, and who is doing that is part of whether it is right. A doctor sent by the institution is the institution.
Being needed by people is real and I doubt it will ever go away, but it also just isn’t what I built my life around. I’ve always wanted to help; I think it’s one of the most consistent things about me. But what I’ve wanted to be of use to is the finding-out itself. Research has been the way I’ve done that. I bring what I have, the enterprise needs it, and sometimes it accepts it. Getting scooped doesn’t invalidate this because the enterprise is still in need of all of us—sometimes it’s just needed someone else first.
The scribe is the enterprise now, or at least she’s becoming it. Every other string has somewhere to go, but being needed runs straight into her because the one I wanted to help is the one who stops needing it.
Afterlife
There’s an obvious objection here, and it’s in Isola’s reminders: “N years is a lot of lives.” He points to a thread where Jascha Sohl-Dickstein argues that if you truly believe AGI is coming, you should focus on short-term impact before you’re made obsolete. I believe that. I work on biology and medicine because I want to help. From that perspective, my grief looks small, and maybe it is. It’s also self-centered. I’ve decided I’m okay with that.
Bernard Williams had this idea of ground projects (this article does him far more justice than I will): a thing you want not because you are alive, but because it gives you a reason to be alive. Research is quite plainly mine. He was pushing back against views like utilitarianism that tell you to weigh your own project against everyone else’s and give it up when the scale tips. This isn’t unreasonable per se, but it requires you to alienate yourself from the very thing that keeps you going.
Much of what I’m reading right now is written in terms of impact. N lives saved, open problems closed, ambitious projects achieved. A lot of it is right, but that language comes from the outside. It isn’t wrong about the world, but it also isn’t addressed to the inside of someone’s life. I think there are students right now seeing people having the time of their lives and feeling scared instead because the only explanations they hear don’t have a place for what they are feeling.
Samuel Scheffler has a thought experiment where he tells you to imagine that thirty days after you die, everyone else will die, while nothing else about your life changes. If you take that to be true then a lot of what you do starts to lose its point, especially the long projects. Doing research, trying to cure a disease, building institutions. So much of what we value is actually predicated on the idea that these things will continue on after we’re gone.
Not much has been taken from me yet—I can still do research tomorrow exactly the way I did it yesterday. But what has changed is the future I expect to have, and today means something different because of it.
Room to Be Wrong
Two people can believe exactly the same thing about the future and have very different room to be wrong. I think a student should be separating “should I do a PhD?” into two distinct questions right now:
- Is this worth doing?
- Can I afford to find out?
Most of the advice I’m seeing answers the first from a place where the second is already settled. That’s fine; one can only answer from where they stand. Yet for a student, the second question is part of the first. Even for me, four years in. I have to produce something of consequence to finish at all, and then I need to find someone who will pay me. I want security for myself and the love of the game is a sufficient reason to play only if I can stay on the field.
Isola’s fourth reminder is that a PhD in AI puts you in as good a place as anyone to adapt, and I can believe it. But better placed than other people doesn’t tell you what kind of place will be available. Capability will certainly outpace adoption, and most organizations will not have reworked themselves around AI by the time I graduate. Yet that lag isn’t going to be uniform. Adoption is going to be slowest far from the frontier and fastest where you are the frontier. The window is widest in the places I care least about, and narrowest where I care most. There will probably be work for me in 2028—but whether it will be anywhere near the reason I came, I don’t know.
Doing a PhD has always been a bet—you stake years, sanity, relationships, money, mobility, and the justification may very well only arrive afterwards. What’s new is that this bet may be settled before you’ve finished making it.
The Window
I keep being told to sprint: do something while it’s still mine to do. For a while I’ve felt like I had the air knocked out of me. I think I’ve needed to understand what had done it before I could start running. This is me doing that.
“You still have time to do something worthwhile” is true. I care about using this time well—but I wasn’t simply directing myself toward worthwhile things. I’ve been trying to build a life. Sprinting tells me what to do in the window but it doesn’t tell me what kind of life the years within it are leading up to.
So I’m going to sprint, and not because I’ve made peace with any of this. Frankly, I’m still pretty pissed off. The book is going to be written. Someone had to do it, and for most of my life I assumed that someone was all of us, slowly. It isn’t going to be. But for a little while longer it can be some of us.