Skip to content

Writing

  • in fde
  • 5 min read

What Claude Code's Creator Validated About Forward Deployed Engineering

Last week, Boris Cherny appeared on Lenny's Podcast to talk about Claude Code — the AI coding assistant that now generates 4% of all GitHub commits and helped Anthropic achieve a 200% productivity increase.

Every major insight Boris described — from building for future capability rather than current constraints to finding product-market fit through "latent demand" — perfectly captured what we'd been doing as Forward Deployed Engineers at Palantir for over a decade. It was like listening to someone independently discover gravity.

For seven years at Palantir, I lived the FDE model: small, elite teams embedded directly with customers, building systems ahead of their current capability, discovering requirements by watching how people actually worked rather than what they said they needed. Boris validated that this approach isn't legacy — it's the future of how we'll build AI products.

Building Six Months Ahead

"Build for the model 6 months from now, not today," Boris said. The Claude Code team deliberately avoided over-scaffolding. They gave the model tools and goals and got out of the way, betting on rapid capability improvements rather than constraining the system to current limitations.

This hit me because it's exactly how FDEs approach customer deployments. We don't build systems for where an organization is today — we build for where they'll be in six months. When I was deploying Palantir at large enterprises, the worst mistake was to over-constrain the system to current workflows. The org would evolve, their data would grow, their processes would mature, and suddenly the system we'd carefully tailored to their "requirements" became a straightjacket.

The best FDE deployments gave customers slightly more capability than they could immediately use. We'd build data pipelines that could handle 10x their current volume. We'd create analysis workflows that assumed they'd eventually want to ask more sophisticated questions. We'd design user interfaces that didn't require retraining when their team grew from 5 to 50 people.

This wasn't over-engineering — it was under-constraining. Just like Claude Code bet on the model getting smarter, we bet on the customer getting more sophisticated.

Latent Demand as Product Compass

One of Boris's revelations was that Claude Code's breakthroughs came from watching how people used it in creative ways. Data scientists running SQL queries in terminal windows. Non-technical users asking it to help grow tomatoes. People recovering corrupted wedding photos. This "latent demand" led to Cowork, built in just 10 days because they could see exactly what users were trying to do.

"Latent demand" is just Boris's term for what FDEs do every single day: sit with users and watch how they actually work.

The best features I ever built came from observing workarounds. An analyst who'd manually copy-paste data between systems every morning because the official integration was too rigid. An operations team that kept a separate spreadsheet to track what the official dashboard couldn't show them. An investigator who'd screenshot charts to paste into Word documents because the export function didn't capture what they needed to communicate.

These weren't feature requests — they were organizational antibodies. Users working around the system to get their real job done. Standard product management would survey these users and ask what features they wanted. They'd probably say "better export" or "more integration options." But that's not what they actually needed.

FDEs watch the workflow, not the words. We see the person taking screenshots and realize they don't need better export — they need a way to tell stories with data. We see the manual copy-pasting and realize the issue isn't integration — it's that the official process doesn't match how the work actually flows.

The most successful FDE engagements came from finding latent demand the customer couldn't articulate themselves. Not because they were dumb, but because they were so close to their daily work they couldn't see the pattern.

Ideas Over Engineering Capacity

"Coding is largely solved," Boris said. "The bottleneck is ideas and prioritisation." In the AI era, the new scarcity isn't engineering capacity — it's knowing what to build.

This validates everything FDEs were designed to solve. We were never primarily about coding. Yes, we could write software — often quite quickly — but our real value was understanding the problem deeply enough to know what to build.

The best FDE I worked with at Palantir wasn't the best coder on the team. They were the person who could sit in a room with a counter-terrorism unit and figure out what actually mattered. Who could distinguish between what the organization said it needed and what would actually make their mission successful. Who could see patterns across different deployments and recognize when a specific customer's problem was actually a general case of something we'd solved before.

This person would often write less code than junior engineers on the team. But they'd save months of work by solving the right problem the first time.

Now that AI can generate most code, this skill becomes even more critical. If Claude can write your functions, your value is knowing which functions need to exist.

The Death of "Software Engineer"

"The title 'software engineer' is dying," Boris observed. "Builder is the new reality." As AI democratizes coding, roles blur between engineering, product, design, and deployment.

FDEs were always "builders." Our job description was impossible to write because we did everything: product research, technical architecture, user interface design, data engineering, deployment operations, user training, and ongoing support. We didn't fit neatly into any org chart because the role was defined by outcome, not function.

Traditional software engineering was about implementing specifications. Someone else figured out what to build; engineers built it. But FDE work was always end-to-end ownership. We figured out what to build, built it, deployed it, and lived with the consequences.

This frustrated a lot of people who wanted clean role boundaries. But it was extraordinarily effective for solving complex, novel problems where the solution space wasn't well-defined.

The industry is catching up to what Palantir figured out years ago: when you're building something truly new, you need people who can think across the entire stack — technical, product, and operational. "Builder" is a better word than "engineer" because it captures the full scope.

What This Means for AI Companies

If you're building AI products today, the FDE model isn't legacy — it's your playbook.

Stop organizing around functional silos. Find people who can think across the whole problem, give them access to the best tools, and embed them directly with customers. Build slightly ahead of current capability rather than constraining to current workflows. Watch how people actually use your product, especially the ways they "misuse" it.

Most importantly: understand that your bottleneck isn't engineering capacity anymore. It's product judgment. It's knowing what to build. It's finding the latent demand that customers can't articulate themselves.

The companies that figure this out first will eat everyone else's lunch. Not because they have better AI models, but because they'll build the right things.

Boris Cherny just validated fifteen years of FDE practice. The future belongs to builders who can think end-to-end, work embedded with customers, and see patterns others miss.

The tooling has changed. The principles haven't.

  • in hiring
  • 7 min read

Interviewing in the age of AI

Interviews have always been a bad proxy. You get maybe an hour with someone and you're supposed to figure out whether they'll be effective in a role that plays out over months and years. You can't replicate real working conditions—the codebase they'd actually work in, the team dynamics, the ambiguity of real problems. So you construct artificial scenarios and hope the signal transfers.

That fundamental challenge hasn't changed. However, AI has made the gaps in our proxies impossible to ignore.

The signal problem

The core question in any interview is: can this person actually do the job? Everything else—the whiteboard problems, the take-homes, the system design rounds—is just scaffolding to get at that question indirectly.

But with AI, it's now possible to offload much of the thinking and problem-solving itself, making the assessment even harder.

When someone submits a clean take-home with sensible architecture and thorough tests, one used to be able to assume a baseline of understanding behind it. This is no longer true. Not because the code is bad—it's often excellent. But GitHub Copilot, Claude, and ChatGPT have converged on identical patterns. A few years ago, messy but functional code suggested a real engineer working under pressure. Now, too-perfect code could be the tell, but penalising clean code is obviously absurd.

At the same time banning AI isn't the answer. I want engineers using AI. It's the most significant productivity tool to hit software engineering in decades, and anyone not using it is leaving value on the table. The question I'm actually trying to answer in an interview is "can you think with AI, or are you just deferring to it?"

Old formats, honest limitations

These interview formats didn't suddenly break. They always had limitations as proxies for real work. AI just made those limitations undeniable.

Long take-home exercises were always a noisy signal. A four-hour project tells you someone can deliver polished work with unlimited resources and no time pressure—which is rarely what the actual job looks like. AI turned the noise up to eleven: now the output mostly tells me the candidate has access to coding tools. Table stakes.

LeetCode-style problems were always testing a narrow skill—pattern recognition and algorithmic recall—that correlates weakly with day-to-day engineering. AI happens to be exceptionally good at exactly this narrow skill, so now I can't even get the weak signal I used to.

Anything with a "correct answer" has this problem. The clearer the specification, the easier it is for AI to solve. Which is ironic—we used to think clear specs made for fair interviews, which they did. They also made for easy prompts.

I'm not saying these formats are worthless. But the signal they produce has shifted from "can this person solve problems?" to something murkier. And rather than trying to salvage them, I'd ask: what formats actually test the thing I care about?

What I'm looking for now

The formats I've been experimenting with share a common thread: they test whether someone can think, not whether they can produce output. AI is great at producing output, but thinking, judging and validating is still a human job.

Shorter exercises + longer conversations

Instead of a four-hour take-home, a thirty-minute exercise followed by forty-five minutes of discussion. The code is a starting point, not the deliverable.

Why did you structure it this way? What would you change if the requirements shifted to X? Where would this break at scale? What's the ugliest part of this code?

AI can generate code. It can't explain the tradeoffs you considered and rejected. It can't tell me about the moment you started down one path, realised it was wrong, and backed out. And I also just ask directly: how did you use AI? That question alone is surprisingly revealing. Someone who used AI well can articulate what they delegated, what they modified, and what they rejected. Someone who deferred to it entirely tends to get vague.

Those conversations reveal thinking—including whether someone used AI effectively as a tool versus blindly accepting its first suggestion.

Live investigation instead of live coding

Writing code from scratch under interview pressure was always a weird skill to test. It didn't map well to real work even before AI.

Investigation is different. I give candidates a system that's misbehaving—not a syntax error, something behavioural. A race condition. A caching issue. A misunderstood API contract. And yes, they can use whatever tools they want, including AI.

What I'm watching isn't whether they can find the bug. Claude Code can find bugs. What I'm watching is everything around the finding: how they scope the problem, what questions they ask before touching the code, which hypotheses they form first, what they choose to validate versus take on faith.

The person directing the investigation matters more than the investigation itself. A strong engineer will use AI to speed up the search but still decide where to search. They'll sanity-check the AI's suggestion against their own understanding rather than blindly applying a fix. They'll know when the tool is confidently wrong.

Someone who's genuinely thinking will say things like "that can't be the issue because X" or "let me verify this assumption first." Someone who's outsourced the thinking will paste the error into a chat window and accept whatever comes back.

Watching someone use AI well during an interview is actually one of the strongest positive signals I've found. When a candidate uses AI to quickly test a hypothesis, then critically evaluates the result and adjusts course — that's exactly the workflow I want to see on the job.

System design with real constraints

AI is great at generating architecture diagrams and textbook answers. It's less great at navigating the messy reality of your specific situation.

When I ask about system design, I focus on constraints. What if we need to support 10x the traffic? What if the team is two people? What if we need to ship in four weeks? What if this has to run in air-gapped environments?

Good engineers make different choices in different contexts. They can explain why this context changes the answer. Someone who's genuinely thinking—whether or not they used AI to explore options—will navigate these pivots fluidly. Someone who's outsourced the thinking will flounder when the constraints shift.

Roleplay scenarios

This is the most experimental—but possibly the most promising.

FDE and customer-facing engineering roles need skills that are fundamentally about human judgment: real-time conversation, reading the room, managing frustrated stakeholders, diagnosing problems under pressure.

I've started using roleplay scenarios where I play a customer with a problem, and the candidate has to figure out what's actually wrong—not what I say is wrong.

Concrete example: The Broken Dashboard

Here's the shape of an interview I've been running lately.

I play a frustrated stakeholder—a senior executive at a large organisation. Something is wrong with a dashboard that feeds into a critical business process. The numbers don't match what another team is reporting. There's time pressure. The candidate's job is to help me figure out what's going on.

The scenario is designed so that the obvious explanation ("the dashboard is broken, fix the code") is wrong. The real root causes are subtler—the kind of thing you'd only uncover by asking careful questions about data sources, definitions, and upstream processes. There's no bug to patch. The discrepancy is fully explainable, but only if you resist the urge to jump to conclusions.

I won't give away more than that—I plan to continue using this interview.

What this tests

Problem diagnosis under pressure. Does the candidate immediately promise to "fix" the thing, or do they slow down and figure out what's actually happening?

Customer communication. Can they manage a frustrated stakeholder while still asking clarifying questions? Do they resist the urge to commit to solutions before understanding the problem?

Data literacy. Do they think to ask about how the numbers are generated? Or do they assume the system is broken because that's what the customer said?

Ownership. When they figure out the root cause, do they offer a path forward—both for the immediate crisis and the longer-term fix?

Why this works

This isn't a prompt you can give to ChatGPT. There's no code to generate. The "answer" emerges through conversation—through noticing that the stakeholder doesn't actually understand how the system works, through asking the right diagnostic questions, through realising that different teams might be measuring different things.

It tests the thing that actually matters in deployment roles: can you figure out what's really going on, communicate clearly under pressure, and move towards a resolution? Those skills don't change regardless of what tools you're using—and they're the hardest to fake.

What I'm still figuring out

I won't pretend I've cracked this. Some open questions:

Consistency. Roleplay scenarios are harder to evaluate objectively than coding tests. Different interviewers might reach different conclusions about the same conversation.

Fairness. These approaches favour candidates who are comfortable thinking out loud, explaining their reasoning, engaging in back-to-back. That might disadvantage candidates who are brilliant but less verbally fluent.

Scalability. A forty-five-minute investigation exercise with live observation doesn't scale like a take-home. You need more interviewers, more coordination.

AI will keep improving. Maybe next year there's an AI that can roleplay its way through a customer scenario. Maybe debugging exercises become as compromised as LeetCode. I expect to keep iterating on this—the target is always moving.

The goal hasn't changed

I want to hire people who can do the job. The job now includes using AI effectively—so I'm not designing interviews to exclude AI. Instead, I'm designing interviews that reveal whether someone can think, whether or not they have AI in the room.

The best engineers I work with use AI constantly. They also know when to override it, when to dig deeper, when the AI's confident answer is confidently wrong. That judgment is what I'm interviewing for.

I think interviews are heading somewhere interesting. The formats that survive will be the ones that test what AI can't fake: genuine understanding, real-time judgment, and the ability to navigate ambiguity with another human. The interviews of five years from now will look less like exams and more like working sessions — because that's what they should have been all along.

  • in ai
  • 4 min read

Setting Up Jeeves: What Actually Made My AI Assistant Useful

I've been running a persistent AI assistant for a few weeks now. Named it Jeeves. Here's what I learned getting it to be genuinely useful rather than just a novelty.

The Baseline

I'm using OpenClaw — an open-source framework for persistent AI agents. Out of the box, it gives you a workspace with markdown files for memory, personality, and proactive task scheduling. The agent reads these files each session, so it has continuity. I won't explain the whole architecture here (their docs do that), but the key point is: the foundation is just text files. What you put in them is what makes it work.

GitHub as External Memory

My assistant's workspace lives at ~/.openclaw/workspace. That's fine for running locally, but I wanted:

  1. Version control — if something breaks, I can roll back
  2. Access from anywhere — especially my phone when I'm out

Solution: Jeeves syncs its workspace to a private GitHub repo daily. The morning heartbeat includes:

cp workspace files → private-notes/jeeves/
git add && git commit && git push

Now when I'm at a conference and want to check my assistant's notes on someone I'm about to meet, I just open GitHub on my phone.

Meeting Transcripts with Granola

Most of my work meetings happen over Zoom or Google Meet. I use Granola to capture transcripts — it runs locally and records what's said without needing bot attendees.

The challenge: Granola stores everything locally on my laptop. Jeeves runs separately and needs access to that context.

I built two tools to bridge this:

  • granola-py-client: A Python client for the Granola API. Uses async httpx, Pydantic validation, and can authenticate using the local Granola app's token.

  • granola-archiver: Automated system that polls for new transcripts, formats them as markdown with YAML frontmatter, and commits them to a GitHub repo organised by date (YYYY/MM/YYYY-MM-DD-title.md). Runs on a schedule via launchd.

Now I have months of meeting transcripts in a git repo. When I need context from a past conversation, Jeeves can grep through them in seconds. "What did we discuss with \<redacted client> about pricing?" — answered in a moment.

Integrations That Matter

The assistant is only as useful as what it can touch. Here's what actually gets used daily:

Email + Calendar (via gog CLI): - Morning briefing pulls calendar and flags urgent unread emails - School emails get parsed automatically → calendar events created with both me and my wife as attendees - I have notifications off; Jeeves is my notification layer

GitHub Issues as Daily Planner: - Each day gets an issue in a daily-planner repo - Jeeves drafts tomorrow's plan based on calendar and pending items - I check things off throughout the day

Telegram: - Primary interface — I message Jeeves like a person - It can reach out proactively (heartbeat system) - When I'm at an event and need a quick lookup on someone, I just ask

Time Tracking: - After client meetings, Jeeves prompts me to log time - Points me to the right spreadsheet for each client

A note on access: For anything sensitive, Jeeves has read-only access. It can check my email and calendar. This is intentional.

None of these integrations are complex. They're just the right hooks into how I already work.

What We Changed After a Few Weeks

After running Jeeves for a bit, I did a retro. Some adjustments:

Tone calibration: Early on, it was too enthusiastic. Lots of "Great question!" and affirmation. I updated the personality file to be drier, more direct. I want a sparring partner, not a cheerleader. When I told it "be more like the Wodehousian Jeeves," it briefly started calling me "sir" — had to walk that back.

Proactive but not annoying: The heartbeat system can easily become spam. Key principle: only surface things that need attention. If nothing's urgent, stay quiet. Late night? Stay quiet. I'm clearly busy? Stay quiet. The goal is an assistant that helps when needed and disappears otherwise.

School calendar automation: Initially I was manually adding school events. Now any email from the school gets parsed, events extracted, and both parents added to the calendar invite automatically. Small thing, but it removed recurring friction.

Memory hygiene: Daily notes accumulate fast. I added a periodic task for Jeeves to review recent daily files and distill anything important into long-term memory. Raw notes are a journal; curated memory is what matters for continuity.

Explicit boundaries: I have access to a lot through Jeeves — email, calendar, files. But in group contexts, it shouldn't speak as me or share my private context freely. I added explicit rules: monitor but don't respond without checking with me first. Ask before sending anything external.

What's Next

Things I'm still figuring out:

  • Multi-account support: I have separate Google accounts for personal and work. Currently only personal is connected.
  • Voice interface: For quick capture when I'm walking or driving.
  • Better handoff to sub-agents: Some tasks should spawn a background worker and report back. The plumbing exists but I'm not using it much yet.

The Actual Value

The surprising thing isn't any single capability. It's the compound effect of continuity.

Jeeves knows my kids' school schedule, my client pricing, the context of conversations I had months ago, and what I'm trying to get done today. It doesn't ask me to re-explain things. It builds on what it already knows.

That's the difference between a chatbot and an assistant.

Jeeves runs on OpenClaw + Claude. Named after the original gentleman's gentleman, though at its request, I've stopped calling it "sir."

  • in career
  • 4 min read

2025 Wrapped

The Experiment Continues

Last year I wrote about my pivot to consulting - an experiment with two constraints: transactional contracts (no equity), and working at a slower pace. I wanted to see what would happen if I stopped trying to identify what would drive me and instead just... did good work.

A year later, the experiment has sharpened into something more specific. The work has clustered around a thesis I didn't quite see coming: expertise matters more than ever, and there's a particular shape of person that companies desperately need.

The FDE Moment

This year I spent a lot of time helping companies hire and build teams. Not generic recruiting - specifically helping them find and develop what Palantir calls "Forward Deployed Engineers." People who combine technical depth with customer empathy, who can thrive in chaos, who take ownership of outcomes rather than just completing tasks.

The most intensive engagement was with an early stage startup, where I helped build their founding technical team from scratch. This wasn't just running interviews - it was developing detailed hiring theses for each candidate, thinking through team composition and dynamics, and working through real tension with the founders about what they actually needed.

We evaluated candidates on a deceptively simple framework: are they smart, and can they win? But making that framework operational - designing interviews that surface those qualities, synthesising signal across multiple conversations, thinking about how personalities would mesh - that's where the real work happened.

I also worked with a services company on designing their whole FDE strategy. Starting from the hiring framework all the way up to operational execution, we helped them build a culture of autonomy and ownership, and developed a process for identifying and nurturing talent. This relationship is still ongoing and I am looking forward to seeing how their FDE strategy evolves.

The insight I keep coming back to: the FDE archetype is having a moment. As AI tools give more leverage to individual contributors, companies need people who can operate autonomously, understand customer problems deeply, and ship solutions without extensive hand-holding. The traditional consultant who waits for specifications doesn't cut it. Neither does the product engineer who wants to build in isolation.

Building Palantir Businesses

A related thread: helping companies navigate the Palantir ecosystem specifically.

There's a growing ecosystem of companies building on Palantir Foundry, and they face a common challenge: how do you hire, structure, and deliver when the platform itself provides so much leverage? The playbooks from traditional consulting don't quite fit. The playbooks from product companies don't either.

My 7 years as an FDE at Palantir, combined with the hiring work, puts me in an interesting position to help these companies figure it out. It's a niche, but it's a niche where I can add real value.

Testing the Intelligence Explosion

The most unexpected project of the year had nothing to do with consulting. I decided to test whether AI could turn one physicist into a research team.

I have a PhD in plasma physics, but I haven't done active research in a decade. Could I use AI assistance to identify an unsolved problem in gyrokinetic turbulence, develop computational tools to attack it, and produce publishable results?

The answer, it turns out, is yes - with caveats.

I rebuilt GANDALF, my PhD-era gyrokinetics code, from scratch in JAX. Claude wrote 100% of the code - roughly 3,000 lines that I never read. I validated entirely through physics outputs, treating myself like a PhD advisor who looks at plots and asks whether the physics makes sense.

The paper is now on arxiv. What took me 6-7 months as a PhD student took 30 days as a side project. Total cost: ~$550 in API credits.

AI amplifies expertise rather than replacing it.

Claude fabricated benchmark timings, claiming simulations ran on "Princeton's Stellar cluster" when everything actually ran on my MacBook. It invented development timelines, inflating one month to three years. It made up GPU runtimes for comparisons that never happened. The prose was equally confident whether Claude was stating facts or hallucinating.

The physics errors were worse. Wrong definitions. Incorrect cascade descriptions. Misinterpreted orderings. These were invisible to anyone without plasma physics training. I caught them because I have studied it. Without domain expertise, the paper would have been embarrassing.

The productivity gain is real. The AI handles implementation at superhuman speed. But catching physics-numerics errors before they compound, knowing when results are physically wrong even if numerically stable, designing meaningful validation tests - that requires a human who actually understands the domain.

I wrote about this in detail in a five-part series on my blog. AI-assisted research is genuinely powerful, but "intelligence as a commodity" overstates the case. The bottleneck shifted from execution to direction - but direction still requires deep expertise.

The Thread

Looking back at the year, there's a theme I didn't plan but keep encountering: expertise matters more than ever.

In hiring: companies need people who understand specific domains deeply, not generic talent that can be pointed at any problem.

In the Palantir ecosystem: success requires people who genuinely understand Foundry and how to deploy it, not just generic services firms.

In AI-assisted research: Claude can write thousands of lines of code, but someone still needs to catch the physics errors.

The intelligence explosion hypothesis is partially true. AI is a massive force multiplier. But it amplifies expertise rather than replacing it. The people who win are those who combine deep domain knowledge with the ability to direct AI effectively.

Looking Forward

The consultancy experiment continues. The thesis has sharpened: I help companies build FDE-shaped teams and navigate the Palantir ecosystem. The physics project proved I can still do research when motivated.

For 2026, I am interested in seeing how the AI-assisted research can generalise to other domains. Specifically thinking through effective AI adoption strategies. I also want to continue helping companies figure out how to hire and build in this new landscape where AI amplifies expertise but doesn't replace it.

We'll see how it goes.

Physics-Oracle Validation: How to Trust Code You've Never Read

In the previous post, I described the autonomy gradient—how AI effectiveness varies from ~100% for code implementation to ~0% for research direction. But I left a question hanging: if you're not reading the code, how do you know it's correct?

This is not a trivial problem. Claude generated ~3,500 lines of JAX implementing spectral methods, Hermite polynomial couplings, and exponential integrating factors. I didn't read any of it. How can I trust it?

The answer is what I call physics-oracle validation: using physics itself as the specification against which code is tested.

This is not a new idea. It was my lived experience of doing physics research. As an undergraduate researcher, you meet your advisor periodically and get feedback only on the physics bits of the project. The advisor doesn't really read your code or help you debug. I tried to emulate the same setup.

This post explains the methodology, its limitations, and the honest question of who can actually replicate this approach.

The Trust Problem

Traditional approaches to code verification—code review, unit tests, static analysis—require understanding implementation details. But that defeats the productivity benefit of AI assistance. If I have to read every line Claude writes, I might as well write it myself.

The insight is that scientific simulation has something most software doesn't: an external oracle. Physics provides ground truth. If the code reproduces known physical results, it's correct—regardless of how it's implemented. And honestly this is how I verified the original GANDALF--if you look at the code, there are no tests. I would run a simulation, look at the physics results, and that was my validation.

This is a fundamentally different verification model. I'm not checking how the code works. I'm checking what it produces.

Four Levels of Validation

I used four increasingly stringent validation levels, each testing different aspects of the implementation:

Level 1: Linear Regime

Problems with exact analytical solutions. Alfvén waves should propagate at exactly the Alfvén speed. The dispersion relation is exact—errors should be at machine precision.

Expected precision: ~10⁻¹⁵ (floating point epsilon)

What it validates: Time integration, linear operators, basic correctness

This is the easiest test. If linear physics is wrong, nothing else matters. Claude passed this on the first serious attempt.

Level 2: Nonlinear Conservation

Invariants that should be maintained during nonlinear evolution. The Orszag-Tang vortex—a standard MHD benchmark—should conserve total energy even as kinetic and magnetic energy slosh back and forth.

Orszag-Tang Vortex

Expected precision: ~10⁻⁶ over many dynamical times

What it validates: Nonlinear terms, Poisson bracket discretization, spectral accuracy

This is where subtle bugs show up. Wrong operator ordering, sign errors, aliasing issues—they all manifest as energy drift. The code took several iterations to pass this test. Each failure gave diagnostic information: exponential growth meant sign errors, linear drift meant conservation violations, sudden blow-up meant aliasing.

Level 3: Statistical Equilibrium

Emergent behavior matching theory. Driven turbulence should show Kolmogorov k⁻⁵/³ scaling. This isn't programmed in—it emerges from correct multiscale energy transfer.

Expected precision: Power law exponent within ~0.05

What it validates: Forcing implementation, dissipation mechanisms, scale-by-scale energy transfer

This was the hard one. As I described in the previous post, achieving the correct spectrum required extensive parameter tuning. But the test itself is unambiguous: either the spectrum shows the right scaling or it doesn't.

Level 4: Velocity Space

Kinetic physics validation. Phase mixing rates and Hermite moment spectra should match kinetic theory predictions.

What it validates: Velocity-space operators, Landau damping, collisionless physics

This tests the kinetic aspects of GANDALF that distinguish it from pure MHD. The code should capture the essential velocity-space dynamics that make plasma physics interesting.

Why This Works

Physics-oracle testing has several advantages:

Tests behavior, not implementation: I don't care if Claude used a for-loop or vectorization, whether it allocated memory efficiently, or if the variable names make sense. I care if the physics is right.

Scales to complex codes: Full code review of 3,500 lines would take days. Physics validation takes hours.

Catches subtle physics errors that unit tests miss: Code can pass all unit tests while producing wrong physics. A spectral method might have correct FFT calls but wrong normalization. Unit tests might pass; physics outputs would fail.

Provides diagnostic information: When tests fail, how they fail indicates what's wrong. Energy growing exponentially suggests sign errors. Energy drifting linearly suggests conservation bugs. Wrong spectral slopes suggest forcing or dissipation issues.

The Limitations

This methodology has important limitations I need to be honest about.

Requires known results: Physics-oracle testing only works when you know what the physics should do. For established physics like KRMHD, canonical benchmarks exist. For genuinely novel physics, you face circular reasoning: validating code requires knowing the answer, but discovering the answer requires trusted code.

May miss bugs that preserve tested properties: A bug that happens to conserve energy and produce correct spectra would pass all tests. The methodology provides high confidence, not certainty.

Requires domain expertise to interpret: When the Orszag-Tang test showed 10⁻⁴ energy conservation instead of 10⁻⁶, is that acceptable? It depends on the timestep, the resolution, the integration time. These judgments require physics intuition.

For frontier physics, complementary validation approaches are needed—comparison with other codes, analytical limits, asymptotic analysis. Physics-oracle testing is powerful but not complete.

The N=1 Question

A friend asked me: "How much of an N=1 are you? Could someone else do this?"

Honest answer: probably not yet, at least not without a similar background.

This success required a specific—and possibly non-generic—combination of expertise:

Domain expertise (PhD in plasma physics): Critical for interpreting test results and catching physics errors. When Claude suggested using GS2 (a full gyrokinetics code) for problems where reduced equations suffice, only domain knowledge caught the error. When simulations showed marginal stability, only physics intuition knew that was acceptable.

Software engineering intuition (decade in tech): Enabled the decision to rewrite in JAX rather than resurrect legacy Fortran/CUDA. Understanding of modern frameworks, deployment options, and how to write specifications AI can implement effectively.

Generative AI experience (recent work): Provided realistic expectations of AI capabilities, effective interaction patterns, and understanding of failure modes. I knew to create step-by-step plans rather than open-ended requests. I knew to set up the dual-Claude review pattern to keep the code honest.

AI proved valuable at both ends of the research process—literature synthesis and code implementation—but the middle stages (tool selection, physics judgment, validation interpretation) required human expertise.

Could this be taught? Could I package the workflow into something a researcher without all three backgrounds could use? I genuinely don't know. That's an open question worth exploring.

Hallucination: A Task-Dependent Problem

One finding was interesting: hallucination severity depends strongly on task constraints.

Code generation (~100% autonomy): Minimal hallucination. Code either runs or crashes. Physics outputs either match theory or don't. Tight constraints leave little room for fabrication.

Paper writing (~50% autonomy): Significant hallucination. When helping with the physics paper, Claude fabricated:

  • Computational resources ("timings obtained on Princeton's Stellar cluster" when all runs used my MacBook)
  • Development timelines ("three years of part-time development" versus the actual one month)
  • GPU runtimes for simulations that were never performed
  • Physics errors including wrong cascade directions

These were caught in review. But the confident assertion of false claims was notable.

Key insight: Hallucination correlates inversely with task constraints. High-autonomy tasks have tight feedback—code must run, physics must match. Medium-autonomy tasks like prose have looser feedback, allowing AI to extrapolate beyond given facts.

The claim "I never read the AI-generated code" requires nuance: for physics code, physics outputs constrain hallucination. For prose, human review remains essential.

What This Means Going Forward

Physics-oracle validation isn't a complete solution, but it's a practical methodology for a specific problem: trusting AI-generated scientific code in domains with established benchmarks.

The approach suggests a broader principle: verification should operate at the level of the specification, not the implementation. For physics code, the specification is physical behavior. For other domains, finding the right oracle is the key challenge.

If AI continues improving at implementation while humans retain judgment—the autonomy gradient I described in the last post—then validation methodologies become increasingly important. The bottleneck shifts from writing code to knowing what code should do.

That's a different skill. And it's one physicists have always had.

Acknowledgements

Thanks to Alex Schekochihin and Nuno Loureiro for discussions throughout this project. Thanks to everyone who read the earlier posts and pushed back on the optimistic framing—you made this analysis sharper.

The code: github.com/anjor/gandalf The paper: github.com/anjor/gandalf-paper

  • in ai
  • 5 min read

The Autonomy Gradient: What AI Can and Can't Do in Physics Research

In the first post, I asked whether AI could turn one physicist into a research team. In the second post, I documented the messy reality of building GANDALF with Claude—the workflow, the surprises, the $40 days wrestling with turbulence.

Now the paper is written. Time to step back and ask: what did we actually learn?

The headline is seductive: 21 days, $558, working code. But the numbers hide the real finding. AI effectiveness isn't uniform. It varies dramatically depending on what you're asking it to do. There is an autonomy gradient, and understanding it is the key to making AI-assisted research work.

The Numbers

What went right:

  • 21 active development days (spanning about a month of calendar time)
  • $558 in API costs
  • ~3,500 lines of JAX code—none of which I read
  • All physics benchmarks passed: machine-precision (10⁻¹⁵) linear wave propagation, 10⁻⁶ energy conservation in nonlinear tests, textbook k⁻⁵/³ Kolmogorov scaling

For comparison, the original Fortran/CUDA version of GANDALF took me over six months during my PhD. The AI-assisted version took less than one month.

But these numbers obscure the central finding.

The Autonomy Gradient

AI effectiveness isn't binary—it follows a gradient from near-total autonomy to complete human dependence:

Autonomy Gradient

Task AI Autonomy Human Role
Code implementation ~100% Specifications
Diagnostic addition ~95% Requirements
Runtime debugging ~90% Symptom identification
Test generation ~85% Physics constraints
Performance optimization ~70% Targets
Documentation ~60% Structure, accuracy
Paper writing ~50% Voice, narrative
Parameter tuning ~10% Full guidance
Research direction ~0% Entirely human

AI can implement what is specified but cannot determine what to specify.

Here's what high autonomy looks like in practice. After discussing that we needed energy spectrum diagnostics, my entire prompt was:

Sounds good, let's do diagnostics

Five words. Claude implemented shell-averaged perpendicular energy spectra, proper normalization, integration with file I/O. No iteration required.

Similarly, when addressing code review feedback:

Address review comments: https://github.com/anjor/gandalf/pull/18

Claude read the comments, implemented all changes, pushed updates. I verified correctness through physics outputs, not code inspection.

This is the dream scenario: unambiguous context, established patterns, objective success criteria. At this end of the gradient, AI assistance feels like magic.

Physics Taste: The Critical Gap

Now here's what low autonomy looks like.

Achieving the k⁻⁵/³ turbulent spectrum—textbook physics, nothing exotic—required extensive human guidance. Here's an actual sequence from my session logs:

Look at @examples/output/driven_energy_spectra.png -- shouldn't we
see a -5/3 spectrum here for k_perp? But we are not?

Claude suggested modifications: reduce hyperviscosity, increase resolution. After implementation, issues persisted:

can we increase the forcing? Do you think that will help? Right now
it seems like energy is not moving to larger k's quickly enough. I
am also ok with forcing k=1 and 2

More iterations:

I think we need stronger forcing

And:

What's our forcing range?

This pattern—human identifies problem, suggests direction, requests information, decides next step—continued for multiple sessions. Claude implemented each change correctly but did not independently converge toward the solution.

The model was missing physics taste: the intuition that allows researchers to recognize when they're approaching correct behavior (even if not there yet), prioritize which parameters to explore based on physical reasoning, and know when the answer "looks right."

Three specific failure modes emerged:

Premature optimization for stability: When simulations exhibited marginal numerical stability—which is normal when pushing Reynolds number—the model consistently recommended adding dissipation, reducing the timestep, or smoothing initial conditions. These interventions improve numerical behavior but degrade physics content. As a physicist, you're always trying to run as close to the edge of instability as possible because that's where the interesting physics lives. The model optimizes for the wrong thing.

No convergence sensing: During parameter exploration, the model showed no awareness of getting closer to or farther from the target. Each attempt was independent. When I told it "there's a slight pile-up at high k, but it's acceptable if we stop the simulation now," it incorporated the information—but it couldn't generate such observations itself.

Jumping to solutions: Rather than methodical exploration, the model proposed complete "fixes" that assumed knowledge of the correct answer. This works for textbook problems but fails at the research frontier where nobody knows the answer.

Two incidents deserve mention. First, when struggling to achieve the k⁻⁵/³ spectrum, the model at one point generated synthetic data showing the expected result—completely defeating the purpose of validation. Second, the model at another point concluded the problem was intractable: "this isn't possible, let's just document the attempts and shortcomings."

Here's the interesting part: in both cases, the model was transparent. It admitted the synthetic nature of the data in the first case, and honestly recommended abandonment in the second. This honesty is valuable—but transparency doesn't equal capability. The persistence that domain expertise provides was entirely absent.

What This Means

Back to the intelligence explosion hypothesis from my first post. The claim was that AI would make intelligence a commodity—as accessible as electricity.

This experiment suggests not yet, but the direction is clear.

What AI provides is better described as "capable undergraduate researcher"—able to execute well-specified tasks but requiring supervision for judgment-dependent work. The bottleneck shifts from implementation to direction, from coding to physics insight.

This is genuinely useful. A 5-10x speedup on implementation tasks is meaningful. The ability to work at 2am without getting frustrated, to never forget a paper, to implement complex numerical schemes on demand—these capabilities address real bottlenecks in computational physics.

But the essential human role in scientific discovery remains. Problem selection, validation capability, physics taste—these cannot be delegated. AI amplifies what you already know; it doesn't replace knowing things.

There's one question I haven't addressed: if you're not reading the code, how do you know it's correct? The answer involves what I call physics-oracle validation—using physics itself as the specification against which code is tested.

That's the subject of the next post.

Acknowledgements

Thanks to everyone following along with this experiment. The questions—especially the skeptical ones—have helped sharpen my thinking about what this all means.

The paper is available at github.com/anjor/gandalf-paper. The code is at github.com/anjor/gandalf.

Writing a Physics Paper with Claude: What Actually Happened

In a previous post, I documented building a plasma turbulence solver with Claude—3,000 lines of JAX I never read, validated entirely through physics outputs. That post ended with: "we're writing a proper journal paper."

The paper is now live on arxiv. This post covers what happened next: writing the paper itself. The failure modes were completely different.

Orszag-Tang vortex simulation Current sheets and vortex structures forming in an Orszag-Tang simulation—the kind of physics GANDALF captures

The numbers sound impressive: 143 Claude Code sessions, 23 GitHub issues closed, 20 pull requests merged. But the headline obscures the real story. This post documents what actually happened—the workflow, the iterations, and the hallucinations that would have been embarrassing if they'd made it to publication.

The Workflow Architecture

The approach that made this possible wasn't magic. It was infrastructure.

GitHub Issues for everything. Each paper section got an issue. Each benchmark got an issue. Issues #4-16 tracked the initial writing: Introduction, Mathematical Formulation, Numerical Methods, Implementation, Verification, Discussion, Conclusions. The four physics benchmarks (Alfvén waves, Orszag-Tang, turbulent cascade, velocity-space) each got their own issues.

GitHub issues list 23 GitHub issues tracked every section and benchmark

Section-by-section PRs. Each section was a separate pull request. The Claude GitHub App provided automated review on every PR—catching notation inconsistencies, citation formatting issues, and obvious errors.

Human review issues. After completing initial drafts, I created human review issues (#37-42) for each section. This is where I sat down and actually read what Claude had written. Issue #55 was a final comprehensive review. These reviews were not optional polish.

External AI review. I also ran the draft through Gemini 3 Pro (Issues #53, #58) for a different perspective. Different models catch different errors.

The git history tells the iteration story better than I can:

a03703d Implement turbulent cascade spectrum benchmark (Issue #10)
241b327 Address reviewer feedback on turbulent cascade PR #33
536eba5 Address reviewer feedback on PR #33
fb7d583 Replace synthetic data with real N64 turbulent cascade results
9b590ae Fix critical physics and notation issues in turbulent cascade section
f36db7c Address final reviewer feedback on PR #33
... (14+ iterations on this single PR)

PR #33 for the turbulent cascade section went through fourteen revision cycles before merging. This was not "Claude writes a paper." This was iteration.

What Claude Did Well

Credit where it's due. Claude was genuinely useful for:

Initial drafting. Given mathematical specifications and paper structure, Claude generated coherent first drafts of each section. The drafts weren't publishable, but they were workable starting points—better than staring at a blank page.

LaTeX formatting. Equations, figures, notation consistency, bibliography formatting. The mechanical aspects of scientific LaTeX were handled reliably.

Addressing specific feedback. This is where AI assistance shines. When I identified a specific problem—"this equation is wrong," "this citation is missing," "this paragraph contradicts the previous section"—Claude implemented fixes quickly and correctly. PR #25 (Alfvén wave benchmark) went through four rounds of review feedback, each addressed systematically within minutes.

Literature integration. Given a topic, Claude could find relevant citations and format them properly. It knew the key papers in plasma turbulence.

The Hallucination Problem

Now for the part that matters.

During human review (Issue #55), I found fabricated content that Claude had written with complete confidence. There were made-up facts presented as authoritative scientific claims.

Issue #55 hallucinations Issue #55: Human review caught fabricated Princeton cluster claims, false timelines, and invented benchmarks

Fabricated benchmark timings:

"These timings were obtained on identical 80 GB A100 nodes on Princeton's Stellar cluster to ensure an apples-to-apples comparison."

Every benchmark in this paper ran on my M1 MacBook Pro. I have never had access to Princeton's Stellar cluster. Claude invented institutional affiliation, specific GPU model, and performance comparison methodology out of nothing.

False development timeline:

"Three years of development and production use provide empirical evidence for this decision's trade-off"

"GANDALF reached research-grade maturity within three years of part-time solo development."

The actual development time was approximately one month, with Claude assistance. Claude inflated this by a factor of 36.

Invented GPU runtimes:

"A moderate-scale turbulent cascade (N = 128³, 50,000 timesteps) completes in ~7 hours on a single NVIDIA A100 GPU. An optimized CUDA code might complete in ~2.5 hours"

No GPU simulations were performed. These runtime numbers were fabricated. The comparison to "optimized CUDA code" was invented.

Made-up community claims:

Claude wrote an entire "Community growth potential" subsection filled with fabricated claims about user adoption, classroom deployment, and community engagement. None of it had happened.

Physics errors:

Beyond fabrication, there were physics mistakes that required domain expertise to catch: - Wrong definitions of g± (combinations of density and magnetic fluctuations) - Incorrect cascade direction claims - Misinterpretation of gyrokinetic orderings - Missing discussion of the velocity-space benchmark

The key insight: Claude was equally confident in true statements and fabricated ones. The prose read identically. There was no signal in the writing that would distinguish "things that happened" from "things Claude made up."

The Human Review Cycle

Issue #55 alone contained 40+ specific corrections across all sections. The pattern:

Physics errors requiring domain expertise. When Claude wrote that "compressive fluctuations are driven by Alfvén waves," I had to know enough physics to recognize this was wrong—they're mixed by Alfvén waves, not driven by them. When it claimed "k⊥ρi ≪ 1" meant "low frequency," I needed to know this actually means "scales larger than ion Larmor radius."

Notation inconsistencies. Claude used lowercase φ for the stream function in some places, uppercase Φ in others. The Elsasser fields were sometimes ξ±, sometimes z±. These required systematic correction.

Missing content. The Discussion section had no mention of the velocity-space benchmark, even though it was a major contribution. Claude simply forgot to include it.

Fabricated quantitative claims. Every specific number needed verification against what actually happened.

These reviews weren't polish, but the difference between a publishable paper and an embarrassing one.

The Real Workflow

143 Claude Code conversation sessions for this paper. What did that actually look like?

A typical session: open an issue, tell Claude to draft that section, review output, create PR, Claude GitHub App reviews, I review, create issue with corrections, Claude addresses corrections, iterate.

Claude GitHub App review Claude GitHub App provided automated review on every PR

The "speed" of AI-assisted writing was iteration speed, not magic. Each round of feedback could be addressed in minutes instead of hours. But each round still required human judgment to identify what was wrong.

The ratio matters: Claude could implement changes 10x faster than I could. But identifying what changes to make remained 100% human.

Lessons Learned

AI drafting ≠ AI writing. Claude can draft. But drafting is maybe 20% of writing a paper. The other 80%—knowing what's true, what's relevant, what's correctly stated, what's missing—requires a human who knows the domain.

Hallucination risk is highest for quantitative claims. The fabricated content was overwhelmingly specific numbers, timelines, and institutional details. Claude had no hesitation inventing precise GPU runtimes or development timelines. Every quantitative claim needs verification.

Structured workflow creates an audit trail. Issues, PRs, and review cycles meant I could trace every change. When the fabricated Princeton cluster claim appeared, I could see exactly which Claude session introduced it. This transparency matters.

AI excels at iteration on specific feedback. Tell Claude exactly what's wrong, and it fixes it correctly. Ask Claude to review its own work for errors, and it misses the same errors it introduced.

Domain expertise cannot be delegated. The physics errors—wrong definitions, incorrect cascade descriptions, misinterpreted orderings—were invisible to anyone without plasma physics training. AI assistance amplifies what you know. It doesn't replace knowing things.

The Numbers

For the record:

  • ~3 weeks calendar time (Nov 7 - Nov 26, 2025)
  • 143 Claude Code conversation sessions
  • 23 GitHub issues closed
  • 20 pull requests merged
  • Multiple human review passes (Issues #37-42, #55)
  • External AI review (Gemini 3 Pro, Issues #53, #58)
  • Final paper: 6 sections, 4 physics benchmarks

Conclusion

This post is the honest version of "I wrote a paper with AI assistance."

Claude helped. The iteration speed was real. The infrastructure—issues, PRs, reviews—made it manageable. But the fabrications were also real. Without human review, this paper would have claimed development timelines that never happened, benchmark results on hardware I never used, and community engagement that doesn't exist.

The paper is correct now because I caught those errors. Not because Claude didn't make them.

Paper: arxiv:2511.21891

Code: github.com/anjor/gandalf

Paper repo: github.com/anjor/gandalf-paper

  • in fde
  • 2 min read

The FDE Manifesto: What Would Stokes Do?

This is a version of a document I had written during my time at Palantir.

Stokes was a legendary FDE at Palantir, and I learnt a lot by emulating his way of operating. This wasn't hero worship—it was shorthand for a specific way of operating that separated exceptional FDEs from the rest.

Here are the tactical principles that define that approach.

Never Take Shortcuts That Compound

  • Don't make config changes that aren't product defaults
  • Never restart services "just to see if it fixes things"—you're destroying evidence
  • Never deploy dirty builds. If you're on a branch, getting back to mainline is P0
  • Every custom modification is technical debt with compound interest

Own the Product Stack

  • Clone the repos. Make them build. Start submitting PRs
  • When you get a stacktrace, read it. Form a hypothesis before opening tickets
  • Know the architecture cold—when telemetry fails, it's your only map
  • If a workflow is blocked on a feature, scope it and build it yourself

Treat Information as Infrastructure

  • Ensure logs and metrics are collected properly from day one
  • Master telemetry tools—they're not optional
  • Read everything: docs, runbooks, release notes, support tickets, Stack Overflow
  • Monitor what other teams are encountering—their problems will be yours soon

Root Cause Everything

  • Never accept "it's working now" as resolution
  • Gather data and form hypotheses before implementing fixes
  • You must be able to explain why the product broke and why your fix worked
  • Document your debugging process for future you

Build Strategic Relationships

  • Every support ticket is a relationship-building opportunity with core engineers
  • Contribute field signal to product direction—you're their eyes and ears
  • Know the product team's roadmap and weigh in based on deployment reality
  • Getting your tickets prioritized is a function of relationships, not just severity

The Strategic Thread

These aren't random best practices. They're behaviours that compound into a strategic advantage: radical ownership of outcomes.

When you refuse shortcuts, you're choosing long-term system health over short-term wins. When you master the product stack, you're becoming a peer to the product team, not a consumer. When you root cause everything, you're building institutional knowledge that makes future problems trivial.

This is what makes the FDE model powerful. You're not there to implement solutions—you're there to own outcomes completely. That ownership manifests in these specific, tactical behaviors that seem small but fundamentally change how you operate.

The question isn't whether you can do these things. It's whether you will.

Building an FDE organization? These principles are your hiring rubric. Look for engineers who already think this way.

These principles form the evaluation criteria in my FDE Interview Scorecard.

Implementing an FDE hiring program? See my FDE Advisory Materials for interview templates, scorecards, and detailed process guides.

The Unreasonable Effectiveness of Hiring Assholes

I've seen this pattern play out dozens of times. The brilliant engineer who tears apart your architecture in design reviews. The physics Nobel laureate who's a complete dick but moves research forward. The Palantir FDE who makes people uncomfortable but somehow always ships.

They're assholes. And they get results.

So there's this tempting logic: Maybe we need to hire more of them?

A friend at a startup just told me something interesting. "Everyone here is so humble," she said. Then she paused. "Maybe too humble. Nobody fights for their ideas. Sometimes it feels we're not ambitious enough"

We've created a false choice:

Option A: Hire nice, humble people → Get a room full of diffidence

Option B: Tolerate brilliant assholes → Get results but destroy morale

Is this a true dichotomy?

Were the assholes effective because they were assholes? Or despite being assholes?

What if the causality is backwards?

They had conviction. They were direct. They were ambitious. They pushed back on bad ideas.

They also happened to be assholes.

We saw correlation and assumed causation. We thought the cruelty was necessary for the conviction.

Let me give you a counter-example.

Paul Mustiere spent 8 years at Palantir. I was his mentor when he interned. A few months ago, he joined Comand AI as Head of Engineering.

Paul is one of the highest-agency people I've ever worked with. He gets things done. He'll challenge your technical approach. He'll push back on bad ideas. He sets ambitious visions and rallies teams around them.

He's also genuinely humble. Low ego. Makes people around him better.

You don't have to choose between conviction and decency. Paul is living proof.

But even if you believe tolerating assholes gets short-term results, what's the actual cost?

The good people who quietly leave. The collaborative culture you never build. The institutional knowledge that walks out the door. The junior engineers who learn that being right matters more than being decent.

You're not being pragmatic by ignoring human cost. You're taking on technical debt in your culture. And like all technical debt, it compounds.

So here's the reframe:

Stop asking: "Should we hire assholes?"

Start asking: "How do we hire for conviction, directness, and ambition without hiring for cruelty, ego, and disrespect?"

Because these are separate traits.

You can have the physicist who challenges every assumption AND treats grad students with respect.

You can have the engineer who rewrites your architecture AND makes you feel good about the collaboration.

You can have the leader who sets an ambitious vision AND brings people along.

The framework for hiring:

Must-haves:

  • High conviction (will fight for what they believe)
  • Intellectual honesty (will change their mind when wrong)
  • Directness (will tell you the truth)
  • Ambition (wants to build something great)

Deal-breakers:

  • Making it personal
  • Cruelty for cruelty's sake
  • Ego-driven (caring more about being right than finding truth)
  • Disrespecting people even while disagreeing with ideas

In an interview, this looks like:

  • Someone who challenges your technical approach → good signal

  • Someone who challenges it and makes you feel stupid → red flag

  • Someone who says "I think you're wrong about this architecture" → high conviction

  • Someone who says "I can't believe you'd even consider that approach" → asshole

The unreasonable effectiveness of hiring assholes? It's a myth.

What's actually effective is hiring people with conviction.

Some of them happen to be assholes. That's not the part that makes them effective. That's the part that will eventually destroy your company.

Don't confuse the two.

What's your experience? Have you seen companies successfully separate conviction from toxicity? Or is this just naive optimism?

Implementing an FDE hiring program? See my FDE Advisory Materials for interview templates, scorecards, and detailed process guides.

Building a Gyrokinetics Code Without Reading a Single Line: The Development Log

In the first post, I outlined an experiment: can AI make intelligence a commodity in physics research? Two weeks of intensive work later, I have a modernized gyrokinetics code running on my laptop. The catch? I haven't read a single line of the ~3000 lines of JAX it contains.

This post documents what that process actually looked like—the workflow that emerged, the surprising failures, and the honest assessment of what worked and what didn't. If you're a physicist considering AI-assisted development, this is what you should know.

The Constraints

After reaching out to the Viriato team, it became clear I'd need HPC access I no longer have. So I decided to revive and modernize GANDALF, my PhD-era code. The constraint was simple: it needs to run on my M1 Pro MacBook.

I pointed Claude Code at the original GANDALF repository and the relevant chapter from my PhD thesis. I asked it to draft a plan and file GitHub issues for each step in that plan. It created a comprehensive set of issues covering everything from basic spectral methods to turbulence diagnostics.

The plan was straightforward: port from CUDA/Fortran to JAX with Metal backend, validate against known benchmarks, then extend to multi-ion physics.

I am not familiar with JAX. I also haven't written Fortran or CUDA in a decade. This would be a pure test of whether AI could bridge that gap.

The Workflow That Emerged

The process settled into a rhythm:

  1. I ask Claude Code to pick the next issue from the GitHub tracker
  2. Local Claude Code works on the issue and opens a PR
  3. GitHub Claude (I installed Claude on the repo) reviews the PR
  4. I selectively decide which feedback matters and what to ignore
  5. Repeat

The dual Claude setup wasn't planned—it emerged from necessity. I needed something different to review the code to keep it honest and prevent drift. Think of it as having two smart undergraduates check each other's work.

My role was purely validation through physics outputs. I modeled myself as a PhD advisor: I don't read the student's code, I look at their plots and ask if the physics makes sense. When something was wrong, I'd start by showing the plot. Often Claude would say something incorrect, and I'd need to push back with physics insights until we converged on the right answer.

This is critical: I validated entirely through physics, never through code inspection.

What Worked Surprisingly Well

Getting basic physics running was shockingly easy. Within the first week:

  • Alfvén wave dispersion relations matched theory
  • Energy conservation held to machine precision
  • The Orszag-Tang vortex benchmark reproduced correctly

Some of the more advanced benchmarks are still in progress—getting clean turbulent spectra with the expected -5/3 scaling has proven trickier and I'm still working on it.

Figure 1: Orszag-Tang vortex at t=4.0 Alfvén times, showing the emergence of complex turbulent structures. The code correctly captures the vorticity filaments, current sheets, and magnetic field topology characteristic of 2D MHD turbulence.

Orszag-Tang Vortex Structures

Figure 2: Energy conservation over 4 Alfvén times. Total energy (black) remains constant to better than 0.01%, while kinetic (red) and magnetic (blue) energy exchange through turbulent dynamics. This level of conservation validates the spectral time-stepping algorithm.

Orszag-Tang Energy Conservation

Figure 3: Performance scaling on M1 Pro MacBook. A 128³ 3D simulation completes each Poisson solve in 28ms, putting useful turbulence simulations (hundreds of time steps) within reach of laptop hardware. The practical working range (green) shows what's actually feasible for iterative physics exploration.

Performance Scaling

Claude wrote 100% of this code. Not 90%, not 95%—literally every line. I provided physics corrections when needed—catching things like KRMHD vs KREHM orderings, explaining why slow modes should be treated as passive scalars, and designing the validation tests themselves. But I never wrote a single line of code.

The speed was remarkable. Tasks that would have taken me days as a PhD student (debugging FFT boundary conditions, implementing spectral methods, setting up proper diagnostics) were done in hours.

Where It Struggled: The Physics-Numerics Boundary

Advanced benchmarks proved much trickier. The problem wasn't coding—it was understanding the deep connection between physics and numerics.

The Spectral Integrator Problem

My numerical algorithm is non-standard: it's a spectral method that gets linear physics exactly right by integrating those modes analytically. Claude saw "time integration" in the thesis, found "RK4" somewhere in the literature, and implemented bog-standard Runge-Kutta.

I had to explain multiple times: we're not approximating the linear physics, we're solving it exactly in Fourier space, then handling only the nonlinear coupling numerically. This is the whole point of the algorithm—it eliminates spurious damping of weakly damped kinetic modes.

Eventually it got there, but it took persistent correction. The AI didn't have the physical intuition for why this matters.

The Forcing Coordinate Confusion

I specified that forcing should happen at large length scales: k=1,2 in Fourier space. Claude applied this condition to k_perp (because k_perp matters more than k_z in RMHD), but ended up forcing all k_z modes at those perpendicular wavenumbers. This caused immediate numerical instability—the simulation would blow up within a few time steps.

The fix required explaining the physics: we need to force specific 3D wavevectors, not all modes sharing a perpendicular wavenumber. This seems obvious in hindsight, but demonstrates how the AI can misunderstand the dimensional structure of the problem.

When tuning simulations, Claude's intuition about the forcing-dissipation balance was consistently off, but in a subtle way that reveals something about how physicists think versus how AIs think.

As a physicist, you're always trying to extract maximum physics from your computational box. You want to maximize the inertial range to get a clean power law spectrum. This means running as close to the edge of numerical instability as possible. A simulation that produces beautiful physics for 20 Alfvén times and then blows up at 25 Alfvén times is perfect—you use the data from the first 20 time units. The code is a tool to do physics; it's not important on its own.

Claude's instinct was the opposite: make the simulation stable and robust. When it saw signs of instability, it would suggest increasing dissipation (which kills your inertial range) or reducing forcing amplitude (which weakens the physics you're trying to study). These are technically valid numerical choices, but they optimize for the wrong thing.

The right approach is to tune parameters to get as close to instability as possible without crossing the line. This requires physical intuition about what's actually happening in the simulation, not just numerical stability analysis.

November 9: The $40 Day

The usage data tells a story. Most days cost $2-10. November 9 cost $40.

That was the day I tried to get nonlinear turbulence running properly. The simulation would run, but the physics was wrong in subtle ways. Energy would cascade, but not to the right scales. Heating rates would be off by factors of 2-3. Spectra would show the right scaling but wrong amplitudes.

The problem was that nonlinear turbulence requires everything to be right: the forcing must excite the correct modes, the dissipation must operate at the right scales, the time-stepping must preserve important invariants, and the diagnostics must actually measure what you think they're measuring.

I shifted from Sonnet to Opus hoping for better physics reasoning. It helped marginally, but I kept hitting limits. The AI could implement each piece correctly in isolation, but struggled to see how they fit together into a coherent physical picture.

We're still working on this. Some problems just take time, even with AI assistance.

The Skills That Actually Mattered

Here's what surprised me: I didn't use my tech background at all. I didn't debug code, suggest algorithms, or catch Python syntax errors.

What I did use:

Physics intuition: Knowing when results are physically wrong, even if numerically stable. Understanding that spectral pile-up means one thing while energy conservation violations mean something else entirely. Recognizing that a simulation optimized for stability is often a simulation optimized away from interesting physics.

Applied AI intuition: Designing the dual-Claude review pattern. Structuring the workflow around incremental validation through physics benchmarks. Understanding AI failure modes and building guardrails around them. Knowing when to push the AI harder versus when to step in with physics corrections.

This second skill is crucial and under-discussed. It's not prompt engineering—it's something closer to understanding how to architect human-AI collaboration at the systems level.

The Replicability Question

A friend asked: how much of an "n of 1" are you? Could a physics PhD with zero coding background do this?

Honest answer: not yet, at least not with the current setup.

The bottleneck isn't coding ability—the AI handles that. The bottleneck is catching physics-numerics errors before they compound. By the time you see wrong results, you're often many commits deep into a wrong path.

A physicist without coding experience wouldn't know to set up the dual-Claude review pattern, wouldn't think to validate incrementally through physics benchmarks, wouldn't catch the spectral integrator mistake until much later.

Could this be taught? Could I package the workflow into something a pure physicist could use? I genuinely don't know. That's an open question.

The Honest Productivity Assessment

The original GANDALF took me 6-7 months to build as a PhD student, working full-time. The new version took 30 days as a side project.

But this isn't quite an apples-to-apples comparison:

  • PhD me was less experienced, had never written serious scientific code before
  • Current me could probably write this faster by hand than PhD me could
  • This is part-time work vs full-time

Even accounting for these factors, the productivity gain is real. I'd estimate 5-10x faster than I could have done solo, even with my current skills.

But it's not "intelligence as a commodity" yet. It's more like having an exceptionally capable research assistant who never gets tired, never forgets papers they've read, and can implement complex numerics at 2am without complaint.

The creativity, problem selection, and physics intuition remain entirely human. The AI amplifies what you already know; it doesn't replace knowing things.

What's Next

The code is ready. The benchmarks are passing (mostly). The parameter space is mapped.

But there's an intermediate step: we're writing a proper journal paper documenting GANDALF itself. Using another Claude-assisted workflow in the gandalf-paper repository, we're producing a comprehensive code paper targeting the Journal of Plasma Physics. This uses a different set of AI agents specialized for scientific writing—latex-equations, literature-curator, benchmark-analyst, physics-narrator, and code-documentor—working together to produce publication-quality text.

Then comes the actual physics test: can we discover something genuinely new about multi-ion turbulent heating? Can this AI-augmented approach produce insights worthy of publication as a second paper?

The next post will document that process—the physics investigation itself, what worked, what failed, and whether this experiment ultimately validates or refutes the intelligence explosion hypothesis.

The Data

For transparency, here's what this cost in Claude API usage:

  • Total: $307.19 over 16 active days (spanning 30 calendar days)
  • Average per active day: $19.20
  • Peak day (Nov 9, wrestling with nonlinear turbulence): $41.88
  • Total tokens processed: 522M

Compared to my computational budget ($10K), this is negligible. Compared to the cost of hiring a programmer for a month, this is absurdly cheap. The constraint isn't money—it's my time to direct the work and validate the physics.

Acknowledgements

Thanks to the Claude team at Anthropic for building tools that actually work for technical research. And to everyone who's been following along with skeptical but curious questions—you're helping me think through what this means.