The Rules Are Half the Physics

In the first post I made a promise that sounded almost too convenient: in a regulated industry, you do not have to infer the rules from a pile of data, because the rules are already written down. Someone published them. You can build your world directly from them, the way a game builds a cave system from cave-generation rules instead of hand-placing ten thousand caves.

I still believe that. We spent the last stretch stress-testing it against a genuinely ugly real-world regulated system, one of the largest and most litigated in the country, and it held up, with a correction stapled to it that turned out to be more useful than the original idea.

The rules give you a skeleton, not a world. They are exact where they apply and completely silent everywhere else, and getting that boundary wrong in either direction is how these projects quietly fail.

And then something I did not expect at all. The same rulebook that makes this work so hard hands you a thing that world-model builders in every other domain would pay enormous money for: a free test set.

Before the argument, here is the whole thing as three experiments. Each one runs the same setup twice in worlds that differ by exactly one thing, and each is scored on how close it gets to a stated goal. Close the streets on the first one and watch which model still gets home.

The rest of this post is why those three experiments are the ones worth running. It starts with taxis.

The Manhattan taxi problem

This is my favorite AI result of the last few years, and it is the drunk-cats story of world models.

A team of researchers trained AI models on taxi routes through Manhattan. Millions of trips, turn by turn. By the obvious measure the models were nearly perfect: next-turn predictions valid close to 100 percent of the time, and the routes they produced legal and sensible between 96 and 99 percent of the time.

Then the researchers reconstructed the map those models had implicitly learned, by asking them a lot of questions and drawing the streets the answers implied. The map was nonsense. Streets running in physically impossible directions. Roads crossing where no overpass exists. The models could navigate Manhattan beautifully while carrying an internal picture of Manhattan that could not exist.

That alone would be a good story. Here is the part that changed how we work. The researchers added detours: close a street at random, force the model off its usual path, and see if it can still get you home.

The model trained on shortest paths went from 0.99 accuracy, to 0.69 when one percent of trips hit a detour, to 0.08 at ten percent, to 0.00 at fifty percent. Not degraded. Gone. A second model trained on near-optimal routes did the same thing. A third had been trained on random walks, aimless wandering, the kind of trajectory no real taxi would ever drive, worse data by every normal measure. It held at 0.99 through a ten percent detour rate and was still at 0.97 when half of all trips were detoured, working fine while both of the “good” models sat at zero.

The model trained on the garbage data was the one that actually understood the city.

We rebuilt this from scratch on a small grid to make sure we believed it, and we do:

Once you see why, you cannot unsee it. The shortest-path models only ever saw optimal routes, so they never visited the streets a detour puts you on, and their internal picture had no reason to include them. They were not modeling Manhattan. They were modeling good trips through Manhattan, which is a much smaller thing that looks identical right up until the moment you need it not to be.

So the real question when you build a world model is not “what should it predict.” It is “where do I get detours.”

Hold that thought. It is the whole post.

What a world model actually needs

Four things. Only the fourth is hard.

1. A state. Something you can be in.
2. An action. Something you can do.
3. A transition. A rule that takes a state and an action and gives you the next state.
4. Coherence under intervention. When you do something the model never saw in training, it still behaves sensibly.

Most things sold as world models have the first three and quietly skip the fourth. A good number do not really have the third either.

One distinction earns its keep in real meetings. Any model of a changing system has two halves: one that says how the world evolves, and one that says what your instruments see. Nearly every “world model” I have reviewed inside a regulated company is entirely the second half, a careful, expensive, genuinely valuable description of what the data means, at what grain, from what source, and how it lies to you.

That is an observation model. Good thing to have, but you cannot run it forward, and calling it a world model starts a conversation that always ends the same way: “so it can forecast?” No. Getting this right is not pedantry. It tells you exactly what you are missing.

Four things that make this hard when there is a regulator in the room

One: nothing runs on the same clock, and you did not pick the clocks

The failure I look for first, because it silently ruins more work than anything else here. In a regulated system, different parts tick at wildly different rates, and most rates are set by law:

• Contracts renew once a year and are frozen in between.
• Eligibility changes annually, in a window.
• Transactions settle over weeks or months, then get revised.
• Some money lands two or three quarters after the event it belongs to, and lands as a lump at a completely different level of detail than the event.
• Scores get restated several times for a period that already ended.
• Policy can change any day at all, in the middle of a contract.

Now watch two of those collide. Almost every performance number in a regulated business is a fraction. If the top counts events on one clock and the bottom is recalculated on another, the fraction moves for reasons that have nothing to do with what it claims to measure. Drop your most expensive customers during the year, compute the year’s rate against a population recalculated at the end, and your numbers improve. Nobody improved anything.

We looked at a case where a genuinely impressive published improvement landed in the same window as a government program that independently pushed behavior the same way, in overlapping regions. Two causes, one effect, no control group. The number is probably not wrong. It is uninterpretable, which is worse if you were planning to train something on it.

Most “model failures” in regulated industries are not model failures. They are two clocks colliding.

Before I believe any rate a regulated system reports about itself, I ask three things:

• What is the denominator, and when was it frozen?
• What else changed inside the same window?
• Which date is this built on: when it happened, when it settled, or when it was last revised?

You do not need a model to ask those. You usually need to be inside the company to answer them, which is itself worth knowing.

Two: the world reads your model

A hurricane does not read the forecast. This is the deepest difference between modeling weather and modeling a regulated business, and almost nothing in the world-model literature has a category for it.

In a regulated system, the quantity you are modeling is produced by people who are paid according to the model. A scoring model that determines payment changes what gets written down. A price benchmark feeds the negotiations that produce next year’s prices, which are your next input. A rule about what gets approved changes what gets requested. Goodhart’s law stops being an aphorism on a slide and becomes a force inside the dynamics.

My favorite evidence is not a paper but a regulator conceding the point inside its own arithmetic. One federal payment formula we worked through contains an explicit term scaling for the fact that people consume more when their coverage is more generous. The regulator wrote the feedback loop into the formula, because pretending it was not there would have been indefensible.

Two rules fall out, both cheap on day one and brutal to retrofit:

Some quantities are things to measure, never things to optimize. The moment a measurement becomes a target in a system where the measured party generates the measurement, you have built a machine for producing the measurement instead of the thing.

Provenance is a required field on every number, not metadata. If a value can arrive by several different routes, and some of those routes are contested, then any total you cannot break down by route is a total you cannot defend.

Three: the fence around your data is legal, not technical

This one surprises engineers every time. Inside a big regulated group you will find data the company unambiguously owns, at enormous scale, that the modeling team is not allowed to use. Not a pipeline problem: a governance firewall, a consent decree, or a statute. The usable data estate is smaller than the data estate, and the most distinctive asset in the building is often the one you are least free to touch.

The compensation is a pattern we have hit three separate times, and it has stopped feeling like coincidence. The compliance-safe design and the technically correct design keep turning out to be the same design. Model groups rather than individuals, and you can answer “what if” without a single person’s record entering the system. Build only on what was published under a disclosure rule, and your provenance story writes itself. Describe a distribution instead of emitting a recommendation, and you avoid an entire category of liability while shipping the more honest object anyway.

When a constraint points at the same architecture from three directions, that is usually the architecture.

Four: if it works, everyone uses it, and then everyone is wrong together

Short, but it belongs here. There is agent-based simulation work on the insurance industry showing that when too few distinct risk models are in use, default risk rises and industry profits fall. Model diversity is not just prudent; it measurably improves outcomes for participants.

For anything benchmark-shaped there is a nastier version: your output becomes next year’s input. A widely adopted benchmark stops measuring an independent market and starts measuring its own past output, and the variation that made it worth having compresses toward whatever number you published.

That is not a distant risk. It is the success case.

The turn: the rulebook is also the fix

Now the good half, and it is better than I expected going in.

Gift one: some transitions are exact, and free

Buried in the statutes are transitions that fire on a calendar and reference nothing about the condition of the thing they act on. After a fixed number of months, responsibility moves from one party to another. A status ends a fixed period after a triggering event. A weight phases in at 33 percent, then 67, then 100.

These need no estimation, no data, and no confidence interval. They are already exact: pure “what if” structure, sitting in a public document, free.

The design move this implies is to split the state into factors that move for different reasons:

• a factor that moves by statute and calendar, which you compile rather than learn, with zero fitted parameters
• a factor that moves randomly, which is the only part you actually fit
• a factor that is simply observed

Then model the arrows between them, and pay particular attention to the arrow running from the rule-governed factor into the random one. That arrow is the feedback loop from hard part two, and every model fitted on raw history gets it wrong, because in raw history the rule change and the behavior change show up at the same moment and look like one event.

This is a refinement of “build the world from the rules”, not a retreat from it. The rules generate the skeleton. Controlled randomness fills in the rest. It is the same shape as writing cave rules instead of drawing caves, moved up one level.

Gift two: free detours

Remember the taxis. You need detours, and in most domains getting them is painful or impossible. You cannot close streets in a live business to see whether your model understands it.

In a regulated industry, the regulator publishes the detours in advance, with dates on them.

They are everywhere once you start looking:

• Phase-ins. A weight moving 33, then 67, then 100 over three years is a dated shock to exactly one part of the system and nothing else.
• Partial mandates. A program that becomes compulsory in 188 named markets and applies nowhere else is very close to a randomized experiment somebody else paid for.
• Calendar transitions. A clock that changes which party pays after a fixed interval, with no reference to anyone’s actual situation.
• Definition changes, which are the sharpest tool in the box and get their own section below.

Hold these out of training. Then check whether the model predicts them. That is a real test of understanding: it needs no ground truth, it cannot be passed by training longer, and a model can ace every accuracy metric you own and fail it completely.

Show me: the definition-change test

Enough theory. Here is the one I would run first, in five steps, on whatever you are building.

Of all the free detours, the sharpest is a change in how something is measured, on a known date, where the underlying reality did not change at all.

We worked through a case where a clinical measurement formula was officially revised on a single day. A large number of people moved between severity categories immediately. Nothing about any of those people changed. The map moved. The territory did not.

That is a scalpel. A model that predicts a real change in the world at that boundary has confused the map with the territory, and you find out in one test. Here is that test run on a simulated population, where we know the truth exactly:

Step 1. Find your date. Every regulated industry has several of these lying around: a reclassified category, a revised index, a changed threshold, a restated definition. Write down the date and the exact scope of what the change touched.

Step 2. Split on the date and train only on “before.” No peeking. The whole point is that the model has never seen the new definition.

Step 3. Pre-register your predictions. This is the step everyone skips and it is the one that makes the test honest. Before you run anything, write down two lists: what must move, and what must not.

change:  measurement formula revised, effective YYYY-MM-DD
scope:   affects how the category is computed, not what is being measured

MUST MOVE (the map)
  - distribution across severity categories shifts on the date
  - shift size matches the published effect of the revision
  - the shift is a step, not a ramp

MUST NOT MOVE (the territory)
  - underlying event rates
  - downstream outcomes for the same people
  - anything measured by an instrument the revision did not touch

Step 4. Roll the model forward across the date. Do not retrain. Do not tune. Just let it run into the change the way it would in production.

Step 5. Score two numbers, not one.

map_score       did the measured quantity move as the rule says?
territory_score did the underlying quantity correctly stay put?

coherent  = map_score high AND territory_score high
confused  = map_score high AND territory_score low   <- the dangerous one
blind     = map_score low

The failure everybody expects is “blind”: the model misses the change. The common failure is confused. The model dutifully predicts the world got worse on the day the ruler changed length, because in every year of training data a rising category count really did mean a worsening population. That model passes your accuracy dashboard forever and is wrong every time a rule changes, which, in a regulated industry, is constantly.

Run this on one dated definition change this quarter. It is cheap, it is decisive, and I have not yet seen a team run it and learn nothing.

Where this lands in a real industry

Abstraction is easy to nod along with and hard to act on, so here is the concrete version in one of the most heavily regulated and most publicly scrutinized industries there is: US health insurance.

These companies are unusually open about where they are deploying AI. Below is a map from what they say they are doing to which hard part it runs into. Everything in the left column is a public statement or a public report, marked as company claim, regulator or congressional finding, or unresolved allegation, three different things, and collapsing them is how commentary about this industry usually goes wrong.

This is not one company’s problem, which is what makes it urgent rather than academic. Across roughly 53 million coverage determinations in 2024, denial rates by parent company ranged from 4.2 percent to 12.8 percent, a threefold spread across companies applying the same federal rules. Of the denials that were appealed, 80.7 percent were overturned. In July 2026, senators sent bipartisan letters to three of the largest insurers asking for records on AI in post-acute denials, and asking each to confirm whether it remains policy that a final denial cannot be made by AI. Read that last clause as a product requirement, because that is what it is.

Notice the shape of the table. Not one of these is a modeling problem. Every one is a clock, a feedback loop, a legal boundary, or an attribution question. That is the argument of this post in miniature, and it is why the interesting research here is not bigger models.

What it looks like when you actually go and get the data

Everything above is a claim about structure. Here is the part that only shows up once you try to build the thing, from our own work pulling the machine-readable price files that federal rules require insurers to publish. These are notes from live runs against real sites, not a design document.

A file can be valid and stale at the same time. One payer’s content network keeps rotated-out index files live for about ninety days after a newer one replaces them. The old URL still returns 200, still parses, still passes schema validation. Any check shaped like “the known URL responds and looks right, so we are done” resolves happily onto a two-month-old file and never learns a newer one exists. Validity is not currency, and only one of those is easy to test. We deliberately seed the checker with a URL old enough to be genuinely dead, so the discovery ladder is forced to escalate rather than settle.

The label describes the payload, not the bytes. One payer serves its index as a gzipped object with a content type announcing plain JSON, because the header describes what you get after decompressing. Read the raw bytes as text, as any reasonable scanner does, and you get binary garbage and conclude the marker you were looking for is absent. The fix is to sniff the magic bytes and ignore what the label claims. This is the same error as the whole post, four layers down: a difference in a label read as a difference in the world.

“The payer’s file” is sometimes not a thing. One national insurer publishes no single master index. Its files are per-employer-group, found through a client-side search tool. So the unit you assumed existed does not, and “ingest the file” quietly becomes “decide which employer groups matter,” which is a modelling decision wearing an engineering costume.

Absence of evidence arrives on a timer. That same search tool populates its results through a slow client-side call. A fixed pause after page load produced twenty-one results on a good run and zero on a bad one, and zero looks exactly like “this payer publishes nothing.” Polling instead of sleeping fixed it. Any pipeline that records a null without recording how hard it looked is manufacturing false negatives.

Some doors are held shut. One payer’s index sits behind a signed URL that expires on a date, so it must be re-resolved on every scheduled run rather than stored as a link. Another renders its download as a button handler rather than a link, so scraping anchors finds nothing while the file sits one click away. A third serves a bot-management challenge to every client we tried. We have not gotten past that last one, and the honest entry in our notes says so rather than substituting a plausible-looking fallback.

None of this is exotic. It is the ordinary texture of a domain where the obligation to publish is legal and the manner of publishing is not specified. Note what every one of these bugs has in common with the model failures above: the system reports something that is true about the label and false about the world, and the only defence is provenance carried on every value, including how it was fetched, when, and how hard you looked.

What we are building next

Built in the open, as always. Two of these now have first versions running, which is what the widget at the top of this post is.

1. A detour benchmark for regulated world models. Coherence under intervention has no standard test, so everyone reports accuracy and nobody reports robustness. We are assembling dated regulatory interventions into a public, versioned, held-out suite: phase-ins, partial mandates, calendar transitions, definition changes. Each gets a date, a scope, and a pre-registered list of what must and must not move, in the format above. Score a model on the detours instead of on the next step. The expensive part of an intervention benchmark is normally running the interventions, and here the regulator already did that for us.

2. Compiled rule skeletons inside procedurally generated worlds. (first version running) Make the factoring literal: compile the statutory factor into an exact transition with zero learned parameters, generate the rest procedurally around it, and measure what the compiled spine buys. In the second experiment above it buys the whole result, same data, same population, and the compiled model is the only one that survives the rule change.

3. Practice worlds with an adversarial clock. (first version running) Every simulator I have seen, including ours, gives the agent clean time: events in order, numbers final when you see them, a past that does not change. Real regulated systems restate values, land money quarters late in a lump, recompute populations retroactively, and shift definitions under a running series. So we built one that does all of that on purpose, as a hazard. The suspicion was that an agent which has only ever seen aligned clocks fails on first contact with a restatement, and that this, rather than any weakness in the model, is what kills these deployments in the field. On the first run the clock alone costs a quarter of the policy’s value:

The only two questions that matter

As promised in the first post, every post in this series ends with the two questions you will actually be asked.

One: what does it cost me when this goes wrong? A world model that is locally accurate and incoherent under intervention does not fail on a dashboard. It fails the first time a rule changes, which in a regulated business is scheduled, published, and inevitable. And it fails in the most expensive possible way: confidently, in the same direction as every historical pattern, right at the moment the historical pattern stopped applying. The taxi model was at 99 percent until the street closed, and then it was at zero. Your regulator will change a definition next year. You already know the date.

Two: can I defend it to a regulator? People assume regulated industries demand mechanistic, hand-written models and will never accept a learned one. Look at what the rules actually require and you find documentation, versioning, validation, reproducibility, and the ability to reconstruct what the model said on the day someone acted on it. Every one of those is about specification: is it written down, is it versioned, can you argue with it. Not one is about mechanism.

In physics-shaped domains the two look identical, because physics hands you the documentation for free. With no physics of behavior you have to produce specification administratively, which is why some of the most audited and most litigated models in the country are ordinary fitted statistical models with published coefficients. Nobody minds, because you can see them and argue with them.

You are not required to be mechanistic. You are required to be specified. Immutable versioned artifacts, provenance on every input, the original bytes kept next to every transformation, a history you can reconstruct. Most good engineers already do this for correctness reasons and have no idea it is also the entire compliance story.

The short version

Regulated industries are a hard place to build world models. The clocks collide. The world reads your model and changes underneath it. The fence around your data is legal rather than technical. And if you succeed widely enough, your own output contaminates your next input.

They are also the best place to build world models, for one specific reason. Everywhere else, the detours you need to find out whether your model understands anything are expensive, rare, or impossible to run. Here they are published in advance, with dates, by somebody else.

The rules are half the physics. The other half is knowing exactly where they stop.

This is part of an ongoing series on building AI for regulated decisions you can explain, reproduce, and defend. Start at Hello World Models, then A World From a Seed and The LLM Learned to Stop Planning.

Sources and further reading

The taxi result, and testing world models

• Keyon Vafa, Justin Y. Chen, Ashesh Rambachan, Jon Kleinberg, Sendhil Mullainathan, Evaluating the World Model Implicit in a Generative Model, NeurIPS 2024 (spotlight). The Manhattan maps, the detour collapse, and the random-walk model that survived. The diagnostics are inspired by the Myhill-Nerode theorem, which is a nice reminder that the sharpest tool here is fifty years old.
• One precision note if you cite this: the 96 to 99 percent figure is route validity across all three models, not shortest-path optimality. Optimality was 97 percent and 94 percent for the two goal-directed models.

The world-model idea itself

• Yann LeCun, A Path Towards Autonomous Machine Intelligence (2022), and Meta’s I-JEPA.
• DreamerV3: learning a latent world model and training a policy inside it.
• MuZero: planning with a learned model when the rules are not given to you. Worth reading precisely because our situation is the opposite, since our rules are given to us.
• DeepMind Genie: generative interactive environments, the “world model as simulator” end of the spectrum.

Doing this properly in a regulated setting

• The ISPOR-SMDM Modeling Good Research Practices task force, a seven-report series from 2012 published jointly in Value in Health and Medical Decision Making. Report 3 is the method spec for state-transition models and Report 7 is transparency and validation. If you are building this in health, do not invent a methodology. This is it.
• CISNET, the National Cancer Institute’s comparative modeling network: multiple independent models of the same disease, run against shared inputs, specifically so the disagreement between them can be studied. Institutionalized model diversity, already sitting upstream of national policy.

Model homogeneity

• Torsten Heinrich, Juan Sabuco, J. Doyne Farmer, A simulation of the insurance industry: the problem of risk model homogeneity, Journal of Economic Interaction and Coordination 17(2), 2022, 535-576. Preprint. Too few distinct models raises default risk and lowers profits.
• Goodhart’s law, which in this setting is not an aphorism but a term in the dynamics.

The industry examples

• KFF on prior authorization determinations and denial rates by parent company, and on post-acute denial and appeal-overturn rates.
• The Senate Permanent Subcommittee on Investigations staff report on automated review in post-acute denials, and the 2026 follow-up letters asking three insurers about AI in denials.
• HHS-OIG on denial rates at the three largest Medicare Advantage organizations. Note the report does not name individual insurers; press coverage that does is going beyond the document.