Drones That Learn Together Survive Together

The goal

Automatically manufacture labels from failure, then act on them.

A drone dies. That death is a label. The fleet works out what preceded it, turns that into a detector, and every survivor is better than it was an hour ago. Come home, pool it, the whole fleet levels up.

Everything below is the road to that sentence being true.

A drone alone cannot learn

It has sensors and no answers. Whatever it sees, nothing tells it whether it saw correctly.

Every method that tries to fix this from inside one aircraft ends up reinforcing what the aircraft already believed. The standard trick, entropy minimization, sharpens your own predictions. Its degenerate optimum is be confident about anything. Measured: it barely works and collapses at higher learning rates. It cannot discover that the world moved, because the only evidence it consults is its own opinion.

The fleet is the only source of truth available. Not because peers are wise. Because they are correlated. Several aircraft looking at the same world at the same time collectively know something no single one of them does.

Three world models, not one

Everything else falls out of this, so it is worth stating flatly. There are three models in play, and they are different objects with different jobs.

1. The assumed world model. What one aircraft believes right now. In the code today that is its running summary of what it has seen, plus the correction it has made to itself because of it. It is assumed because nothing checks it. A hypothesis held by a single aircraft with no way to test it, rewritten every tick. This is precisely the thing rung 0 below tries to improve from the inside, and cannot.

2. The shared world model. What the aircraft currently flying together agree on. What each has learned, folded together, plus the pooled record of what they each saw, kept deliberately separate from it.

The property that matters: the shared model has an extent. It is not fleet-wide truth. It is only valid across the ground its contributors actually looked at. Push it past that and it makes things worse, measurably, which is the subject of the next section. Overlap is the measurement of that extent, so the label-free overlap estimate is not really a safety check. It is the shared model telling you how far it reaches.

3. The global world model. What has ever been known. Built on the ground, not in the air. Public data first: spectrum allocations, device spec sheets, published emitter signatures, open imagery, terrain. Private data later: our own footage, our own recorded signatures, our own hardware. This is the frozen base every aircraft takes off carrying, and it is the only one of the three that already exists in finished form, because it is the frozen encoder.

The loop is the point. Take off carrying the global model. Diverge from it locally, because the world here is not the world it was built from. Reconcile with whoever is nearby, within extent. Come home with evidence, including whatever was auto-labelled by things going wrong. Fold that into the global model. Next sortie takes off smarter than this one landed.

That is what “the whole fleet levels up” means, written as a data path instead of a slogan.

Disagreement between tiers is a signal, not a fault. If an aircraft diverges from its neighbours, either the world changed here or that aircraft is broken. If a formation diverges from the global model, either this theatre is genuinely new, which is worth bringing home, or the formation has drifted together, which is worth rolling back. Neither is decidable from inside a single tier. That, more than storage, is the argument for three rather than two.

So: one question, asked at five strengths, across three models.

The ladder of labels

Rungs 0 and 1 are done. They are the safety floor everything above stands on, so they are worth stating before the interesting part.

Rung 1, and the two years of bad news in it

Here is the mechanic, in English. Every so often, each aircraft broadcasts how it has changed its mind since takeoff. Not what it saw, which the radio could never carry, and not what it decided, which is stale by the time it lands. Just the correction: about 8 KB of here is how I now differ from the model we all took off with (the adapter delta). Anyone in range folds it into their own.

It helps a lot when the neighbours are looking at a world like yours. It actively hurts when they are not, ending up worse than never adapting at all. Take advice from someone flying over a different world and you get worse at your own.

So listening is not safe by default. Which raises the question the whole paper is about: can an aircraft work out, with nobody labelling anything, whether a neighbour’s world resembles its own?

Asking “did we change in the same way?” does not work. Compare the corrections themselves and the answer bands overlap at every setting. No threshold exists anywhere. Everyone started from the same frozen model, so once you subtract the part of the change that everyone shares, what is left is too faint to read.

Asking “have we seen the same things?” works. Each aircraft keeps a running sketch of what has actually passed in front of it: the average of each kind of thing it has seen (the centroid bank). Compare sketches instead of corrections.

The readings either side of the helps/hurts crossover do not touch. First time an absolute cutoff existed anywhere in this project. And when two aircraft share nothing at all, the sketch comparison lands on the true answer, 0.2385 measured against 0.2410 actual, so it is not merely correlated with the truth. It reports it.

Now let each aircraft ask a question before it listens, and score how much worse it does than an aircraft that was simply told the right answer. Lower is better:

Only the last two are ever better than keeping to yourself (0.234).

Why the bottom row exists. The third question is asked of a neighbour’s decisions, and it sticks: once an aircraft has decided a neighbour is a stranger, it never reconsiders. Fly one into a different flight’s world halfway through a sortie and it scores 0.1875, worse than guessing at random. Not unhelpful. Confidently wrong. An aircraft that cannot change its mind about who to trust is more dangerous than one that trusts everybody.

The reason it cannot reconsider generalizes well past this project: taking someone’s advice destroys the evidence you needed in order to judge whether taking it was wise. Once you have folded a neighbour’s correction into your own, your correction and theirs are nearly identical by construction, 0.9995, and the comparison of decisions actually inverts. But the sketch of what you saw is untouched, 0.6257 against 0.2369, because nothing you were told changes what passed in front of your own sensor. So keep the sketch as its own thing and never let advice contaminate it. Same aircraft, same mid-sortie re-tasking: 0.8516.

One idea died on the way. We tried having aircraft send each other actual samples and ask “what do you make of this?” It needs 128 samples to reach a 0.16 margin where comparing sketches reaches 0.21 by tick 25. Asking for an opinion compresses everything the neighbour knows into one number per sample. You throw the information away, then pay bandwidth to estimate it back.

The multiplier

Rung 3 on its own is nearly useless. One death, one example, and the victim’s own data died with it.

Rung 4 is what makes it work.

The fleet knows who else was there. Every aircraft whose observations overlapped the victim’s in space and time is holding a buffer of the same world from a different angle. The death labels all of those buffers at once, retroactively. One loss becomes N positive examples, plus a much larger set of confirmed negatives from everyone who was elsewhere and lived.

That is auto-labelling. Nobody annotated anything. The world did, and geometry distributed the annotation.

And it runs backwards just as well. This is the risk the three tiers make visible, and it is the one that worries me most. The multiplier does not check whether the death was real. Mistake a radio shadow for a shoot-down and the same machinery turns that mistake into N labelled buffers, held by N different aircraft, all agreeing with each other because they were all looking at the same nothing. That agreement then reads as corroboration. It gets promoted into the shared model, carried home, folded into the global model, and shipped to every aircraft that takes off next week.

Consensus does not filter a bad label. Consensus is what launders it. So anything crossing from a sortie into the global model has to pass a check that is capable of failing, the same discipline as everywhere else here.

Which makes overlap the whole project

Overlap started as the safety problem. Share with a peer who sees your world and you gain; share with one who does not and you lose. That is rung 1.

The same quantity is the labelling mechanism at rung 4. High overlap with the victim means your buffer is evidence about what killed it. Low overlap means your buffer is a negative example.

Same number, used twice, in opposite directions.

And the fleet can already measure it with no labels at all. That result, the one in the table above, is the hinge the entire story turns on, and it is the one thing here that is already done.

The modality is a slot

Nothing above says what a sensor is. An aircraft carries a frozen model that turns whatever its sensor gives it into a vector, keeps a running sketch of those vectors, and compares sketches with neighbours. Radio, camera, acoustic, whatever. The frozen model is the global world model’s half that does the perceiving, which is why that tier is the one already real.

Which is why the last stretch of work counts even though it looks like plumbing. It proves that slot takes a real model on real data, end to end:

• a stock ResNet-50 loaded in C++, reproducing its author’s own reference logits to 2.9e-6
• the classifier head cut off the graph to expose the 2048-d features underneath, verified on the real 98 MB file
• preprocessing checked against a published number: predicted 58–66% before running, measured 62.12% top-1

Swap the encoder, keep everything else.

The world we are building to test this

Rungs 2 through 4 cannot be tested in the world we have. Not because the harness is weak, but because it is too kind in exactly the places reality is cruel.

(One disambiguation, since “world model” is now doing two jobs on this page and a third in the neighbouring posts. The three above are what an aircraft, a formation, and the organisation each believe. What follows is the simulated environment we run them inside. Neither is a learned latent dynamics model in the Dreamer sense. Whether an aircraft should be gossiping a learned model of how the world moves, rather than a correction to how it perceives, is a real question and a parked one.)

Note that the third tier, the global model, mostly does not need this simulator. It is built on the ground from data that already exists. What the simulator has to produce is the thing that feeds it: evidence coming home from a sortie, labelled by what happened, trustworthy enough to fold in.

Four things have to change.

1. Overlap must stop being a dial.

Today ω is a number we set. That is the single biggest weakness in every result above: the mechanism is understood, but the axis it is measured along was invented by us.

In the new world, aircraft observe a world that has things in it. Sources at positions. What you see is a function of which sources are within your range. Two aircraft close together see the same sources, so their observation distributions overlap. Overlap stops being a parameter and becomes a consequence of where you are.

That is the same goal we had for measuring overlap across real corruption types, arrived at from a better direction: it comes out of the geometry rather than out of a dataset’s taxonomy.

2. Time has to exist.

Everything we have built is per-frame and memoryless. Rung 4 needs a rolling buffer, because the label arrives after the evidence and points backwards. Nothing in the codebase currently remembers anything.

3. Death has to be a consequence, not a command.

We can already remove an aircraft mid-sortie from a replayable order list. That is useful, and it is not the same thing. Death has to be caused by the world, at a place and a time, by a hazard with an envelope, so that “what preceded it” is a question with a real answer the simulator knows and the fleet has to work out.

Critically, whether the precursor is observable at all must be a parameter. If the shot has no detectable signature, in band and in time, then there is nothing to learn and a system that insists on learning anyway will manufacture a superstition from three examples. We need to be able to run that case deliberately and confirm the fleet learns nothing rather than something false.

4. The world has to lie about deaths.

This is the one that decides whether any of it works.

A silent peer might be dead. Or radio-shadowed. Or jammed. Or out of range. Or gone home. Train on false deaths and you build an excellent detector for terrain.

We already have the confound and did not build it for this: aircraft carry positions, delivery falls off with distance, and a fleet can be flown apart until it partitions. Across one split, 74 frames cross while together, 0 while apart, 112 on rejoin. Every one of those zero-delivery ticks is a false death waiting to be swallowed.

So the auto-labeller’s first job is not detection. It is distinguishing killed from merely gone, and it has to be scored on that separately, with a confusion matrix, before any downstream number means anything.

Where this goes

Next rung is 2, not 3. Consensus among overlapping peers is rung 4 without needing anyone to die first: same machinery, same geometry, no aircraft spent. If agreement among co-located peers cannot produce a usable label, then labelling from a death certainly cannot, and we find that out cheaply.

Then the world model above, in the order listed, because each item makes the next testable.

Running alongside, because it needs no aircraft at all: the global tier, assembled on the ground from public data. Spectrum allocations, spec sheets, published signatures. It is the one piece of this that can be built today by someone with a laptop and no fleet.

Parked deliberately: routing advice rather than absorbing it, as a general policy. Sketches as the thing that gets shared rather than only the thing that decides whether to share. And whether an aircraft should be gossiping a learned model of how the world moves instead of a correction to how it sees.

Running note on method

Four times now the useful move has been the same one: predict the number before running, then refuse to move the threshold afterwards.

It costs something every time. One deliberately-broken control slipped through a window I could trivially have tightened after the fact, so the honest report is that the check has a four-point blind spot rather than that everything passed.

The alternative is a project where every check passes, which is the same as a project with no checks in it.

The short version

• A drone alone cannot learn. Its own confidence is not evidence.
• A fleet can, because it is correlated. Correlation is the raw material for labels nobody wrote down.
• Three models, not one: what I think, what we here think, what has ever been known. They update at different rates and fail in different ways, and their disagreements are the only diagnostic you get.
• Taking a neighbour’s advice is not safe by default, and you cannot tell from the advice whether it is safe. You can tell from what each of you has seen.
• An aircraft that cannot change its mind about who to trust fails worse than random.
• Absorbing advice destroys the evidence you needed to judge it. So keep them apart.
• One death should teach every aircraft that witnessed it, and geometry is what says who did.
• The same multiplier launders a wrong label just as efficiently. So the way home needs a gate that can fail.
• All of it is currently conditional on a world we invented, which is why the next build is a world that gets to be cruel in the specific ways reality is.