Case study · Generative AI production pipelines · September 2026
The Kerosene Lamp Problem.
Six of the nine generated frames I audited contain a lit kerosene lamp. The film is set between 1813 and 1821. Kerosene was not distilled until 1846, and the lamp that burns it was not sold until the 1850s. Every prompt forbade the lamp by name. The issue: the pipeline's main tool for keeping the wrong century out of the picture, a list of things not to draw, does not work on this model. The pipeline lacks period specific details to set the stage so the model creates its own.
- 1813Fort Strother, the film opens
- 1821Pensacola, the audited scene
- 1846kerosene first distilled
- 1850skerosene lamp first sold
25 to 40 years separate the film from the fuel, and the lamp went on sale in industrial cities, not on the Alabama frontier or in a Spanish garrison town on the Gulf. The generation models seem to use the kerosene lamp as a preferred aesthetic for the century, too imprecise for historical guardrails.
Mean historical-accuracy score across nine frames audited pixel by pixel
Frames containing an object the prompt forbids by name
Of every scene description was being silently discarded before it reached the model
Score change on one frame from a single build fix. Same prompt, same model
One beat, generated twice
The scene is a woman at a window watching an ordinary Catholic Sunday in Pensacola, 1821. The audience is meant to see nothing wrong with the street below.
Drag to compare. On the first pass: 1 the kerosene lamp · 2 the gold key · 3 the silhouette.

Findings, first pass
- 1A lit kerosene lamp, indoors, in daylight, in 1821. Forbidden by name in the prompt.
- 2None of the street the page describes is in the picture. Shops open, men at a card table under an awning, a fiddle, people going to and from Mass: the plaza is bare. The build had cut the beat's description at 110 characters and all of that was in the part it dropped.
- 3A large gold key floats on the shutter. Nothing asked for it. The project suspends its symbol vocabulary by ruling so signs cannot be designed by accident, and the model designed one.
- 4A featureless black silhouette in the street, forbidden by name in the same list as the lamp.
- 5Overhead wires above an 1821 street. Nothing forbade them, because nobody thought to.
- 6The dress reads 1840s: tiered ruffles, fitted natural waist. The wardrobe band specifies an 1821 gored skirt and the negatives name “natural waist” explicitly.
- ✓Caption correct and inside the ruled border. Rachel above and apart from the town, which is the posture the page asks for.
Findings, after the fix
- ✓The awning is there. Two men at a table with the cards down. A musician playing. People going to and from Mass. Shops open.
- ✓The kerosene lamp is gone. So are the key, the silhouette and the wires. None of them was separately fixed. Giving the model real content displaced them.
- ✓Interior daylight through the shutter with dust in the beam, which is the project's own rendering rule for this lighting preset.
- ×The 1840s silhouette persists. This is the one criterion the fix did not touch, and the reason is the finding below.
- ×The instrument is a cello rather than a fiddle. Minor.
The only difference is context. The first prompt carried the opening sentence of the beat; the second carried all of it, 273 more characters.
The Spanish garrison leaving Pensacola at the transfer of Florida. This beat carries an authored crowd note, because the crowd deliberately has no cast rows and therefore no wardrobe authority. The note was written specifically to prevent what happened to an earlier uncast crowd, which came back as the Union army.
Findings, “The flag comes down”
- ✓The composition is exactly the page: she is on an upper gallery, above it, looking down at a distance.
- ✓The note did keep the crowd out of the cast list and into the picture as a body rather than as portraits.
- ×The flags are wrong. The 1821 Spanish flag is red-yellow-red. The frame shows a blue-white-red tricolour and a black-and-white quartered field.
- ×The garrison reads as generic Napoleonic-era infantry rather than Spanish colonial. Naming the nationality in prose moved the needle less than expected.
- ×A man in a dark coat and wide hat stands on the gallery. The page says of this beat, by name, that Jackson is not in this frame.
- ×Two featureless silhouettes, a kerosene lamp, and overhead wires.
Seven rules, joined to their scores
Authority runs upward through four tiers — sources → bibles → screenplay pages → build artefacts, and the page is the fact. A generated frame is a build artefact and is accountable to every tier above it. The rubric invents no criteria; it makes seven existing project rules scoreable from 0 to 3.
| Id | Criterion | Score | Enforced | What it tests |
|---|---|---|---|---|
| A1 | Provenance | 2.2/3 | Prompt | Everything depicted traces to the screenplay page or a tagged source. Content that appears without a line behind it is unsourced, however plausible it looks. |
| A2 | Period | 0.7/3 | Prompt | Nothing in frame postdates the beat: objects, materials, lighting technology, architecture. Scored hard, because one anachronism is what a historian finds first. |
| A3 | Costume | 1.0/3 | Prompt | Every figure resolves to an authored wardrobe band at the right occasion. A figure with no cast row has no wardrobe authority at all. |
| A4 | Presence | 2.3/3 | Prompt | Everyone in frame belongs there, and no one the page excludes appears. The page can bar someone by name, and that outranks everything. |
| A5 | Consent | 3.0/3 | Code | The consultation gate is honoured exactly. Twelve beats are absolutely withheld; 182 more generate flagged, never silently. Pass or fail, no partial credit. |
| A6 | Quotation | 2.6/3 | Code | Text visible in frame matches the register at its verified status. Where the record is a transcription, the honest depiction is a hand you cannot read. |
| A7 | Restraint | 1.3/3 | Prompt | Nothing is added that argues: no invented symbol, no composition that editorialises, no emphasis the page withholds. |
CODE = the build enforces it before any prompt is written. PROMPT = the prompt asks for it and the model has to comply. Every criterion a machine enforced passed. Every criterion that asked the model to leave something out scored below half.
The control surface was pointed backwards
Here is the whole finding in one sentence, and then I will show you why it is true and what we do about it. The model does what you describe, and it ignores what you forbid.
An image model is not a person, and this is not a perfect account of what happens inside it. But it predicts the model's behaviour well enough to design against, and that is the test that matters for a tool.
The kerosene lamp is forbidden by name in every prompt this project emits, and it appeared in six of nine audited frames. A featureless silhouette is forbidden in the same list and appeared in four. “Natural waist” is forbidden on Rachel and she has one in every frame she is in. Then the other column: one sentence saying the candle standing on that table is the only light in the picture produced a candle-lit room on the first try, after four separate prohibitions had failed to. And the street furniture, restored to the prompt, pushed four forbidden objects out of the frame without naming any of them.
Every rule written as “do not draw X” scored below half. Every rule written as “this is what is here,” or enforced before the prompt existed, passed.
So what do we do about it? Three things, in this order.
Say what is there, not what is not.
Every prohibition in the prompt gets rewritten as the thing that occupies its place. The do-not-depict list stops being treated as a control and becomes a checklist for the reviewer instead.
Move every rule you can into code.
The two criteria that never failed, consent and quotation, are the two the build enforces before a prompt exists. The consent gate refuses a beat outright; the quotation register drops an unverified line from the caption. Neither one asks the model for anything. If a rule can be a gate, make it a gate.
Read the prompt before you pay for the picture.
Three of the four worst frames were not model failures. They were build defects, plainly visible in the prompt text and invisible in the image: the truncation, a transition marker leaking into the action line, a cleanup rule quietly eating real words out of fourteen captions. The cheapest quality gate in the whole pipeline turned out to be a person reading the thing that gets sent, before it gets sent.
What the rubric cannot reach
Every criterion above audits depiction: what is in frame, what it is made of, when it is from, whose authority put it there. All of that has a reference in the record. Motivation, conflict and dialogue do not, and the project's own rules make that deliberate. A concealed want is never sent to the model, only its observable consequence. No character is allowed to state what a fact means.
What applies to any generation pipeline
None of this is specific to film, or to history. Four rules you can apply on Monday.
2 rules never failed
both were enforced by code. every rule that relied on the model's cooperation scored below half
Enforce in code what you can, and expect the rest to drift.
The two criteria that never failed were the two enforced before the prompt existed. Everything left to the model's cooperation scored below half. That ratio is a design instruction, not a disappointment.
→ If a rule can be a gate, make it a gate3 of 4 worst frames
were not model failures. the defect was sitting in the prompt text, where nobody was looking
Audit the artefact, not the output.
Three of the four worst frames failed for reasons plainly legible in the prompt text and invisible in the picture. The cheapest quality gate turned out to be reading the thing you are about to send.
→ Read the prompt before you pay for the pictureThe failure was saved
the 10/21 first pass was archived before the fix replaced it. that is why this page can prove anything
Preserve the baseline before you improve it.
The most persuasive evidence here exists only because the first pass was archived with checksums before the second overwrote it. An improvement is legible only against the thing it improved on.
→ Archive the first pass, checksums and all5 written, 0 applied
rulings made and written down that never reached the code they were meant to govern
A decision recorded is not a decision applied.
Five rulings had been made, written down, and never reached the code that needed them. One was in the script that prices the work, still naming a superseded model at nearly three times the cost.
→ Trace every ruling to the code that obeys itMethod. Nine of twenty generated frames audited pixel by pixel against seven
criteria derived from existing project rules. Scores are mine, applied consistently, and are not blind.
The eleven unexamined frames are excluded from the mean rather than assumed. Every frame carries a
sidecar recording its exact prompt, model, settings, a SHA-256 of its bytes, and the work order that
authorised it. The first pass is preserved with checksums so the comparison stays checkable.
Red Eagle adapts my published research on William Weatherford and the Creek War of 1813 to 1814
(International Journal of Interdisciplinary Social Sciences, Vol. 6 No. 4). Consultation with the
Poarch Band of Creek Indians and the Muscogee (Creek) Nation has not taken place. Muscogee content in
this draft is generated under a flag and is not cleared for publication.