Orelon logoOrelon
Preise

Turn Video References Into AI Prompts That Actually Work

6. Nov. 2026 · Von Orelon Team

KI-Video-Vorlagen entdecken

Lass dich von ein paar Community-Kreationen inspirieren und öffne dann eine Vorlage, um in Orelon weiterzuerschaffen.

Learn how to break a video reference into precise AI prompts: shot analysis, camera and motion cues, lighting language, and repeatable iteration workflows.

Most people who try to turn a video into a prompt start by describing what they see. That is the wrong starting point. A finished video is the output of hundreds of decisions — lens choice, blocking, key light position, motion cadence, grade, texture — and a description of the visible surface tells a generator almost nothing about how to rebuild the structure underneath it. The useful skill is not describing; it is decomposing. You watch a clip, strip it into layers, and then write instructions that describe the mechanics of each layer rather than the impression it leaves.

This guide walks through a practical method you can run on any reference clip: a four-pass breakdown, a fixed set of prompt layers, assembly rules that keep prompts readable for a model, and an iteration loop that tells you exactly what to change when the output drifts. It is written for people who already have a reference in hand and want an output that holds up next to it.

Start With a Shot List, Not a Vibe

The first mistake is writing one long prompt for an entire reference. A thirty-second clip usually contains three to six distinct shots, and each one has its own camera behavior, lighting logic, and pacing. When you collapse them into a single prompt, the model averages everything and produces something that resembles none of it.

Cut the reference into shots first. A shot ends when the camera cuts, when the subject changes, or when the visual logic changes meaningfully — a lamp turning on, a color shift, a change in depth of field. Number them. For each, note only four things before you write anything: duration in seconds, whether the camera moves or is locked, whether the subject moves or is still, and where the light comes from.

That skeleton alone solves most of the fidelity problem. If a shot is a locked wide with a subject walking toward camera in hard side light, the prompt needs exactly those words. If the next shot is a slow push-in on a face in soft window light, it needs entirely different ones. Two prompts, two generations, one edit.

A useful rule: if you cannot describe a shot in one sentence of mechanics, you have not watched it closely enough yet.

The Five Layers Every Reference Prompt Needs

Once a shot is isolated, build the prompt in layers. Keeping them in a consistent order makes prompts easier to edit and easier to reuse across models. The five layers below cover almost everything that meaningfully affects a generated frame.

Layer one: subject and silhouette

Describe what occupies the frame in terms a camera would register, not terms a story would. "A courier in a rain-dark jacket" is weaker than "a lone figure in a boxy dark jacket, shoulders squared, occupying the left third of frame." Silhouette, posture, framing position, and scale relative to the frame do more work than adjectives. If the reference has two people, say where each sits in the composition.

Layer two: camera behavior

Locked-off, handheld, slow dolly in, orbit, crane down, whip pan — pick one primary motion and, if needed, one secondary. Add lens language when it is visible: wide angle with barrel distortion, long lens compression, shallow depth of field with the background falling off fast. Camera terms are some of the most reliably understood tokens a video model receives, so spend words here rather than on mood.

Layer three: light and color logic

The reference rarely has "nice lighting." It has a specific setup. Identify the source (window, practical streetlamp, overhead fluorescent, hard sun), the direction (backlit, side-lit, top-lit), the quality (soft and wrapped, or hard and specular), and the resulting contrast ratio. Then note the color logic: is the palette warm skin against cool shadows, monochrome with one saturated accent, or high-key neutral? Describe ratio and direction before naming colors.

Layer four: motion physics

This is the layer most prompts skip and most outputs fail on. How does movement behave? Is the walk heavy with weight transfer, or light and quick? Does fabric trail the motion or stay stiff? Does the camera lag the subject slightly or track it precisely? Words like "deliberate," "weighted," "floaty," and "snappy" are vague on their own — pair them with a physical cause, such as "heavy boots on wet pavement, short stride, minimal arm swing."

Layer five: texture and finish

Grain, halation around highlights, slight lens breathing, chromatic fringing on edges, filmic highlight rolloff, or a clean digital finish. This is where a generation either feels cinematic or feels synthetic, and a single clause usually covers it: "fine 35mm grain, soft halation on practicals, gentle highlight rolloff."

A Four-Pass Breakdown Method

Watching once and writing immediately produces shallow prompts. Four passes take about the same time as three failed generations and give you far better material.

Pass one: silent viewing

Watch with sound off and no writing. Your only job is to feel the rhythm — where the clip accelerates, where it holds, which shot you remember afterward. Memory is a good filter for what actually carries the visual weight.

Pass two: segment mapping

Watch again and mark cut points. Note duration and the one mechanical sentence per shot. Do not judge anything yet; just map.

Pass three: attribute extraction

Now go shot by shot and pull attributes into the five layers. This is where you pause, step frames, and look at edges of the image: what is in focus, where the falloff begins, which side the shadow falls on. Take screenshots of two or three representative frames per shot — you will use them later as comparison stills.

Pass four: prompt assembly

Only now do you write prose. Assembly is compression, not creation: you are choosing the strongest detail from each layer and putting them in a fixed order. If a detail did not survive the cut, it was not doing structural work.

Prompt Assembly: Order, Weight, and Specificity

Order matters because most video models weight earlier tokens more heavily. A reliable sequence is: shot type and subject, camera motion, lighting, motion physics, texture, then style and finish. Keep the opening clause the most concrete thing in the prompt.

Specificity has a ceiling. Three precise details beat ten approximate ones. If a prompt has four competing adjectives for the light, the model picks one at random between generations, which destroys consistency. Pick the two you would defend in an edit review and delete the rest.

Avoid stacking negations. Long lists of what you do not want tend to leak the unwanted concept into the frame. Instead of "no crowds, no neon, no reflections," write the positive state: "empty street, matte surfaces, flat ambient light."

Keep prompts to roughly 40–80 words per shot. Long enough to carry five layers, short enough that no clause gets diluted. When you need more control, generate more shots, not longer sentences.

Adapting One Prompt Across Different Models

The same reference prompt will behave differently across generators, and rewriting from scratch each time is wasted effort. Adjust in three places only.

First, camera syntax. Some models respond well to technical camera vocabulary; others respond better to plain description of what the viewer sees. If a dolly-in prompt produces a static frame, rewrite it as "the frame slowly grows closer to the subject."

Second, motion density. Models that handle complex motion well can take two simultaneous movements in one prompt. Constrained models do better with one motion per generation, composited later.

Third, texture clauses. Heavier finish language — grain, halation, anamorphic flare — sometimes overwhelms a model and pushes it toward a stylized look. If the output looks illustrated, strip the texture layer, generate clean, and add grain in post.

If you are comparing engines on the same reference, run the identical prompt across each and grade the results against your screenshot stills rather than against memory. A side-by-side such as Orelon vs Runway is more useful after you have a controlled prompt than before.

Common Mistakes That Cost You Fidelity

Describing emotion instead of mechanics. "Melancholy atmosphere" gives a model nothing to render. "Overcast flat light, desaturated cool shadows, subject standing still with lowered gaze" gives it everything.

Ignoring the edit. Many references feel cinematic because of cut rhythm, not individual frames. If your output will be assembled in an editor, plan the pacing there instead of trying to force it into a single generation.

Over-stuffing the subject. References often have busy frames because of set design, not because the subject is complex. Describe the one or two elements that define the composition and let the rest be suggested.

Assuming a still frame is enough. A screenshot tells you composition but not motion cadence. Always extract at least a short moving segment per shot, even if it is two seconds.

Skipping the comparison still. Without a reference still pinned next to your output, you will unconsciously drift toward whatever the model likes to make.

Worked Example: A Night Street Shot

Take a reference shot: six seconds, a person walking away from camera down a wet street, streetlamps overhead, shallow focus, slight handheld float.

Weak prompt: "cinematic night street, person walking away, moody lighting, film look."

Strong prompt: "Wide shot from behind, a single figure in a dark knee-length coat walking away down the center of a wet asphalt street, steady medium stride. Handheld camera following at walking pace with slight lateral float. Overhead sodium streetlamps creating pools of hard top light, deep shadows between pools, cool blue ambient fill. Wet surface reflecting lamp highlights in broken vertical streaks. Shallow depth of field, background lamps blooming softly. Fine grain, gentle halation on the brightest reflections."

Both prompts describe the same shot. The second one gives the model decisions to make in the right order, which is why it holds up across multiple generations instead of drifting.

Once a prompt like this works, save it and swap the subject and location. That is how a personal prompt library becomes genuinely useful — not as a list of pretty phrases, but as tested structural templates you can recombine. Browsing a curated prompt library is a good way to see how other people structure the same five layers.

Reviewing Output Against the Reference

Grade each generation on four axes, one at a time, rather than judging the whole clip at once: composition, camera behavior, light direction, and motion quality. Score each as close, partial, or off.

Then change one thing. If composition is close but light direction is wrong, edit only the lighting clause. Changing three things at once makes it impossible to learn which clause caused the improvement — and over a few sessions that learning is worth more than any single result.

Discard generations honestly. Keeping a near-miss in the timeline because it took several attempts is a sunk-cost trap that flattens the whole edit.

Building a Reusable Prompt Library

After ten or fifteen reference breakdowns, patterns emerge. Group them by structural situation, not by subject: locked wide in hard light, handheld follow in ambient light, slow push-in on a face, overhead top-light interior, backlit silhouette exterior. Each becomes a template with three variable slots — subject, location, and palette.

Store the reference still next to each prompt. Six months later, the still explains the prompt faster than the words do. Starting from video templates is a reasonable shortcut when you need a structural baseline before customizing.

FAQ

Can I use a whole video as one prompt? Not effectively. Split it into shots. Each generation handles one shot's worth of decisions well; stacking several shots into one prompt averages them into something unrecognizable.

How long should a reference-based prompt be? Around 40–80 words per shot. Long enough to carry subject, camera, light, motion, and texture; short enough that every clause still influences the result.

Do I need editing software? Usually yes if the reference has multiple shots or complex motion. Generating single strong shots and assembling them gives you far more control than forcing a model to do the whole sequence.

Why does the same prompt give different results each time? Generation is probabilistic. Reduce variance by trimming competing adjectives, keeping one primary camera motion, and reusing the exact same prompt text instead of retyping it from memory.

What if the reference has a style I cannot name? Describe its mechanics instead: contrast ratio, color temperature split, grain size, halation, lens compression. Style names are shortcuts; mechanics are instructions.

Should I include negative prompts? Sparingly. Positive descriptions of the desired state work better than long lists of exclusions, which tend to introduce the excluded element.

Put Your Reference to Work in Orelon

You already have the hardest part: a reference that shows exactly what you want. The rest is mechanical — isolate the shots, extract the five layers, assemble short prompts in a fixed order, and iterate one variable at a time.

Orelon is built for that workflow. Open the AI video generator, paste a layered shot prompt, and compare the result against your reference still. When you need a clean comparison frame or a style plate to anchor the palette, the AI image generator gets you there in seconds. Keep a running file of what worked — and when you want more breakdowns like this one, the Orelon blog has you covered.