I’ve open-sourced the Skill I use to make animations with AI video: https://github.com/oil-oil/oil-motion…

The overall flow of the Skill looks like this:

reference materials, expression goal, and control method ↓ first confirm what the start, middle, and end should look like ↓ use AI video to generate continuous motion between those frames ↓ check frame by frame, removing pauses, duplicates, and anomalous frames ↓ organize and compress assets according to the actual display size on the page ↓ map scrolling, mouse, drag, touch, or device orientation to animation progress

Since the whole flow is still a bit complex, I’ve bundled some of the implementations I actually use into scripts, which makes the post-generation animation processing more stable. I’ve also added some creative reference documents — for example, the scroll-driven exploded-view apple effect you can see in the video. Welcome to try it out~

oil-oil/oil-motion

Source: https://github.com/oil-oil/oil-motion

English: README.en.md · 中文: README.md

Oil Motion is a general-purpose interactive animation Skill for Agents. It handles designing motion, generating continuous frames, organizing animation assets, and connecting the final result to page scrolling, mouse, drag, touch, or device orientation.

To use it, you only need to describe what you want to express, which materials to provide, and what operation the animation should follow. The actual generation, checking, compression, and front-end implementation are all done by the Agent.

Demo

https://github.com/user-attachments/assets/08e26ad6-ca23-4f31-ac53-44c7692ba99d

Installation

Tell the Agent: “Help me install the Skill at https://github.com/oil-oil/oil-motion.”

Animations It’s Well-Suited For

• As the page scrolls down, the product gradually expands, disassembles, or switches states.
• As the mouse moves, a character, pet, or product naturally adjusts its orientation.
• When dragging a progress control, the animation follows the gesture forward, backward, or stops at a specified position.
• On mobile, use touch or device orientation to control the visuals.
• When clicking, hovering, or when a component state changes, play the corresponding action.
• Plays naturally when there’s no interaction; switches to follow mode once the user starts interacting.

These animations can be used for product introductions, character interaction, interactive demos, data changes, and section transitions. Oil Motion also handles motion continuity, visual clarity, loading size, and mobile performance.

How It Works

AI-generated video can only play from start to finish on its own. To make it follow scrolling or mouse movement, Oil Motion first generates a continuous motion sequence, then maps the user’s input position to the animation progress.

For example, a product disassembly animation has 100 frames total. When the page scrolls to 30%, frame 30 is shown; when it reaches 70%, frame 70 is shown. If the user scrolls back, the animation returns naturally along the same set of frames. Mouse, drag, and device orientation all use the same mapping approach — only the input source differs.

The complete process is as follows:

Reference materials, expression goal, and control method
↓
First confirm what the start, middle, and end should look like
↓
Use AI video to generate continuous motion between those frames
↓
Check frame by frame, removing pauses, duplicates, and anomalous frames
↓
Organize and compress assets according to the actual display size on the page
↓
Map scrolling, mouse, drag, touch, or device orientation to animation progress

This workflow is split into three parts.

Confirm the Key Frames First

Key frames are the states that must remain accurate throughout the motion — for example, the product intact, parts expanded, and the final disassembly. The Agent first generates and checks these frames to confirm that the subject’s identity, structure, logo, composition, and style remain unchanged.

If the full video is generated directly, the model may change the product structure, character proportions, or the end point of the motion along the way. Confirming key frames first gives the continuous motion a clear start, process, and end.

Then Generate the Continuous Motion

Once the key frames are confirmed, AI video generates the transitions between them. Body rotation, product deformation, material changes, and front-to-back occlusion are all handled at this stage, because these changes usually look unnatural if done only by moving or scaling images on the front end.

Position, scale, cropping, playback speed, follow damping, and maximum rotation speed are controlled programmatically. These changes don’t need to be regenerated by the video model — handling them in code is more stable and easier to adjust later.

Finally, Convert to Controllable Web Assets

The generated video needs to be cleaned up before it can be used for interaction. The Agent examines every frame, trims pauses at the beginning and end, removes near-duplicate frames, detects flicker or structural changes, and compresses according to the actual display size on the page.

After processing, the browser doesn’t regenerate video when the user interacts; instead, it locates positions within the already-prepared continuous frames. This lets the animation respond to input immediately and avoids different outcomes on each interaction.

Asset Formats

All animations start from a green-screen master, with the page background kept separate. The Agent automatically chooses the asset format based on frame size, animation length, access pattern, and device budget — the user doesn’t need to understand or decide the technical approach.

The Agent first runs a budget script and directly adopts the single most suitable primary approach; it doesn’t leave the atlas-vs-video choice to the user, nor does it implement both redundantly. Either way, the final background is provided by the page, so changing the background doesn’t require regenerating the subject.

Compression is based on the actual display size on the page. Larger display areas keep higher resolution; smaller areas drop unnecessary data. When file size and clarity conflict, the priority is the actual on-page visual experience.

Quick Start

When you already have materials and a motion direction, you can simply say:

Use $oil-motion to turn these two product images into an animation that gradually unfolds as the page scrolls.
The desktop display area is large and needs to stay sharp; use lighter assets for mobile.

When you only have product materials and no motion plan yet, you can have the Agent design first:

Use $oil-motion to design a scroll-driven continuous animation for this product landing page.
First provide three directions, each explaining what it expresses, how it follows scrolling, and the implementation cost.
After the direction is confirmed, generate the key frames and video.

When a character or pet needs to follow the mouse, you can specify the movement range and desired feel:

Use $oil-motion to make this character adjust its orientation to follow the mouse.
The motion needs to respond promptly while limiting rotation speed; it must not flicker or jitter on fast reversals.

What the Agent Determines

Before starting, the Agent confirms the following:

• What the animation is meant to express, and its role on the page.
• The subject’s identity, structure, proportions, and the parts that must not change.
• Whether the animation is controlled by scrolling, mouse, drag, touch, or another state.
• Whether the motion is a single back-and-forth path, or needs to respond to both horizontal and vertical directions simultaneously.
• The actual display size on the page, and whether desktop and mobile need different assets.
• What to show when the animation hasn’t loaded yet, fails to load, or the user has reduced motion enabled.

If both the mouse’s horizontal and vertical positions change the subject’s pose, 2D directional frames are needed. A single left-right looping video can’t accurately express up-down or near-far changes.

Deliverables

Depending on the project’s needs, the Agent delivers some or all of the following:

• Confirmed key frames and generation prompts.
• Raw video that can be further processed.
• Animation assets with pauses, duplicates, and anomalous frames removed.
• Atlases or video suited for desktop and mobile loading.
• Interaction code that can be integrated directly into the project.
• A principle preview page showing the full frame, current frame, and interaction input.
• Configuration files for animation assets, interaction ranges, and loading behavior.

These files retain the information needed for generation, processing, and acceptance, making it easy to modify the motion, replace frames, or adjust interaction parameters later.

Quality Checks

Before delivery, check that:

• Subject identity, product structure, proportions, position, and lighting remain consistent across the continuous frames.
• There are no extra limbs, duplicated parts, hard cuts, flash frames, or abnormal pauses.
• Transparent assets have clean edges, and internal white areas and thin lines haven’t been accidentally removed.
• During fast scrolling, fast reversals, and frequent mouse movement, the animation doesn’t lag, jitter, or go out of bounds.
• After page scrolling, zooming, or device rotation, the input position still maps correctly to the animation direction.
• There’s a clear first frame on initial load, and it can fall back to a static image if asset loading fails.
• The final assets remain sharp once placed in the real page, and mobile doesn’t visibly stutter from oversized assets.
• When the user chooses reduced motion, the page can show static alternative content.

The program can handle slight drift, color differences, duplicate frames, and encoding issues, but it cannot fix incorrect motion design. If errors are found in subject structure, limb relationships, or the motion direction, the key frames need to be reworked or the video regenerated.

First-Time Animation Generation

The first time animation generation is needed, the Agent will guide you through configuring the required API Key. The key is stored only locally and will be read automatically afterward — no need to enter it again.

Technical Reference

Everyday use doesn’t require running scripts manually. If you need to modify generation parameters, asset processing, or web runtime logic, see SKILL.md and references/.

License

MIT