Skip to main content
Built on FLUX 3’s image and video training, FLUX 3 Action learns to predict actions and their visual outcomes together. Given camera images, the current state, and an instruction, it predicts a chunk of actions (32 for DROID and games, 42 for SO-101) alongside future video latents. DROID and game plans cover about two seconds at 15 Hz; SO-101 uses 30 Hz. The current API returns actions; see predicted video for what is available at inference. Start with a prepared robot checkpoint, or fine-tune the model on demonstrations of your own controls and tasks. Run the model or train it for your task.
A pick-and-place skill learned from about 200 demonstrations. FLUX 3 Action controls the SO-101 arm; footage is shown at 4× speed with pauses between plans removed. Teach your arm a task.

​
Try the control loop

Given “put the red cube in the left bin,” the model uses camera images and joint positions to predict motor commands and their visual outcomes. The application executes a set number of actions (n_action_steps), then uses fresh observations to plan again. Try it below: start a mission, drop a cube, and watch how the arm recovers. Change how often it looks to see how that affects its response. This scripted demo illustrates the control loop, cameras, and predicted frames. The videos above show FLUX 3 Action running.

​
What it runs on

An embodiment is the system being controlled: its cameras, its available controls, and the values used to describe its current state. Dimensions above are width × height; JSON canvas_hw uses height, width. DROID and SO-101 have prepared checkpoints. The game and drone examples fine-tune the shared model with new input and output layers for their controls, using recorded demonstrations.

Run the model

Get your first action prediction, then connect the model to a control loop.

Fine-tune

Teach the model new controls and tasks with your own demonstrations.

SO-101 with LeRobot

Train your SO-101 arm for a new task using LeRobot and a LoRA.

Example: a video game

Train on a scripted bot’s demonstrations, then test the model in a shooter and a racing game.

Example: a drone

Train a drone to follow flight instructions, then test it in new simulated rooms.