# Video encoding parameters

When **video storage** is on, LeRobot stores each camera stream as an **MP4** file rather than saving **every timestep as its own image file**. **Video encoding compress across time**, which usually cuts **dataset size and I/O** compared to heaps of PNGs, and MP4 stays a **familiar format** for players and loaders. Incoding frames into a MP4 file is a full FFmpeg pipeline: choice of encoder, pixel format, GOP/keyframes, quality vs speed, and
optional extra encoder flags. **Many of those knobs are user-tunable** and are exposed on the dataset config as
**`dataset.camera_encoder_config`** — a nested **`VideoEncoderConfig`** (`lerobot.datasets.video_utils.
VideoEncoderConfig`) passed through **PyAV**.

You can set these parameters from the CLI with **`--dataset.camera_encoder_config.<field>`** (e.g. `lerobot-record`, `lerobot-rollout`). The same block applies to **every** camera video stream in that run. **Video storage must be on** — **`use_videos=True`** in Python APIs or **`--dataset.video=true`** (recording default); with video off, inputs stay as images and **`camera_encoder_config` is ignored.**

For **when** frames are written vs encoded (streaming vs post-episode), queues, and other top-level **`--dataset.*`** switches, see [Streaming Video Encoding](./streaming_video_encoding). For codec/size/speed experiments, see the [video-benchmark Space](https://huggingface.co/spaces/lerobot/video-benchmark).

---

## Tuning Parameters

| Parameter       | CLI flag                                        | Type                 | Default       | Description                                                                                                                                             |
| --------------- | ----------------------------------------------- | -------------------- | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `vcodec`        | `--dataset.camera_encoder_config.vcodec`        | `str`                | `"libsvtav1"` | Video codec name. `"auto"` picks the first available hardware encoder from a fixed preference list, else `libsvtav1`.                                   |
| `pix_fmt`       | `--dataset.camera_encoder_config.pix_fmt`       | `str`                | `"yuv420p"`   | Output pixel format; must be supported by the specified codec in your FFmpeg build.                                                                     |
| `g`             | `--dataset.camera_encoder_config.g`             | `int \| None`        | `2`           | GOP size (keyframes every `g` frames). Emitted as FFmpeg option `g`.                                                                                    |
| `crf`           | `--dataset.camera_encoder_config.crf`           | `int \| None`        | `30`          | Abstract **quality**; mapped per codec in the table below (CRF, QP, `q:v`, etc.). Lower → higher quality / larger output where the mapping is monotone. |
| `preset`        | `--dataset.camera_encoder_config.preset`        | `int \| str \| None` | `12`\*        | Video encoding speed preset; meaning depends on the specified codec. \*Unset + `libsvtav1` → LeRobot sets `12`.                                         |
| `fast_decode`   | `--dataset.camera_encoder_config.fast_decode`   | `int`                | `0`           | `libsvtav1`: `0–2` passed in `svtav1-params`; `h264` / `hevc` (software): if `>0`, sets `tune=fastdecode`; other codecs: often unused.                  |
| `video_backend` | `--dataset.camera_encoder_config.video_backend` | `str`                | `"pyav"`      | Only `"pyav"` is implemented for video encoding today.                                                                                                  |
| `extra_options` | (nested config / non-scalar)                    | `dict`               | `{}`          | Extra FFmpeg options merged after the built-in mapping; **cannot** override keys already set from structured fields above.                              |

---

## Validation

| What                 | Behavior                                                                                                                                                                                        |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Video codec presence | `vcodec` must exist as a video encoder in the local FFmpeg build (after resolving `"auto"`).                                                                                                    |
| Pixel format         | `pix_fmt` is checked against the encoder’s reported pixel formats when available.                                                                                                               |
| Options              | `get_codec_options()` output (including values originating from `extra_options`) is checked against PyAV/FFmpeg option metadata (ranges, integer constraints, string choices) where applicable. |

---

## Mapping: `VideoEncoderConfig` → FFmpeg options

From **`get_codec_options()`** after `vcodec` resolution. Only fields on `camera_encoder_config` are listed here (no global thread / queue flags).

| Resolved `vcodec`                        | `g` | Quality from `crf`          | `preset` | `fast_decode`                              |
| ---------------------------------------- | --- | --------------------------- | -------- | ------------------------------------------ |
| `libsvtav1`                              | `g` | `crf`                       | `preset` | `svtav1-params` includes `fast-decode=0…2` |
| `h264`, `hevc` (software)                | `g` | `crf`                       | `preset` | `tune=fastdecode` if `fast_decode > 0`     |
| `h264_videotoolbox`, `hevc_videotoolbox` | `g` | `q:v` (derived from `crf`)  | —        | —                                          |
| `h264_nvenc`, `hevc_nvenc`               | `g` | `rc=constqp` + `qp` ← `crf` | `preset` | —                                          |
| `h264_vaapi`                             | `g` | `qp` ← `crf`                | —        | —                                          |
| `h264_qsv`                               | `g` | `global_quality` ← `crf`    | `preset` | —                                          |

---

## `extra_options`

- Merged **after** structured options; keys **already** set by `g`, `crf`, `preset`, etc. are **not** replaced by `extra_options`.
- Values are strings or numbers as FFmpeg expects; numeric values are validated when the codec exposes option metadata.

---

## Example

```bash
lerobot-record \
    --robot.type=so100_follower \
    --robot.port=/dev/tty.usbmodem58760431541 \
    --robot.cameras="{laptop: {type: opencv, index_or_path: 0, width: 640, height: 480, fps: 30}}" \
    --robot.id=black \
    --teleop.type=so100_leader \
    --teleop.port=/dev/tty.usbmodem58760431551 \
    --teleop.id=blue \
    --dataset.repo_id=<my_username>/<my_dataset_name> \
    --dataset.num_episodes=2 \
    --dataset.single_task="Grab the cube" \
    --dataset.streaming_encoding=true \
    --dataset.encoder_threads=2 \
    --dataset.camera_encoder_config.vcodec=h264 \
    --dataset.camera_encoder_config.preset=fast \
    --dataset.camera_encoder_config.extra_options={"tune": "film", "profile:v": "high", "bf": 2} \
    --display_data=true
```
