One engine for embodied inference and RL rollout.

Documentation · Quick start · Models · Performance · 简体中文

EmbodiInfer is an inference and RL-rollout engine for embodied models. Run manipulation policies, world-action models, and recurrent navigation policies through a common Python interface, with model-specific execution optimizations and shared batching, session, and multi-GPU machinery.

Embed it in your application or trainer, or serve predictions to EmbodiRun for robot deployment and execution.

Why EmbodiInfer?

Optimize the model path. Reuse the engine from robot inference to RL rollout.

⚡ Capture, compile, replay CUDA Graphs · torch.compile / Inductor Replay static prefix and decode paths and compile model computation. Policy-specific profiles target the repeated work in denoising and token generation. Execution paths →	🎚️ Precision to fit your GPU BF16 · FP8 · INT8 · NVFP4 Choose hardware-specific π0.5 and StreamVLN profiles: INT8 on AGX Orin, FP8 on RTX 4090 and Thor, and NVFP4 on Thor. Precision and backend guide →
🔥 Optimized down to the operators Triton attention · fused model operations Specialized attention kernels and fused normalization, rotary embeddings, and gated activations accelerate supported native model paths. Native optimizations →	🧠 Reuse context, retain history Prefix KV reuse · recurrent sessions Reuse fixed observation context across denoising steps. Navigation policies retain episode history, with transactional updates and explicit reset and cancellation. Context and session lifecycle →
🚀 From one request to many environments Request batching · multi-GPU execution Batch observations, aggregate asynchronous requests, and distribute work across replicas. Supported policies also offer tensor parallelism. Parallel execution →	🔄 Inference that fits the RL loop Action sampling · log probabilities · weight refit Use RL-capable decoders to collect rollouts and refresh policy weights. Keep your trainer and learning objective; reuse model execution. RL integration →

Choose features by policy and hardware in the capability reference. Adapters own model-specific optimizations; the shared scheduler stays model-independent.

Performance

Lower latency on a single RTX 4090

Native inference optimizations reduced mean end-to-end latency by 47.7% for π0.5 and 31.2% for GR00T N1.7, compared with the earlier EmbodiInfer path:

Policy	Before	Optimized	Optimized throughput
π0.5	74.33 ms	38.89 ms	25.71 observations/s
GR00T N1.7	45.51 ms	31.30 ms	31.95 observations/s

Recorded on September 9, 2026: RTX 4090, BF16, batch size 1, and 1,600 LIBERO-10 observations per model. π0.5 uses pi05_libero_finetuned_v044_gitcode with 10 denoising steps; GR00T uses nvidia/GR00T-N1.7-LIBERO with 4. Both paths use Inductor and denoising CUDA graphs. The baseline uses SDPA; the optimized paths add native inference and prefix graphs, with Triton prefix and denoising attention for π0.5 and SDPA for GR00T.

Timing covers decoded CPU observations through CPU action output, after warmup on 10 cross-task observations. The linked reports include checkpoint details, environments, commands, numerical checks, and batch-size sweeps. At batch size 32, the optimized GR00T path reached 59.33 observations/s on the same GPU.

Explore navigation and world-action benchmarks, quantized π0.5, quantized StreamVLN, and multi-GPU measurements.

Supported models

✓ Implemented · ◐ Experimental · ○ Planned

🦾 Manipulation	🧭 Navigation	🌐 World-action models
✓ π0.5 ✓ GR00T N1.7 ✓ OpenVLA-OFT ✓ LingBot-VLA ✓ DM0.5 Manipulation policies with RL decoder interfaces.	✓ StreamVLN ✓ Qwen R2R · low / panoramic ✓ NaViDA ◐ ActiveVLN Episode-scoped history and recurrent inference.	✓ Cosmos Policy Diffusion-based action generation and candidate planning.

Network serving: π0.5, DM0.5, and StreamVLN through HTTP / WirelessComm, at batch size 1. All model families have Python entry points. The capability reference covers batching, CUDA graphs, RL interfaces, and installation profiles. ActiveVLN's real-checkpoint GPU parity is pending.

Planned models

• SmolVLA — model adapter, input processing, and serving integration.
• OpenVLA — base-model inference, separate from OpenVLA-OFT.

See the model roadmap for the implementation steps. Robot and simulator integrations live in EmbodiRun.

Quick start

Run without a checkpoint

For a first run without a checkpoint, install the core package with Python 3.10+ and uv 0.12.x:

git clone https://github.com/BUAA-CI-LAB/EmbodiInfer.git
cd EmbodiInfer
uv sync --frozen
uv run python examples/quickstart.py

This CPU-friendly example uses a synthetic policy to walk through single and batched requests and print their action-chunk shapes. The Linux lock includes CUDA-enabled Torch wheels, so installation can be large even on a CPU-only machine.

Load π0.5

Use the isolated π0.5 profile and a CUDA GPU:

uv sync --python 3.12 --frozen --no-dev --group pi05
uv run --no-sync python examples/pi05_inference.py \
  --ckpt lerobot/pi05_base --envs 1

This example loads real weights but uses synthetic observations to demonstrate the engine API. Follow Quick start for the input contract and Serving to connect real observations and configure their mapping to a checkpoint.

Choose an integration

Workflow	Interface	Start here
Inference inside an application	Vvla.act(observation) or Vvla.act(observations)	Python API
Network inference for a robot runtime	HTTP / WirelessComm sessions	Serve and send your first observation
RL rollout and weight updates	Rollout / refit interfaces	RL integration
Multi-GPU execution	Data parallelism / tensor parallelism	Parallelism

The network launchers support π0.5, DM0.5, and StreamVLN at batch size 1. Other policies use the Python API. See the capability table for policy-specific support.

The distribution is named embodiinfer; Python imports remain under embodiinfer. Existing vvla-* command aliases and vvla.policy.* wire schemas remain supported.

Documentation

Task	Guide
Install a model profile	Installation
Run inference	Quick start · Examples
Deploy a service	Serving
Use multiple GPUs	Parallelism
Integrate or extend the engine	Architecture · Python API
Inspect model behavior	Models
Evaluate performance	Benchmarks

Contributing

See CONTRIBUTING.md for development setup, model integration, and numerical validation requirements. Use GitHub issues for bugs and feature requests, and SECURITY.md for private vulnerability reports. Community participation follows our Code of Conduct.

License

Apache-2.0. See LICENSE, NOTICE, and third-party notices. Model weights and datasets retain their upstream licenses.