One loop from pixels to physical outcomes
Physical AI repositories combine a physics engine, renderer, robot assets, datasets, a training framework, a policy runtime, and a controller. A complete AMD port keeps their semantics aligned: the model receives the intended images and state, produces actions in the trained scale and order, and the simulator executes them at the correct cadence.
Task definition, observation schema, action schema, control frequency, checkpoint identity, seeds, result JSON, and media are versioned together. That contract turns a working experiment into a reusable system.
Six layers with explicit acceptance gates
Each layer has a concrete input, executable check, and artifact. The same sequence works for a new simulator, robot, or policy without coupling every issue into one environment.
Runtime
Pin ROCm, HIP, Python, PyTorch or JAX, simulator, and plugin versions; record the detected GPU backend.
Simulation
Run official reset, step, and render calls headlessly, then export a stable MP4 from the original scene.
Data
Audit observation and action dimensions, temporal order, episode boundaries, padding, and normalization statistics.
Training
Preserve configuration, metrics, checkpoints, hashes, and resume semantics under one output directory.
Inference
Bind the checkpoint to its tokenizer, stats, camera map, action chunk, controller, and execution frequency.
Evaluation
Use fixed tasks, seeds, process topology, full denominators, videos, result JSON, and SHA-256 manifests.
DISCOVERSE: expert trajectories, HD replay, and 3DGS
DISCOVERSE is the broadest simulator workstream in the suite. MuJoCo tasks, AIRBOT and MMK2 assets, expert trajectories, policy entry points, ROS2, LiDAR, Gaussian Splatting, and MP4 export share one reproducible AMD integration path.
3.1 Reachable expert motion
The block_bridge_place trajectory exposed an interaction between inverse-kinematics reachability, support collisions, and discontinuous stage targets. The repaired workflow checks end-effector reachability first, support clearance second, and only then freezes the trajectory and task definition. The resulting strict replay completes 31 of 31 stages.
3.2 Headless, multi-view video
MuJoCo offscreen rendering and platform-specific headless backends generate native RGB frames before FFmpeg encoding. Four MMK2 tasks are captured at 1920x1080 from synchronized views so end-effector motion, object state, and scene changes remain readable.
3.3 Gaussian Splatting on ROCm
The isolated ROCm renderer loads 1,334,537 RM2 points and 1,026,855 SkyRover points, renders non-empty 640x360 frames, and drives dynamic Franka and UR5e replay. Separating the simulator and renderer runtimes keeps their graphics and ML dependencies stable.
source local_tools/activate_amd_runtime.sh
python -c "import torch, gaussian_renderer; print(torch.__version__, torch.version.hip)"
python local_tools/verify_gaussian_renderer.py \
--robot rm2 --output results/gaussian/rm2.png \
--width 640 --height 360RoboCasa365: assets, policy protocols, and four-view replay
The RoboCasa365 workstream brings the complete official asset archive, environment registration, MuJoCo rendering, Pi0.5 JAX inference, GR00T inference, and a shared 16-task protocol onto AMD 395. The same run produces per-task statistics, aggregate JSON, and synchronized center, left, right, and wrist videos.
| Workstream | Protocol | Visual output | Artifacts |
|---|---|---|---|
| Pi0.5 | 16 tasks × 50 | Multi-task closed-loop replay | JSON · MP4 · SHA-256 |
| GR00T | 16 tasks × 50 | Household manipulation | JSON · MP4 · SHA-256 |
| Long-horizon gallery | 4 tasks × 10 | Steam, restock, pack, and coffee tasks | Four-view task films |
4.1 Asset integrity before policy inference
The asset archive is verified by byte size, checksum, extracted file count, scene reset, and render output. This makes environment identity an explicit part of every policy result.
4.2 One run, two readable outputs
The fixed protocol supports model comparison; the four-view gallery explains the behavior. Both originate from the same model and environment run, and the result index connects them by task and seed.
ROCm JAX and Pi0.5: from backend test to robot actions
The native environment follows AMD's official ROCm JAX route: Python 3.12, ROCm 7.14, JAX and JAXlib 0.10.0, and a rocm:0 backend on gfx1151. Matrix, convolution, attention, Orbax restore, tokenizer, normalization, vision input, action decoding, and MuJoCo execution are validated in sequence.
A visible GPU backend establishes device access. A robot policy additionally depends on parameter-tree compatibility, tokenizer and normalization identity, camera tensors, attention kernels, action decoding, and simulator timing.
5.1 Version the runtime as one unit
ROCm, JAX, jaxlib, the ROCm plugin, PJRT, and model code move together. This resolves target-GFX kernels, shared-memory pressure, convolution compilation, and checkpoint metadata at their actual interface boundaries.
5.2 Validate one task, then the multi-task protocol
- Restore Orbax parameters, tokenizer, and normalization statistics.
- Run one diagnostic task with saved observations, actions, and MP4.
- Freeze the model and environment identity in the run manifest.
- Run the official task set and aggregate results without changing the protocol.
Training, checkpoints, and closed-loop evaluation
Every Embodied provides native notebooks for SmolVLA, Pi0, and ACT training, protected fine-tuning, evaluation, and video generation. RoboWits adds the W7900 ACT 100k route with checkpoint restore, HD replay, task evaluation, and a final hash manifest.
Approach, contact, grasp, lift, transport, place, release, and recovery statistics map a task result back to data coverage, action bridging, chunk execution, cameras, and normalization.
Nine recurring interfaces in Physical AI ports
ROCm through framework APIs
Record HIP version, rocminfo, GPU model, backend, and measured memory use together.
Offscreen rendering
Select EGL, OSMesa, or the simulator's native offscreen path, then validate frames and encoding.
Target-GFX coverage
Exercise matrix, convolution, attention, plugin, and PJRT paths on the actual GPU architecture.
Chunk execution
Chunk length, overlap, temporal ensembling, joint convention, gripper encoding, and control rate define execution.
Checkpoint statistics
Archive state dimensions, action scales, camera shape, color order, and normalization with the checkpoint.
Accumulated control error
Action traces and stage metrics reveal where local prediction error becomes a physical task outcome.
Scene identity
Version official assets, licenses, BlenderKit references, and task distributions with the environment.
Repeatable evaluation
Fix seeds, process count, cold or warm startup, worker reuse, and output layout.
PVC and restart recovery
Keep datasets, checkpoints, manifests, environment records, and resume commands on persistent storage.
Reproduce the smallest complete loop first
Start with runtime identity, official scene reset, one rendered episode, and artifact validation. Then run training or the fixed evaluation with the same environment and output contract.
export PROJECT_ROOT=/path/to/radeon-physical-ai-evidence-suite
export DATA_ROOT=/path/to/data
# 1. Verify the AMD runtime
python -c "import torch; print(torch.__version__, torch.version.hip, torch.cuda.is_available())"
rocminfo | head -n 30
# 2. Run an official reset/render and save an MP4
python scripts/robocasa_amd_smoke.py --output results/smoke
# 3. Validate the intended checkpoint and evidence files
sha256sum -c SHA256SUMS
ffprobe -v error -show_streams results/formal/success.mp4
# 4. Run the fixed task and seed protocol
python scripts/evaluate_official.py \
--checkpoint /path/to/checkpoint \
--seeds 0 1 2 3 4 \
--save-video --output results/formal| Gate | Artifact | Following gate |
|---|---|---|
| import + backend | Pinned dependencies and AMD device record | reset / render |
| reset / render | Frame and MP4 with scene identity | Policy load |
| Policy load | Checkpoint, tokenizer, stats, and manifest | Closed loop |
| Closed loop | Actions, stages, videos, and result JSON | Fixed protocol |
| Fixed protocol | Aggregate metrics and SHA-256 | Publish |
A review path from result to reproduction
The public package organizes each workstream around five linked objects: source, report, model or checkpoint manifest, evaluation data, and video. A reviewer can start with the film, open the task result, verify the artifact hash, and continue into the exact environment and command.
- Source — versioned adapters, launchers, validators, and notebooks
- Runtime — AMD GPU, ROCm, framework, simulator, and dependency matrix
- Results — fixed protocol, aggregate metrics, stage statistics, and task videos
- Integrity — tagged release, manifests, and SHA-256 verification