pi05-piper1h-ea β€” Ο€β‚€.β‚… finetuned on Piper H (three checkpoints)

Ο€β‚€.β‚… (flow matching) finetuned from pi05_base for a 14-DoF Piper H robot, on real Piper H teleop and retargeted EgoDex human video.

This repo started as a single run and now holds three checkpoints from three runs that share one recipe and differ only in what data they saw. They are kept together because they are only interpretable against each other.

# Run / config Data Draw Steps Checkpoint Path in this repo W&B
A pi05_piper1h_ea 1 h mixture: 20.9 min teleop + 40 min ego 24 teleop + 40 ego 2,400 2399 repo root 4ksr9cp0
B pi05_piper1h_teleop teleop only (same 20.9 min) 64 teleop 600 599 piper1h_teleop/599/ blk1ni8c
C pi05_piper20m_ea 20 min mixture: 10 min teleop + 10 min ego 32 teleop + 32 ego 800 799 piper20m_ea/799/ 9wyobg5w

All three are raw Orbax training checkpoints (params + train_state + norm stats), not converted HF-format models. They are meant to be loaded by openpi. ~13 GB each, ~39 GB total.

NOTE ON "BIMANUAL": the rig is bimanual and the tensors are 14-D, but action dims 7–13 are identically zero in every dataset used here β€” min and max are exactly 0.0. This is a single-arm capture (the right arm, matching the pick_and_lift_right task name) padded into a bimanual container. Only 7 of the 32 model action dims carry any signal. Read every loss number below with that in mind.


⚠️ Read this first

Checkpoints Final step only, for each of the three runs. No intermediates.
Evaluation None, for any of the three. Every number in this card is training loss.
Holdouts Withheld correctly and verified in the logs β€” but never scored.

All three runs completed cleanly (EXIT=0). Because there is no evaluation, "final" means last, not best, and nothing here establishes that any one of the three is better than the others.

The three losses are NOT directly comparable. This is the single most important thing on this page, and it is explained in Comparing the runs below. In short: they sit at different points on different cosine schedules, and β€” worse β€” each run normalizes its actions with its own quantiles, so the losses are not even measured on the same scale.


Shared recipe

Everything below is identical across A, B and C. Only the data and step count differ.

Field Value
Model Pi0Config(pi05=True) β€” PaliGemma backbone + flow-matching action expert
PaliGemma variant gemma_2b_lora
Init gs://openpi-assets/checkpoints/pi05_base/params (12 GB)
Action dim 32 (14 real, 18 zero-padded β€” and 7 of the 14 are themselves constant zero)
Action horizon 30 (not the openpi default of 50 β€” see Why H=30)
Max token len 200
Batch size 64
Optimizer AdamW, clip_gradient_norm=1.0
LR schedule Cosine, warmup β†’ peak 3.5e-5 β†’ decay 3.5e-6 over the full run
EMA off (ema_decay=None)
Dataloader workers 16
Normalization quantile (use_quantile_norm=True, q01/q99), recomputed per run
Augmentation ImageAugment(brightness=0.15, contrast=0.15, saturation=0.1, hue=0.03, crop_scale=(0.95,1.0)), geometric crop on base_0_rgb only
Hardware 2Γ— A100-SXM4-80GB (NVLink), data-parallel, --fsdp-devices 1
Throughput ~3.0 s/step on all three runs
Env XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 (required β€” batch 64 OOMs at the 0.75 default)
Code angkul07/openpi, branch piper1h-pi05 (A: dd8fa13, B: 1946f26, C: a079613)

The LR schedule was inherited unchanged from the YAM arms of this series so that the mixture and the embodiment stay the only moving variables. It was not tuned for Piper.

Why H=30 and not the default 50

Two independent reasons, either sufficient:

  1. openpi's LeRobot path clamps delta_timestamps at episode ends β€” it repeats the final action rather than dropping the sample β€” and nothing in openpi consumes is_pad. So clamped steps enter the flow-matching loss as genuine regression targets, not ignored padding. The clamped share of supervised action steps is (Hβˆ’1)/2L. Per dataset:

    mean episode length clamped @ H=30 clamped @ H=50
    teleop (1 h, 154 eps) 162.8 8.9% 15.1%
    ego (1 h, 613 eps) 78.2 18.5% 31.3%
    teleop (20 min, 75 eps) 164.7 8.8% 14.9%
    ego (20 min, 153 eps) 78.8 18.4% 31.1%

    This is signal corruption, not reweighting.

  2. 20 Hz. H=30 at 20 Hz is 1.5 s, the same physical horizon H=50 gave the 30 Hz YAM arms (1.67 s). Inheriting 50 here would have silently asked for a 2.5 s chunk.

Observation space (all three runs)

Two real cameras, not three. The rig stores three, but the key names do not describe their content β€” this was established by extracting frames and looking, not by reading metadata:

dataset key teleop content ego content model slot
front top-down workspace full egocentric frame base_0_rgb β€” real
right right-arm wrist camera tight crop on grasp point left_wrist_0_rgb β€” real
top sideways view of the robot wide crop on grasp point dropped
β€” (no camera) (no camera) right_wrist_0_rgb = zeros, masked off

front and right genuinely correspond across the two halves. top cannot: a head-mounted ego camera can never produce a sideways view of the robot, so that key holds two unrelated images under one name.

It is dropped in the repack rather than masked, because Pi0.embed_prefix runs SigLIP on every entry of obs.images regardless of the mask β€” masking costs the same compute while keeping the contradictory content. The surviving wrist view goes in slot 1 with slot 2 as the masked padding slot, matching droid_policy and libero_policy.

Note the teleop-only run (B) keeps this exact two-camera repack even though it has no ego half, so the observation space is identical across all three.

Caveat: dropping top at the repack does not skip its video decode β€” LeRobotDataset decodes every video feature before transforms run.

Also: 14-D joint state fed as a discrete state token (TokenizePrompt(discrete_state_input=True), the Ο€β‚€.β‚… convention), and a language prompt from the LeRobot task string (prompt_from_task=True).

Action space (all three runs)

  • 14-D, horizon 30, zero-padded to 32.
  • DeltaActions(mask=(TrueΓ—6, False, TrueΓ—6, False)) β€” the 12 arm-joint dims are predicted relative to current state; the 2 gripper dims stay absolute.
  • Verified empirically that both halves store absolute actions, which is what this mask assumes: mean|action βˆ’ state| is 0.005 (teleop) and 0.022 (ego) against mean|action| of 0.73 and 1.16.
  • The mask is symmetric per arm (6 joints + 1 gripper, twice), so it is correct regardless of which arm occupies dims 0–6 β€” which matters, because the active arm is the right one.
  • Zero-variance dims are safe: both normalize paths carry a + 1e-6 guard, so the 25 constant dims produce no NaN.

Run A β€” pi05_piper1h_ea (1-hour mixture, 2,400 steps)

Checkpoint 2399, at the root of this repo. Wall clock 2 h 02 m 59 s for 2,400 steps.

Source Train frames % frames Per batch Gradient share Epochs
Piper H teleop (real) 23,347 32.7% 24 37.5% 2.47
EgoDex β†’ Piper H (retargeted) 47,953 67.3% 40 62.5% 2.00
Total 71,300 100 64 153,600 frames seen

Teleop is 154 episodes / 25,075 frames before the holdout; 15 episodes (holdout_fraction=0.1, seed 0 β€” indices 2, 5, 10, 25, 38, 44, 72, 76, 89, 92, 97, 119, 121, 137, 149) are withheld. Ego is 613 episodes / 47,953 frames, no holdout. Both LeRobot v2.1 at 20 fps.

Teleop's gradient share (37.5%) is 1.15Γ— its storage share β€” a deliberate but mild oversample toward the real-robot half. The draw was chosen to put both sources at ~2 epochs; with pools 2.05Γ— apart, a 32/32 draw would have reached ego 2.0 epochs only at 3,000 steps, by which point teleop is at 4.24.

LR warmup 60. Checkpoints every 250 (max_to_keep=2, keep_period=1000); 1000/2000/2250 were written during the run and are not uploaded. Norm stats piper1h_p50.

Step window Loss chunk_first chunk_last grad_norm param_norm
0 – 100 0.04832 0.03718 0.07005 0.20740 1803.16
100 – 250 0.02526 0.00909 0.04255 0.09605 1803.18
250 – 500 0.02232 0.00819 0.03783 0.09034 1803.21
500 – 1,000 0.01893 0.00666 0.03138 0.08734 1803.28
1,000 – 1,500 0.01540 0.00577 0.02495 0.08917 1803.33
1,500 – 2,000 0.01286 0.00519 0.02074 0.08621 1803.36
2,000 – 2,400 0.01170 0.00489 0.01844 0.08489 1803.36

First-step loss 0.19416; minimum single-step loss 0.00741 at step 2,182; param_norm drift +0.2074 (+0.0115%). Gradient clipping bound on exactly 3 steps β€” 0, 1, 2 (1.170, 1.244, 1.052) and never again.

Run B β€” pi05_piper1h_teleop (teleop only, 600 steps)

Checkpoint 599, at piper1h_teleop/599/. Wall clock 34 m 28 s.

The single-source control for A. Same embodiment, same cameras, same horizon, same LR β€” the only variable removed is the ego half.

Source Train frames Per batch Epochs
Piper H teleop (real) 23,347 64 1.64

Same 154-episode teleop dataset and the same 15-episode holdout as A (selection is deterministic in total_episodes/fraction/seed, and the source dataset is unchanged, so these are literally the same episodes). LR warmup 30. Norm stats piper1h_teleop_only β€” recomputed, and not optional; see Comparing the runs.

This is less teleop exposure than A got. A drew 24 teleop for 2,400 steps = 57,600 teleop frames (2.47 epochs); B sees 38,400 (1.64 epochs), i.e. 0.67Γ—, on 4Γ— fewer optimizer steps. So B-vs-A is not a clean ablation of "ego added or not" β€” it also cuts teleop exposure and total steps. To isolate the ego contribution properly, a teleop-only arm needs 973 steps to match teleop frames, or 2,400 to match optimizer steps. 600 was chosen for cost.

Step window Loss chunk_first chunk_last grad_norm param_norm
0 – 100 0.04236 0.03132 0.07582 0.18417 1803.16
100 – 250 0.02452 0.01071 0.04983 0.09861 1803.18
250 – 400 0.02000 0.00755 0.04015 0.08651 1803.20
400 – 600 0.01669 0.00677 0.03310 0.08965 1803.21

First-step loss 0.16931; minimum 0.01209 at step 504; param_norm drift +0.0493 (+0.0027%). Clipping bound on steps 0 and 1 only (1.188, 1.252).

Run C β€” pi05_piper20m_ea (20-minute mixture, 800 steps)

Checkpoint 799, at piper20m_ea/799/. Wall clock 44 m 42 s.

A small, cheap mixture on the same recipe as A β€” roughly 1/3 the data and 1/3 the steps.

Source Train frames % frames Per batch Gradient share Epochs
Piper H teleop (real) 11,035 47.8% 32 50% 2.32
EgoDex β†’ Piper H (retargeted) 12,049 52.2% 32 50% 2.12
Total 23,084 100 64 51,200 frames seen

Teleop is 75 episodes / 12,048 frames with 8 withheld (indices 1, 2, 5, 19, 22, 35, 43, 57); ego is 153 episodes / 12,049 frames, no holdout. LR warmup 40. Norm stats piper20m_p50.

The draw is 32/32, not A's 24/40, and that is the same rule rather than a different one. A's 24/40 was derived from wanting ~2 epochs per source against 1:2 pools. C's pools are already 1:1 (12,048 vs 12,049 frames), so the same rule gives 32/32. C is therefore not a proportional scale-down of A: A is 1:2 teleop:ego on disk and 37.5/62.5 by gradient, C is 1:1 and 50/50. Anything read off C about mixture ratio does not transfer to A.

Step window Loss chunk_first chunk_last grad_norm param_norm
0 – 100 0.04306 0.03204 0.06342 0.19434 1803.16
100 – 250 0.02208 0.00884 0.03722 0.09758 1803.18
250 – 400 0.01839 0.00760 0.03034 0.08793 1803.20
400 – 600 0.01523 0.00650 0.02526 0.09158 1803.22
600 – 800 0.01286 0.00631 0.02009 0.08412 1803.22

First-step loss 0.19165; minimum 0.00786 at step 773; param_norm drift +0.0656 (+0.0036%). Clipping bound on steps 0 and 1 only (1.258, 1.224).

C's data needed no repair β€” unlike A and B's teleop half, all 75 parquet files were free of the datasets 4.x "List" metadata and all 684 videos already started at pts 0.000000. This was checked, not assumed.


Comparing the runs

Final 200-step trailing means, side by side:

A 1 h mix (2,400) B teleop (600) C 20 min mix (800)
flow_loss 0.01142 0.01669 0.01286
chunk_first 0.00455 0.00677 0.00631
chunk_last 0.01788 0.03310 0.02009
chunk_last / chunk_first 3.93Γ— 4.89Γ— 3.18Γ—
grad_norm 0.08374 0.08965 0.08412
param_norm drift +0.0115% +0.0027% +0.0036%
clipped steps 0, 1, 2 0, 1 0, 1

Do not read this table as a ranking. Three confounds, in descending order of severity:

  1. The losses are not on the same scale. Each run computes its own q01/q99 normalization over its own pool, and the flow-matching loss lives in that normalized space. B's teleop-only quantiles are 5–9Γ— narrower than A's mixture quantiles (e.g. actions dim 3: B spans [βˆ’0.126, 0.251], A spans [βˆ’1.832, 1.552]). Under a tighter normalizer the same physical error produces a larger normalized error. So B's higher loss does not mean B fits its data worse β€” a good part of that gap is the denominator. C's stats land within ~10% of A's, so this confound is small for A-vs-C and large for anything involving B.
  2. Different points on different cosine schedules. A ran 2,400 steps, C 800, B 600. Each decays 3.5e-5 β†’ 3.5e-6 over its own length, so the final windows are not comparable snapshots. A has had 3–4Γ— more optimizer steps.
  3. Different data. Different pool sizes, different mixture ratios, different epoch counts.

What the table does support: all three optimized cleanly and near-identically in shape β€” grad_norm settles to ~0.085 by step 100 in every run, clipping binds only in the first 2–3 steps, and no run shows instability. The recipe is reproducible across three data configurations.

The holdouts overlap β€” and there is exactly one clean comparison

C's 75 teleop episodes are a strict subset of A/B's 154: all 75 match by exact content hash of observation.state + action. Its 153 ego episodes likewise match into A's 613. So the two holdouts interleave, and this was measured rather than assumed:

  • C's 8 held-out episodes map to A/B indices 2, 4, 10, 39, 45, 72, 89, 118. Of those, 4 were in A and B's training set (A/B indices 4, 39, 45, 118). So scoring A or B on C's holdout is 50% contaminated.
  • Conversely, 4 of A/B's 15 held-out episodes (25, 76, 97, 149) are in C's training set. So scoring C on A/B's holdout is 27% contaminated.

The clean protocol: score all three checkpoints on the 11 episodes held out by both β€” A/B indices 2, 5, 10, 38, 44, 72, 89, 92, 119, 121, 137. Every one of those was withheld from all three runs. That is the only comparison across these three checkpoints that is not contaminated, and it has not been run.


Data preparation β€” what we changed

Two defects in the teleop half blocked training outright for runs A and B. Both were fixed locally on the training box, not upstream, so a fresh pull of the 1-hour dataset will reproduce them. Ego was clean on both counts, and run C's data was clean on both counts (verified).

1. Parquet metadata: datasets 4.x "List" β†’ 3.x "Sequence"

The teleop converter embedded HuggingFace feature metadata written by datasets β‰₯ 4.x, which renamed Sequence to List. openpi pins datasets==3.6.0, whose Features.from_dict has no List type, so load_dataset() raised before training began:

ValueError: Feature type 'List' not found. Available: ['Value', 'ClassLabel', ..., 'Sequence', ...]

Fixed by rewriting "_type": "List" β†’ "Sequence" in the schema metadata of all 154 teleop parquet files. Only the metadata string changed; every file was verified for identical row count and bit-identical observation.state / action values before the swap. Ego carried no huggingface metadata at all, so its types are inferred from the arrow schema and it was unaffected.

2. Video PTS rebase: a uniform one-frame timestamp offset

Every one of the 462 teleop videos (154 episodes Γ— 3 cameras) started at pts 0.05 instead of 0.0 β€” exactly one frame at 20 Hz. All 613 ego videos started correctly at 0.0. With lerobot's default tolerance_s=1e-4 this is a hard crash mid-training:

AssertionError: query timestamps violate tolerance (0.0500 > tolerance_s=0.0001)
queried: 0.0   loaded: 0.05

No frames were missing. Teleop frame counts matched parquet rows exactly (195/195, 188/188, 174/174), spacing was a uniform 0.05, and the last frame sat +0.05 past the last parquet timestamp. The entire PTS timeline was shifted by one frame β€” the encoder started its clock at 1/fps, not 0.

Fixed losslessly with ffmpeg -c copy -bsf:v setts=PTS-STARTPTS (no re-encode). All 462 were then verified: frame count preserved, first pts 0.0, last pts equal to the episode's last parquet timestamp.

Why rebase rather than widen tolerance_s. Raising the tolerance to ~0.05 would have "worked" with a one-line change, but it maps dataset frames 0 and 1 onto video frame 0, leaving every teleop sample with a one-frame-stale image while ego stays correctly aligned. The two halves would then disagree about image/action alignment β€” the exact class of silent inconsistency that quietly degrades a co-trained policy.

Note for anyone repeating this: -itsoffset -0.05 looks equivalent and silently drops a frame (194 vs 195, measured). Use setts.

3. No gripper rescale β€” measured as unnecessary

The YAM arms of this series needed a gripper range alignment. That is not the case here, and it was checked rather than assumed. Both halves store strictly binary {0.0, 1.0} grippers with the same polarity:

start mid end fraction at 1
teleop 0.745 0.677 0.083 0.50
ego 0.472 0.466 0.411 0.44

Same encoding, no inversion, so no rescale was applied. But see Caveats β€” the ego gripper carries almost no temporal signal, which is a different and arguably worse problem than a scale mismatch.

What was deliberately not changed

  • Ego retargeting error was not filtered. Mean IK error is 4.770 cm over the 613 clips, and it is bimodal, not uniform: 0.701 cm where the wrist-orientation constraint is never pinned, versus 6.944 cm where it is pinned in >50% of frames. The cause is Piper's joint5 range (Β±1.22 rad) against the Β±1.571 the source retarget assumed. _PIPER1H_EGO_EXCLUDE is empty in all three runs.
  • No resolution matching between halves. Teleop is 640Γ—480 portrait, ego is 224Γ—224 square; the resize to 224 squashes teleop ~1.33Γ— vertically.
  • Teleop was not rescaled or resampled. It is the reference.

Learnings and takeaways

1. Absolute loss is barely interpretable here, because 25 of 32 action dims are constant zero. 18 are standard zero padding, where x_t = tΒ·noise makes the target u_t = noise = x_t/t a closed-form function of the input β€” free to fit. On top of that, 7 more are the all-zero left arm. Roughly 78% of the objective is trivially predictable. The headline loss falling ~0.19 β†’ ~0.043 in the first 100 steps is mostly the model learning to emit zeros, in all three runs.

2. Normalization scale is a confound between runs, not just a preprocessing detail. This is the lesson B taught. Recomputing norm stats per pool is mandatory β€” reusing A's piper1h_p50 for a teleop-only run would have squeezed teleop into Β±0.1 of the [βˆ’1,1] axis and shifted the gripper midpoint. But the necessary fix has a consequence people forget: it changes the units of the loss. Any cross-run loss table where the runs have different norm stats is comparing different quantities. Report the normalizer ranges alongside the losses, or compare only in physical units after denormalizing.

3. chunk_last is the metric that carries information; chunk_first is nearly free. At the end of each run chunk_last is 3.2–4.9Γ— chunk_first. Predicting the action at t=0 is close to trivial when actions are deltas from current state. The 1.5 s-ahead prediction is where the difficulty lives. Note the caveat, though: 9% (teleop) to 18% (ego) of supervised steps are clamped repeats of the final action, and those land disproportionately in the late part of the chunk β€” so chunk_last is partly measuring how well the model predicts "hold still at the end", which is easier than real motion.

4. All three runs barely moved the base model, and the evidence is unambiguous. param_norm drifted +0.0115% (A), +0.0027% (B), +0.0036% (C). With LoRA, a 3.5e-5 peak and ~2 epochs, essentially all capability is inherited from pi05_base. That was the stated intent, and it means none of these runs is likely to have overfit. It equally means you should not expect strong Piper-specific specialisation from any of them. If eval comes back bland rather than wrong, under-training is the first hypothesis, and more steps or a higher peak LR is the lever β€” not more data.

5. No run was length-limited. Loss fell monotonically in every window of every run with no plateau, and chunk_last was still descending at the final step in all three. Step counts were chosen to hit ~2 epochs per source, not because a curve flattened.

6. The recipe transfers across data scales without retuning. A, B and C differ by 3Γ— in data and 4Γ— in steps, and all three produced the same optimization signature: grad_norm ~0.085 from step 100, clipping only in the opening 2–3 steps, monotone descent, no spikes. The warmup fraction was the only knob that needed rescaling (2.5% at 2,400 steps, 5% at 600–800, because 2.5% of 600 is a 15-step ramp to peak). Whatever else is uncertain here, the optimizer settings are not the fragile part.

7. Subset datasets silently break holdout hygiene. C was built as a small cut of the same source data as A and B, which is sensible β€” but it re-derives its holdout from total_episodes=75 instead of 154, so the two splits interleave rather than nest. The result is 50% contamination in one direction and 27% in the other, which would have quietly inflated any cross-run eval. If you cut a subset of a dataset you already have runs on, carry the parent's holdout episode IDs into the subset instead of re-deriving the split. The 11-episode intersection is the workaround here; carrying IDs is the fix.

8. Aggregate loss cannot answer what these mixtures were built to ask. A single scalar over a mixture cannot distinguish learning from the teleop half, the ego half, or fitting the 25 constant dims. Per-source (loss_teleop / loss_ego) and real-vs-padded-dim splits remain the highest-value cheap change: split jnp.square(v_t βˆ’ u_t) before the final mean and carry a source_index through MixtureDataset into the train step.

9. Check what the camera keys actually contain before mapping them. The teleop rig's names are actively misleading β€” front is the top-down view, top is a sideways view of the robot, and only right is honest. One frame extracted per key catches it; metadata does not.

10. Infrastructure notes worth reusing.

  • 2Γ— A100 SXM4 uncapped at 400 W ran 3.07 s/step vs 3.528 for an otherwise identical pair capped at 275 W β€” ~14% throughput for free. Check nvidia-smi -q -d POWER before benchmarking a rented box.
  • nproc reported 128 on this container; the real quota from /sys/fs/cgroup/cpu.max was 30.7 cores. Size worker pools off cgroup, never nproc. num_workers=16 is a non-lever β€” the loader uses ~3 cores and the run is compute-bound.
  • XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 is required for batch 64 on 80 GB; the 0.75 default OOMs.
  • Ο€β‚€.β‚… checkpoints measured 13 GB (6.7 GB params + 5.9 GB train_state), not the 16–17 GB budgeted.
  • A dataset authored with datasets β‰₯ 4.x will not load under openpi's pinned 3.6.0 if the converter embedded HF feature metadata. Cheapest check before renting a GPU: pq.read_schema(f).metadata.
  • Setting save_interval equal to num_train_steps yields exactly one checkpoint: a run of N steps covers indices 0…Nβˆ’1, so the interval never fires mid-run and only the final write happens.

Caveats and open questions

These bound what the numbers above can support.

  • No evaluation of any kind, for any checkpoint. Holdouts exist and were correctly withheld (verified in the training logs), but none was ever scored. Loss curves only.
  • The data is single-arm in a bimanual container. Dims 7–13 are exactly zero, so the model learns 7 real dims out of 32. Any claim about "bimanual" capability is unsupported.
  • The ego gripper channel is close to noise, and ego is 50–62.5% of the gradient in A and C. Teleop shows clean pick semantics (open 0.745 β†’ closed 0.083); ego is flat at ~0.44 throughout with only a slight downward drift. Under Ο€β‚€.β‚…'s few-step flow matching, a near-random gripper target on half or more of the samples is precisely the setup that mode-averages into a half-open hand. This is the first thing to suspect if grasping underperforms, and the fix is upstream in the ego gripper labels, not in hyperparameters.
  • Most of the gradient in A and C comes from synthesized, retargeted data whose IK residual averages 4.770 cm. The sim-to-real gap of that share is unmeasured.
  • Language conditioning comes almost entirely from the ego half. Teleop has 1 task string (pick_and_lift_right); the 1 h ego half has 100 and the 20 min ego half has 79. prompt_from_task=True therefore conditions the ego half and does essentially nothing for the half an eval would score. Run B, being teleop-only, is effectively unconditioned.
  • B is not a clean ablation of A. It cuts teleop exposure (1.64 vs 2.47 epochs) and optimizer steps (600 vs 2,400) at the same time as removing ego.
  • C is not a scale-down of A. Different mixture ratio on disk (1:1 vs 1:2) and different draw (32/32 vs 24/40).
  • Both data fixes were applied to the local copy of the 1-hour dataset, not upstream. Pulling it fresh gives back the List metadata and the 0.05 s PTS offset.
  • Single seed per configuration, no repeats. Nothing here isolates seed variance, and with three runs differing in several variables at once, no single-variable conclusion is available.

Repo layout

# Run A β€” 1-hour mixture, step 2399, at the repo ROOT
_CHECKPOINT_METADATA
params/                              # Orbax OCDBT β€” inference weights (6.7 GB)
train_state/                         # optimizer state β€” resuming only (5.9 GB)
assets/piper1h_p50/norm_stats.json
train_metrics_full.log               # per-step metrics, steps 0–2399
train_console.log

# Run B β€” teleop only, step 599
piper1h_teleop/
  599/
    _CHECKPOINT_METADATA
    params/
    train_state/
    assets/piper1h_teleop_only/norm_stats.json
  train_metrics_full.log             # per-step metrics, steps 0–599
  train_console.log

# Run C β€” 20-minute mixture, step 799
piper20m_ea/
  799/
    _CHECKPOINT_METADATA
    params/
    train_state/
    assets/piper20m_p50/norm_stats.json
  train_metrics_full.log             # per-step metrics, steps 0–799
  train_console.log

Run A's checkpoint is at the repo root, not under a 2399/ directory. That is a historical artifact of how it was first uploaded (upload_large_folder takes no path_in_repo). It is left as is; B and C are namespaced. Adjust paths accordingly β€” this is the one asymmetry in the layout.

For inference you only need params/ and assets/ β€” train_state/ is ~46% of the bytes and is needed only to resume training.

Usage

# Run A (1-hour mixture) β€” files live at the repo root
hf download angkul07/pi05-piper1h-ea \
  --include "params/*" "assets/*" "_CHECKPOINT_METADATA" \
  --local-dir ./ckpt_A

# Run B (teleop only)
hf download angkul07/pi05-piper1h-ea \
  --include "piper1h_teleop/599/params/*" "piper1h_teleop/599/assets/*" \
            "piper1h_teleop/599/_CHECKPOINT_METADATA" \
  --local-dir ./ckpt_B

# Run C (20-minute mixture)
hf download angkul07/pi05-piper1h-ea \
  --include "piper20m_ea/799/params/*" "piper20m_ea/799/assets/*" \
            "piper20m_ea/799/_CHECKPOINT_METADATA" \
  --local-dir ./ckpt_C
from openpi.policies import policy_config
from openpi.training import config as _config

# pick the matching config name for the checkpoint you downloaded
cfg = _config.get_config("pi05_piper1h_ea")        # A
# cfg = _config.get_config("pi05_piper1h_teleop")  # B
# cfg = _config.get_config("pi05_piper20m_ea")     # C

policy = policy_config.create_trained_policy(cfg, "./ckpt_A")
#                                            B:  "./ckpt_B/piper1h_teleop/599"
#                                            C:  "./ckpt_C/piper20m_ea/799"

action_chunk = policy.infer({
    "observation/front_image": front_rgb,   # HxWx3 uint8 β€” top-down / egocentric view
    "observation/right_image": right_rgb,   # HxWx3 uint8 β€” wrist / grasp-point view
    "observation/state":       state_14d,
    "prompt":                  "pick_and_lift_right",
})["actions"]                               # (30, 14)

The config must match the checkpoint. Each run shipped its own norm stats under its own asset id, and loading a checkpoint under the wrong config will silently apply the wrong normalization β€” the failure mode is plausible-looking but mis-scaled actions, not an error.

Note the observation keys: two images, not three. observation/top_image is not consumed β€” PiperInputs reads only front and right, and fills right_wrist_0_rgb with a masked zero array. Expect dims 7–13 of the returned actions to be ~zero, since they were constant in training.

Requires branch piper1h-pi05 of angkul07/openpi (revision a079613 has all three configs), along with src/openpi/policies/piper_policy.py.

To resume training, point openpi at the parent directory:

uv run scripts/train.py pi05_piper20m_ea --exp-name piper20m_ea_v1 --fsdp-devices 1 --resume

License

Apache-2.0, inherited from openpi and pi05_base. The underlying datasets carry their own terms; the EgoDex-derived half inherits EgoDex's license.

Downloads last month

-

Downloads are not tracked for this model. How to track
Video Preview
loading