tt-openvla

A TT-Metal / TTNN bring-up of OpenVLA-7B -- a vision-language-action model for robot manipulation, combining a fused DINOv2 + SigLIP vision backbone with a LLaMA-2-7B language model -- for Tenstorrent Blackhole hardware. Code: tsingletaryTT/tt-openvla.

No separate checkpoint is hosted here. The port uses the real, original openvla/openvla-7b weights directly -- no stripping, conversion, or fine-tuning was done, so there's no new derived artifact that isn't already correctly hosted and licensed at the source. This repo is a model card pointing at the implementation and the real checkpoint it runs, not a weights mirror.

What's validated

Every stage checked against a real reference implementation, each at PCC >= 0.995 except where noted:

• DINOv2 ViT-L/14, register-token variant (the exact tower OpenVLA actually uses, not the plain checkpoint) -- PCC 0.9992
• SigLIP ViT-So400M/14 -- PCC 0.9967
• Fused DINOv2+SigLIP vision backbone (matching OpenVLA's real fusion mechanism) -- PCC 0.9986
• Projector MLP, validated against the real fine-tuned weights -- PCC 0.99999
• Full 32-layer LLaMA-2-7B backbone, real fine-tuned weights, reusing tt-metal's own tt_transformers attention/RoPE/KV-cache kernels -- PCC 0.9966
• End-to-end "Grounded Check": a real image + OpenVLA's own documented prompt format, through the whole pipeline -- vision backbone, fused embeddings, one PREFILL pass plus 6 real autoregressive DECODE steps across all 32 real LLaMA layers, on a real 2-device Blackhole mesh -- producing an actual, deterministic decoded 7-DoF action from real weights.

See the GitHub repo's tt/ directory for the individual correctness tests and tt/demo_grounded_check.py for the end-to-end demo, including notes on real bugs found and fixed along the way (a tile-alignment issue in SigLIP's attention on this hardware, and a non-determinism bug in the decode loop's output-tile handling).

Try it

Packaged with tt-model-manager (v6 thin -- a pip/venv install, not a container)

tt-model pull episod/tt-openvla --with-weights
tt-model serve episod/tt-openvla

serve opens the URL for a Gradio UI (not a REST API) -- upload an image, write an instruction, click "Predict action". Every number comes from a real forward pass through all 32 real LLaMA-2-7B layers with the real fine-tuned openvla-7b weights, not a scripted or precomputed response. Confirmed live: a real 7-DoF action (dx/dy/dz/droll/dpitch/dyaw/gripper) decoded in ~45s on a cold cache, 2-chip P300 mesh.

REST API: POST /act

Mirrors openvla/openvla's own reference server (vla-scripts/deploy.py) exactly, so existing OpenVLA client code and robot-control loops written against that server work against this bundle unmodified -- no chat/ completions shape, since a continuous action vector isn't natural-language output:

import json_numpy; json_numpy.patch()  # pip install json-numpy
import requests, numpy as np
from PIL import Image

image = np.array(Image.open("my_photo.jpg").convert("RGB"))
resp = requests.post("http://localhost:8000/act", json={
    "image": image, "instruction": "pick up the remote control",
})
action = resp.json()["action"]  # 7-DoF: dx, dy, dz, droll, dpitch, dyaw, gripper

Confirmed live: two identical requests returned bit-for-bit identical actions (deterministic greedy decode), ~0.93s wall clock on a warm backend.

Locally, from the GitHub repo

gradio_app/app.py is the same interactive demo, runnable directly -- upload an image, write an instruction, get back a real decoded 7-DoF action from an actual forward pass (not a scripted response). Runs against real Blackhole hardware by default, or --backend reference for CPU-only. Also registered with tt-discolike (.disco/app.yaml) for one-click start/stop.

Benchmarks

Real end-to-end latency for the full pipeline, measured on this port's own hardware (tt/benchmark.py / tt/cpu_benchmark.py):

	latency/call	relative
Blackhole (TTNN, 2-chip mesh, kernel cache warm)	~320 ms	1x
same, first-ever call (one-time kernel compilation, cached to disk after)	~9-11 s	~30x slower, once
CPU reference (composed real PyTorch, same host machine)	~10.0 s	~31x slower

License

This port's own code is MIT, matching openvla/openvla's license. The openvla/openvla-7b checkpoint itself is a fine-tune of Meta's Llama-2-7B, so using those weights is subject to the Llama Community License separately from this repo's own MIT terms.