PruhaNLP/needle-3-esp32-s3

Needle 3 on an ESP32-S3

A ~10 MB tool-calling model running fully offline on a microcontroller. You type English, the board emits a schema-valid JSON call. No network, no cloud. Official Cactus engines skip ESP32 — this is a separate C99 runtime.

Code, flash guide, TUI: github.com/PruhaNLP/needle-3-esp32

On an Artontop ESP32-S3 N16R8 (16 MB flash / 8 MB octal PSRAM), 6 layers, that request was ~1.5 tok/s and ~23 s wall. Not a chatbot. Not transformers. Not a finetune and not new weights — inference port. Weights stay on Cactus-Compute/needle3; the GitHub repo slices that archive to the depth that fits.

git clone https://github.com/PruhaNLP/needle-3-esp32
cd needle-3-esp32
# then the Setup section in that README