docs/theme_lab_app.md

# Theme Lab App

Theme Lab is a FastAPI web app for browsing the theme dictionary, playing theme
note events in the browser, and generating new samples from the available theme
generation engines.

## Local Run

Install runtime dependencies:

```bash
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt
```

Start the app:

```bash
.venv/bin/python -m apps.theme_lab.app
```

Open:

```text
http://127.0.0.1:7860
```

The app serves corpus MIDI and ABC files directly from this repository. Catalog
score previews are rendered on demand from SQLite note events through
MusicXML/Verovio and cached under `outputs/web_app_runs/catalog_scores/`;
existing `svgs/*.svg` files remain the fallback when dynamic rendering is not
available. Generated files are written under `outputs/web_app_runs/`.

## Current Features

- Search and filter the theme dictionary by free text, composer, and key.
- View catalog metadata and dynamically rendered SVG score previews.
- Play catalog themes through browser Web Audio using note events from SQLite.
- Generate new samples with the variable-order Markov engine.
  The web UI defaults to max order 3 and caps Markov max order at 3; order 4+
  can take tens of seconds on CPU and is better run from the offline scripts.
  Docker builds precompute the default order-3 backend under
  `models/theme_lab_markov_cache/` so the deployed app can load and warm it at
  startup.
- Run the transformer engine from the same UI. The committed
  `models/theme_transformer_default.pt` checkpoint is loaded at startup so
  normal generation samples from a pretrained model instead of retraining.
- Export generated samples as MIDI, ABC, and MusicXML.
- MusicXML score exports are rendered as complete excerpts: final measures are
  padded when needed and end with a final barline.
- Render generated MusicXML to SVG when the `verovio` command-line tool is
  available on the host.

## Generated Register

Both generation engines emit key-relative pitch classes and duration labels,
not fully specified MIDI pitches:

```text
symbol = (relative_pitch_class, duration_value)
```

The shared export/playback layer realizes those pitch classes into concrete
octaves. Generated Markov and transformer samples are currently constrained to
the readable treble range C4-C6 during this realization step, so MIDI playback,
ABC, MusicXML, and Verovio score previews all use the same register policy.
This is an output-realization choice, not a learned register model.

## Docker / Hugging Face Spaces

Build locally:

```bash
docker build -t theme-lab .
docker run --rm -p 7860:7860 theme-lab
```

For a Hugging Face Docker Space, push this repository with the `Dockerfile` at
the root. The container listens on `PORT`, defaulting to `7860`.

## Deployment Notes

The Docker build installs:

- `muses` from `https://github.com/fpachet/muses.git`
- `vo-regular-bp` from the public optimized branch
  `https://github.com/fpachet/vo-regular-bp.git@codex/lsdb-virtual-bp-optimization`
- `torch` for the transformer engine

Verovio SVG rendering is optional. If `verovio` is missing, generated MusicXML
downloads still work and the UI shows the export links without an SVG preview.

## Transformer Checkpoint

The repository includes a compact default checkpoint trained with 200 CPU steps.
Train and save a replacement checkpoint with:

```bash
.venv/bin/python scripts/run_theme_generation.py transformer \
  --samples 2 \
  --length 24 \
  --steps 300 \
  --output-dir outputs/transformer_checkpoint_smoke \
  --save-checkpoint models/theme_transformer_default.pt \
  --write-musicxml
```

Generate from the checkpoint without retraining:

```bash
.venv/bin/python scripts/run_theme_generation.py transformer \
  --samples 4 \
  --length 24 \
  --load-checkpoint models/theme_transformer_default.pt \
  --output-dir outputs/transformer_checkpoint_generation \
  --write-musicxml
```

The app falls back to on-demand training only when this checkpoint is absent or
cannot be loaded.