ms180/espnet3_falar_owsm_lora

ESPnet3 owsm_whisper_publication model

Packed model bundle generated from /work/hdd/bbjs/someki1/30_shared/owsm_whisper_publication.

Model

• Repository: ms180/espnet3_falar_whisper_lora
• Recipe: /work/hdd/bbjs/someki1/30_shared/owsm_whisper_publication
• Task: owsm_whisper_publication
• System: None
• Creator: someki1
• Created: 2026-05-20T15:14:47

Usage

from espnet3.publication import InferenceModel

model = InferenceModel.from_pretrained("ms180/espnet3_falar_whisper_lora", trust_user_code=True)
result = model(sample)

Packaging

• Bundle: model_pack
• Exp dir: ./exp/owsm_peft_finetune
• Strategy: copy experiment outputs; include extra recipe assets; apply exclude filters

Results

Metrics were not bundled. Run the measure stage before pack_model to include evaluation results.

Training config

expand

num_device: 1
num_nodes: 1
task: null
recipe_dir: .
data_dir: ./data
exp_tag: owsm_peft_finetune
exp_dir: ./exp/owsm_peft_finetune
stats_dir: ./exp/owsm_peft_finetune/stats
dataset_dir: ''
create_dataset:
  func: src.creating_dataset.create_dataset
  dataset_dir: ''
dataset:
  _target_: espnet3.components.data.data_organizer.DataOrganizer
  recipe_dir: .
  train:
  - name: train_falar
    dataset:
      _target_: src.data.dataset.FalarPortugalDataset
      split: train_0
      num_shards: 16
      world_shard_size: 4
      _convert_: all
  valid:
  - name: valid_falar
    dataset:
      _target_: src.data.dataset.FalarPortugalSingleDataset
      split: dev
      _convert_: all
  test:
  - name: test_falar
    dataset:
      _target_: src.data.dataset.FalarPortugalSingleDataset
      split: test
      _convert_: all
  preprocessor:
    _target_: src.data.dataset.WhisperTokenizeTransform
    model_tag: openai/whisper-large-v3
    _convert_: all
  _convert_: all
tokenizer:
  vocab_size: 50002
  character_coverage: 1.0
  model_type: bpe
  save_path: data/bpe_50000
model:
  _target_: src.peft_model.OWSMFinetune
  model_tag: espnet/owsm_v4_medium_1B
  peft:
    type: lora
    r: 32
    lora_alpha: 32
    lora_dropout: 0.05
    task_type: seq_2_seq_lm
    target_modules:
    - linear_q
    - linear_k
    - linear_v
    - linear_out
    - w_1
    - w_2
  _convert_: all
optimizer:
  _target_: torch.optim.AdamW
  lr: 5.0e-05
  weight_decay: 1.0e-06
  _convert_: all
scheduler:
  _target_: torch.optim.lr_scheduler.ConstantLR
  warmup_steps: 6000
  factor: 1.0
  total_iters: 1
  _convert_: all
scheduler_interval: step
scheduler_monitor: null
best_model_criterion:
- - valid/loss
  - 3
  - min
seed: 2024
init: null
parallel:
  env: local
  n_workers: 1
dataloader:
  collate_fn:
    _target_: espnet2.train.collate_fn.CommonCollateFn
    int_pad_value: -1
    _convert_: all
  train:
    total_shards: 1
    dist_world_size: 1
    iter_factory: null
    batch_size: 2
    num_workers: 2
    shuffle: true
    pin_memory: true
    prefetch_factor: 2
  valid:
    total_shards: 1
    dist_world_size: 1
    iter_factory: null
    batch_size: 2
    num_workers: 2
    shuffle: false
    pin_memory: true
    prefetch_factor: 2
trainer:
  accelerator: auto
  devices: 1
  num_nodes: 1
  accumulate_grad_batches: 1
  check_val_every_n_epoch: 1
  gradient_clip_val: 5
  log_every_n_steps: 100
  max_epochs: 1
  logger:
  - _target_: lightning.pytorch.loggers.WandbLogger
    project: Whisper_Finetuning_Portugal
    save_dir: ./exp/owsm_peft_finetune/wandb
    name: default_lr5e-05
    _convert_: all
  strategy: auto
  precision: bf16
fit: {}
recipedir: .
lr: 5.0e-05

Citing ESPnet

@inproceedings{watanabe2018espnet,
  author={Shinji Watanabe and Takaaki Hori and Shigeki Karita and Tomoki Hayashi and
    Jiro Nishitoba and Yuya Unno and Nelson Yalta and Jahn Heymann and Matthew Wiesner
    and Nanxin Chen and Adithya Renduchintala and Tsubasa Ochiai},
  title={{ESPnet}: End-to-End Speech Processing Toolkit},
  year={2018},
  booktitle={Proceedings of Interspeech},
  pages={2207--2211},
  doi={10.21437/Interspeech.2018-1456}
}