Ephraimmm/pidgn_llama_modelv1

Pidgin Llama Model v1

Overview

pidgn_llama_modelv1 is a LoRA (Low-Rank Adaptation) adapter for Meta's Llama 3.2 3B Instruct model, fine-tuned to understand and generate Nigerian Pidgin English (Naija, ISO 639-3: pcm).

Nigerian Pidgin is spoken by tens of millions of people across Nigeria and the wider West African diaspora, and functions as a lingua franca in everyday conversation, media, and online communication. Despite this reach, it remains a low-resource, underrepresented language in NLP — most large language models are trained almost exclusively on standard English and struggle to produce fluent, natural Pidgin. This project adapts a compact, efficient open-weight Llama model so it can better follow and respond in conversational Nigerian Pidgin, contributing a small step toward more linguistically inclusive language technology.

Training Details

Based on the metadata shipped in this repository:

• Base model: unsloth/llama-3.2-3b-instruct-unsloth-bnb-4bit (Meta Llama 3.2, 3B parameters, Instruct-tuned, 4-bit quantized)
• Fine-tuning method: Parameter-efficient fine-tuning with LoRA via PEFT (v0.18.1)
  • Rank (r): 16
  • LoRA alpha: 16
  • LoRA dropout: 0
  • Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
  • Task type: CAUSAL_LM
  • Bias: none
• Frameworks/libraries: Unsloth (for ~2x faster, memory-efficient training), Hugging Face TRL, PEFT, and Transformers
• Reported training loss: ~0.7 (as noted by the author in the original release)
• Adapter only: this repository contains the LoRA adapter weights (adapter_model.safetensors), not a merged full checkpoint. The base model is loaded separately at inference time.

Detailed dataset composition, number of training steps/epochs, and evaluation metrics are not recorded in the repository metadata and are therefore not reproduced here to avoid overstating what is documented.

Intended Use

This adapter is intended for:

• Generating conversational responses in Nigerian Pidgin English
• Prototyping chatbots, assistants, or dialogue agents that need to communicate in Pidgin
• Research and experimentation on low-resource / underrepresented African languages in NLP
• Educational or cultural applications that benefit from Pidgin-fluent text generation

It is not intended for high-stakes decision-making, medical/legal/financial advice, or use cases requiring guaranteed factual accuracy.

How to Use

Since this repo contains a LoRA adapter, load the base model first and then apply the adapter with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

base_model_id = "unsloth/llama-3.2-3b-instruct-unsloth-bnb-4bit"
adapter_id = "Ephraimmm/pidgn_llama_modelv1"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    device_map="auto",
    torch_dtype=torch.float16,
)
model = PeftModel.from_pretrained(base_model, adapter_id)

messages = [
    {"role": "user", "content": "Abeg, tell me wetin dey happen for Naija today."}
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

outputs = model.generate(inputs, max_new_tokens=200, do_sample=True, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Alternatively, using Unsloth (matches the framework used to train the model, and is faster/lighter):

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Ephraimmm/pidgn_llama_modelv1",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

messages = [{"role": "user", "content": "How you dey? Wetin be your name?"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to("cuda")

outputs = model.generate(inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

• No formal evaluation metrics (accuracy, BLEU, human eval, etc.) are published for this model — quality claims beyond the reported training loss should not be assumed.
• The training dataset and its size/coverage are not documented in this repository, so the breadth of Pidgin dialects, topics, and registers actually learned is unclear.
• As a 3B-parameter base model, it has more limited reasoning and world knowledge than larger LLMs, and may mix in Standard English or produce inconsistent Pidgin.
• Like any fine-tuned LLM, it can hallucinate facts and should not be relied on for accuracy-critical use.
• No dedicated safety/toxicity filtering specific to Pidgin content has been documented; standard LLM safety caveats apply.

Author

Developed by Ephraimmm