kv-rane/text2cypher-smollm2

Text2Cypher — SmolLM2-135M Fine-tuned with LoRA

A fine-tuned version of SmolLM2-135M-Instruct that generates Cypher queries from natural language questions and a graph schema.

Model Details

• Base model: HuggingFaceTB/SmolLM2-135M-Instruct
• Model type: Causal Language Model with LoRA adapter
• Language: English
• License: Apache 2.0
• Fine-tuned by: Kartik Rane

Training Data

• Dataset: RomanTeucher/text2cypher-curated
• 1000 training samples, 75 validation, 50 test
• Each sample contains a graph schema, a natural language question, and a target Cypher query

Training Details

• Fine-tuning method: LoRA (Low-Rank Adaptation)
• LoRA rank: 16, alpha: 32
• Target modules: q_proj, v_proj
• Trainable parameters: 921,600 (0.68% of total)
• Epochs: 3
• Learning rate: 2e-4
• Batch size: 4
• Max sequence length: 512
• Hardware: CPU (Windows)
• Final train loss: 0.76
• Final val loss: 0.87

Evaluation

Evaluated on 50 test examples:

• Exact Match: 0.00% (I expected that for 135M model)
• Average Token F1: 27.4%
• Structural Validity: 74.0%

Token F1 improved 64% when scaling from 200 to 1000 training examples (16.7% → 27.4%), confirming genuine learning.

Note - 200 sample training was done just to check cpu performance and loss behaviour

How to Use

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained(
    "HuggingFaceTB/SmolLM2-135M-Instruct"
)
model = PeftModel.from_pretrained(base_model, "kv-rane/text2cypher-smollm2")
tokenizer = AutoTokenizer.from_pretrained("kv-rane/text2cypher-smollm2")
tokenizer.pad_token = tokenizer.eos_token

schema = "Movie {title, year}, Person {name}, (Person)-[:DIRECTED]->(Movie)"
question = "Which movies did Christopher Nolan direct before 2010?"

prompt = f"""### Schema:
{schema}

### Question:
{question}

### Cypher:
"""

inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False)
generated = outputs[0][inputs["input_ids"].shape[1]:]
print(tokenizer.decode(generated, skip_special_tokens=True))

Key Differences from Full Fine-tuning

This model uses LoRA instead of full fine-tuning:

• Only 0.68% of parameters trained
• Faster training on CPU
• Smaller adapter artifact (few MB vs full model)

Limitations

• 135M parameter model generates Cypher that is not the best
• Exact match 0% due to sensitivity to exact syntax
• Repetition loops on some complex queries
• Property hallucination on unseen schemas
• No execution validation against real Neo4j database
• This model is not suitable for production use