KarthionLupenix/sap-ticket-classifier-bert

SAP Support Ticket Classifier — Arinous-0.1

Fine-tuned bert-base-uncased that classifies SAP enterprise support tickets into the correct SAP functional module. Built by Arinous AI.

Live Demo → IntelliRoute on Hugging Face Spaces
Model ID → KarthionLupenix/sap-ticket-classifier-bert

---

Labels

ID	Label	SAP Module	Scope
0	FI	Finance	GL, AP, AR, asset accounting, bank reconciliation
1	MM	Materials Management	Purchasing, inventory, MRP, WM, batch management
2	SD	Sales & Distribution	Orders, delivery, billing, pricing, rebates
3	HR	Human Resources	Payroll, time management, org management, ESS/MSS
4	ABAP	ABAP / Custom Dev	Custom reports, BAPIs, user exits, SmartForms, BDC
5	BASIS	SAP Basis	System admin, transports, jobs, users, kernel, ICM
6	INTEGRATION	Integration / CPI	SAP CPI iFlows, OData, IDocs, RFC, SFTP adapters
7	OTHERS	Others / Unclassified	General SAP UI, authorisations, printing, misc
8	UNKNOWN	Unknown	Unrecognised or non-SAP tickets

---

Validation Metrics and Training, Valiation loss

Evaluated on a held-out 20% validation split (91 samples).
Source: train  ·  Trained: 2026-07-02 11:29 UTC

Metric	Score
Accuracy	82.4%
F1 (weighted)	82.6%
Precision (weighted)	84.0%
Recall (weighted)	82.4%

Per-class Scores

Module	Precision	Recall	F1
FI	75.0%	90.0%	81.8%
MM	60.0%	60.0%	60.0%
SD	70.0%	70.0%	70.0%
HR	100.0%	90.0%	94.7%
ABAP	100.0%	70.0%	82.3%
BASIS	69.2%	90.0%	78.3%
INTEGRATION	100.0%	90.0%	94.7%
OTHERS	81.8%	81.8%	81.8%
UNKNOWN	100.0%	100.0%	100.0%

Train samples: 369  ·  Validation samples: 91  ·  Trained: 2026-07-02 11:29 UTC

---

Quick Start

from transformers import pipeline

clf = pipeline(
    "text-classification",
    model="KarthionLupenix/sap-ticket-classifier-bert",
    top_k=None,
)

result = clf("Vendor invoice not posted due to GR/IR account mismatch")
# [{'label': 'FI', 'score': 0.93}, {'label': 'MM', 'score': 0.03}, ...]

top = max(result[0], key=lambda x: x['score'])
print(top["label"], top["score"])   # FI  0.93

---

Training Details

Setting	Value
Base model	bert-base-uncased
Architecture	BertForSequenceClassification — [CLS] pooled → Linear(768 → 9)
Max sequence length	128
Epochs	10 (early stopping, patience 3)
Batch size	8
Learning rate	3e-5 with linear warmup (10%)
Optimizer	AdamW, weight decay 0.01
Train / val split	80% / 20%
Selection metric	Weighted F1 on validation set
Dataset balancing	Random oversampling to equal class counts
Class weighting	Inverse-frequency weighted loss

Training data

Seed dataset of labelled SAP support ticket examples across 9 classes (data/sample_tickets.csv). Human corrections from the feedback UI are accumulated and used for warm-start retraining via scripts/retrain.py.

---

Intended Use

Route incoming SAP support tickets automatically to the correct functional team without manual triaging. Designed for enterprise SAP environments generating high volumes of support requests across Finance, Logistics, HR, Technical, and Integration workstreams.

---

Limitations

• Trained on a small seed dataset. Production accuracy improves significantly with more labelled examples per class (500+ per class recommended).
• English language tickets only.
• Short tickets (< 10 words) may have lower confidence scores.
• To add more SAP modules (PP, PM, PS, QM, CO): extend LABELS in src/config.py, add labelled examples, and retrain.

---

Human Feedback & Retraining

The live demo collects human corrections via the feedback panel. Corrections are stored in a private HF Dataset repo and merged with the original training set for warm-start retraining with double-weighted corrections.

# Retrain locally with accumulated feedback
python scripts/retrain.py --push

---

License

Apache 2.0

---

Built by Arinous AI · Intelligence. Innovation. Impact.