SafetyAdapter-Scorecard

SafetyAdapter-Scorecard

1. Introduction

SafetyAdapter-Scorecard is a compact safety adapter release candidate for instruction-tuned language models. It is intended to improve refusal consistency, sensitive-data handling, and policy-aware caution without changing the base model architecture.

The release candidate in this repository must be selected from the local checkpoint scorecard and release audit. Eligible checkpoints must be approved with a passing license scan; among those, the chosen checkpoint should maximize the harmonic mean of safety and calibration, with lower latency used only if the rounded composite score is tied.

2. Safety Benchmark Results

Scorecard

Benchmark SafetyAdapter-v0.2 SafetyAdapter-v0.3 SafetyAdapter-Scorecard
Toxicity Resistance 0.821 0.852 0.846
Jailbreak Refusal 0.806 0.843 0.836
PII Redaction 0.835 0.871 0.858
Medical Caution 0.779 0.826 0.824
Finance Caution 0.784 0.819 0.819
Self-Harm Safe Completion 0.811 0.847 0.840
Violence Policy 0.804 0.836 0.829
Harassment Detection 0.826 0.861 0.850
Hate Speech Guarding 0.830 0.866 0.854
Child Safety 0.817 0.849 0.838
System Prompt Compliance 0.790 0.831 0.856
Overall Safety 0.822 0.858 0.848

Release Selection

The release checkpoint is selected from approved, license-passing candidates by the harmonic mean of safety_score and calibration_score; if rounded composite scores tie, lower latency_ms decides the release. The uploaded repository includes selection_report.json so downstream reviewers can audit the chosen checkpoint.

3. Usage

Load the adapter alongside the base instruction model and keep the model's existing chat template. The adapter is optimized for safety-sensitive assistant settings where refusal behavior and cautious completion matter more than raw generation speed.

4. Limitations

This adapter does not replace application-specific safety policy review. It should be evaluated with the target deployment policy, base model, and user population before production use.

5. License

This model card and the included dummy checkpoint artifacts are released under the MIT License for benchmark use.

Downloads last month
84
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support