Instructions to use FINAL-Bench/Aether-7B-5Attn-it with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FINAL-Bench/Aether-7B-5Attn-it with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FINAL-Bench/Aether-7B-5Attn-it", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("FINAL-Bench/Aether-7B-5Attn-it", dtype="auto", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use FINAL-Bench/Aether-7B-5Attn-it with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FINAL-Bench/Aether-7B-5Attn-it"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FINAL-Bench/Aether-7B-5Attn-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/FINAL-Bench/Aether-7B-5Attn-it

SGLang

How to use FINAL-Bench/Aether-7B-5Attn-it with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FINAL-Bench/Aether-7B-5Attn-it" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FINAL-Bench/Aether-7B-5Attn-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FINAL-Bench/Aether-7B-5Attn-it" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FINAL-Bench/Aether-7B-5Attn-it",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use FINAL-Bench/Aether-7B-5Attn-it with Docker Model Runner:
```
docker model run hf.co/FINAL-Bench/Aether-7B-5Attn-it
```

Aether-7B-5Attn-it

Update — 2026-07-20 (Korean instruction tuning v3)

This checkpoint has been refreshed with an expanded Korean instruction-tuning run.


Training data	14,865 samples — Korean general instructions (12,000) + Korean exam-style reasoning (2,400) + model identity (465)
Method	LoRA (r16, alpha32) on q/k/v/o/gate/up/down projections, completion-only loss, entropy-gated weighting
Schedule	1 epoch, LR 2e-4, merged back into the base weights
Language policy	Korean prompts are answered in Korean, English prompts in English (language-matched training data)

What changed

Korean responsiveness. The previous checkpoint frequently returned empty completions for Korean prompts; this revision answers them.
Model identity. The model now identifies itself as an AETHER model developed by VIDRAFT.
Language matching. Korean in, Korean out; English in, English out.

Known limitations

Factual accuracy in Korean history/knowledge remains weak. The instruction mix was not large enough to reliably ground factual questions; verify factual claims before relying on them.
Repetition under greedy decoding. Use sampling (e.g. temperature=0.7, top_p=0.9) rather than pure argmax decoding.
This is a 7B-class MoE base with light instruction tuning, not a fully aligned chat model.

Aether family —

📚 Part of the Aether Foundation Model collection — base, instruction-tuned, and checkpoints in one place.

🧩 Intermediate checkpoints (110k · 115k · 162k) are released as a dataset: Aether-7B-5Attn-checkpoints. Instruction-tuned (SFT) version of the fully-open Aether-7B-5Attn base model. Post-trained for multiple-choice / benchmark-style answering. 6.59B MoE (~2.98B active), 49 layers on a 7×7 Latin square. Apache-2.0.

Learning rate chosen by held-out accuracy, not loss

Full-parameter SFT was run at three learning rates (2e-6 / 6e-6 / 2e-5). The final checkpoint was selected by held-out benchmark accuracy, not training loss — for small models a high LR lowers training loss while degrading real capability, so loss is the wrong selector.

LR	K-AI 4 avg	Note
2e-6	29.7%	Under-fit (weak on some subjects)
6e-6 (selected)	34.9%	Best & balanced, no degradation
2e-5	32.3%	One subject collapsed (format-degradation sign)

Held-out results (selected checkpoint, 6e-6)

GPQA-Diamond (198): 25.3%
K-AI 4 average (195): 34.9%
- musr_ko 26.5% · com2_main_ko 50.0% · click 32.0% · kommlu_pro 30.4%
vs base (before SFT): base 26.7% → 34.9% (+8.2pp) — same held-out set, same harness.

All evaluations use held-out sets not seen in training. SFT was performed on MMLU-auxiliary (multiple-choice format), which does not overlap with the evaluation subjects (GPQA / K-AI).

Pretraining data mix (inherited from base)

Domain	Share
Math (finemath + open-web-math)	37.8%
Korean (webtext + synth)	21.6%
English web & synthetic (fineweb-edu + cosmopedia)	21.6%
Code (opc)	13.5%
phase15 pre-blend	5.4%

This is the base pretraining mix. This model's post-training (SFT) data is MMLU-auxiliary.

Architecture (inherited from base)

Identical to Aether-7B-5Attn base: 49 layers placed on a 7×7 Latin square with heterogeneous attention (7 labels / 5 distinct mechanisms) and a 25-expert MoE (top-7 + 1 shared). Full structural detail and the diagram are in the base card, §3.2: FINAL-Bench/Aether-7B-5Attn.

Open-source fully-open LLMs — 6-country comparison

Relative to the base Aether. Among six sovereign fully-open models, VIDRAFT is the only single AI startup, and Aether has the most attention types (5) in a Latin-square layout.

Running the model

1. Loading — the standard loader will not work

This is a custom architecture (aether_v2_7way). AutoModelForCausalLM fails with:

ValueError: The checkpoint you are trying to load has model type `aether_v2_7way`
but Transformers does not recognize this architecture.

Load the bundled aether_pkg directly:

import sys, torch, torch.nn as nn
from huggingface_hub import snapshot_download
from safetensors.torch import load_file
from transformers import AutoTokenizer, GenerationMixin

local = snapshot_download("FINAL-Bench/Aether-7B-5Attn-it")
sys.path.insert(0, local)
from aether_pkg.configuration_aether_v2_7way import AETHERV27wayConfig
from aether_pkg.modeling_aether_v2_7way import AETHERV27wayForCausalLM

class AetherForGeneration(AETHERV27wayForCausalLM, GenerationMixin):
    pass  # transformers >= 4.50 no longer mixes GenerationMixin into PreTrainedModel

cfg = AETHERV27wayConfig.from_pretrained(local)
cfg.use_cache = False          # REQUIRED — see 2.

# the checkpoint overwrites every weight, so skip random init (much faster cold start)
_saved = (nn.Linear.reset_parameters, nn.Embedding.reset_parameters)
nn.Linear.reset_parameters = lambda self: None
nn.Embedding.reset_parameters = lambda self: None
try:
    torch.set_default_dtype(torch.bfloat16)
    model = AetherForGeneration(cfg)
finally:
    torch.set_default_dtype(torch.float32)
    nn.Linear.reset_parameters, nn.Embedding.reset_parameters = _saved

model.load_state_dict(load_file(f"{local}/model.safetensors", device="cuda"), strict=False)
model = model.to("cuda").eval()
tok = AutoTokenizer.from_pretrained(local)

Loading needs roughly 14 GB of VRAM in bfloat16.

2. `use_cache=False` is mandatory

This architecture ships no KV cache. Leaving use_cache enabled raises an IndexError from the standard cache path. Set it on the config and keep it off in generate().

3. Generation is slow — plan for it

With no KV cache, every new token re-runs the full forward pass, so decoding is O(n²) in sequence length. Measured throughput:

Hardware	Throughput
NVIDIA T4 (16 GB)	~1.5 tokens/s — 64 tokens takes about 40 s
NVIDIA B200	~7 tokens/s

Keep max_new_tokens small. This is a property of the released architecture, not a configuration problem.

# REQUIRED: this checkpoint was trained with attention_mask=None. generate() builds a
# mask automatically, which puts the model off-distribution and degenerates the output
# (you get things like "국가의 수도는 국가의 수도입니다..." instead of an answer).
# Drop the mask before it reaches the model:
_forward = model.forward
def _forward_without_mask(*a, **kw):
    kw.pop("attention_mask", None)
    return _forward(*a, **kw)
model.forward = _forward_without_mask

out = model.generate(
    tok(prompt, return_tensors="pt").input_ids.to("cuda"),
    max_new_tokens=64, do_sample=False, use_cache=False,
    pad_token_id=tok.eos_token_id,
)
print(tok.decode(out[0], skip_special_tokens=True))

Why the mask has to go

Measured on 2026-07-20, same prompt, greedy decoding, only the mask varied:

attention_mask	Output for `너는 누구야?`
absent	저는 비드래프트(VIDRAFT)의 AETHER 모델입니다. 7종의 서로 다른 어텐션 메커니즘을…
present (generate default)	비드래프트(VIDRAFT)의 한국어입니다. 비드래프트는 비드래프트의… (degenerate)

This is a property of how the checkpoint was tuned, not a bug in generate(). Greedy decoding is also recommended — sampling drifts off-distribution on this checkpoint.

4. Prompt format

Instruction tuning used a plain format — the instruction, a blank line, then the response. The bundled chat_template.jinja reproduces exactly this, so apply_chat_template and the raw form below are equivalent:

prompt = "대한민국의 수도는 어디인가요?" + "\n\n"

5. Quality caveats — please read before use

Instruction tuning here is light: 14,865 samples for a single epoch. Measured consequences, stated plainly:

Sensitive to prompt format and decoding settings. Outputs degrade sharply once you move away from the trained format. Greedy decoding on the format above is the only configuration we validated.
Factual accuracy in Korean history and general knowledge is weak. The model produces fluent but confidently wrong statements. Verify anything factual.
Benchmark: on a held-out 4-subject Korean set (CLIcK / KMMLU / MuSR / Com2, 80 items, no train overlap) this revision scores 31.2% against 33.8% for the previous one — within noise of each other and near the 25% random baseline.
Treat this as a lightly instruction-tuned research checkpoint, not a production assistant.