PolyKode Community
PolyKode Community is ProCogia's community release channel for the PolyKode code-translation product family. This 35B mixture-of-experts checkpoint is the open foundation that ProCogia builds PolyKode's SAS / R / Python translation and execution-parity capabilities on top of.
This release is the community checkpoint. It is a supervised fine-tune for SAS / R / Python translation, released under Apache 2.0. It does not include the execution-parity training, harness, or delivery commitments of the Enterprise edition. See License and attribution below.
For the fully-adapted, execution-parity-trained models with the PolyKode translation stack, evaluation harness, audit trails, and regulated-delivery commitments, see PolyKode Enterprise (contact ProCogia).
About PolyKode
PolyKode is ProCogia's LLM product for translating code across SAS, R, and Python in regulated industries. It emphasizes:
- Bidirectional translation across SAS ↔ R ↔ Python
- Execution-parity validation — row/column checks, exact-value parity, formatting preservation, and SAS/R behavioral-difference handling
- Compliance-oriented workflows for clinical pharma, health sciences, insurance/BFSI, and manufacturing
- License-aware training corpora with permissive/copyleft/CC-BY-SA segregation
- SAS-exact numerical contracts, including MT19937 stream reproduction with original
sgenrandseeding
Community vs Enterprise
| PolyKode Community | PolyKode Enterprise | |
|---|---|---|
| Weights | ✅ Open (this repo) | ✅ Open + proprietary adapters |
| Translation SFT | ✅ Included | ✅ Included |
| RLVR / execution-parity training | ❌ | ✅ |
| Execution-parity harness | ❌ | ✅ Full harness |
| SAS 9.4 / Viya certification runs | ❌ | ✅ |
| Audit trails and compliance artifacts | ❌ | ✅ |
| SLA and regulated-delivery support | ❌ | ✅ |
| Deployment | Self-serve | ZeroBoxx on-prem / cloud |
The Community edition is the adoption funnel: use it to try the model, evaluate memory/latency on your hardware, and prototype pipelines. Migrate to Enterprise when you need the parity harness, certification, and delivery commitments.
Base model flexibility
The PolyKode adaptation is not tied to one set of base weights, and the base can be changed on demand. ProCogia has built the translation stack on several different open bases and can port it onto a base of your choosing — to fit your hardware envelope, your licensing posture, or a model you already run in production. Contact ProCogia to discuss a build on a specific base.
A port is a retraining exercise rather than a swap of the weights in this repository, and results are base-dependent: the choice of base materially changes translation performance.
Model card
PolyKode Community is a 35B-parameter mixture-of-experts model for bidirectional program translation between SAS, Python, and R, covering all six directions.
It is a supervised fine-tune on paired statistical and data-processing programs, trained with LoRA and merged into full weights. It is not a general-purpose assistant and has not been trained or evaluated as one.
| Parameters | ~35B total, ~3B active per token (256 experts, 8 active) |
| Method | LoRA (attention, GatedDeltaNet, shared expert, routed experts), merged |
| Precision | bfloat16 |
| Architecture | Qwen3_5MoeForConditionalGeneration / qwen3_5_moe |
| Files | ~72 GB, 26 safetensors shards |
| Trained sequence length | 8,192 tokens |
| Recommended serving context | 32,768 tokens |
| Modality | Text only |
Internally tracked as run s6 checkpoint-600. That identifier is what the training and
checkpoint-selection sections below refer to.
Intended use
In scope. Translating SAS, Python, and R programs for statistical and data-processing work into one of the other two languages, one program per request, as a first-draft aid for a human reviewer.
Out of scope.
- General chat, reasoning, agentic use, or tool calling. None of these appear in the training data.
- Languages other than SAS, Python, and R.
- Any multimodal use. See "Vision tower" below.
Architecture
Unchanged by fine-tuning:
| Property | Value |
|---|---|
| Hidden layers | 40 (30 linear-attention / GatedDeltaNet, 10 full attention at every 4th layer) |
| Hidden size | 2,048 |
| Full attention | 16 heads, 2 KV heads, head_dim 256, output gating on |
| Linear attention | 16 key heads × 128, 32 value heads × 128, conv kernel 4 |
| MoE | 256 experts, 8 active per token, expert intermediate size 512, one shared expert |
| RoPE | theta 1e7, partial rotary factor 0.25, mrope interleaved |
| Vocab | 248,320 (tokenizer defines 248,077 ids; 243 embedding rows unused) |
max_position_embeddings |
262,144 |
Roughly 32B of the ~35B parameters are routed experts (256 experts × 40 layers × 3.15M each), which is why the fine-tune targets them explicitly rather than attention alone.
Loader note. The configuration class is qwen3_5_moe; loaders and tooling should target
that, and follow Qwen3.5-MoE documentation for this family. transformers>=5.2.0 is required.
Vision tower
A vision_config is present in config.json because this is a VLM repository shape. The
vision tower was never loaded, never trained, and is not supported. Serve this model text-only.
No MTP / speculative decoding head
The text-only load drops the MTP head, so the merged model has no NEXTN speculative decoding.
mtp_num_hidden_layers still reads 1 in the config; that is inherited metadata, not a usable
head. This is fine for greedy batch translation and is the intended serving mode.
Prompt format
The model was trained on a single fixed shape and is sensitive to it. Deviating from it will degrade output.
Chat template. Use the template shipped in tokenizer_config.json, with
enable_thinking=False. Reasoning is off; there are no reasoning traces in the training data.
The rendered assistant header is:
<|im_start|>assistant\n<think>\n\n</think>\n\n
The training-time header and the serving-time header are byte-identical.
System prompt. A fixed 1,394-character system prompt was present in every training row and should be sent verbatim at inference.
You are a code translation engine for statistical and data-processing programs written in SAS, Python, and R.
You are given one complete program in a source language and produce the equivalent program in the target language. Behavioural equivalence is the only criterion: given the same inputs, your program must compute the same values and place them in results carrying the same names.
- Preserve the source program's structure, step order, and intent. Carry its comments across as comments in the target language, and add a brief comment where the target expresses a source construct non-obviously.
- Name every result exactly as the source names it, so results can be compared name for name.
- SAS semantics decide the answer even when SAS is not the target language. A SAS date is whole days since 1960-01-01 and a datetime is seconds since 1960-01-01, both stored as plain numbers. A SAS FORMAT changes only how a value is displayed, never what is stored. Missing (.) sorts below every number, so `x < 5` is true when x is missing. Character values are blank-padded to a declared length and compare ignoring trailing blanks.
- A step that only prints or plots produces no data and has no translation outside SAS; leave it out rather than inventing an equivalent.
- Respond with the translated program and nothing else: no prose, no explanation, no markdown code fences, no placeholders.
User turn. Header line, then the target-language constraint paragraph, then the source program in a fenced block:
Translate the following {SOURCE} program to {TARGET}.
{constraint paragraph for TARGET}
```{python|r|sas}
{source program}
Constraint paragraphs. One of the three below goes into every user turn, chosen by the target language. These were present in every training row and are part of the trained prompt, not optional guidance. Send the one matching your target verbatim.
Target: SAS
Use DATA steps for row-wise logic, and PROC steps for sorting, summarising, joining and modelling. Every input table is already present in the WORK library under its one-level name, so refer to tables by one-level names only, write no LIBNAME, FILENAME or PROC IMPORT statements, and read no external files. Create each result as a SAS dataset named after the result it corresponds to, and express date, format and missing-value handling with SAS's own facilities rather than transliterating the source's workarounds for them.
Target: Python
Write a single Python program using pandas and numpy; the Python standard library, scipy and statsmodels are also available. Any input table is already loaded as a DataFrame bound to a variable of that name, and a two-level SAS name such as LIB.TABLE is bound as TABLE - do not read or write files, do not print, and do not use randomness, the current date, or plotting. Assign each result to a module-level DataFrame named after the result it corresponds to.
Target: R
Write a single R program, using dplyr where it is natural and base R otherwise; tidyr, tidyverse, stringr, purrr, lubridate, data.table and ggplot2 are not available. Any input table is already loaded as a data.frame bound to a variable of that name, and a two-level SAS name such as LIB.TABLE is bound as TABLE - do not read or write files, do not print or cat, and do not use randomness, the current date, or plotting. Assign each result to a data.frame named after the result it corresponds to.
Training
Data
| Split | Rows |
|---|---|
| Train (as staged) | 11,365 |
| Train (after over-length drop) | 11,329 |
| Held-out eval | 544 |
| Watched validation subset | 120 (20 per direction) |
All six directions are present in both splits. Rows longer than the 8,192-token limit were dropped, never truncated: 36 train rows (0.32%, from 13 programs), 0 eval rows.
Loss was computed on assistant tokens only (response-only masking). The supervised share of a rendered row averages 0.293 on train and 0.250 on eval.
Direction rebalancing and →SAS upsampling were both off; natural direction counts are within 1.44×. No general-coding replay data was mixed in.
Corpus availability
The training corpus is internal to ProCogia and is not being released. It is the product of the curation, cleaning and pairing work that PolyKode is built on, and it stays with the product rather than the weights.
What is published instead: the weights, the full training configuration above, the prompt format, and the measured results. Everything needed to serve, reproduce the prompt contract, and check the claims is in this card. What is not published is the corpus itself and the harness fixtures.
LoRA configuration
| Group | Modules | Rank / alpha |
|---|---|---|
| Attention | q_proj, k_proj, v_proj, o_proj |
32 / 64 |
| GatedDeltaNet | in_proj_qkv, in_proj_z, out_proj (adapted together) |
32 / 64 |
| Shared expert | shared_expert.{gate,up,down}_proj |
32 / 64 |
| Routed experts | fused gate_up_proj, down_proj |
8 / 16 |
Both groups have effective scaling 2.0. Routed experts get a smaller rank deliberately: rank is multiplied by 10,240 expert MLPs, and r=32 there would mean ~1.8B trainable parameters.
Frozen: the MoE router (mlp.gate), the shared-expert scalar gate, and the vision tower.
Freezing the router is intentional — training it over-specializes routing on a narrow task.
Hyperparameters
| Optimizer | adamw_8bit, weight decay 0.01 |
| Learning rate | 1e-4, cosine, 31 warmup steps |
| Max grad norm | 0.5 |
| Batch | 1 × grad accumulation 32 (effective 32) |
| Epochs scheduled | 3 (1,065 optimizer steps) |
| Sequence length | 8,192, packing disabled (GatedDeltaNet is recurrent — never pack) |
| Precision | bf16 |
| Seed | 3407 |
| Early stopping | patience 4, threshold 0.005, on validation loss |
Run
| Hardware | 1 × NVIDIA H200 SXM 141 GB |
| Steps completed | 600 of 1,065 (56.3%), stopped early at epoch 1.69 |
| Training time | 3.05 h (10,969 s) |
| Peak reserved memory | 89.7 GiB (63.6% of card) |
| Throughput | ~17.5 s/step, ~3,080 tok/s |
Final train_loss |
0.2531 |
| Total FLOPs | 6.83e18 |
| Gradient norm | median 0.304; 7.2% of steps clipped at 0.5; no NaN |
Training loss by 100-step window: 0.370 → 0.296 → 0.265 → 0.227 → 0.186 → 0.175.
Validation loss (120 watched rows)
| Step | 100 | 200 | 300 | 400 | 500 | 600 |
|---|---|---|---|---|---|---|
| eval_loss | 0.3335 | 0.3058 | 0.3190 | 0.3244 | 0.3466 | 0.3178 |
Per direction:
| Direction | 100 | 200 | 300 | 400 | 500 | 600 |
|---|---|---|---|---|---|---|
| Python→R | 0.1893 | 0.1700 | 0.1671 | 0.1680 | 0.1692 | 0.1652 |
| Python→SAS | 0.5364 | 0.4944 | 0.5427 | 0.5521 | 0.6020 | 0.5317 |
| R→Python | 0.1689 | 0.1608 | 0.1555 | 0.1576 | 0.1562 | 0.1531 |
| R→SAS | 0.5582 | 0.5035 | 0.5500 | 0.5582 | 0.6224 | 0.5426 |
| SAS→Python | 0.2915 | 0.2634 | 0.2629 | 0.2706 | 0.2874 | 0.2807 |
| SAS→R | 0.2622 | 0.2412 | 0.2368 | 0.2431 | 0.2411 | 0.2362 |
Repeat measurements on identical rows and weights differ by 0.0013–0.0032 per direction; that is the noise floor, and smaller differences above should not be read as real.
Loss levels are not comparable across target languages of different entropy. Read trends within a direction, not levels between them. The overall curve is driven by the two →SAS directions, which are the only ones that degrade after step 200.
Checkpoint selection
This model is checkpoint-600, which is not the lowest-validation-loss checkpoint.
The trainer's load_best_model_at_end selected step 200 (0.3058 vs 0.3178). Step 600 was shipped
instead because selection was made on execution parity — whether a translated program runs
and reproduces the original's output tables cell by cell — not on token loss. Validation loss is
a proxy for the objective and was never the ship gate.
On the execution benchmark the two candidates are 4 pairs apart on the 396-pair subset (186 vs 182) and 3 apart on the full 424 (194 vs 191). Step 600 gains in SAS→R (+5), Python→R (+2) and Python→SAS (+1), loses in SAS→Python (−3) and R→Python (−1), and ties in R→SAS. That margin is inside the benchmark's noise and should not be read as a demonstrated difference between the two checkpoints. See Evaluation below.
Evaluation
Method
Translations are scored by execution parity, not text similarity and not an LLM judge. Each translated program is executed against frozen fixtures, and a pair is accepted only if every declared output table matches the original program's captured output cell by cell.
The held-out set is 424 translation pairs, roughly 70 per direction. The 396 column is a
subset of it used for historical comparability; the two columns are reported separately and
should not be mixed.
Results
Counts of accepted pairs. Per-direction denominators in parentheses.
| Model | SAS→Py (68) | SAS→R (67) | Py→SAS (63) | R→SAS (66) | Py→R (64) | R→Py (68) | Subset (396) | Full (424) |
|---|---|---|---|---|---|---|---|---|
| Claude Opus, thinking off | 46 (67.6%) | 51 (76.1%) | 42 (66.7%) | 54 (81.8%) | 55 (85.9%) | 64 (94.1%) | 312 (78.8%) | 337 |
| Claude Opus, thinking on | 51 (75.0%) | 55 (82.1%) | 38 (60.3%) | 49 (74.2%) | 52 (81.2%) | 64 (94.1%) | 309 (78.0%) | 329 |
| GPT-5.6 Sol | 44 (64.7%) | 47 (70.1%) | 56 (88.9%) | 57 (86.4%) | 32 (50.0%) | 61 (89.7%) | 297 (75.0%) | 320 |
| PolyKode Community (this model) | 23 (33.8%) | 20 (29.9%) | 20 (31.7%) | 18 (27.3%) | 48 (75.0%) | 57 (83.8%) | 186 (47.0%) | 194 |
| Prior checkpoint (step 200) | 26 (38.2%) | 15 (22.4%) | 19 (30.2%) | 18 (27.3%) | 46 (71.9%) | 58 (85.3%) | 182 (46.0%) | 191 |
| Muse Glimmer 30B * | 15 (22.1%) | 22 (32.8%) | 29 (46.0%) | 35 (53.0%) | 22 (34.4%) | 58 (85.3%) | 181 (45.7%) | 194 |
| Prior internal fine-tune (2.7) | 24 (35.3%) | 17 (25.4%) | 16 (25.4%) | 14 (21.2%) | 47 (73.4%) | 58 (85.3%) | 176 (44.4%) | 183 |
| Pre-fine-tuning base | 9 (13.2%) | 14 (20.9%) | 41 (65.1%) | 32 (48.5%) | 15 (23.4%) | 51 (75.0%) | 162 (40.9%) | 169 |
| Qwen3.8-27B | 20 (29.4%) | 17 (25.4%) | 31 (49.2%) | 23 (34.8%) | 12 (18.8%) | 54 (79.4%) | 157 (39.6%) | 167 |
| Qwen3-Coder-30B-A3B | 18 (26.5%) | 14 (20.9%) | 31 (49.2%) | 22 (33.3%) | 14 (21.9%) | 48 (70.6%) | 147 (37.1%) | 155 |
| Codestral 25.08 | 11 (16.2%) | 13 (19.4%) | 31 (49.2%) | 26 (39.4%) | 11 (17.2%) | 51 (75.0%) | 143 (36.1%) | 149 |
| Prior internal fine-tune (2.6, Devstral 24B) | 13 (19.1%) | 5 (7.5%) | 5 (7.9%) | 5 (7.6%) | 27 (42.2%) | 28 (41.2%) | 83 (21.0%) | 83 |
| Prior internal fine-tune (2.0) * | 4 (5.9%) | 1 (1.5%) | 12 (19.0%) | 8 (12.1%) | 14 (21.9%) | 28 (41.2%) | 67 (16.9%) | 75 |
| Nemotron 3.5 Lightning base * | 5 (7.4%) | 3 (4.5%) | 8 (12.7%) | 7 (10.6%) | 8 (12.5%) | 26 (38.2%) | 57 (14.4%) | 59 |
| PolyKode V1 * | 4 (5.9%) | 0 (0.0%) | 12 (19.0%) | 8 (12.1%) | 8 (12.5%) | 18 (26.5%) | 50 (12.6%) | 58 |
| Nemotron 3 Nano * | 3 (4.4%) | 4 (6.0%) | 4 (6.3%) | 5 (7.6%) | 8 (12.5%) | 21 (30.9%) | 45 (11.4%) | 46 |
Usage
vLLM (reference deployment)
Served with vLLM 0.28.0, one GPU, tensor parallel size 1.
vllm serve /models/ProCogia/PolyKode-Community \
--served-model-name PolyKode-Community \
--host 0.0.0.0 --port 8001 \
--dtype bfloat16 \
--tensor-parallel-size 1 \
--max-model-len 32768 \
--max-num-seqs 1 \
--kv-cache-dtype fp8 \
--language-model-only \
--limit-mm-per-prompt '{"image":0}' \
--default-chat-template-kwargs '{"enable_thinking": false}' \
--gpu-memory-utilization 0.90
Flags that are not optional:
--language-model-onlyand--limit-mm-per-prompt '{"image":0}'— the repo carries a vision config that was never trained.--default-chat-template-kwargs '{"enable_thinking": false}'— matches the trained header.--max-num-seqs 1on an 80 GB card. Raising it will OOM.
Weights stay bfloat16; fp8 applies to the KV cache only. First load takes several minutes;
poll /health.
Example request:
curl -sS http://127.0.0.1:8001/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "PolyKode-Community",
"temperature": 0,
"max_tokens": 2048,
"messages": [
{"role": "system", "content": "<the system prompt above, verbatim>"},
{"role": "user", "content": "Translate the following SAS program to Python.\n\n<constraints>\n\n```sas\ndata a; x=1; run;\n```"}
]
}'
Hardware requirements
| GPU memory | 80 GB minimum at bf16 / 32k context. 40 GB is not enough. |
| Disk | ~100 GB free (weights are ~72 GB) |
| System RAM | 64 GB (unified-memory machines share the budget) |
Verified on A100 80 GB, H100 94 GB, and DGX Spark GB10 128 GB unified.
License and attribution
Released under the Apache License 2.0, inherited from the base model.
The base model's license file is the unmodified Apache 2.0 text carrying
Copyright 2026 Alibaba Cloud. Apache 2.0 §4 obligations that apply to redistributing this
derivative:
- A copy of the license accompanies this distribution (
LICENSE). - These weights are a modified version of Qwen3.6-35B-A3B, changed by LoRA fine-tuning as described above.
- The base model's copyright and attribution notices are retained.
Apache 2.0 §6 grants no trademark rights; "Qwen" is used here only to identify the origin of the base model.
Citation
@misc{polykode_community_2026,
title = {PolyKode Community: A SAS/R/Python Code Translation Model},
author = {{ProCogia}},
year = {2026},
howpublished = {\url{https://huggingface.co/ProCogia/PolyKode-Community}},
note = {Fine-tune of Qwen3.6-35B-A3B. PolyKode Community edition.}
}
Built on Qwen/Qwen3.6-35B-A3B, loaded from
the unsloth/Qwen3.6-35B-A3B bfloat16 mirror. PolyKode Community is a LoRA fine-tune of those weights,
merged; the training configuration is described above.
- Downloads last month
- 309
Model tree for ProCogia/PolyKode-Community
Base model
Qwen/Qwen3.6-35B-A3B