Hugging Face's logo

AnandRago
/
PHPko-31M

Text Generation
Transformers
Safetensors
qwen3
php
code
code-completion
from-scratch
small-model
text-generation-inference
Model card Files
xet
 Community

PHPko-31M

A 31M-parameter PHP code-completion model, trained from scratch (random initialisation — not a fine-tune of any existing model).

It writes the next lines of PHP given the code you have typed so far. It is small enough to run comfortably on a laptop CPU.

Architecture

A Qwen3-style decoder-only transformer, implemented from scratch in PyTorch. It loads as a stock Qwen3ForCausalLM, so no trust_remote_code is required (needs transformers>=4.51).

Component Choice
Normalisation RMSNorm (pre-norm)
Positions RoPE, theta = 1000000
Attention Grouped-Query Attention (GQA)
QK-Norm Yes (RMSNorm on Q and K, before RoPE)
Feed-forward SwiGLU
Biases None
Embeddings Tied (input = output)

Size

Parameters 30.7M
Layers 8
Hidden size 512
Attention heads 8 (KV heads: 2)
Head dim 64
FFN size 1408
Context length 1024 (trained on 512-token windows)
Vocab 16000 — byte-level BPE trained on PHP
Weights model.safetensors (float32)

The tokenizer was trained on this PHP corpus rather than reused, so it packs PHP efficiently (~4.1 characters per token) and can represent any byte sequence.

Training

Data ~24.5M tokens of PHP (29,844 files from 41 open-source projects)
Objective Next-token prediction (causal LM)
Steps 4000
Validation loss 1.1826 (perplexity ~3.3)
Optimiser AdamW, cosine LR decay with warmup
Hardware Single free-tier GPU

What it does well

  • Continues PHP you have started: class bodies, method signatures, property declarations, docblocks.
  • Produces syntactically valid, idiomatic modern PHP — typed signatures (: string, ?int), constructor property promotion, fluent return $this;, PSR-style formatting.
  • Knows conventions of major frameworks it trained on (Symfony, Laravel, Doctrine, PHPUnit style).
  • Fast: roughly 0.3 s for a short completion on a CPU — usable as live editor autocomplete.

What it does NOT do

Please read this before using it — it is a small model and these limits are real:

  • It is not a chatbot. It does not follow instructions. Prompting it with "write me a function that sorts users" will not work — give it the start of code and it continues.
  • It does not understand your program's logic. It writes code that looks right more reliably than code that is right. Example: asked to complete add(Money $other), it may return 0 instead of summing.
  • It repeats itself. It sometimes emits the same method twice. Use a repetition penalty (~1.15) and keep max_new_tokens modest.
  • No fill-in-the-middle (FIM). It only sees code before the cursor. Do not send a suffix; use prefix-only completion.
  • It does not know your codebase, private APIs, or anything outside its training data.
  • Not for security-sensitive code. Always review output. It can produce insecure or non-functional code.
  • PHP only. Other languages will be poor.

Usage 

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("AnandRago/PHPko-31M")
tok = AutoTokenizer.from_pretrained("AnandRago/PHPko-31M")

prompt = "<?php\n\nclass Invoice\n{\n    private array $lines = [];\n\n    public function getTotal(): float\n    {\n        return "
ids = tok(prompt, return_tensors="pt").input_ids
out = model.generate(ids, max_new_tokens=64, do_sample=True,
                     temperature=0.2, top_p=0.9, repetition_penalty=1.15)
print(tok.decode(out[0], skip_special_tokens=True))

Recommended settings: temperature 0.2-0.5, top_p 0.9, repetition_penalty 1.15, max_new_tokens 48-96. Very low temperature (<0.1) makes it loop.

Training data & provenance

Trained only on permissively licensed (MIT / BSD-3) open-source PHP. GPL projects (e.g. WordPress, Drupal) were deliberately excluded so the corpus carries no copyleft obligations.

Files were filtered to remove vendor directories, tests, generated code, minified files, translation/lookup tables, and exact duplicates. License header comments were stripped.

Sources (each retains its original license) 

Corpus sources. Each retains its original license.

Carbon                   MIT      https://github.com/briannesbitt/Carbon.git
FastRoute                BSD-3    https://github.com/nikic/FastRoute.git
PHP-Parser               BSD-3    https://github.com/nikic/PHP-Parser.git
Slim                     MIT      https://github.com/slimphp/Slim.git
Sylius                   MIT      https://github.com/Sylius/Sylius.git
Twig                     BSD-3    https://github.com/twigphp/Twig.git
bagisto                  MIT      https://github.com/bagisto/bagisto.git
cakephp                  MIT      https://github.com/cakephp/cakephp.git
collections              MIT      https://github.com/doctrine/collections.git
collision                MIT      https://github.com/nunomaduro/collision.git
commonmark               BSD-3    https://github.com/thephpleague/commonmark.git
composer                 MIT      https://github.com/composer/composer.git
core                     MIT      https://github.com/api-platform/core.git
csv                      MIT      https://github.com/thephpleague/csv.git
dbal                     MIT      https://github.com/doctrine/dbal.git
filament                 MIT      https://github.com/filamentphp/filament.git
flysystem                MIT      https://github.com/thephpleague/flysystem.git
framework                MIT      https://github.com/laravel/framework.git
guzzle                   MIT      https://github.com/guzzle/guzzle.git
horizon                  MIT      https://github.com/laravel/horizon.git
laminas-mvc              BSD-3    https://github.com/laminas/laminas-mvc.git
laravel                  MIT      https://github.com/laravel/laravel.git
laravel-medialibrary     MIT      https://github.com/spatie/laravel-medialibrary.git
laravel-permission       MIT      https://github.com/spatie/laravel-permission.git
livewire                 MIT      https://github.com/livewire/livewire.git
log                      MIT      https://github.com/php-fig/log.git
migrations               MIT      https://github.com/doctrine/migrations.git
monolog                  MIT      https://github.com/Seldaek/monolog.git
oauth2-server            MIT      https://github.com/thephpleague/oauth2-server.git
orm                      MIT      https://github.com/doctrine/orm.git
phpdotenv                BSD-3    https://github.com/vlucas/phpdotenv.git
phpstan-src              MIT      https://github.com/phpstan/phpstan-src.git
phpunit                  BSD-3    https://github.com/sebastianbergmann/phpunit.git
promises                 MIT      https://github.com/guzzle/promises.git
psr7                     MIT      https://github.com/guzzle/psr7.git
rector-src               MIT      https://github.com/rectorphp/rector-src.git
symfony                  MIT      https://github.com/symfony/symfony.git
symplify                 MIT      https://github.com/symplify/symplify.git
telescope                MIT      https://github.com/laravel/telescope.git
uuid                     MIT      https://github.com/ramsey/uuid.git
yii2                     BSD-3    https://github.com/yiisoft/yii2.git

Files kept per source repository

Repo Files
symfony 7,498
Sylius 3,600
rector-src 3,077
filament 2,458
phpstan-src 1,884
framework 1,619
bagisto 1,433
core 1,357
phpunit 1,131
cakephp 788
livewire 596
Carbon 470
orm 459
yii2 459
dbal 435
Twig 335
composer 311
commonmark 298
PHP-Parser 271
flysystem 181
migrations 164
horizon 140
csv 125
monolog 123
uuid 114
laravel-medialibrary 93
telescope 86
oauth2-server 85
laminas-mvc 81
Slim 72
guzzle 47
laravel-permission 42
psr7 42
phpdotenv 41
FastRoute 35
collision 35
laravel 19
promises 18
collections 14
log 7
symplify 1

Total: 30,044 files, ~25.6M BPE tokens.

License

Apache-2.0 for the model weights. Each training source retains its own license (listed above); please respect them.

Downloads last month
112
Safetensors
Model size
30.7M params
Tensor type
F32
·
Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support