A transformer inference engine, written from scratch

Qwen2.5-0.5B-Instruct, running on WebGPU. No PyTorch, no transformers, no tokenizer library: the safetensors reader, the bfloat16 conversion, the BPE tokenizer, the KV cache, the attention kernels and the quantization are all in the repository.

Generating plausible text proves nothing; the reference implementations already do that. Every claim below is checked against an independent implementation or measured on named hardware, and the ones that went against me are here too.

Loading measurements…