PolyStrata
Large mixture-of-experts models on the graphics card you already have. Open source, MIT, built on Strata.
Download the source (6.6 MB, commit b0e3b57a)
Or clone the full repository with its history and tags from this Space (install git-lfs first, the binaries are stored with it):
git clone -b source https://huggingface.co/spaces/VecSzn/PolyStrata12 October: the GitHub pages of this project (github.com/VecSzn/PolyStrata, its releases and issues) are not visible at the moment: GitHub has put the account under review, and an appeal is in. The code is unchanged and is here meanwhile. The zip above is the source at the latest release, 0.1.43.1-poly.2, without the benchmark records and the media files. Unpack it and run START-HERE.bat on Windows or ./setup.sh on Linux. The ready-made Windows engine (strata-poly-windows-x64-bb7d247465918148.zip) will be put next to it; until then setup compiles the engine on your PC, which needs Visual Studio's build tools and the CUDA toolkit (setup offers to install them). Once the engine zip is here, set STRATA_POLY_PREBUILT_URL=https://huggingface.co/spaces/VecSzn/PolyStrata/resolve/main/ before running setup and it downloads it from here instead of GitHub.
A mixture-of-experts model only uses a few of its experts for each token. PolyStrata keeps the experts a model uses most on the GPU and computes the rest on the CPU from RAM, so a 35B model runs next to an 8 GB card and 32 GB of RAM. It reads the same GGUF files as llama.cpp, and its output is checked against llama.cpp token by token.
Same laptop, same model file, same request
PolyStrata on the left, llama.cpp b11540 on the right, on a laptop with an RTX 4070 (8 GB) and 32 GB of RAM, Qwen3.6-35B-A3B UD-Q3_K_M. On this laptop PolyStrata reads prompts 2.7 to 3.5 times as fast as llama.cpp and writes 1.5 to 1.8 times as fast; the comparison script is in the repository.
Tokens per second
Laptop: RTX 4070 (8 GB), Core i7-14700HX, 32 GB RAM
| Model | Size | Writes, tokens/s | Reads a 10K prompt, tokens/s |
|---|---|---|---|
| Qwen3.6-35B-A3B | UD-Q3_K_M | 94-100 | 1,690 |
| Qwen3.6-35B-A3B | UD-Q2_K_XL | 123-124 | 1,590 |
| Qwen3.6-35B-A3B | UD-Q4_K_M | 82-86 | 1,560-1,570 |
| Ornith 1.5 | APEX-I-COMPACT | 76-82 | 1,620 |
| Ornith 1.5 | APEX-I-MINI | 80-86 | 1,550-1,560 |
| Ornith 1.5 | APEX-I-QUALITY | 59-63 | 1,570-1,580 |
| Gemma 4 26B-A4B | UD-Q3_K_M | 92-93 | 2,240 |
| Gemma 4 26B-A4B | UD-Q2_K_XL | 101-104 | 2,150-2,160 |
| Gemma 4 26B-A4B | UD-Q4_K_M | 85 | 2,150 |
| Tiel Coder 35B-A3B | UD-Q3_K_XL | 83 | 1,700 |
| Tiel Coder 35B-A3B | UD-IQ3_XXS | 96-97 | 1,610 |
| Tiel Coder 35B-A3B | UD-Q4_K_XL | 74-79 | 1,660-1,670 |
Workstation: RTX 5090 (32 GB), Xeon Platinum 8470Q, 120 GB RAM
| Model | Size | Writes, tokens/s | Reads a 10K prompt, tokens/s |
|---|---|---|---|
| GLM 5.3 Flash | 3.0BIT | 59-66 | 820 |
| GLM 5.3 Flash | UD-IQ2_XXS | 50-52 | 1,060-1,080 |
| GLM 5.3 Flash | REAP50-Q3_K_M | 59-66 | 1,300-1,310 |
| Qwen3.6-35B-A3B | UD-Q3_K_M | 394-395 | 6,350-6,550 |
| Qwen3.6-35B-A3B | UD-Q2_K_XL | 450 | 6,400-6,590 |
| Qwen3.6-35B-A3B | UD-Q4_K_M | 363-401 | 6,030-6,690 |
| Ornith 1.5 | APEX-I-COMPACT | 316-319 | 6,830-6,860 |
| Ornith 1.5 | APEX-I-MINI | 289-300 | 6,300-6,380 |
| Ornith 1.5 | APEX-I-QUALITY | 289-298 | 6,040-6,390 |
| Gemma 4 26B-A4B | UD-Q3_K_M | 353-356 | 7,880-8,230 |
| Gemma 4 26B-A4B | UD-Q2_K_XL | 327-340 | 7,380-8,000 |
| Gemma 4 26B-A4B | UD-Q4_K_M | 335-346 | 7,040-8,050 |
| Tiel Coder 35B-A3B | UD-Q3_K_XL | 337-345 | 6,570-6,830 |
| Tiel Coder 35B-A3B | UD-IQ3_XXS | 339-351 | 6,090-6,520 |
| Tiel Coder 35B-A3B | UD-Q4_K_XL | 316-322 | 6,750-6,870 |
GLM 5.3 Flash is a 79 to 114 GB file and needs about 100 GB of RAM beside a 32 GB card. Qwen3.8-Flash-Next runs on Strata's own engine; its numbers are in the README.
Works with Claude Code
The server speaks the OpenAI and Anthropic APIs on localhost, so Claude Code, Codex and other apps run against the local model. This is Claude Code on the laptop model, in real time:
Install
Windows: download the source from the latest release and run START-HERE.bat; it installs a ready-made engine and the model. Linux: ./setup.sh builds the engine. AMD cards work on Linux as a beta. Details, every option and the community results are in the README inside the zip (docs/ has the rest).