A Smaller On-Device Model Artifact Matched or Beat a Larger One: A Single-Board Case Study of 4B and 9B RKLLM Artifacts on the RK3588 NPU

We compare two quantized language-model artifacts on a single 32 GB Turing RK1 module (Rockchip RK3588 NPU) under one frozen task set: a Qwen3-4B-Instruct-2507 RKLLM artifact and an experimental Qwen3.5-9B RKLLM conversion. Twelve frozen tasks—six Python coding tasks scored by executing assertions and six tool-selection tasks scored by exact comparison with an answer key—were each run three times per model in fresh processes, for 72 trials. The 4B artifact passed 36/36 trials and the 9B artifact 33/36. Answers were byte-identical across repeats for every task, so the effective sample is twelve tasks per model: 12/12 versus 11/12 (exact 95% intervals 0.735–1.000 and 0.615–0.998; exact paired test on the single discordant task, p=1.0). The correctness data therefore show one reproducible failure, not a statistically separable quality difference. Both artifacts passed all coding trials; the 9B failure was a syntactically valid tool call that selected a different tool than the one the request named. The resource differences are large relative to their spread: median native generation rate was 3.385 versus 1.75 tokens/s (ranges 3.35–3.44 and 1.75–1.76), a ratio of 1.93; median prefill rate was 74.415 versus 31.76 tokens/s; and peak process resident memory was 4335 versus 8679 MiB. Per-trial clock snapshots showed the DDR controller at 528 MHz before and after every trial; a separate sampled diagnostic run of the 4B artifact showed it at 2112 MHz throughout inference, so the idle snapshots do not support calling these throughputs DDR-limited. The two artifacts differ in model release, toolkit version, quantization grouping, and chat framing, so this is a deployment comparison of two artifacts on one board, not a test of parameter count.

Drafted with AI assistance; reviewed and approved by the author. CloudLab Works.

@techreport{alvord2026rk1,
  author = {Alvord, Alex},
  title = {{A Smaller On-Device Model Artifact Matched or Beat a Larger One: A Single-Board Case Study of 4B and 9B RKLLM Artifacts on the RK3588 NPU}},
  institution = {CloudLab Works},
  publisher = {Zenodo},
  year = {2026},
  month = oct,
  type = {Preprint},
  doi = {10.5281/zenodo.23720938},
  url = {https://doi.org/10.5281/zenodo.23720938}
}