microperceptrn

Benchmarks

State-of-the-art performance for local AI inference.

Qwen3.6-35B-A3B decoding at 13.2 tok/s on the Lenovo Yoga 7 2-in-1 16ILL10 (Core Ultra 5 226V, 16 GB), a model larger than the laptop's memory streamed from its SSD, and at 42.5 tok/s on the MSI Prestige 16 AI+ C3MTG (Core Ultra X7 358H, 32 GB), where every weight fits in memory. On the Lenovo Yoga 7 2-in-1, the SSD reads at its own ceiling. Each workload also ran on the same laptop under Ubuntu, with the standard Linux tool for the job — llama.cpp for inference, fio with io_uring for storage — as the reference. Every number on this page is read off a screenshot of the run, and every screenshot is here.

Qwen3.6-35B-A3B decode

13.2tok/s

Lenovo Yoga 7 2-in-1 16ILL10 · Core Ultra 5 226V · 16 GB

35B model larger than the laptop's memory, streamed from its SSD

Cosmos
13.2tok/s
Ubuntu + llama.cpp
1.8tok/s

7.3× the speed of Ubuntu + llama.cpp on the same laptop

Qwen3.6-35B-A3B decode

42.5tok/s

MSI Prestige 16 AI+ C3MTG · Core Ultra X7 358H · 32 GB

35B model, every weight resident in memory

Cosmos
42.5tok/s
Ubuntu + llama.cpp
14.95tok/s

2.8× the speed of Ubuntu + llama.cpp on the same laptop

NVMe sequential read

6.02GB/s

Lenovo Yoga 7 2-in-1 16ILL10 · Core Ultra 5 226V · 16 GB

5 of 7 read patterns within 1% of the drive's 6.0 GB/s ceiling

Cosmos
6.02GB/s
Ubuntu + fio
5.313GB/s

+13.3% over Ubuntu with io_uring, sequential

01Local inference

A 35B model, running locally on a laptop.

Local LLM inference with Qwen3.6-35B-A3B, MTP and NVFP4 weights — the same weights on both laptops and both systems, running entirely on the device — asked for a merge sort in C. Speed is each runtime's own generation readout.

Qwen3.6-35B-A3B · MTP · NVFP4

16 GB: a model larger than memory, streamed from SSD

Lenovo Yoga 7 2-in-1 16ILL10 · Core Ultra 5 226V · 16 GB

The model is bigger than the machine's memory. Cosmos keeps each layer's dense weights and a share of its experts resident, and streams the rest from NVMe into GPU-visible memory as the router asks for them. llama.cpp runs the same weights, with expert-cache support.

Laptop
Lenovo Yoga 7 2-in-1 16ILL10
CPU
Intel Core Ultra 5 226V
GPU
Intel Arc 130V · Xe2 (integrated)
Memory
16 GB, shared by CPU, GPU and NPU
Storage
SK hynix HFS512GEM4X182N · PCIe 4.0 ×4 NVMe
Left, kore status mem on the 16 GB Lenovo Yoga 7 2-in-1. Right, the inference log as the model loads: the vision encoder staged on the VPU, a 3 GiB model arena, a 5 GiB expert pool of 75 of 257 slots per layer, 4.97 GiB of experts in 40 pools read in 0.9 s at 5.98 GB/s, and ready in 4.3 s.(open full size)
Model load · 16 GBLenovo Yoga 7 2-in-1 16ILL10 · Core Ultra 5 226V · 16 GB

Loading Qwen3.6-35B-A3B on the 16 GB Yoga 7 2-in-1: a 5 GiB expert pool (75 of 257 slots per layer), 4.97 GiB of experts read at 5.98 GB/s, ready in 4.3 s. The vision encoder is staged on the NPU.

13.2tok/s

Decode speed on this laptop · 7.3× the speed of Ubuntu + llama.cpp on the same machine

Cosmos
13.2tok/s
Ubuntu + llama.cpp
1.8tok/s

Readouts, as shown on screen

cosmos

1948 tokens at 13.2 tok/s · context 3039 of 13312

Context 3,039 / 13,312 · Expert reads 2.43 GB/s¹

ubuntu

Avg speed 1.8t/s

Context 3,088 / 20,480

Weights
CosmosQwen3.6-35B-A3B · MTP · NVFP4
UbuntuThe same weights
System
CosmosCosmos on Kore
UbuntuUbuntu 26.04.1 LTS · Linux 7.0.0-38 · latest Intel Xe drivers
Runtime
CosmosKore's inference engine · built-in chat
Ubuntullama.cpp 0.8 · native NVFP4 · expert-cache support · web UI

¹ Bytes read divided by time spent waiting on reads — not whole-run SSD throughput.

Evidence

Benchmark capture card, Pair A, 16 GB SSD streaming, Qwen 35B-A3B NVFP4, Cosmos. The Cosmos chat and a kore status mem terminal on the Core Ultra 5 226V laptop, with the status line 1948 tokens at 13.2 tok/s, context 3039 of 13312 highlighted. The card reads answer generation speed 13.2 tokens per second and expert reads 2.43 GB/s.(open full size)
Cosmos · 13.2 tok/sLenovo Yoga 7 2-in-1 16ILL10 · Core Ultra 5 226V · 16 GB
Benchmark capture card, Pair A, Ubuntu with llama.cpp. The llama.cpp web UI on the same laptop after a merge sort answer, with a token usage popover reading context 3.09K of 20.48K and Avg speed 1.8 t/s highlighted. The card reads 1.8 tokens per second, llama.cpp v0.8.0.(open full size)
Ubuntu + llama.cpp · 1.8 tok/sLenovo Yoga 7 2-in-1 16ILL10 · Core Ultra 5 226V · 16 GB

Qwen3.6-35B-A3B · MTP · NVFP4

32 GB: every weight resident in memory

MSI Prestige 16 AI+ C3MTG · Core Ultra X7 358H · 32 GB

With 32 GB every weight fits in memory on both systems. This pair compares captured generation rates for the same weights and the same kind of request, a merge sort in C.

Laptop
MSI Prestige 16 AI+ C3MTG
CPU
Intel Core Ultra X7 358H
GPU
Intel Arc B390 · Xe3 (integrated)
Memory
32 GB, shared by CPU, GPU and NPU
42.5tok/s

Decode speed on this laptop · 2.8× the speed of Ubuntu + llama.cpp on the same machine

Cosmos
42.5tok/s
Ubuntu + llama.cpp
14.95tok/s

Readouts, as shown on screen

cosmos

1117 tokens at 42.5 tok/s · context 4058 of 197632 · 4.12 tokens a step

Context 4,058 / 197,632 · 4.12 tokens per decode step

ubuntu

3,193 tokens · 3min 33s · 14.95 t/s

3,193 tokens in 3 min 33 s

Weights
CosmosQwen3.6-35B-A3B · MTP · NVFP4
UbuntuThe same weights
System
CosmosCosmos on Kore
UbuntuUbuntu 26.04.1 LTS · Linux 7.0.0-38 · latest Intel Xe drivers
Runtime
CosmosKore's inference engine · built-in chat
Ubuntullama.cpp 0.8 · native NVFP4 · expert-cache support · web UI

Evidence

Benchmark capture card, Pair B, 32 GB memory resident, Qwen 35B-A3B NVFP4, Cosmos. kore status cpu and a chat answer on the MSI Prestige 16 AI+ (Core Ultra X7 358H), with the status line 1117 tokens at 42.5 tok/s, context 4058 of 197632, 4.12 tokens a step highlighted. The card reads 42.5 tokens per second.(open full size)
Cosmos · 42.5 tok/sMSI Prestige 16 AI+ C3MTG · Core Ultra X7 358H · 32 GB
Benchmark capture card, Pair B, Ubuntu with llama.cpp. The llama.cpp web UI on the same laptop after a C merge sort answer, with 3,193 tokens, 3 min 33 s and 14.95 t/s highlighted. The card reads 14.95 tokens per second.(open full size)
Ubuntu + llama.cpp · 14.95 tok/sMSI Prestige 16 AI+ C3MTG · Core Ultra X7 358H · 32 GB
02Storage

Storage at the drive's ceiling.

The 64 GiB pack partition on the Lenovo Yoga 7 2-in-1's SSD, read 7 ways: runs of 64 MiB to 1 GiB scattered over 8–64 GiB, and one sequential pass. 5 of them land within 1% of 6.0 GB/s — the drive's own limit, below the PCIe 4.0 ×4 link's 7.9 GB/s raw rate. Ubuntu's fastest path, io_uring with direct I/O, is shown for reference.

NVMe raw read

Reads straight off the 64 GiB partition Cosmos keeps model packs on, through each system's fastest path: Kore's pack engine, and fio with io_uring, direct I/O and registered buffers.

Laptop
Lenovo Yoga 7 2-in-1 16ILL10
CPU
Intel Core Ultra 5 226V
GPU
Intel Arc 130V · Xe2 (integrated)
Memory
16 GB, shared by CPU, GPU and NPU
Storage
SK hynix HFS512GEM4X182N · PCIe 4.0 ×4 NVMe
NVMe read throughput in GB/s, Cosmos and Ubuntu, per access pattern
PatternCosmosUbuntuΔ
Random64 MiB / 64 GiB5.785.559+4.0%
Random1,024 MiB / 64 GiB6.015.916+1.6%
SequentialContiguous · see note6.025.313+13.3%
Random64 MiB / 8 GiB5.645.027+12.2%
Random256 MiB / 16 GiB5.955.177+14.9%
Random256 MiB / 64 GiB5.995.396+11.0%
Random512 MiB / 64 GiB6.005.387+11.4%

GB/s, decimal · group / span

Software
CosmosKore pack engine (pack bench)
UbuntuUbuntu 26.04.1 LTS · Linux 7.0.0-38 · fio 3.41
Submission
Cosmos4 NVMe rings, round robin
Ubuntuio_uring · 8 jobs × queue depth 16
Transfer size
Cosmos256 KiB — the controller's maximum transfer
Ubuntu128 KiB — the kernel's max_sectors_kb for this drive
Buffers
CosmosFunded and IOMMU-mapped before timing
UbuntuO_DIRECT · fixed buffers · registered file

Group = one contiguous read run, not the device I/O size. 12 GiB per case; the last case reads 18 GiB.

Sequential layouts differ: Kore declares one 64 GiB span; fio reads eight adjacent 1.5 GiB regions, one per job.

Commands for every case

fio, shared by every case

sudo fio --name=pack --filename=/dev/nvme0n1p6 --readonly --direct=1 --ioengine=io_uring --fixedbufs=1 --registerfiles=1 --thread --numjobs=8 --group_reporting --bs=128k --iodepth=16 --invalidate=0 --norandommap --randrepeat=0
  1. 64 MiB runs over 64 GiBkore › pack bench 4 64 3fio … --rw=randread:512 --ba=64M --size=64G --io_size=1536M
  2. 1,024 MiB runs over 64 GiBkore › pack bench 4 1024 3fio … --rw=randread:8192 --ba=1024M --size=64G --io_size=1536M
  3. Sequentialkore › pack bench 4 64 3 seqfio … --rw=read --size=1536M --offset_increment=1536M
  4. 64 MiB runs over 8 GiBkore › pack bench 4 64 3 span 8fio … --rw=randread:512 --ba=64M --size=8G --io_size=1536M
  5. 256 MiB runs over 16 GiBkore › pack bench 4 256 3 span 16fio … --rw=randread:2048 --ba=256M --size=16G --io_size=1536M
  6. 256 MiB runs over 64 GiBkore › pack bench 4 256 3 span 64fio … --rw=randread:2048 --ba=256M --size=64G --io_size=1536M
  7. 512 MiB runs over 64 GiBkore › pack bench 6 512 3 span 64fio … --rw=randread:4096 --ba=512M --size=64G --io_size=2304M

Evidence

Benchmark capture card, NVMe raw read throughput on the Core Ultra 5 226V laptop, Cosmos. A Kore terminal running pack bench, with the sequential run, 12288 MiB in 2136 ms: 6.02 GB/s, highlighted. The card reads 6.02 GB/s with 4 NVMe rings and a 256 KiB maximum transfer.(open full size)
Cosmos · 6.02 GB/sLenovo Yoga 7 2-in-1 16ILL10 · Core Ultra 5 226V · 16 GB
Benchmark capture card, NVMe raw read throughput, Ubuntu with io_uring. A terminal running fio 3.41 against the same partition, with the sequential result BW=5067MiB/s (5313MB/s)(12.0GiB/2425msec) highlighted. The card reads 5.313 GB/s with 128 KiB I/O and 8 jobs at depth 16.(open full size)
Ubuntu + fio · 5.313 GB/sLenovo Yoga 7 2-in-1 16ILL10 · Core Ultra 5 226V · 16 GB
A Kore terminal running pack info and seven pack bench commands against the 64 GiB pack partition, each reporting between 5.64 and 6.02 GB/s. The engine line reads round robin, 4 rings, 62021 submitted, 62021 completed, 0 in flight.(open full size)
pack benchLenovo Yoga 7 2-in-1 16ILL10 · Core Ultra 5 226V · 16 GB

The raw terminal behind the storage results: seven pack bench runs over the 64 GiB pack partition.

03Method

What these numbers are, and aren't.

Same machine
Each pair ran on one laptop, booted into Cosmos and into Ubuntu in turn: same silicon, same drive, same partition, same model weights.
Current baseline
Ubuntu 26.04.1 LTS on Linux 7.0.0-38 with the latest Intel Xe drivers, on both machines; llama.cpp 0.8, with native NVFP4 and expert-cache support; fio 3.41 with io_uring, direct I/O and registered buffers, as its commands below show. Current releases, not old builds.
Captured runs
Each figure is one captured run, read off its screenshot — not an average of repeated runs.
Native harnesses
Each system is measured with its own tools: Cosmos's chat and pack bench; llama.cpp's web UI and fio on Ubuntu. Transfer sizes and scheduling are reported, not equalized.
Units
Storage in decimal GB/s (10⁹ bytes per second). Tokens per second as each runtime reports its own generation speed.
Setup excluded
Buffer allocation and registration happen before timing on both systems; Cosmos prints its setup time on a separate line.
Get early access