32 GB RAM laptop: Cosmos as a guest of Ubuntu, on one virtual function of the GPU

MSI Prestige 16 AI+ C3MTG · Core Ultra X7 358H · 32 GB

Captured November 8 – 9, 2026 · One captured run per system

Cosmos ran as a QEMU/KVM guest on the laptop's Ubuntu, driving one SR-IOV virtual function of its Arc B390 GPU. Ubuntu stayed in charge of the hardware: its driver set the GPU's clocks and power, and its scheduler ran the guest's CPUs. Against llama.cpp running natively on the same host, with the same weights and the same request, Cosmos still decoded several times as fast, and about as fast as Cosmos booted on the laptop itself.

Laptop: MSI Prestige 16 AI+ C3MTG
CPU: Intel Core Ultra X7 358H
GPU: Intel Arc B390 · Xe3 (integrated)
Memory: 32 GB, shared by CPU, GPU and NPU

Two windows on an Ubuntu desktop. Left, htop: CPUs 0 and 1 at 98% and 80%, and qemu-system-x86_64 threads at the top of the process list. Right, a window titled QEMU (Kore) showing the Cosmos chat running AxionML/Qwen3.5-9B-NVFP4 with a 65536-token context, thinking, speculating 7 tokens a step; its system context ready in 8938 ms; the request, Write an example of merge sort in C; Thought for 3 s; and the start of the answer, a C listing with merge and mergeSort. — The request · Cosmos as a guestMSI Prestige 16 AI+ C3MTG · Core Ultra X7 358H · 32 GB
Cosmos in a QEMU window on the Ubuntu host's desktop, the request its first message. Beside it, htop on the host shows QEMU's threads carrying the load.

37.9tok/s

Decode speed as a guest · 3.4× the speed of Ubuntu + llama.cpp on the same machine · 97% of Cosmos on metal

Cosmos, as a guest: 37.9tok/s
Cosmos on metal: 38.9tok/s
Ubuntu + llama.cpp: 11.11tok/s

Readouts, as shown on screen

Cosmos, as a guest

806 tokens at 37.9 tok/s · context 2055 of 65536 · 3.88 tokens a step

Context 2,055 / 65,536 · 3.88 tokens per decode step · The run in the screencast

Cosmos on metal

824 tokens at 38.9 tok/s · context 2073 of 65536 · 4.18 tokens a step

Context 2,073 / 65,536 · 4.18 tokens per decode step

Ubuntu + llama.cpp

11.11 t/s · Tokens generated 930 tok · Avg speed 11.1t/s

930 tokens generated · Thinking on

Single runs. With MTP, decode speed depends on how many drafted tokens each step keeps, which varies with the text generated: the guest run kept 3.88 tokens a step, the metal run 4.18. A second guest run, three minutes later, read 39.0 tok/s at 3.97; it's with the evidence.

llama.cpp's figure is the response's own rate; its token panel averages 11.1 t/s over the same turn.

What each system ran

Weights: AllAxionML/Qwen3.5-9B-NVFP4, with its MTP head
System: Cosmos, as a guestCosmos on Kore, as a QEMU/KVM guest; Cosmos on metalCosmos on Kore; Ubuntu + llama.cppUbuntu 26.04.1 LTS · Linux 7.0.0-38
Host: Cosmos, as a guestUbuntu 26.04.1 LTS · Linux 7.0.0-38 · GNOME 50.1 running; Cosmos on metalNone: booted on the laptop; Ubuntu + llama.cppItself · GNOME 50.1 running
GPU: Cosmos, as a guestOne SR-IOV virtual function of the Arc B390, through VFIO; Cosmos on metalThe whole Arc B390; Ubuntu + llama.cppThe whole Arc B390 · Intel Xe driver
Clocks, power: Cosmos, as a guestSet by the host's Xe driver; Cosmos on metalSet by Kore; Ubuntu + llama.cppSet by Ubuntu's Xe driver
CPU, memory: Cosmos, as a guest4 vCPUs, not pinned: left to the host's scheduler across all 16 cores · all guest memory in preallocated 2 MiB hugepages; Cosmos on metalThe whole machine; Ubuntu + llama.cppThe whole machine, under Ubuntu's scheduler
IOMMU: Cosmos, as a guestMandatory: the guest's virtual IOMMU, over the host's VT-d; Cosmos on metalMandatory: VT-d; Ubuntu + llama.cppOn: VT-d in Linux's default DMA-FQ mode
Context: All65,536 tokens
Runtime: Cosmos, as a guestKore's inference engine · built-in chat; Cosmos on metalKore's inference engine · built-in chat; Ubuntu + llama.cppllama.cpp 0.8.0-dev (build 1, 418f905) · Vulkan, Mesa 26.0.8 · llama-server web UI
Thinking: Cosmos, as a guestOn; this model has no effort levels; Cosmos on metalOn; this model has no effort levels; Ubuntu + llama.cppOn; the web UI's effort selector read Medium, a level this model doesn't have

Evidence

Benchmark capture card D, 32 GB, Cosmos as a guest: Qwen3.5-9B MTP NVFP4 in a QEMU/KVM guest on Ubuntu, with one SR-IOV virtual function of the Arc B390 passed to the guest. Left, htop on the Ubuntu host: CPUs 0 and 1 near full load and qemu-system-x86_64 threads at the top of the process list. Right, a window titled QEMU (Kore) showing the Cosmos chat running AxionML/Qwen3.5-9B-NVFP4: the end of a C merge sort answer with its complexity and compile command, then the next request, Now try one in Python and run it too. The status line 806 tokens at 37.9 tok/s, context 2055 of 65536, 3.88 tokens a step is highlighted. The card reads 37.9 tokens per second, C merge-sort response, first in a fresh chat. — Cosmos as a guest · 37.9 tok/sMSI Prestige 16 AI+ C3MTG · Core Ultra X7 358H · 32 GB

Benchmark capture card D, 32 GB, Cosmos on metal: Qwen3.5-9B MTP NVFP4 with the whole GPU, booted on the laptop. Left, kore status mem: 10.1 of 30.8 GiB of physical memory in use, page classes of 4 KiB, 2 MiB and 1 GiB, and DRAM stream bandwidth per core class. Right, the Cosmos chat running AxionML/Qwen3.5-9B-NVFP4: the end of a C merge sort answer, a table of its components, its complexity and how to compile it. The status line 824 tokens at 38.9 tok/s, context 2073 of 65536, 4.18 tokens a step is highlighted. The card reads 38.9 tokens per second, C merge-sort response, first in a fresh chat. — Cosmos on metal · 38.9 tok/sMSI Prestige 16 AI+ C3MTG · Core Ultra X7 358H · 32 GB

Benchmark capture card D, 32 GB, native on the host: Ubuntu with llama.cpp, Qwen3.5-9B MTP NVFP4, with the whole GPU through Ubuntu's Xe driver. Left, fastfetch: Ubuntu 26.04.1 LTS on a Prestige 16 AI+, Linux 7.0.0-38, GNOME 50.1, an Intel Arc B390. Right, llama.cpp's web UI at 127.0.0.1:8080 after a C merge sort answer, with its context panel open. Highlighted: the response's rate, 11.11 t/s; tokens generated, 930; the panel's average speed, 11.1 t/s; and the web UI's effort selector, Medium, which the card notes isn't a level this model has. The card reads 11.11 tokens per second, C merge-sort response, thinking on. — Ubuntu + llama.cpp · 11.11 tok/sMSI Prestige 16 AI+ C3MTG · Core Ultra X7 358H · 32 GB

Screencast · Cosmos as a guestMSI Prestige 16 AI+ C3MTG · Core Ultra X7 358H · 32 GB0:26 · real time

The guest run from request to readout, in real time, recorded on the Ubuntu host: Cosmos in its QEMU window, htop beside it. It ends on the readout the guest's capture card shows.

Edited for disclosure: one line of the guest's startup log, describing how its speculative decoding is structured, is covered where it appears, for the first 11 seconds, and the cover says so. Nothing else is changed: no cuts, no speed-up.

Left, fastfetch on the host: Ubuntu 26.04.1 LTS on a Prestige 16 AI+ C3MTG, Linux 7.0.0-38-generic, GNOME 50.1 on Wayland, an Intel Core Ultra X7 358H, and two Intel Arc B390 entries, one marked integrated. Right, the QEMU window with the Cosmos chat: the end of a C merge sort answer, how it works and key notes; its status line reads 838 tokens at 39.0 tok/s, context 2087 of 65536, 3.97 tokens a step. — A second guest run · 39.0 tok/sMSI Prestige 16 AI+ C3MTG · Core Ultra X7 358H · 32 GB
Three minutes later, a fresh chat in the same guest: 838 tokens at 39.0 tok/s, 3.97 tokens a step. Beside it, fastfetch on the host.

Left, gputop on the host: DRM minor 0 lists gnome-shell and Xwayland; DRM minor 128 lists gnome-shell, Xwayland, mutter, ibus, ptyxis, firefox and llama-server, which holds 10 GB. Right, llama.cpp's web UI after a Python version of the answer, with its token panel: 1,505 tokens generated across all turns, 575 this turn, and an average speed of 11.0 t/s. — llama.cpp on the GPUMSI Prestige 16 AI+ C3MTG · Core Ultra X7 358H · 32 GB
After llama.cpp's turns on the host, gputop lists llama-server holding 10 GB on the GPU.

How it was measured

Who runs the hardware: For the guest run, Ubuntu stays in charge. Its Xe driver owns the GPU's physical function and sets its clocks and power; Kore's driver makes no clock or power request from a virtual function, and only reads the clock configuration the host reports. Ubuntu's scheduler runs the guest's four vCPUs as ordinary host threads, unpinned, on whichever of the 16 cores it picks, performance, efficient or low-power. What differs from the llama.cpp run is the software that runs the model.
Guest setup: QEMU with KVM on the Ubuntu host: a q35 machine with 4 vCPUs (-cpu host), not pinned to any core; all guest memory backed by 2 MiB hugetlb pages, preallocated at launch; one SR-IOV virtual function of the Arc B390 passed to the guest through VFIO. The host's GNOME desktop kept running throughout.
IOMMU: Neither run turned the IOMMU off. Kore keeps its IOMMU mandatory as a guest: the GPU reaches only the memory Kore mapped, through the guest's virtual IOMMU, and the host's VT-d confines the guest on top. On the host, Linux translated the GPU's memory access too, in its default DMA-FQ mode, which batches invalidations rather than issuing each one as it happens.
Same request: All three runs answer a request for merge sort in C as the first message of a fresh chat, with the same weights and a 65,536-token context.
Same machine: Every comparison ran on one laptop, with the same silicon, drive and model weights on each side. On metal, Cosmos and Ubuntu were booted in turn; a result that ran Cosmos as a guest says how the guest was set up.
Current baseline: Ubuntu 26.04.1 LTS on Linux 7.0.0-38 with the latest Intel Xe drivers, on both machines; llama.cpp 0.8, with native NVFP4 and expert-cache support; fio 3.41 with io_uring, direct I/O and registered buffers, as the storage result's commands show. Current releases, not old builds, with one exception: the 27B result's llama.cpp run used an earlier build whose version wasn't recorded, and its settings say so.
Captured runs: Each figure is one captured run, read off its screenshot — not an average of repeated runs.
Native harnesses: Each system is measured with its own tools: Cosmos's chat and pack bench; llama.cpp's web UI and fio on Ubuntu. Transfer sizes and scheduling are reported, not equalized.
Units: Storage in decimal GB/s (10⁹ bytes per second). Tokens per second as each runtime reports its own generation speed.
Setup excluded: Buffer allocation and registration happen before timing on both systems; Cosmos prints its setup time on a separate line.

More results

Download Cosmos