DocsMemory and devices

Kore ABI

Memory and devices

Userland drivers on Kore: DMA buffers by stated properties, IOMMU-mediated device access, and why physical addresses never reach a process.

Source: device.rs

Drivers first

Kore runs its drivers in user space, so the device contract is the part of the ABI it serves first. Its rules are few:

  • No physical address crosses the boundary. The wire carries stamped capabilities and op-codes, never a physical address, kernel pointer or borrow. Device addresses are their own type, IovaAddr: an I/O virtual address, never a physical one.
  • The IOMMU is mandatory. A device's DMA is translated by the IOMMU from the moment it is bound; bus mastering opens only after a live IOMMU window exists.
  • A driver gets exactly what it declares. Register windows, interrupt routes and DMA windows are derived from the device capability one by one.

Stating what you need

A program states requirements, not placements. AllocProps is 24 bytes:

FieldTypeMeaning
sizeu64Bytes
alignu64Alignment
numaNumaNodeA NUMA hint; NumaNode::ANY (0xFFFF) lets the kernel choose
dmaDmaClassNone, Dma32 (below 4 GiB) or Dma64
attrsMemAttrsWRITE_BACK, WRITE_COMBINE, UNCACHED, READ, WRITE
contiguousboolPhysically contiguous backing

The kernel's placement policy picks the backing region from these properties. The memory service's wire uses the same vocabulary.

Leases and mappings

Memory is held as a lease: a capability over a physical extent with one owner, which can be mapped into several address spaces and device domains at once. Revoking the lease unmaps it everywhere and frees it exactly once.

OperationPayloadEffect
Carve a leaseLeaseReq (size, flags, align_log2)A MemLease from a physical region, aligned to 4 KiB, 2 MiB or 1 GiB
Map for the CPUShareReq (va, bytes, write, access)Install a lease in an address space
Map into a windowVIEW_MAP with ViewMapReqInstall into the map, surface, heap or device window
Narrow permissionsPROTECTClear read, write or execute bits; never set them

Lease flags mark a lease as sealed (encrypted when swapped), initially executable (which needs a live executable mint) or tracked on release. A request for write and execute together is refused, and the user bit is never selectable on the wire. Memory changes between writable and executable only through the EXEC and EXEC_RETURN revoke flavors.

DMA

A DMA buffer is a lease mapped into a device's IOMMU domain with DeviceShareReq: the device slot, the I/O virtual address, the length, and an access mode of read, read-write or write (posted writes only). DEVICE_ACTIVATE then opens PCI bus mastering, and refuses until such a window exists. Peer-to-peer apertures between devices are sponsored by a DeviceMmu capability and mapped with DERIVE_KIND_PEER_IOVA.

Registers and interrupts

  • Registers. BarReq claims one of a PCI function's six BARs as an MmioRegion, which is then mapped with ShareReq or VIEW_MAP. On x86, an IoPortGrant holder can use in and out directly; the kernel admits it through the per-CPU I/O permission bitmap.
  • Interrupts. IrqReq (first_entry, a power-of-two count, affinity) derives an IrqRoute, preferring MSI-X and falling back to MSI. IRQ_ROUTE_MSIX insists on MSI-X, and IRQ_ROUTE_LINE takes the firmware's INTx line. A driver parks on its route with WAIT_IRQ, and the interrupt wakes it.

Finding devices

The kernel enumerates devices; drivers claim them. DeviceRecord describes a function as data, never authority, with flags such as HAS_BINDING (a live IOMMU binding), HAS_REMAPPED_MSI, HAS_REMAPPED_MSIX, HAS_BAR0, IS_BRIDGE and whether it is a driver endpoint. InventoryReq queries records, counts, I2C admission evidence and claims; a claim hands the driver one affine device capability.

Two read-only surfaces support this: the device census, bounded rows of PCI functions for system tools, and I2C admission, evidence pages that let an input driver confirm a controller before claiming it.

Writing a driver

libkore's driver feature exposes kore-driver: submission and completion rings, cookie tables, typed register access, DMA pools, the common device host and bounded completion waits. The host contract is the same for every driver: consume one claimed device and a DMA supply, derive the declared windows and routes, bind an owner, quiesce through a class hook, and retire children first.