mlx-runtime-rs

mlx-runtime-rs is a small, safe Rust runtime boundary for MLX on Apple silicon. It provides runtime initialization, owned f32 arrays, evaluated matrix multiplication, runtime capabilities, memory snapshots, and a fail-closed capability boundary for thread-local streams.

The default build links to a real, system-installed MLX runtime. It does not silently substitute a mock backend. A separate stub feature exists for platform-independent fail-closed tests; the stub does not emulate MLX operations and is not evidence of Metal or GPU execution.

Requirements

• A Mac with Apple silicon

• macOS

• Rust 1.83 or newer

• A C++20 compiler (the Apple Clang toolchain included with Xcode Command Line Tools is sufficient)

• MLX installed by Homebrew:

  brew install mlx

If MLX is installed under a nonstandard prefix, set MLX_PATH to that prefix before building.

Quick start

Run the real Metal-backed example:

cargo run -p mlx-runtime --example gpu_matmul

The example selects the GPU, reports the MLX version and Metal device, computes a 2x2 f32 matrix multiplication, evaluates the lazy result, copies it back to Rust, verifies [19, 22, 43, 50], and prints MLX active, peak, and cache memory.

Run the test suite against the installed MLX runtime:

cargo test --workspace
cargo clippy --workspace --all-targets -- -D warnings

Run the complete repository-local release-confidence gate—including the real Metal-backed example and the explicit stub contract—with:

just ci

Run platform-independent contract tests with the explicit stub backend:

cargo test -p mlx-runtime --no-default-features --features stub

The stub verifies that attempting to initialize an unavailable MLX runtime returns a descriptive error instead of silently running a different backend.

Safety and concurrency

Rust values own their native MLX handles and release them on drop. Errors from the C++ boundary are copied into Rust-owned strings, and C++ exceptions do not cross the C ABI.

The supported concurrency posture is intentionally conservative: serialized use is the safe default. Version 0.1 reports scoped thread-local streams as unsupported and fails installation closed. MLX 0.32 only exposes thread-wide stream destruction, which cannot safely implement a Rust guard without risking streams owned by another MLX user on the same thread. The capability fields and guard API reserve an explicit path for a future MLX lifecycle that can be owned and restored safely; they do not imply cross-thread array or graph safety.

Scope

Version 0.1 focuses on a narrow runtime and ownership surface:

• CPU, GPU, or automatic device selection
• runtime and device capability readback
• owned f32 arrays
• matrix multiplication, explicit evaluation, and host readback
• active, peak, and cache memory observation
• explicit garbage collection
• a reserved, fail-closed capability boundary for scoped thread-local streams

It is not a model server, tensor framework, model loader, training runtime, or general-purpose Metal binding.

License

Licensed under either of:

• Apache License, Version 2.0
• MIT License

at your option.