Author(s): Enzo Lombardi

Originally published on Towards AI.

Crossing into C

Part 10 got Eugene talking to DwarfStar without a socket, but it didn’t get rid of the process boundary. DwarfStarProvider still spawns ds4 as a child, writes a line to its stdin, and reads lines back until a ds4> prompt reappears in stdout. That works, and it’s honest about its own limit: it’s parsing terminal output that was never meant to be a wire format, and it trusts a banner string to mean “this turn is done.” The REPL keeps its KV cache warm across turns, which was the whole point, but Eugene still can’t ask it for a sampling temperature, a token’s logprob, or a clean signal that generation actually finished versus the pipe just going quiet for a moment.

This post shows how to replace the subprocess/REPL text protocol with direct in-process FFI by using DwarfStar’s public C header boundary (ds4.h). It explains that DwarfStar exposes an engine and per-session KV-cache timeline, including prefix-aware session sync that reuses KV state by diffing the provided token prefix against the live checkpoint. With that seam, the author builds a Rust Provider that calls ds4_engine/session functions directly (no pipes), compiles the required C/Objective‑C sources into a Rust binary via build.rs (targeting Metal/CUDA as DwarfStar recommends), and wraps unsafe FFI narrowly to ensure resources are freed exactly once and C-side errors are surfaced as Rust Result failures. The resulting complete() implementation ends generation using the engine’s EOS token (not prompt strings), gaining token-level sampling control and explicit cache-reuse semantics while paying costs in C build integration and carefully pinned/unstable ABI assumptions. The article closes by emphasizing that the agent loop remains unchanged across integration strategies—HTTP, subprocess, or FFI—because the Provider trait isolates transport and execution details.

Read the full blog for free on Medium.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI