API Reference
Typed, composable GStreamer pipelines in C++20 topics with brief descriptions are:
| Pipeline runtime | Session, PipelineRun, options, reports, errors |
| Builder and graph | Node interfaces and STL-only graph/builder utilities |
| Nodes | Pipeline building blocks |
| Common nodes | Common GStreamer utility nodes (caps, convert, appsink, etc.) |
| I/O nodes | Appsrc/RTSP/file/UDP source and sink nodes |
| Node groups | Higher-level recipes composed of multiple nodes |
| Sima nodes | SimaAI-specific nodes (decode/encode/preproc/postproc) |
| RTP nodes | RTP helpers (depay/payload/caps fixups) |
| MPK integration | Model pack loading and stage fragments |
| Contracts | Builder-level validation rules and reports |
| GStreamer helpers | Thin wrappers and utilities around GStreamer APIs |
| Tensor | Tensor core types, constraints, conversions, adapters |
| Diagnostics | PipelineReport, debug probes, and observability helpers |
| Model | Loaded MPKs, preprocess plans, and the simplified inference entry point |
| Runtime graph | Actor-style runtime DAG, named-port stages, and the GraphSession/GraphRun pair |
| Runtime graph nodes | Stage, fan-out, join, scheduler, and adapter nodes for runtime graphs |
| Runtime graph internals | Queue and mailbox primitives used by runtime graph execution |
| Policy | Decoder/encoder/RTSP/memory policy decision helpers used by the builder |
| Framework internals (SiMa reach-through) | Reach-through layer between core/ and the SiMa SoC pipeline internals |
Description
The Neat framework is the C++/Python library and runtime that runs models on the SiMa Modalix chip. It loads compiled model packs (MPKs), assembles them into deterministic GStreamer pipelines, and executes inference at frame rate on heterogeneous compute (A65 host CPU + EV74 vector cores + MLA accelerator + hardware codec).
This site is the reference documentation for the framework's public API. For design rationale and worked examples, see the Architecture deep dive, which mirrors the architect-level documentation.
Neat has two parts
People at SiMa use the word "Neat" to mean two related but distinct things. Both exist; both are called Neat; they're complementary:
- Neat framework — the library you #include, the runtime that loads models, the GStreamer-based pipelines that execute inference. The subject of this site.
- Neat environment — a Docker container with shared filesystem mounts to the DevKit, integrated agent skills, and a single-point-of-contact workflow that lets developers (and AI agents) compile on a fast x86 host and test on real Modalix hardware. The environment is what enables the agentic workflow — but it works because the framework underneath is built for agents.
The framework runs on the chip, in production, on every Modalix everywhere. The environment is a developer convenience that exists alongside. Production ships the framework; developers also use the environment.
What the framework solves
Modern AI inference on the edge has hard constraints: frame-rate throughput on small power budgets, heterogeneous compute (CPU + vector + accelerator), zero-copy memory across processors, deterministic latency, and the ability to integrate cleanly with media pipelines (cameras, codecs, RTSP). Solving all of these by hand for every application is a year of work. The Neat framework absorbs that work into a single typed C++/Python API.
A typical Neat-framework application looks like this:
A dozen lines from "RTSP camera → real-time YOLO detection running on Modalix accelerator." Behind that simplicity is a lot of carefully-designed machinery — that's what this reference documents.
The eight main concepts
The framework's surface is eight primary concepts. They form a clean progression from "the file on disk" to "the data that flows between processors at runtime":
| # | Concept | Purpose | Reference |
|---|---|---|---|
| 1 | MPK | Sealed model file on disk (.tar.gz from the compiler — kernels + weights + manifest). | Model, MPK contract |
| 2 | Model | Loaded form of an MPK; the simplified entry point. | Model |
| 3 | Tensor | Typed data unit that flows between stages. | Tensor, Memory model, dtype contract |
| 4 | Nodes | Smallest building blocks; each wraps one (or a few) GStreamer elements. | Node, GStreamer underneath |
| 5 | NodeGroups | Pre-made bundles of Nodes capturing common patterns. | Node groups |
| 6 | Session | Assembly stage that turns Nodes into a runnable pipeline. | Session, SessionOptions |
| 7 | Run | Live, running pipeline produced by Session::build(). | Run, Async vs sync timing, Threading model |
| 8 | Graph | Composition of pipelines for non-linear shapes. | Builder Graph, Runtime Graph, Two graph systems |
Where to read what
| What you want | Where to look |
|---|---|
| Conceptual overview, architecture, design rationale | Architecture deep dive — the architect-level documentation, surfaced as navigable site pages |
| C++ public API — every class, method, field | C++ API Reference — generated from Doxygen comments |
| Python public API — pyneat module surface | Python API Reference — generated from nanobind bindings |
| Conceptual deep-dives (dtype contract, memory model, error codes, etc.) | dtype contract, memory model, graphs, timing model, threading, processor backends, GStreamer underneath, CVU kernels, MPK contract, error codes, build options, agentic workflow |
| Glossary, environment variables, scripts, error format | Glossary, env vars, scripts inventory, plugin error format |
| Onboarding, build, minimal example | Installation, Build, Minimal example |
| How to do specific things (debugging, runtime tuning, plugin failures) | How-to: runtime tuning, How-to: diagnostics, How-to: plugin failures |
| Coding standards, MPK contract, contribution policy | Coding standard, MPK contract, Architecture |
Why the framework is built for agents
A consequence of how the framework was designed: it's an exceptionally good substrate for AI code generation. Every framework error produces a structured SessionReport (machine-readable error code + reproducer command). Every public symbol has stable naming. Validation runs before any pipeline starts. Pipelines are serializable JSON. The public API is ABI-stable. Errors fail fast with actionable messages instead of silent fallbacks.
These properties weren't picked for AI agents specifically — they came from "make the framework deterministic, debuggable, and never hang the process" (see docs/contribute/architecture.md). But they happen to be exactly what an AI agent needs to write code that converges quickly. That's why the agentic workflow in the Neat environment delivers what it does — the framework is the substrate, the environment is the workshop.
The full story of why the framework is good for agents — fifteen specific design properties, each tied to architect-doc rationale — is in the Architecture deep dive.
Conventions in this reference
- All public types live under the simaai::neat namespace.
- Nodes live under simaai::neat::nodes::{common,io,rtp,sima,groups}.
- Headers are organized by responsibility: model/, pipeline/, mpk/, builder/, graph/, nodes/, contracts/, policy/, gst/, plus convenience umbrellas under neat/.
- Every public header has a file-level @file / @ingroup / @brief block. Group definitions live in docs/doxygen/groups.dox.
- Errors are returned as SessionError exceptions carrying a structured SessionReport. See the Error code catalog for the full taxonomy.
The rest of this site is the API itself. Start by browsing classes, or jump to the headline types: Model, Session, Run, Tensor, Node.
Generated via doxygen2docusaurus 2.0.0 by Doxygen 1.9.8.