Build a Production-Ready Pipeline

Field	Value
Difficulty	Advanced
Estimated Read Time	20-25 minutes
Labels	production, reliability, deployment

Concept

Assemble a production-style run loop from the patterns earlier chapters taught: explicit ModelOptions, explicit ModelSessionOptions, explicit RunOptions, and one async push/pull loop. Not a full product framework — a reliable skeleton you can adapt.

The template makes three things explicit that defaults leave implicit:

• Input bounds on ModelOptions (input_max_width/height/depth) so contract failures surface at build time, not runtime.
• Stage naming (name_suffix) so diagnostics in a multi-model system stay readable.
• Queue policy on RunOptions (queue_depth, overflow_policy=Block, metrics on) so pipeline behavior under load is observable.

APIs introduced

• pyneat.ModelOptions() + pyneat.ModelSessionOptions() + pyneat.RunOptions() as a composed unit.
• model.build(tensor, session_options, run_options) — one-call path from Model to Run.
• runner.push(tensor) returning bool + runner.pull(timeout_ms) + runner.close() — the production loop.

When to use this

• Adapting an earlier tutorial into real deployment code.
• Establishing a consistent runtime skeleton across multiple models in the same app.
• As a starting point for apps/examples repo integrations.

Prerequisites Chapters 002 (async), 004 (ModelOptions), 007 (ModelSessionOptions), 015 (RunOptions and queues).

References

• Model
• Session
• Pipeline

Learning Process

1. Prepare deterministic input and shared runtime options for production-like behavior.
2. Execute model-backed blueprint when MPK exists.
3. Execute session fallback blueprint when model assets are unavailable.

Run

Python:

python3 share/sima-neat/tutorials/016_build_production_pipeline/build_production_pipeline.py \
  --mpk /tmp/resnet_50_mpk.tar.gz --iters 4

C++ (prebuilt):

./lib/sima-neat/tutorials/tutorial_016_build_production_pipeline \
  --mpk /tmp/resnet_50_mpk.tar.gz --iters 4

C++ (build from source):

./build.sh --target tutorial_016_build_production_pipeline
./build/tutorials-standalone/tutorial_016_build_production_pipeline \
  --mpk /tmp/resnet_50_mpk.tar.gz --iters 4

To integrate this chapter's C++ source into your own project with a custom CMakeLists.txt (no extras folder required), see How to Run Tutorials on the landing page.

Code

tutorials/016_build_production_pipeline/build_production_pipeline.cpp

// Production blueprint: wrap a Model in a Runner with production-grade RunOptions.
//
// Usage:
//   tutorial_016_build_production_pipeline --mpk /path/to/resnet_50.tar.gz [--iters 4]

#include "neat.h"

#include <opencv2/core.hpp>

#include <iostream>
#include <stdexcept>
#include <string>

namespace {

bool get_arg(int argc, char** argv, const std::string& key, std::string& out) {
  for (int i = 1; i + 1 < argc; ++i) {
    if (key == argv[i]) {
      out = argv[i + 1];
      return true;
    }
  }
  return false;
}

int parse_int_arg(int argc, char** argv, const std::string& key, int def) {
  std::string value;
  if (!get_arg(argc, argv, key, value))
    return def;
  return std::stoi(value);
}

} // namespace

int main(int argc, char** argv) {
  try {
    std::string mpk;
    if (!get_arg(argc, argv, "--mpk", mpk)) {
      std::cerr << "Usage: tutorial_016_build_production_pipeline --mpk <path> [--iters <n>]\n";
      return 1;
    }
    const int iters = parse_int_arg(argc, argv, "--iters", 4);

    cv::Mat rgb(224, 224, CV_8UC3, cv::Scalar(16, 96, 196));
    if (!rgb.isContinuous())
      rgb = rgb.clone();

    // CORE LOGIC
    // Production defaults: bounded queue, blocking overflow, owned output memory,
    // metrics on. Model::build returns a Runner that owns the async pipeline.
    simaai::neat::RunOptions run_opt;
    run_opt.queue_depth = 8;
    run_opt.overflow_policy = simaai::neat::OverflowPolicy::Block;
    run_opt.output_memory = simaai::neat::OutputMemory::Owned;
    run_opt.enable_metrics = true;

    simaai::neat::Model::Options model_opt;
    model_opt.preprocess.kind = simaai::neat::InputKind::Image;
    model_opt.preprocess.enable = simaai::neat::AutoFlag::On;
    model_opt.preprocess.color_convert.input_format = simaai::neat::PreprocessColorFormat::RGB;
    model_opt.preprocess.input_max_width = rgb.cols;
    model_opt.preprocess.input_max_height = rgb.rows;
    model_opt.preprocess.input_max_depth = rgb.channels();
    model_opt.preprocess.normalize.enable = simaai::neat::AutoFlag::On;
    model_opt.preprocess.normalize.mean = {0.485f, 0.456f, 0.406f};
    model_opt.preprocess.normalize.stddev = {0.229f, 0.224f, 0.225f};
    model_opt.preprocess.normalize.has_explicit_stats = true;
    model_opt.name_suffix = "_prod";

    simaai::neat::Model model(mpk, model_opt);

    simaai::neat::Model::SessionOptions sess_opt;
    sess_opt.include_appsrc = true;
    sess_opt.include_appsink = true;
    sess_opt.name_suffix = "_prod";

    auto runner = model.build(
        simaai::neat::TensorList{simaai::neat::Tensor::from_cv_mat(
            rgb, simaai::neat::ImageSpec::PixelFormat::RGB, simaai::neat::TensorMemory::EV74)},
        sess_opt, run_opt);

    int ok = 0;
    for (int i = 0; i < iters; ++i) {
      if (!runner.push(simaai::neat::TensorList{simaai::neat::Tensor::from_cv_mat(
              rgb, simaai::neat::ImageSpec::PixelFormat::RGB, simaai::neat::TensorMemory::EV74)}))
        continue;
      auto out = runner.pull(/*timeout_ms=*/2000);
      if (!out.empty())
        ++ok;
    }
    runner.close();

    std::cout << "outputs=" << ok << "\n";
    std::cout << "[OK] 016_build_production_pipeline\n";
    return 0;
  } catch (const std::exception& e) {
    std::cerr << "[FAIL] " << e.what() << "\n";
    return 1;
  }
}

Source

• C++
• Python
• README