Diagnose and Profile a Pipeline

Field	Value
Difficulty	Intermediate
Estimated Read Time	<10 minutes
Labels	diagnostics, debugging, observability

Concept

Run three checks — session.validate(), one metrics-enabled run.run(), and run.stats() + run.report() — to answer whether a pipeline is wired correctly and how it is performing. This is the triage baseline before deep debugging.

The three checks answer three questions:

1. Is the session contract/build valid? (validate())
2. Does one run succeed with metrics enabled? (build(..., RunOptions(enable_metrics=True)) + run.run())
3. What do runtime diagnostics report? (run.stats(), run.report(), run.diagnostics_summary())

Especially useful when onboarding new models or environments — repeatable, fast, and catches most misconfiguration before it becomes a multi-hour debugging session.

APIs introduced

• session.validate() — contract-level check, returns a report with error_code.
• pyneat.RunOptions() with enable_metrics=True and output_memory=OutputMemory.Owned.
• run.stats() — inputs_enqueued, outputs_pulled, avg/min/max_latency_ms.
• run.report(), run.diagnostics_summary() — structured runtime diagnostics.

Prerequisites Chapter 002 or 003 (Session/Run basics).

References

• Session
• Pipeline

Learning Process

1. Validate session contract and backend parse path (validate()).
2. Run one deterministic frame with metrics enabled.
3. Inspect runtime stats/report/diagnostic summary outputs.

Run

Python:

python3 share/sima-neat/tutorials/011_diagnose_a_pipeline/diagnose_a_pipeline.py

C++ (prebuilt):

./lib/sima-neat/tutorials/tutorial_011_diagnose_a_pipeline

C++ (build from source):

./build.sh --target tutorial_011_diagnose_a_pipeline
./build/tutorials-standalone/tutorial_011_diagnose_a_pipeline

To integrate this chapter's C++ source into your own project with a custom CMakeLists.txt (no extras folder required), see How to Run Tutorials on the landing page.

Code

tutorials/011_diagnose_a_pipeline/diagnose_a_pipeline.cpp

// Three diagnostic commands: Session::validate, Run::stats, Run::report / diagnostics_summary.
//
// Usage:
//   tutorial_011_diagnose_a_pipeline

#include "neat.h"

#include <opencv2/core.hpp>

#include <iostream>
#include <stdexcept>

int main() {
  try {
    cv::Mat rgb(96, 128, CV_8UC3, cv::Scalar(22, 44, 66));
    if (!rgb.isContinuous())
      rgb = rgb.clone();

    simaai::neat::Session session;
    simaai::neat::InputOptions in;
    in.format = "RGB";
    in.width = rgb.cols;
    in.height = rgb.rows;
    in.depth = rgb.channels();
    session.add(simaai::neat::nodes::Input(in));
    session.add(simaai::neat::nodes::Output());

    // CORE LOGIC
    // 1) validate() checks the Session before build() and prints any caps problems.
    auto report = session.validate();
    std::cout << "validate.error_code=" << report.error_code << "\n";

    // 2) Run the Session with metrics enabled so stats() has data.
    simaai::neat::RunOptions run_opt;
    run_opt.enable_metrics = true;
    run_opt.output_memory = simaai::neat::OutputMemory::Owned;
    auto run = session.build(std::vector<cv::Mat>{rgb}, simaai::neat::RunMode::Sync, run_opt);
    simaai::neat::TensorList out = run.run(std::vector<cv::Mat>{rgb}, /*timeout_ms=*/1000);
    if (out.empty())
      throw std::runtime_error("missing output tensor");

    // 3) Post-run diagnostics: counters, per-element report, and a summary string.
    auto stats = run.stats();
    std::cout << "stats.inputs_enqueued=" << stats.inputs_enqueued
              << " outputs_pulled=" << stats.outputs_pulled << "\n";
    std::cout << "report.size=" << run.report().size() << "\n";
    std::cout << "diagnostics_summary=" << run.diagnostics_summary() << "\n";

    std::cout << "[OK] 011_diagnose_a_pipeline\n";
    return 0;
  } catch (const std::exception& e) {
    std::cerr << "[FAIL] " << e.what() << "\n";
    return 1;
  }
}

Source

• C++
• Python
• README