AI Computer Vision: Profiling

AI-first map for profiling CV and script performance with MiniProfiler.

Profiling / Performance Discovery Map

Reference map for measuring script and computer-vision performance with MiniProfiler-backed APIs, CV profiling helpers, manual steps, and profiler output.

For broader profiler selection, startup/app-level MiniProfiler usage, and JetBrains dotTrace/dotMemory capture workflows, start with core/profiling.md.

User Intents

  • Find why a CV loop is slow.
  • Measure image search, OCR, color search, ML search, or pixel processing.
  • Compare two implementations.
  • Add timings around custom script code.
  • Render profiling data as text or custom UI.
  • Avoid unbounded profiling memory growth.

Concept Model

  • EyeAuras uses MiniProfiler-style hierarchical timings.
  • ICvAccessor has a Profiler property.
  • EnableProfiling() enables timing for one CV accessor instance.
  • CV operations create internal profiler steps only when profiling is enabled.
  • Custom code can use Profiler.Step("name") around measured blocks.
  • Profiling data is held in memory and should be bounded by scenario/lifetime.
  • First calls often include initialization and are slower than steady-state calls.

API Details

  • ICvAccessor.Profiler - MiniProfiler instance associated with the accessor.
  • ICvAccessor.EnableProfiling() - enables internal CV timings for this accessor.
  • MiniProfiler.Step(...) - creates a timed child step.
  • MiniProfiler.RenderPlainText() - text report for logs.
  • IProfilerProvider - shared profiler provider for app/global scenarios.
  • ProfilerProvider.Default / ProfilerProvider.Startup - common app-level profiler entry points.
  • CV internal step names usually include operation, key, and phase: Renting, Preparations, Refresh, Resolution, OSD.

Report three CV calls

Run this top-level C# Script in EyeAuras with the OpenCV module and primary-screen capture available. Counting red pixels requires no template file, OCR engine or model:

csharp
using System.Drawing;
using EyeAuras.OpenCVAuras.Scripting;
using EyeAuras.Shared.Scaffolding;
using StackExchange.Profiling;

{
    var api = GetService<IComputerVisionExperimentalScriptingApi>();
    using var cv = api.ForScreen().EnableProfiling();
    for (var i = 0; i < 3; i++)
    {
        cancellationToken.ThrowIfCancellationRequested();
        using (cv.Profiler.Step($"Pass {i}"))
        {
            var count = cv.CountPixels(Color.Red);
            Log.Info($"Red pixels: {count}");
        }
    }
    Log.Info(cv.Profiler.RenderPlainText());
}

The report appears after the using steps close, alongside the pixel counts. The first pass may include capture preparation. This is one accessor's timing tree, not a statistically reliable comparison. Logging inside a step contributes to its time. RenderPlainText() formats the report.

The token check prevents the next pass; it does not interrupt synchronous CountPixels, which accepts no token. The accessor leases the shared CV cache and the profiler accumulates a timing tree. Bound the run. DisableProfiling() stops new internal steps without promising to remove old ones. Clear() from EyeAuras.Shared.Scaffolding clears MemoryCacheStorage, not necessarily the profiler's timing tree.

Common Flows

Profile CV Loop

  1. Create a CV accessor for the target window/source.
  2. Call EnableProfiling() on that accessor.
  3. Wrap each loop iteration in Profiler.Step("Loop #...").
  4. Let CV calls add their own internal child steps.
  5. Render or inspect Profiler output after a bounded run.

Profile Custom Work

  1. Choose an existing profiler or create/resolve a shared one.
  2. Wrap expensive blocks in using (profiler.Step("Meaningful step name")).
  3. Keep step names stable enough for aggregation.
  4. Compare warm-up and steady-state timings separately.

Compare Implementations

  1. Run one warm-up pass if the API/model/image pipeline initializes lazily.
  2. Run candidate implementations under named profiler steps.
  3. Compare average/min/max/percentile data, not only one sample.
  4. Check memory allocations and data movement when CPU timings look surprising.

Performance Notes

  • First CV/ML calls may include model creation, capture initialization, image allocation, or OCR engine setup.
  • Direct memory/span access can be much faster than per-pixel accessor APIs.
  • OSD/debug drawing is useful but should be measured separately from detection.
  • Profiling every iteration forever can become expensive.
  • Profilers store timing trees in memory; long-running mini-apps should bound retention or periodically reset/report.

Prefer

  • Prefer EnableProfiling() before writing custom timers for CV APIs.
  • Prefer stable step names for aggregating repeated loop timings.
  • Prefer measuring warm-up separately from steady-state loops.
  • Prefer RenderPlainText() for quick logs and custom renderers for dashboards.
  • Prefer targeted profiling around suspected slow blocks.

Avoid

  • Avoid profiling unbounded loops indefinitely.
  • Avoid comparing first-call initialization with steady-state work.
  • Avoid measuring input/mouse/OSD time as part of pure CV detection time unless that is intentional.
  • Avoid one-off timing conclusions from a single iteration.
  • Avoid expensive logging inside tight loops when measuring performance.

Research Anchors

  • ICvAccessor
  • EnableProfiling
  • MiniProfiler
  • MiniProfiler.Step
  • RenderPlainText
  • IProfilerProvider
  • ProfilerProvider
  • ImageSearchRaw
  • ColorCheckRaw
  • PixelSearchRaw
  • TextSearchRaw
  • CountPixels
  • ProcessPixels
  • MLSearchRaw

Search Synonyms

  • profiling
  • performance
  • MiniProfiler
  • profiler step
  • CV profiler
  • render plain text
  • image search timing
  • pixel processing performance
  • first call warmup
  • steady state
  • percentile
  • profiling memory