Skip to content

feat(engine): temporal validation of alerts on the Pi - #415

Open
MateoLostanlen wants to merge 17 commits into
developfrom
feat/temporal-validator
Open

MateoLostanlen wants to merge 17 commits into
developfrom
feat/temporal-validator

Conversation

@MateoLostanlen

@MateoLostanlen MateoLostanlen commented Sep 8, 2026 •

Copy link
Copy Markdown
Member
  • While an alert is ongoing, the engine scores the last 11 frames of the camera pose with the temporal smoke model every round (sends them with their boxes to pyro_temporal_api, reads the verdict on the next round) and uploads each box with the probability of the tube covering it (temporal_model_scores, feat: accept the temporal score computed on the device pyro-api#723, which applies its own threshold instead of scoring the sequence). Boxes below 0.1 or outside every scored tube are dropped; only the frames a verdict scored are uploaded on it, and frames of an ended event the model rejected are dropped for good. A service error, a lost job or a job pending longer than 120 s fails open: the frames go out without scores and the API scores them itself. An event that ends while its job is still pending is kept until the verdict is in, so the last alert of an event is never dropped.
  • The service runs the torch-free ONNX runtime from temporal-model (feat: ONNX export and torch-free runtime for the tube classifier temporal-model#76, merged, pinned to commit 7c1500e until a release ships model_onnx.zip), behind the compose profile temporal. Unset TEMPORAL_API_URL keeps today's behaviour. make run-no-temporal starts the stack without it whatever .env says.
  • The service binds to loopback and bounds what it accepts: at most 11 frames of 4 MB per job, 50 boxes per frame, 16 jobs pending, 100 results kept. Boxes must be finite, normalized xyxy; frames must be valid images with unique ids. /health returns 503 when the scoring worker died or is stuck.
  • model_onnx.zip must be exported from the merged code: the graph now has a dynamic batch axis and older exports fail with more than one tube. A format 2 package (split backbone/head) lets the service cache ViT embeddings across jobs.
  • Engine and camera API images move to bookworm since bullseye's security repo no longer resolves.
  • Temporal API tests run in CI and make test.

Tested on a Pi 5 by replaying 8 dataset sequences (5 smokes, 3 non-smoke) through the real engine and the service, one per camera pose, with the format 2 0.6.0 package. Every smoke was held one round and uploaded on the next after a positive verdict, the 3 non-smoke sequences never reached the gate, 0 errors. Scoring takes about 9 s per 10-frame job on 4 cores, the service uses about 530 MB of RSS and the host peaks at 2.4 GB of 4 GB with the engine running.

…X temporal model

POST /jobs takes a window of frames plus YOLO boxes for one camera pose
and returns a job id at once; a worker thread scores it with
OnnxTemporalModel and GET /jobs/{id} serves the verdict. No torch.
With TEMPORAL_API_URL set, the engine keeps the last 10 inference frames
and boxes per camera pose, submits them when an alert becomes ongoing and
reads the verdict on the next round. Negative verdicts resubmit the
longer window; service errors fail open.
bullseye LTS ended in August 2026: its security repo now serves an
expired release file and apt-get update fails in every build.
…ge, compose profile

Per-frame size limit and pending-job cap on POST /jobs, service bound to
127.0.0.1 on the host network, image added to the build-and-push matrix
so make run can pull it, and the service placed behind the temporal
compose profile so disabled stacks do not start it.
@codecov

codecov Bot commented Sep 8, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.10345% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.85%. Comparing base (4ba8abb) to head (3c70979).

Files with missing lines Patch % Lines
pyroengine/temporal_client.py 57.89% 8 Missing ⚠️
pyroengine/engine.py 98.41% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #415      +/-   ##
===========================================
+ Coverage    80.51%   82.85%   +2.33%     
===========================================
  Files            6        7       +1     
  Lines          847      968     +121     
===========================================
+ Hits           682      802     +120     
- Misses         165      166       +1     
Flag Coverage Δ
unittests 82.85% <93.10%> (+2.33%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@MateoLostanlen MateoLostanlen changed the title feat: temporal validation of alerts through a sidecar service feat: temporal validation of alerts Sep 12, 2026
@MateoLostanlen MateoLostanlen changed the title feat: temporal validation of alerts feat: temporal validation of alerts on pi Sep 12, 2026
An event ending while the temporal job was pending was dropped by _end_event,
so a positive verdict never uploaded the frames held during scoring. The state
now survives until the verdict: positive stages the held frames, negative drops
the event without resubmitting.
@MateoLostanlen MateoLostanlen changed the title feat: temporal validation of alerts on pi feat(engine): temporal validation of alerts on the Pi Sep 14, 2026
# Conflicts:
#	pyro_camera_api/Dockerfile
…split export

Pins temporal-model-core to 7c1500e (feat/onnx-embedding-cache): the ONNX
package is now backbone + head, the runtime embeds only real slots and keeps
an LRU of per-slot embeddings keyed by (frame id, crop box). The engine
resubmits the same window one frame longer each round, so a job now mostly
pays the ViT for the new frame. The verdict reports embeddings_cached /
embeddings_new; a fused (format 1) package still loads with a warning.

README points at the model 0.6.0 release (pyro-dataset v4.7.0), to be
re-exported with temporal-export-onnx for the split graphs.
- clear the job on every fail-open path so a lost job cannot keep an event alive
- fail open when a job stays pending longer than temporal_job_timeout (120 s)
- submit the engine's own prediction window (widened to 10 frames) instead of a second buffer
- service: at most 10 frames per job, box and JPEG validation, duplicate frame ids rejected
- service: /health is 503 when the worker died or is stuck, results published under the lock
- 5 s client timeout, temporal API tests in CI and make test, make run-no-temporal
@MateoLostanlen
MateoLostanlen requested a review from fe51 October 22, 2026 12:59
…hreshold

Matches pyro-api PR 723 (temporal_model_scores on POST /detections):
- the service verdict lists each scored tube with its union box and probability
- the engine scores the window every round while an alert is ongoing, uploads the boxes
  covered by a tube above temporal_min_score (0.1) with that probability, drops the others
- fail open (service down, lost or stuck job) sends the frames without scores so the API
  scores the sequence itself
- pyroclient pinned to the PR 723 commit (repoint to main once merged), requirements
  regenerated with uv 0.13.4
…frames for good

- a job remembers the frame ids it submitted; its verdict stages those frames only, frames
  captured afterwards wait for the next verdict
- a verdict rejecting an ended event marks the held frames as consumed so a later event cannot
  upload them under its own score
- window of 11 frames, all submitted (service MAX_FRAMES=11)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In progress

Development

Successfully merging this pull request may close these issues.

2 participants