Repository navigation
feat(import): temporal-filtered, day-by-day alert API import - #398
Merged
Merged
Conversation
…mport Run the released pyronear/temporal-model on each fetched sequence and keep only those scoring above --min-temporal-score (make default 0.1). Torch deps live in an opt-in 'temporal' uv group.
Known URLs keep their source tag; any other URL must pass --source-api so foreign data is never tagged as French. Image transfer defaults to url for every alert API except the French one. Makefile exposes SOURCE_API and TEMPORAL_MODEL_VERSION.
Each day is fetched, scored, imported and annotated before the next one, so nothing is downloaded ahead. A failed day is reported and the run moves on, exiting 1 at the end; Ctrl+C stops the range. --max-sequences is now per day. The temporal model is loaded once for the whole range.
…ty days Stop slicing --sequence-list once for the whole range (the per-day fetch already caps it), skip temporal model loading when a day has no records, and document that sibling dedup does not cross day boundaries.
Lower the --frames-limit default from 30 to 20 and expose it as FRAMES_LIMIT in the Makefile.
The alert_api modules import app.core.config, which reads POSTGRES_URL and S3_* at import time, before import.py called load_dotenv(). A fresh setup with only annotation_api/.env crashed with KeyError.
# Conflicts: # annotation_api/Makefile
Move the tutorial next to the ingestion guide and make it alert-API agnostic. Document the day-by-day loop, the temporal filter and the new flags in data-ingestion-guide.md.
app.core.config requires POSTGRES_URL and S3_* at import time, but the ingestion and export scripts only use HTTP. Default them to a placeholder so a fresh .env with only API credentials works.
A download or model error dropped the sequence like a below-threshold one and the day still exited 0, so unscored sequences were silently lost. Count them, report them and mark the day failed so it gets re-run.
nan or a value above 1 imported nothing while reporting success; a negative value let every sequence through.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
pyronear/temporal-modelon the first 20 images of each alert sequence (FRAMES_LIMIT) and import only sequences scoring above a threshold (default 0.1), so annotators only see likely smoke.ALERT_API_URL,SOURCE_API,REMOTE_API,TEMPORAL_MODEL_VERSION), so Chile can run the same script from their alert API into a local annotation API.temporaluv group; Docker images are unchanged.