An all-sky camera can watch through the night. The interesting engineering problem is teaching the software which parts of that night deserve a closer look.
I have been using astroplate-analysis with real captures from my AllSky setup, and it works in practice. This is a catch-up on that research: how the processing grew from basic streak detection into a pipeline for measuring frames, selecting activity windows, and experimenting with machine learning. The eventual goal is analysis that can run alongside AllSky as images arrive.
This is a progress report on a working research project, with the next stage still open.
Explore astroplate-analysis on GitHub · Browse the development history
So, what is AllSky?
An all-sky camera watches a broad view of the sky from a fixed position, recording changing weather, stars and passing activity over time. Allsky is an open-source, Raspberry Pi-based camera software project that automates capture and provides a web interface for monitoring and reviewing the results. It is the capture side of this experiment: my real AllSky images become the input to astroplate-analysis.
For an introduction and setup information, visit the official Allsky documentation and Allsky source repository. Astroplate-analysis is my separate research pipeline for measuring those captures and concentrating attention on useful activity windows.
⚙ Engineering note / the camera controls the evidence
Exposure duration, capture cadence, gain, lens distortion and foreground obstructions all affect what an analysis can see. A streak can occupy one exposure or several; a brightness change can come from exposure adjustment as well as the sky. Keeping capture settings and timestamps alongside the images helps interpret candidates and compare nights.
From an overnight image collection to a useful question
Capturing images is only the beginning. A night can contain quiet sky, changing cloud, exposure shifts, foreground lighting, aircraft and other moving streaks. Looking through the entire sequence is possible, but the software can do useful preparatory work: measure what changes, collect candidate events, and preserve enough surrounding frames to make those candidates understandable.
In this project, astroplates are the captured sky-image frames and their sequences. The code under review concentrates on image activity and transient candidates. Its practical value today is organizing attention: deciding where to look next in a night of observations.
The capture-to-review workflow
Start with a single night in data/night_YYYY-MM-DD/frames/. Either download it with python fetch.py YYYYMMDD from tools/data_fetch/, pointing the fetcher at your AllSky image server, or copy your captures into that folder. The analyzer reads *.jpg files in filename order, so timestamped names or zero-padded sequence numbers preserve the observation sequence.

For a first pass, run python analyze.py ../data/night_YYYY-MM-DD/ --all-tools from analysis_simple/. This uses interest windows by default and requests plots, overlays, videos and window artifacts. Inspect the saved masks before interpreting candidate streaks: the mask defines the part of the scene the detector can examine.
The optional ML route starts with the basic metrics and interest windows: run train.py in analysis_ml_activity_classifier/, then infer_windows.py in analysis_ml_windows/, passing the same night directory to each. Rerun the analyzer with --windows-source ml --all-tools or --windows-source hybrid --all-tools to render the selected windows through the common review path.
What gets drawn on an image of interest

The overlay tool reads data/events.json, opens each referenced capture, draws its stored [x1, y1, x2, y2] line segments, and adds Streaks: N. It saves a new image with annotated_ prepended to the original filename. These are candidate segments for visual review; several segments can belong to the same physical trail.
One implementation detail matters when reading that count: the analyzer stores up to ten segments per event, and the overlay counts those stored segments. The per-frame streak_count in metrics.csv can therefore be larger. The image overlay and the ML activity probability are separate outputs; the probability lives in the ML prediction files.
The current command-line overlay default produces blue markings because of its RGB/BGR conversion. For red, explicitly run python tools/overlay_streaks.py ../data/night_YYYY-MM-DD/ --color 255,0,0 from analysis_simple/.
Where to find the exports
| What to find | Inside the night folder | What it shows |
|---|---|---|
| Original captures | frames/*.jpg | Imported or downloaded frames, sorted by filename. |
| Analysis region | masks/sky_mask.png; masks/persistent_edges.png; masks/combined_mask.png | Check which pixels were retained or excluded. |
| Frame measurements | data/metrics.csv | Brightness, contrast, focus, streak count and interest score. |
| Candidate coordinates | data/events.json | Source filename and stored line-segment endpoints. |
| Marked images | annotated/annotated_<original filename> | For example, annotated_frame_0123.jpg. |
| Focused review | activity/window_XX_START_END/ | Window timelapse, keogram and startrails. |
| Full-night review | timelapse/ | timelapse.mp4 and, when available, timelapse_annotated.mp4. |
If you are searching an older analysis computer, look first for annotated_*.jpg, combined_mask.png, keogram.png and startrails.png. These are generated image files rather than screenshots. A selected activity window may span many frames; annotated stills are created from the event records, so the two collections need not contain the same frames.
Implementation references: analysis and artifact generation · image overlay and filename rules · capture download options.
How the project developed
| When | Milestone | What changed | History |
|---|---|---|---|
| Dec. 25, 2025 | First image-analysis scripts | Sky masking, contrast statistics, streak candidates and structured output. | Initial analysis |
| Dec. 25–26 | Repeatable acquisition and tools | Download retries, configuration, visualization and input validation. | Data fetching |
| Dec. 26 | Masking experiments | Multi-frame percentiles, persistent edges, dimension checks and clearer unreadable-frame reporting. | Multi-frame mask |
| Dec. 27 | Reviewable activity | Annotated timelapses and interest-based windows with focused review artifacts. | Activity windows |
| Dec. 28–29 | Learning activity | Initial classifier work, ML windows, then peak-seeded probability windowing. | Initial ML work |
| Dec. 30 | Unified review pipeline | Merged interest, ML and hybrid source selection with consistent artifact output. | Window-source integration |
The sequence matters. Acquisition and masking came before the learning experiment. Review artifacts came before the unified window-source interface. Each step made the next experiment easier to inspect and repeat.
First, decide which pixels belong to the sky
The early mask used a single image’s brightness to separate usable sky from obstructions. The history then records a move to sampling multiple frames across a night. A low-percentile image helps identify persistently dark foreground features. The current code also considers bright regions, removes an overlay region, cleans the mask with morphological operations, and expands excluded boundaries to reduce leakage.
A second mask looks for edges that persist across sampled frames. A roofline, wire or tree branch can look like a candidate line in one image; repeatedly finding it at the same pixels is a reason to exclude that structure from streak detection. The combined mask gives the detector a more useful region to examine.
⚙ Engineering note / the scene is part of the detector
Masking is a research choice with consequences. Aggressive exclusion can remove usable sky, while permissive masking lets foreground structures generate candidates. Saving the masks makes that tradeoff visible. The history also records fixes for inconsistent image dimensions and unreadable frames, because a trustworthy analysis begins with knowing which inputs were actually processed.
Measure the night before trying to explain it
The basic pipeline uses OpenCV, with Canny edges and Hough line detection for streak candidates. Alongside those candidates it records brightness, star contrast, focus and streak counts. CSV metrics and JSON event records turn a folder of pictures into a sequence that can be plotted, compared and used for feature engineering.
The interest score combines streak information with changes in brightness, contrast and focus. The documented weights emphasize streaks at 0.60, with brightness and contrast changes at 0.15 each and focus change at 0.10. Robust normalization based on the median and median absolute deviation makes these quantities easier to compare within a sequence.
This gives the first useful review mechanism: contiguous activity windows. Thresholds, minimum length, padding and merging turn individual interesting frames into intervals with context. Padding matters because the frames just before and after a candidate often explain more than its brightest instant.
Make the output something a person can inspect
The pipeline can generate annotated frames, full-night timelapses and per-window timelapses. Keograms condense a sampled image column through time; startrail images use a maximum projection to expose accumulated tracks. Together with plots and the original frames, these provide several ways to inspect the same interval.
The video history includes a practical cleanup: consolidating the timelapse builder, reporting unreadable frames instead of silently skipping them, and checking that an annotated directory contains actual images. Those changes make a research result easier to interpret when part of the input is missing or malformed. Read the timelapse review and fixes.
The first ML experiment: learn what activity looks like
The implemented learning model is a lightweight logistic regression classifier trained with NumPy. It consumes eight features: normalized brightness, contrast, focus, streak count and interest score, plus normalized absolute changes in brightness, contrast and focus. It predicts a frame-level activity probability.
Its first labels come from the rule-based activity windows: frames inside a selected window are positive and frames outside are negative. This bootstraps an end-to-end learning workflow without requiring every frame to be manually labeled first. The training tool writes raw and smoothed predictions, a report with precision, recall, accuracy and confusion counts, feature weights, and a ranked list of frames.
🔧 Engineering note / what the model learns first
The initial target is agreement with the activity-window labels. Because the labels and several input features come from the same rule-based pipeline, validation against those labels measures how well the model reproduces that activity definition. Independently reviewed events and held-out nights are the next step for measuring astronomical detection performance.
There is a useful distinction between the broader identification goal and this first model. The classifier currently separates activity from background. Event-type identification, such as deciding whether a candidate is a meteor, satellite or aircraft, needs additional labels, temporal evidence and evaluation.
A probability curve becomes a review window
The classifier’s common module implements peak-seeded windowing. It smooths the probability sequence with an exponential moving average, finds local peaks above height and prominence thresholds, expands around each peak while the signal stays above a lower tail threshold, and merges neighboring intervals. Windows can be ranked by accumulated probability across the interval.
This gives a different way to organize review from a single threshold on the original interest score. A peak identifies the center of an interval, while the lower tail threshold preserves surrounding evidence. The settings expose a practical balance between keeping brief candidates and producing too many noisy review intervals.
The repository also contains a separate ML-window inference tool with configurable smoothing and thresholding. These are related experimental paths, and their documentation should be read alongside the implementation when reproducing a particular run.
Interest, ML, and hybrid in one pipeline
The latest merged milestone makes window selection explicit with --windows-source interest, --windows-source ml and --windows-source hybrid. The selected source drives the downstream artifacts in the common activity/ output location.
Hybrid selection begins with ML windows, adds interest windows whose overlap with them is below a configured threshold, and merges the resulting intervals. The default overlap threshold is 0.3 using temporal intersection over union. This retains a route for rule-based candidates that the model’s selected windows do not already cover.
The implementation also handles prerequisites: ML-only mode reports missing ML windows, while hybrid mode can fall back to interest windows. A common interface makes it easier to compare approaches without rebuilding the rest of the review workflow.
What has been exercised, and what the history preserves
The pipeline has been used successfully on real AllSky captures in my own testing. The repository preserves the code and the sequence of engineering experiments behind that work: single-frame versus multi-frame masks, persistent-edge exclusion, image-read and dimension handling, activity scoring, video generation, classifier training, probability windowing and hybrid selection.
The major tests for the next research pass follow directly from these experiments: compare masks on the same real capture sequence; compare interest, ML and hybrid candidates against human-reviewed intervals; inspect brief and intermittent candidates; and measure failures as well as detections. Recording false alarms per night and missed reviewed events would make threshold changes much easier to judge.
From overnight batches toward live AllSky analysis
The present structure processes a night as a batch. That is a useful place to develop the analysis because the full sequence is available for building masks, normalizing features and inspecting events. A live implementation has to make those decisions using only the frames received so far.
My proposed next stage is a persistent worker that accepts completed AllSky frames, updates a bounded history, computes features, scores activity and maintains open event windows. It would preserve a short pre-event buffer, append evidence while activity continues, and finalize an event after a configurable quiet period. Review images and notifications could follow from that event record.
🌐 Engineering note / real time changes the experiment
The current code builds feature normalization before splitting the night into training and validation frames, and its split shuffles frame indices. For stronger evaluation, fit preprocessing on training data and hold out entire nights or contiguous event groups. For live inference, persist the trained model and preprocessing parameters, use a past-only mask or calibrated mask, and measure processing latency against the camera’s capture interval.
Other practical work includes an explicit model-loading path, restart recovery, stable timestamps, protection against partially written image files, and recording the configuration and model version with every event. Peak confirmation and event closure also introduce delay, so the live system should distinguish a provisional alert from a completed event.
Where this leaves the research
There is already a useful connection between a real camera and an inspectable analysis process. The code can turn overnight imagery into metrics, candidates and concentrated review windows, and there is an implemented learning experiment that can be compared with the original rules.
The next ambition is to carry that usefulness into the night itself: an AllSky companion that notices changes as they happen, retains the evidence, and gradually gets better at identifying what crossed the camera’s view. This repository is the working research trail toward that goal.
Source snapshot: main at 912b668 · Classifier training · Features and peak windowing · Window-source architecture
