Home / Research / Scout / Code
The released codebase behind the Scout experiments: what is in it, how the acquisition loop maps onto modules, and the rules the code enforces on itself rather than merely documenting.
The repository holds the experiments the paper reports and the evidence behind them. The scout/ package is 36 modules: a typed set of contracts and the acquisition environment; the learned policy with its 19-dimensional node features, alongside six heuristics, three probing protocols from prior work and two capacity-ablated scorers; five dynamic-network benchmarks and two downstream tasks; and the runners, trainer, table builders and figure builder that carry a run through to a reported number.
Next to it, outputs/ holds the evidence itself: the 16 checkpoints the reported numbers were produced with, the per-step result caches the figure and tables read, and 1,394 run cards recording the configuration, package versions and graph fingerprint of every run. Because that evidence is committed, a reader can rebuild the results rather than take a description of them on trust.
No data download, no training, no GPU. Seconds, from the committed evidence.
pip install -r requirements.txt python3 -m scout.plots.plot_paper_baseline_curves # the budget figure python3 -m scout.tables.build_paper_ablation # the ablation table python3 -m scout.tables.build_paper_task_dependence # the task-dependence numbers
Each builder prints the numbers that appear in the paper and then verifies itself: it refuses to emit a table whose Scout arm disagrees with the figure's, whose paired rows come from different seeds, or whose two arms turn out not to have run on the same graph. A disagreement is an error, not a footnote.
To check the pipeline rather than the artefacts:
python3 -m scout.tools.validate_contracts # 17 structural checks (C1–C17) python3 -m scout.tools.check_paths # path / styling / atomic-write rules
One step of the method, and the file that owns it.
core/contracts.py types the whole system: the true Snapshot, the stale ObservedGraph the model actually holds, an Observation, a PolicyDecision, a StepRecord. core/env.py is the single place allowed to see both graphs at once. Everything online sees only the stale one — and a contract check enforces that, rather than a comment asking for it.
policies/features.py builds a 19-dimensional state per node out of legally observable quantities only. policies/learned.py maps it through an MLP with a few rounds of neighbourhood mixing to s_v, then selects Top-B[s_v + λ·log(1 + stale_v)]. The feature version is stored inside the weights: change the features without retraining and loading fails loudly.
core/utility.py is the only implementation of counterfactual utility — set gain, leave-one-out, singleton, greedy lookahead. It is the training signal for the learned policy and the oracle's notion of value, deliberately the same code, so teacher and student cannot drift apart unnoticed. It necessarily touches the true snapshot, so it may be called only offline; that restriction is a checked contract.
Scout is scored against 6 heuristics (policies/heuristics.py), 3 probing protocols from prior work (prior_work.py), and two capacity-ablated scorers (learned_a2.py), on 5 dynamic networks (benchmarks/) under 2 downstream tasks — link prediction by Adamic–Adar and clamped label propagation (tasks/).
scout/ registry.py the single source of truth: benchmarks, policies, tasks, experiments, and the paper's artefacts configs/paths.yaml the only place a filesystem path is declared core/ context.py ExpContext — paths are resolved here and nowhere else contracts.py Snapshot / DynamicGraphStream / ObservedGraph / Observation / PolicyDecision / StepRecord env.py the acquisition environment: the one place that sees both the true graph and the maintained graph audit.py run ids, idempotence, query traces, the ledger utility.py counterfactual utility — the training signal aggregation.py paired statistics results.py result loading io.py atomic writes style.py one visual language benchmarks/ synthetic_drift.py, snapshot_series.py (AS-733), realworld_temporal.py (Reddit / email-Eu / MathOverflow) policies/ features.py (19-dim node state), heuristics.py (6), prior_work.py (3 probing protocols), learned.py (SCOUT), learned_a2.py + a2_nets.py (capacity-ablation scorers) tasks/ link_prediction.py (Adamic–Adar, AUC), node_classification.py (clamped label propagation) runners/ run_experiment.py, train_acquisition.py, eval_checkpoints.py tables/ plots/ tools/ outputs/ cache/ per-experiment step tables — what the figure and tables read run_records/ one card per run + ledger.jsonl (config, environment, artefacts, stream fingerprint) models/ the 16 checkpoints the reported numbers were produced with tables/ plots/ the built artefacts datasets/raw/ downloaded source data (not committed)
Two constraints shape that layout. Every filesystem path is declared in one YAML file and resolved in one class, so no module invents its own location; and every write goes through core/io.py, atomically, so a crashed run cannot leave a half-written result that a later table would quietly read.
Every artefact in the paper has one command that builds it, and the experiments it reads are named.
| In the paper | Command | Experiments it reads |
|---|---|---|
| Budget–performance curves, 10 methods × 5 benchmarks | scout.plots.plot_paper_baseline_curves |
budget_sweep, realworld_*, a2_downstream_*, utility_ablation |
| Ablation table (supervision target, scorer capacity) | scout.tables.build_paper_ablation |
utility_ablation, realworld_*, a2_downstream_* |
| Task dependence of acquisition | scout.tables.build_paper_task_dependence |
task_dependence_lp, task_dependence_nc |
Which policy key is Scout on which line is pinned once, in scout/tables/paper_lines.py, and both the figure and the ablation table read that single definition; the ablation builder then verifies cell by cell that they agree and refuses to write a table if they do not. Identifiers were kept as they were run rather than renamed for readability, so every number stays traceable to the run card that produced it.
These are rules the code checks, not conventions it describes. Each one exists because it is a way results can quietly become wrong.
Train → select on validation episodes → report on the held-out test window. Test performance is recorded during checkpoint evaluation but never used to choose. Contract C14 checks mechanically that every learned arm in every experiment loads a validation-selected checkpoint; C16 that all reported lines share one selection protocol.
On the real networks a seed selects a time window, not a random redraw, and the time axis is partitioned into non-overlapping bands with a buffer between them: evaluation seeds < 100, training 100–199, validation 200+. Disjointness is asserted at start-up, not assumed.
Every policy runs on the same streams, so all reported differences are paired by seed and reported as a mean Δ with win counts, a paired t and a 95% CI. Unpaired means would hide effects smaller than the seed-to-seed spread — which, at these budgets, most of them are.
Every run records a content fingerprint of the graph it ran on. Results from different experiments are pooled only after the fingerprints are asserted equal for each (benchmark, seed). If they disagree, the table is refused rather than produced.
A learned policy with no checkpoint raises instead of degrading to random selection — the failure mode that turns a broken run into a plausible-looking baseline. Running on untrained weights requires an explicit flag, and that flag enters the config hash and the ledger.
run_id = <experiment>__<config_hash>, derived from configuration content with no timestamp — so the same configuration is the same run, and re-running is idempotent. Each run leaves a card with its full configuration, package versions and stream fingerprint, plus a per-step query trace from which any metric can be recomputed offline.
The synthetic benchmark is generated in code. The four real networks come from SNAP and are not redistributed here; the README gives the download layout, and a benchmark whose file is missing is skipped with a note rather than silently dropped. With the data in place, the evaluation sweeps re-run against the committed checkpoints, and a cell whose configuration has already completed is skipped and recorded as skipped:
python3 -m scout.runners.run_experiment --experiment all --dry-run python3 -m scout.runners.run_experiment --experiment realworld_email
Retraining the scorer rather than using the shipped weights is a three-step chain — train with a per-epoch checkpoint trajectory, evaluate every checkpoint on the validation and test windows, then write out the argmax-validation checkpoint — and the capacity ablation has a driver for the whole chain. Every module also runs its own self-test when executed directly (python3 -m scout.core.contracts, and so on). The full sweep is a few hours on a laptop.
git clone https://github.com/AmbitiousK/Scout-code.git