Skip to content

DEMBenchmark: reuse a translated DEM newer than its inputs - #215

Merged
bpurinton merged 2 commits into
mainfrom
dem-benchmark-reuse-translated-dem
Oct 22, 2026
Merged

bpurinton merged 2 commits into
mainfrom
dem-benchmark-reuse-translated-dem

Conversation

@bpurinton

Copy link
Copy Markdown
Member

DEMBenchmark.run(pc_align=True) already reuses the pc_align products under its folder, so a re-run makes no pc_align call. It still rewrote each DEM's translated copy (dem_benchmark/<label>/<stem>_pc_align_translated.tif) on every run, a full-size raster write per DEM. Re-scoring the 39 full-scene DEMs of #207 from cached alignments spent about 6 of its 20 minutes on those writes.

Change

  • The translated copy is reused when its mtime is at least that of the DEM and of every pc_align/<prefix>-* file (_is_newer_than). Otherwise it is rewritten.
  • A rewrite goes to *_pc_align_translated.partial.tif and is then moved into place with os.replace, so an interrupted run never leaves a partial copy that the next run would take as current.
  • The docstrings of the class's directory argument and of run(pc_align=...) say so.
  • One [Unreleased] / Changed line in the CHANGELOG.

Tests

TestRun now also checks three things:

  • a second run leaves the translated DEM's mtime unchanged and returns the same table;
  • a translated DEM older than its pc_align output is rewritten;
  • no .partial file is left behind.

pytest tests/test_dem_benchmark.py: 34 passed. pre-commit run --all-files: passed.

Commits

  1. A cherry-pick (-x) of ac1db60 from Full-scene scene-combination benchmarks: Atlanta and UCSD, lidar check, Atlanta jitter (#207) #210, where the change was first made.
  2. A rewording of the CHANGELOG line, which had quoted that study's numbers.

#210 itself will be closed: its notebooks and write-up are moving out of this repo.

bpurinton and others added 2 commits October 22, 2026 15:37
Re-scoring from cached pc_align runs rewrote every ~500 MB translated
copy, about 6 of the 20 minutes the lidar notebook spent re-scoring the
39 full-scene DEMs. Reuse the copy when it is newer than its DEM and
pc_align output, and write new copies under a temporary name then
rename, so an interrupted run leaves no partial file to reuse.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit ac1db60)
The cherry-picked entry quoted one study's numbers (39 full-scene
DEMs, ~500 MB copies, minutes in one notebook). A package changelog
says what the package does differently; the study and its timings
live elsewhere.
@bpurinton
bpurinton merged commit a5d000a into main Oct 22, 2026
1 check passed
@bpurinton
bpurinton deleted the dem-benchmark-reuse-translated-dem branch October 22, 2026 23:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant