Repository navigation
fix(worker): debounce save queue so burst zones batch correctly - #1
Conversation
Without a debounce, the save worker picked up the first queued item and immediately started processing it. If backend writes completed before the next items arrived (common when backends are fast), get_nowait() returned Empty and the batch closed at 1/1. Subsequent items each opened their own single-item batch instead of being grouped with the burst. Adds BATCH_DEBOUNCE_SECONDS (default 0.5 s): after the blocking get() picks up the first item, the worker sleeps briefly so burst items from DirectAdmin (multiple zones pushed within the same second) can accumulate in the queue. The existing drain loop then processes them all in a single batch. Also fixes short-form `from app.utils import …` to fully-qualified `from directdnsonly.app.utils import …` so the module is importable without a sys.path hack, enabling the new test_worker.py suite.
Additional fixes pushed to this branchTwo more issues found while tracing a Let's Encrypt DNS-01 challenge failure (both cluster nodes received the push, but LE still failed). Reload logs promoted to INFONSD and BIND reload success was logged at Reload failure now triggers retry
Test mock corrected
|
Problem
When DirectAdmin pushes multiple zones within the same second, the save worker was processing them in separate 1/1 batches instead of one grouped batch.
Root cause: The batch drain loop (
get_nowait()) only worked if backend processing was slow enough to act as an accidental debounce gate. With fast backends, item 1 would finish and hitEmptybefore items 2 and 3 from the same burst had landed in the queue — each item then opened its own batch.Fix
Adds
BATCH_DEBOUNCE_SECONDS = 0.5toworker.py. After the blockingget()fires on the first item, the worker sleeps 500 ms so the rest of the burst can accumulate. The existing drain loop then processes all of them in a single batch.Also fixes
from app.utils import …→from directdnsonly.app.utils import …(short-form import was unresolvable without a runtimesys.pathhack, which blocked writing worker tests).Tests (TDD)
New
tests/test_worker.py:test_burst_items_land_in_single_batch— 3 items arriving within the debounce window → asserts exactly 1 batch logged as3/3test_items_outside_debounce_window_start_new_batch— items separated by more than the window → asserts 2 separate batchestest_preloaded_queue_still_batches— 5 pre-loaded items → still drains as one batchTest plan
pytest tests/ -v)📦 Batch complete — N/Nwith N > 1 instead of repeated1/1