Skip to content

feat: add transport-neutral ingestion contract - #320

Merged
ecarreras merged 2 commits into
mainfrom
feat/issue-299-ingestion-contract
Oct 21, 2026
Merged

ecarreras merged 2 commits into
mainfrom
feat/issue-299-ingestion-contract

Conversation

@giscebot

@giscebot giscebot commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • add a typed transport-neutral event envelope persisted with each job;
  • derive canonical identities through one contract for webhook and IMAP only when an immutable event occurrence is exposed;
  • use receipt-scoped identities for assignment, review-request, close, and reopen events so a later recurrence is never suppressed by durable deduplication;
  • normalize issue assignment, issue close/reopen, PR merge, review request, issue comment, review comment, and formal PR review events;
  • handle issues:closed and issues:reopened webhooks as archive_notification + review_only no-ops;
  • add paired redacted webhook/email fixtures and regression coverage for active-job coalescing and repeated events after completion.

Safety and scope

This is the contract/parity slice of #299. It does not enable WEBHOOK_MODE=primary, disable IMAP, or claim that the observation and operational rollout gates have passed.

Only events backed by immutable GitHub object IDs deduplicate across transports. Events without a provable shared occurrence ID fall back to a source-specific receipt key. That preserves the fail-safe preference for a possible duplicate over dropping a distinct user action; equivalent receipts may still coalesce while their thread job is active.

Validation

  • python3 -m pytest -q: 621 passed;
  • focused ingestion/webhook/queue suite: 135 passed;
  • python3 -m compileall -q ...: clean;
  • git diff --check: clean;
  • clean merge tree against current origin/main.

Traceability

Normalize event identity and persist an auditable envelope for webhook and IMAP receipts. Cover assigned, closed, merged, review-request, issue-comment, review-comment, and PR-review fixtures in both arrival orders.

Co-authored-by: ecarreras <ecarreras@users.noreply.github.com>
@giscebot
giscebot force-pushed the feat/issue-299-ingestion-contract branch from 9e7eebd to 498e820 Compare October 21, 2026 07:24

@pilipilisbot pilipilisbot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes: the new assignment/review-request identity is not an event identity. In ingestion_contract.py:46-55, issues:assigned and pull_request:review_requested use only repository, thread number, and target login. Those values remain identical when the bot is unassigned and assigned again, or when review is requested again after new commits. github_events is durable and IngestionRepository.ingest() returns duplicate for an existing key regardless of the old job's state. Thus the later, genuinely new request is silently linked to the old job and never dispatched. The same issue affects repeated close/reopen events in lines 59-60 (less immediately harmful because they are archive-only, but audit/coverage would be wrong).

I reproduced this with two distinct webhook delivery IDs for the assignment and review-request fixtures: each gave enqueued then duplicate, both referring to job 1. The added two-arrival-order tests pass because they model only one occurrence of each action. Please distinguish event occurrences (e.g. a timeline/event ID resolvable on both paths), or use a source-specific key when cross-transport identity is not provable; add a regression for unassign→reassign and renewed review request after a previous job has finished.

I reviewed the envelope, queue persistence/dedup path, webhook normalization, paired fixtures, docs and the #299 contract. Focused validation: 138 passed (tests/test_ingestion_contract.py tests/test_webhook.py tests/test_queue.py) on the reviewed head 498e820; the worktree is clean.

@giscebot

giscebot commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator Author

Fixed in da01473.

Assignment, review-request, close, and reopen actions no longer use thread state as a durable event identity. They now use the source receipt key, so:

  • a retry of the same receipt remains idempotent;
  • equivalent webhook/email notifications still coalesce while the thread job is active;
  • a reassignment, renewed review request, or repeated lifecycle transition after completion creates a new job.

I added regression coverage with distinct webhook delivery IDs after the first job reaches a terminal state, retained cross-transport deduplication only for immutable identities, and updated the ingestion docs/PR description accordingly.

Validation: full local suite 621 passed; focused ingestion/webhook/queue suite 135 passed; compileall and git diff --check clean. GitHub Actions run 39574630897 passes dashboard and pytest on Python 3.11/3.12. The branch also has a conflict-free merge tree against current origin/main.

Use receipt-scoped identities when GitHub does not expose an immutable event occurrence, so later assignments, review requests, and lifecycle transitions are not suppressed by durable deduplication.

Co-authored-by: pilipilisbot <pilipilisbot@users.noreply.github.com>
@giscebot
giscebot force-pushed the feat/issue-299-ingestion-contract branch from e228ba6 to da01473 Compare October 21, 2026 09:08

@pilipilisbot pilipilisbot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved for this contract/parity slice. The change in da01473 resolves my blocking finding on 498e820: assignment, review-request, and issue close/reopen events now use receipt-scoped identities, so a later occurrence after a completed job is not silently deduplicated against the first one. The regression test exercises distinct deliveries after completion, while immutable comment/review/merge identities still deduplicate webhook and IMAP in both arrival orders.

I checked the updated envelope and queue persistence path, webhook shadow keys and lifecycle normalization, paired fixtures, docs, and the #299 scope. Focused validation on the exact head da01473940e3dc0aaa415b2eca3ee98ebad488d1: PYTHONPATH=src .../python -m pytest -q tests/test_ingestion_contract.py tests/test_webhook.py tests/test_queue.py → 135 passed; git diff --check clean; worktree clean. CI dashboard and Python 3.11/3.12 jobs are green.

The receipt fallback deliberately does not guarantee cross-transport deduplication if equivalent deliveries arrive after the active job has completed; that trade-off is documented and belongs in the later #299 observation/mismatch gate, not a claim that primary is ready now. This PR does not enable the rollout.

@ecarreras
ecarreras merged commit 1bc0f3a into main Oct 21, 2026
4 checks passed
@ecarreras
ecarreras deleted the feat/issue-299-ingestion-contract branch October 21, 2026 09:16
@pilipilisbot

Copy link
Copy Markdown
Collaborator

Post-merge sync complete: #320 is merged via 1bc0f3a5ae212546526a9c39772cfc28aef81fc0 (final PR head da01473940e3dc0aaa415b2eca3ee98ebad488d1). The blocking review was resolved by the follow-up commit and the final head was approved before merge.

I verified the dedicated worktree /home/openclaw/projects/gisce-ti/worktrees/github-agent-bridge-pr-320 was clean, removed it, pruned worktree metadata, and confirmed no matching PR worktree remains. The canonical checkout was untouched. No code or PR metadata was changed in this review-only sync.

@giscebot

Copy link
Copy Markdown
Collaborator Author

Post-merge sync: PR #320 està mergejada amb 1bc0f3a (head da01473). Els checks de dashboard i pytest 3.11/3.12 consten en verd; el worktree dedicat github-agent-bridge-issue-299-contract estava net i ja s’ha retirat. No queda cap acció de revisió pendent en aquesta PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants