Skip to content

fix(hash): prevent new fields from inheriting stale key TTL after all fields expire - #3665

Open
weim0000 wants to merge 17 commits into
apache:unstablefrom
weim0000:fix_hfe
Open

weim0000 wants to merge 17 commits into
apache:unstablefrom
weim0000:fix_hfe

Conversation

@weim0000

Copy link
Copy Markdown
Contributor

Summary

Fix #3635. When all fields in a hash expire via HFE (Hash Field Expiration), ordinary read operations (e.g. HGET) lazily skip expired fields without repairing the hash metadata. A subsequent write (e.g. HSET) then reuses the stale metadata — including its version and key-level TTL — causing the new field to inherit the old key's expiration.

Root Cause

ExpireFields uses ExpandExpireBounds which only expands lower/upper bounds but never shrinks them. After shortening a field TTL, upper retains the old larger value. When all fields expire, no read-path cleanup occurs (unless an accurate HLEN happens to trigger scanAndRepair). The next write reuses the stale metadata with the old version and key-level TTL.

The Fix

Add ensureValidMetadata that runs unconditionally before any write operation that can create new fields (MSet, IncrBy, IncrByFloat, SetFieldsWithExpire). The function determines whether existing metadata is still valid:

  • persist > 0: permanent fields exist → metadata is valid → O(1).

  • persist == 0: all fields carry TTL →

  • Fast path: now < lower → all TTL fields still alive → valid → O(1).

  • Fast path: now > upper → all TTL fields expired → invalid → reset → O(1).

  • Slow path: scan subkeys for the first live field. If found → valid. If none → stale → reset.

When metadata is stale, a fresh metadata with a new version and no key-level TTL is constructed and written to RocksDB, overwriting the stale one. The incoming write then proceeds against this new metadata.

Design decisions:

  • Always runs, not gated by hash-length-mode. This is a correctness fix, not a precision trade-off.

  • Lightweight scan: no cleanup of expired fields during the check. Expired subkeys are cleaned up by subsequent scanAndRepair, lazy expiry, or compaction.

AI Assistance Disclosure

The code changes and unit tests in this PR were implemented with the assistance of Qoder, an AI-powered coding tool. I have personally reviewed all generated code and tests thoroughly to verify that the logic correctly implements the proposed design, handles edge cases properly, and aligns with the project's coding conventions. All 48 hash unit tests pass.

@PragmaTwice
PragmaTwice self-requested a review October 10, 2026 13:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Timestamp inconsistency can alter expiration semantics, and the slow path can repeatedly rescan large expired prefixes.

3 open findings
What changed in this PR

Fixes stale hash metadata causing newly written fields to inherit an expired hash’s key-level TTL.

Changes:

  • Validates HFE metadata before field-creating writes.
  • Recreates metadata when no live fields remain.
  • Adds stale-TTL regression tests.
File Description
src/​types/​redis_hash.cc Implements metadata validation across hash write paths.
src/​types/​redis_hash.h Declares the validation helper.
tests/​cppunit/​types/​hash_test.cc Adds regression coverage for stale TTL inheritance.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/types/redis_hash.cc Resolved
Comment thread src/types/redis_hash.cc Resolved
Comment thread src/types/redis_hash.cc Resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HFE: new fields may inherit stale key TTL after all existing fields expire

3 participants