Skip to content

feat(routing): opt-in persistence of the DHT routing table - #11481

Open
guillaumemichel wants to merge 5 commits into
masterfrom
feat/dht-rt-persistence
Open

guillaumemichel wants to merge 5 commits into
masterfrom
feat/dht-rt-persistence

Conversation

@guillaumemichel

@guillaumemichel guillaumemichel commented Oct 21, 2026 •

Copy link
Copy Markdown
Member

Companion to libp2p/go-libp2p-kad-dht#1299, ipfs/boxo#1220 and ipfs/boxo#1223.

This adds an opt-in Routing.PersistDHTRoutingTable flag. With it, Kubo remembers its Amino DHT routing table across restarts and rejoins the network through the peers it already knew, instead of through the bootstrap peers.

Motivation

A running node knows hundreds of DHT servers. The longer it runs, the more of them are stable peers, likely to still be online after a restart. Today Kubo forgets them and starts every time from the same few bootstrap peers, then slowly works its way back to stable peers. The bootstrap peers carry load the rest of the network could easily absorb, a node can't rejoin when they are unreachable, and every restart trusts them for the node's first view of the network. Restoring the routing table skips that climb back, makes restarts faster, and makes the network less dependent on a few operators.

Kubo already saves backup bootstrap peers at /local/temp_bootstrap_peers, but they serve a narrower purpose: up to 20 of the node's connected peers, not necessarily DHT servers, dialed only after the configured bootstrap peers fail.

How it works

  • Saving. With the flag on, the WAN DHT saves its routing table to the repo after 10 minutes, then every hour, and on shutdown.
  • Restoring. On the next start it dials the saved peers first (dht.BootstrapOrder(dht.PersistedFirst)), and the bootstrap peers only if fewer than 10 saved peers answer within 20 seconds.
  • Startup doesn't wait for a first bootstrap round. The builder sets boxo's BootstrapConfig.DeferFirstRound, so the periodic bootstrapper first runs after one Period, as a backstop. Direct IpfsNode.Bootstrap callers keep the blocking first round.
  • The snapshot replaces the backup bootstrap peers. Kubo doesn't pass backup peers to boxo, and deletes /local/temp_bootstrap_peers on startup.
  • With the flag off, nothing changes. The DHT gets dht.RoutingTablePersistence(0), and the backup bootstrap peers are saved and used as before.

Rollout

The plan is to ship this feature as opt-in in the next release, and if it behaves as expected to make it the default in an upcoming version.

TODO

@socket-security

socket-security Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Updatedgithub.com/​ipfs/​boxo@​v0.43.0 ⏵ v0.43.1-0.20261021124930-a3f9255caf5674 +1100100100100
Updatedgithub.com/​libp2p/​go-libp2p-kad-dht@​v0.42.2 ⏵ v0.43.0100 +1100100100100

View full report

@guillaumemichel

guillaumemichel commented Oct 21, 2026 •

Copy link
Copy Markdown
Member Author

Note: CI fails for Interop / ipfs-webui but it is unrelated to this specific PR.

See #11479

@guillaumemichel
guillaumemichel marked this pull request as ready for review October 28, 2026 13:05
@guillaumemichel
guillaumemichel requested a review from a team as a code owner October 28, 2026 13:05
@guillaumemichel

Copy link
Copy Markdown
Member Author

Measurements

Measured on the Amino DHT, 2026-10-21 to 2026-10-28. Full tables and method are in libp2p/go-libp2p-kad-dht#1299.

Kubo restart comparison

Kubo with its default connection manager, restarted 3 times with the flag on and 3 times with it off:

On Off
Routing table reaches 100 peers 2.1 s 11.5–22 s
Peak file descriptors 493–805 801–1012
Peak conntrack entries 2561–3025 3428–4277

How long saved peers stay useful

Routing tables saved by nodes that had run for 30 minutes, re-dialed at increasing ages:

Snapshot age Saved peers still reachable 10 saved peers answered after Fell back to bootstrap peers
0 94–99% 0.2–0.3 s never
1 day 81–86% 0.4–0.5 s never
3 days 72–75% 0.4–0.5 s never
7 days 68–72% 0.3–0.5 s never

So a Kubo node that was down for a week still rejoins in under a second without the bootstrap peers.

Longer-running nodes save more stable routing tables

A routing table keeps its peers until they stop answering, so the longer a node runs, the more its table holds stable, long-lived peers. A routing table saved by a node that had run for 24 hours, compared with the 30-minute ones:

Snapshot age Still reachable: 30-minute nodes Still reachable: 24-hour node Answered within 20s: 30-minute nodes Answered within 20s: 24-hour node
0 94–99% 97% 138–143 142
1 day 81–86% 88% 120–124 128
3 days 72–75% 85% 105–109 124
5 days 69–72% 83% 97–105 121

The gap widens with time: the 30-minute snapshots shed their short-lived peers, while the 24-hour one had already settled on peers that stay online.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant