Repository navigation
Store Worker request bins in MongoDB Atlas - #247
Conversation
Keep request bin routes enabled on the Cloudflare Worker and persist them with @keyv/mongo when MONGODB_URI is set, so bins are shared across isolates through MongoDB Atlas. Co-authored-by: Jared Wray <me@jaredwray.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #247 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 41 41
Lines 1393 1404 +11
Branches 286 287 +1
=========================================
+ Hits 1393 1404 +11 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
The Cloudflare options table uses a single option name in the first column. Co-authored-by: Jared Wray <me@jaredwray.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 867e6524d6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (uri) { | ||
| mockHttp.bins = createWorkerBins(uri); |
There was a problem hiding this comment.
MONGODB_URI gives all users and isolates the same store. Yet GET /bins lists every ID, and request routes have no authentication (src/routes/bins/management.ts:211-259). Any caller can read captured headers and bodies from other users. Add access control or tenant separation before this assignment.
Useful? React with 👍 / 👎.
| return new MockHttpBins({ | ||
| keyv: new Keyv({ | ||
| store: new KeyvMongo(uri, workerBinMongoOptions), |
There was a problem hiding this comment.
Separate isolates create separate MockHttpBins instances, so their _locks maps cannot serialize writes. recordRequest() reads and replaces the complete document (src/mock-http-bins.ts:153-155,198-212). Concurrent webhook captures can overwrite each other, lose requests, and set an incorrect requestCount. Use an atomic MongoDB update or shared lock.
Useful? React with 👍 / 👎.
| "secrets": { | ||
| "required": ["MONGODB_URI"] |
There was a problem hiding this comment.
When MONGODB_URI is absent, this declaration blocks the documented memory fallback during deployment. The workflow also runs plain wrangler deploy at .github/workflows/deploy-site.yaml:99-103. Wrangler fails deployment when a required secret is missing, per the Cloudflare configuration documentation. Remove this requirement, or provision the secret in every supported deployment path.
Useful? React with 👍 / 👎.
| if (!listening) { | ||
| const pending = (async () => { | ||
| try { | ||
| await listenWorkerApp(await createWorkerApp(env)); |
There was a problem hiding this comment.
ensureWorkerListening() keeps the MockHttpBins MongoDB pool from the first request for all later requests in that isolate. Cloudflare forbids TCP sockets shared across requests, so later bin operations can fail with cross-request I/O errors (TCP socket documentation). Create each direct client inside its request, or place the persistent client in a Durable Object.
Useful? React with 👍 / 👎.
| return new MockHttpBins({ | ||
| keyv: new Keyv({ | ||
| store: new KeyvMongo(uri, workerBinMongoOptions), |
There was a problem hiding this comment.
MockHttpBins stores all requests for one bin as one Keyv value. With the defaults, 96 maximum binary bodies alone become 16,777,344 base64 bytes. Thus, MongoDB rejects the document before the advertised 100-request limit under its 16 MiB BSON limit. Store requests separately, or lower the Mongo-backed limits.
Useful? React with 👍 / 👎.
Atlas bin storage is how the free mockhttp.org service runs. The deploy guide again describes the in-memory store used by any other Worker. Co-authored-by: Jared Wray <me@jaredwray.com>
Please check if the PR fulfills these requirements
What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Feature. The free mockhttp.org Worker stores request bins in MongoDB Atlas through
@keyv/mongowhenMONGODB_URIis set. The Cloudflare deploy guide stays generic.Summary
Bin routes were already on, but each isolate kept bins in memory.
createWorkerAppnow buildsMockHttpBinswith Keyv and@keyv/mongowhen the Atlas URI is present. The collection isbins, the pool is 5 connections, and server selection times out after 5 seconds.The URI is read from the Worker binding
env.MONGODB_URI, then fromprocess.env.MONGODB_URI. Fastify starts on the first request so that secret is available. Later requests in the same isolate reuse that server and client.Without a URI, bins stay in memory. That is what a Worker deployed from the generic Cloudflare guide does. The public service sets
MONGODB_URIin Cloudflare. About mockhttp.org is where that is described.Use an Atlas connection string that includes the database name. Atlas network access has to allow Cloudflare Worker egress.
Test plan
pnpm test(Biome and Vitest with coverage, 529 tests)MONGODB_URIis set on the env binding orprocess.envPOST /binsthenGET /bins/:idon mockhttp.org after this Worker is deployed