Repository navigation
Make the query list scale with query history; fix N+1 on the log page - #722
Merged
Merged
Conversation
The query list prefetched every QueryLog for every query just to show a run count and the last run, and recently_viewed() iterated all of the user's logs in Python. Both load the full log history on each page view: with 300k logs the page took ~9.7s; demo.sqlexplorer.io (1.15M logs) takes ~5.5s. - Run count, last run time and last run status are annotated in SQL. - Recently viewed queries are computed with one aggregate plus one fetch. - select_related database_connection removes a per-row query. Same 300k-log benchmark: 0.9s, and a constant 7 SQL queries regardless of how many queries or logs exist. The rendered list is identical to before. Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Each log row looked up its run_by_user and database_connection separately (25-35 SQL queries per page); select_related brings it to a constant 5. Co-Authored-By: Claude Code <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The home page (query list) loaded the entire query log history on every view:
get_queryset()didprefetch_related("querylog_set"), fetching every log row for every query just to display a run count and the last run.recently_viewed()looped over all of the user's logs in Python. Starting that loop evaluates the whole queryset.str(q.database_connection)ran one extra query per row.So the page gets slower as history grows. On a local benchmark (125 queries, 300k logs) it took 9.7 s. demo.sqlexplorer.io, with 1.15M logs, takes ~5.5 s to first byte, while its other pages take ~0.3 s.
Changes
CountplusSubqueryannotations. A query that has never run still shows its creation time as a successful run, matchingQuery.last_run_log().select_related("created_by_user", "database_connection")replaces the prefetch and removes the per-row query.order_by("title"), because a model'sMeta.orderingisn't applied once the query has an aggregate.Also: N+1 on the query log page
Each row on
/logs/looked up itsrun_by_useranddatabase_connectionseparately, which meant 25–35 SQL queries per page.select_relatedbrings that to a constant 5. With this PR, the query list, logs, favorites, connections, query detail and playground pages each run a fixed number of queries as data grows.Results (same 300k-log benchmark)
Tests
test_last_run_status: a never-run query falls back to its creation time and success, and a failed last run shows as failed.test_recently_viewed_is_latest_run_of_distinct_queries: one entry per query, using the latest run, and only the user's own runs.test_query_count_does_not_grow_with_queries_or_logs: this fails onmaster(9 → 14 SQL queries as rows are added) and passes here.test_log_page_query_count_does_not_grow_with_rows: fails onmaster(17 vs 7) and passes here.settingsandsettings_base. The list view and log view tests also pass on Postgres 16.ruff==0.16.8is clean.🤖 Generated with Claude Code