Skip to content

perf(registry): fetch only the requested page of tag names - #24343

Open
perfloop-agent wants to merge 3 commits into
goharbor:mainfrom
perfloop-oss:perfloop-pr-open-94k3mbtqe8
Open

perfloop-agent wants to merge 3 commits into
goharbor:mainfrom
perfloop-oss:perfloop-pr-open-94k3mbtqe8

Conversation

@perfloop-agent

@perfloop-agent perfloop-agent commented Nov 9, 2026 •

Copy link
Copy Markdown

Comprehensive Summary of your change

For positive n below the maximum integer, local registry tag lists now fetch names strictly after last in byte order, with one lookahead row, instead of hydrating every tag. The existing response helper still produces JSON and continuation links. Explicit byte ordering preserves Go comparisons under non-byte database collations, including arbitrary cursor bytes.

With PostgreSQL 17 and n=1000, median Go-allocated bytes for the existing two-tag v1/v2 input fell from 13071 to 9175 B/request over 87 paired runs. Larger inputs extend that naming pattern as constructed scaling workloads.

Times are medians in ns/request; brackets bound the paired median time saved:

Tags Handler before → after [bounds] Handler change PostgreSQL before → after [bounds] PostgreSQL change Pairs
2 401623 → 385151 [6676, 47464] −4.6% (−11.8% to −1.7%, n=87) 33106 → 33751 [−1561, 1019] ~ (−3.1% to +4.7%, n=87) 87
1500 4079196.5 → 1530553 [2337894, 2637931] −62.1% (−64.7% to −57.3%, n=16) 1057470.5 → 657691.5 [326619, 446871] −37.9% (−42.3% to −30.9%, n=16) 16
10000 22071357 → 3121612 [16870971, 20084972] −86.2% (−91.0% to −76.4%, n=16) 5230749.5 → 2193850 [2447397, 3445275] −58.0% (−65.9% to −46.8%, n=16) 16

The two-tag paired SQL estimate is 743 ns/request slower, but its bounds span zero; zero overhead is not established. Lower allocations and large-list gains motivate the bounded query.

PostgreSQL time includes SELECT planning and execution, not driver/network waits or CPU time.

Native benchmark: src/server/registry/tag_benchmark_test.go. Further reproduction from src, with the DAO tests' PostgreSQL settings: go test ./server/registry -run '^$' -bench '^BenchmarkTagList$' -benchtime=8s -count=1 -benchmem.

The database may still scan or sort repository tags. Unpaginated requests, maximum-integer counts and proxy local/remote merging retain their existing paths. Production traffic savings are not measured.

Issue being fixed

Paged local tag lists retrieve and hydrate the whole repository before selecting a page.

Validation and contribution checklist

Go tests and race checks passed for tag DAO, manager and controller behavior, registry handling and registry pagination utilities. The database differential test compares response status, JSON and headers against the whole-collection response across mixed names, cursor positions and page boundaries. The native benchmark also completed a one-iteration smoke run at each size.

Please indicate you've done the following:

  • Well Written Title and Summary of the PR
  • Label the PR as needed. "release-note/ignore-for-release, release-note/new-feature, release-note/update, release-note/enhancement, release-note/community, release-note/breaking-change, release-note/docs, release-note/infra, release-note/deprecation"
  • Accepted the DCO. Commits without the DCO will delay acceptance.
  • Made sure tests are passing and test coverage is added if needed.
  • Considered the docs impact and opened a new docs issue or PR with docs changes if needed in website repository.

Case and evidence


Made by Perfloop. Human sponsor: Tomás Senart.

Signed-off-by: Perfloop Agent <agent@perfloop.ai>
Signed-off-by: Tomás Senart <ts@perfloop.ai>
Signed-off-by: Perfloop Agent <agent@perfloop.ai>
Signed-off-by: Tomás Senart <ts@perfloop.ai>
@perfloop-agent
perfloop-agent requested a review from a team as a code owner November 9, 2026 09:29
@tsenart

tsenart commented Nov 9, 2026 •

Copy link
Copy Markdown

Hi, I'm Tomás, I build Perfloop. I approved this contribution and will handle review feedback here.

@chlins, this is in the registry tag list handler (src/server/registry/tag.go): when a client sets a positive n, Harbor now reads only the requested page of a local repository's tags instead of loading every tag first. Requests without n keep the existing path. We found it while hunting performance improvements with Perfloop. We've had changes merged in Distribution, Prometheus, ClickHouse, Redis, and more.

If you'd like anything in this patch changed, just let me know and I'll update it.

Signed-off-by: Perfloop Agent <agent@perfloop.ai>
Signed-off-by: Tomás Senart <ts@perfloop.ai>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants