Skip to content

Improve file-cache search performance for large caches - #65986

Open
Daddyoftwins1 wants to merge 3 commits into
nextcloud:masterfrom
Daddyoftwins1:prep/bounded-files-search-master
Open

Daddyoftwins1 wants to merge 3 commits into
nextcloud:masterfrom
Daddyoftwins1:prep/bounded-files-search-master

Conversation

@Daddyoftwins1

Copy link
Copy Markdown

This PR improves file-cache search performance for the tested large-cache workloads while preserving the compared search results and maintaining compatibility with the current upstream architecture.

The changes update the search implementation for current connection handling and sharding, integrate optional indexes into Nextcloud's native missing-index lifecycle, and add upstream-native regression tests.

Performance

The patched implementation was compared with unchanged Nextcloud 36 development code at the same commit. The controlled synthetic benchmark used 100,000 target files and approximately 504,000 total cache rows.

Selected warm-cache median results:

Search Unchanged Patched Result
Common name 76.2 ms 5.9 ms 12.9× faster
Recent selective name 64.9 ms 8.8 ms 7.4× faster
No matches 64.3 ms 24.8 ms 2.6× faster
Small-cache control 3.1 ms 5.6 ms 2.5 ms slower

All search results compared within the benchmark fixture matched. Adding indexes alone did not produce a consistent material improvement in this experiment; the search implementation changes were necessary for the measured gains.

These are controlled synthetic benchmarks. Performance will vary with cache size, workload, and environment. The small-cache slowdown is a known tradeoff.

Validation

  • The compatibility regression suite completed with 126 tests, 626 assertions, and one Normalizer-related environment skip.
  • Formatting, syntax, and autoloader checks passed.
  • I personally completed live testing of the change on my running Nextcloud system.

Please review the search implementation, optional-index lifecycle, connection and sharding compatibility, and regression coverage.

AI assistance

I was assisted by Codex AI with implementation, automated testing, benchmarking, and code review.

Evidence

Code and benchmark evidence bundle

@Daddyoftwins1
Daddyoftwins1 requested a review from a team as a code owner October 28, 2026 16:23
@Daddyoftwins1
Daddyoftwins1 requested review from Altahrim, leftybournes, salmart-dev and sorbaugh and removed request for a team October 28, 2026 16:24
Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Josh Duffy <joshduff.jd@gmail.com>
Assisted-by: Codex:gpt-6.1-sol
Signed-off-by: Josh Duffy <joshduff.jd@gmail.com>
@Daddyoftwins1
Daddyoftwins1 force-pushed the prep/bounded-files-search-master branch from 4037466 to 9389000 Compare October 28, 2026 16:45

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant