feat: query separate course and library search indexes - #3248
blarghmatey wants to merge 1 commit into
Conversation
Studio's single studio_content index is being split into a course index and a library index so library authoring isn't slowed down by indexing cost that grows with course content (openedx/openedx-platform#38993). The search config endpoint now returns course_index_name and library_index_name. Each search surface picks the index for the content it shows. When the backend only returns index_name, both fall back to it, so this works against unsplit backends unchanged. No surface searches courses and libraries together: the Studio search modal has filtered to type = "course_block" since openedx#1148, so no multi-index query is needed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01J43t1WXNsmVV5mbdzxv6iT
|
Thanks for the pull request, @blarghmatey! This repository is currently maintained by Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review. 🔘 Get product approvalIf you haven't already, check this list to see if your contribution needs to go through the product review process.
🔘 Provide contextTo help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:
🔘 Get a green buildIf one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green. DetailsWhere can I find more information?If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources: When can I expect my changes to be merged?Our goal is to get community contributions seen and reviewed as efficiently as possible. However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:
💡 As a result it may take up to several weeks or months to complete a review and merge your PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #3248 +/- ##
========================================
Coverage 96.03% 96.03%
========================================
Files 1407 1407
Lines 34287 34292 +5
Branches 7882 8140 +258
========================================
+ Hits 32928 32933 +5
+ Misses 1318 1303 -15
- Partials 41 56 +15 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
🟢 Approval recommended
Index routing and backward compatibility are correctly implemented and covered by focused tests; remaining feedback is non-blocking.
Pull request overview
Updates Studio search consumers for separate course and library Meilisearch indexes while preserving compatibility with older backends.
Changes:
- Adds explicit course/library index selection across search hooks and providers.
- Parses split index names with legacy
index_namefallback. - Adds tests for configuration parsing and index routing.
File summaries
| File | Description |
|---|---|
src/search-modal/SearchUI.tsx |
Routes modal searches to the course index. |
src/search-manager/SearchManager.ts |
Requires and propagates the index type. |
src/search-manager/index.ts |
Exports the index type. |
src/search-manager/data/apiHooks.ts |
Selects the requested index in search hooks. |
src/search-manager/data/apiHooks.test.tsx |
Tests hook routing for both indexes. |
src/search-manager/data/api.ts |
Parses split and legacy configuration responses. |
src/search-manager/data/api.test.ts |
Tests configuration parsing and fallback. |
src/search-manager/data/api.mock.ts |
Updates shared search configuration mocks. |
src/library-authoring/LibraryContent.tsx |
Queries course placeholders from the course index. |
src/library-authoring/LibraryAuthoringPage.tsx |
Routes library browsing to the library index. |
src/library-authoring/library-filters/CollectionDropdownFilter.tsx |
Queries collections from the library index. |
src/library-authoring/import-course/ImportDetailsPage.tsx |
Queries imported course blocks from the course index. |
src/library-authoring/generic/manage-collections/ManageCollections.tsx |
Searches collections in the library index. |
src/library-authoring/data/apiHooks.ts |
Selects indexes based on content keys. |
src/library-authoring/data/apiHooks.test.tsx |
Tests key-based index selection. |
src/library-authoring/collections/LibraryCollectionPage.tsx |
Routes collection searches to the library index. |
src/library-authoring/collections/CollectionDetails.tsx |
Loads collection facets from the library index. |
src/course-outline/outline-sidebar/info-sidebar/InfoSection.tsx |
Loads course block counts from the course index. |
src/course-outline/outline-sidebar/info-sidebar/CourseInfoSidebar.tsx |
Loads course summary counts from the course index. |
src/course-libraries/ReviewTabContent.tsx |
Searches downstream course blocks in the course index. |
Review details
- Files reviewed: 20/20 changed files
- Comments generated: 2
- Review effort level: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| useContentSearchResults, | ||
| buildSearchQueryKey, | ||
| type SearchIndexType, | ||
| } from '../../search-manager'; |
| * Load the Meilisearch connection details from the CMS: the URL to use, the index name, and an API key specific | ||
| * to the current user that allows it to search all content he have permission to view. | ||
| * Load the Meilisearch connection details from the CMS: the URL to use, the name of the course or library index, and | ||
| * an API key specific to the current user that allows it to search all content he have permission to view. |
bradenmacdonald
left a comment
There was a problem hiding this comment.
Looks good and works well. I just have one suggestion on top of the two very minor things that Copilot already pointed out.
| export const useContentFromSearchIndex = (contentIds: string[]) => { | ||
| const { client, indexName } = useContentSearchConnection(); | ||
| const extraFilter = [`usage_key IN ["${contentIds.join('","')}"]`]; | ||
| let indexType: SearchIndexType = 'course'; |
There was a problem hiding this comment.
Since this code is in src/library-authoring/data/apiHooks.ts, it's primarily used for libraries and I think we should leave library as the default index. 4 of 5 usages are library-only.
The one exception is ComponentUsage.tsx where we use this to list the various places in courses that use these library blocks. In that case, I think it would be cleaner to explicitly pass index = 'library' as a parameter to this function to override the default rather than using the auto-detection.
The reason is that leaving it as auto-detection, it's hard to tell where and why this code would even deal with courses at all. With an explicit parameter, it's easier to trace the usage.
Description
Pairs with openedx/openedx-platform#39103, which splits the Studio Meilisearch index into a course index (
studio_content) and a library index (studio_library_content) to fix slow library authoring on large instances (openedx/openedx-platform#38993).The search config endpoint now returns
course_index_nameandlibrary_index_name.SearchContextProvider,useContentSearchConnection,useGetBlockTypesanduseGetContentHitstake a requiredindexType('course' | 'library'), so each caller has to say which index it searches and TypeScript flags any that don't. Querying the wrong index after the split returns nothing rather than an error, which is why it isn't defaulted.useContentFromSearchIndexpicks the index from the key it is given.No combined course-and-library search is needed: the Studio search modal has filtered to
type = "course_block"since #1148, and nothing else searches both.If the backend only returns
index_name, both index types use it, so this works unchanged against a backend without the split.Roles affected: Course Author (no visible change), Operator (deploy with the backend change).
Supporting information
Testing instructions
./manage.py cms reindex_studio --libraries-only: library home lists and searches components, collections and units; the Studio search modal finds course content; the course libraries Review tab lists out-of-sync blocks.Automated:
npm run typesandnpm run lintpass. Jest (withTZ=UTC, as thetestscripts set it) oversrc/search-manager,src/search-modal,src/library-authoring,src/course-libraries,src/course-outlineandsrc/course-unit: 133 suites, 1451 tests pass. New tests cover parsing both index names (including theindex_namefallback), which index each hook queries, and the key-based choice inuseContentFromSearchIndex.Other information
release/verawoodbackport branch is ready and will be opened alongside the backend backport.Best Practices Checklist
.ts,.tsx).propTypesanddefaultPropsin any new or modified code.src/testUtils.tsx(specificallyinitializeMocks)apiHooks.tsin this repo for examples.messages.tsfiles have adescriptionfor translators to use.../in import paths. (library-authoring/data/apiHooks.tsonly widens its existing'../../search-manager'import; no new relative imports.) To import from parent folders, use@src, e.g.import { initializeMocks } from '@src/testUtils';instead offrom '../../../../testUtils'