Skip to content

feat: query separate course and library search indexes - #3248

Open
blarghmatey wants to merge 1 commit into
openedx:masterfrom
blarghmatey:tmacey/split-studio-content-index
Open

blarghmatey wants to merge 1 commit into
openedx:masterfrom
blarghmatey:tmacey/split-studio-content-index

Conversation

@blarghmatey

Copy link
Copy Markdown

Description

Pairs with openedx/openedx-platform#39103, which splits the Studio Meilisearch index into a course index (studio_content) and a library index (studio_library_content) to fix slow library authoring on large instances (openedx/openedx-platform#38993).

The search config endpoint now returns course_index_name and library_index_name. SearchContextProvider, useContentSearchConnection, useGetBlockTypes and useGetContentHits take a required indexType ('course' | 'library'), so each caller has to say which index it searches and TypeScript flags any that don't. Querying the wrong index after the split returns nothing rather than an error, which is why it isn't defaulted.

  • Course index: search modal, course libraries Review tab, course outline block-type counts, course import details, migration placeholders, component usage.
  • Library index: library home and component picker (also used by the course AddSidebars), collection pages, manage collections, collection filter and details, adding items to containers.
  • useContentFromSearchIndex picks the index from the key it is given.

No combined course-and-library search is needed: the Studio search modal has filtered to type = "course_block" since #1148, and nothing else searches both.

If the backend only returns index_name, both index types use it, so this works unchanged against a backend without the split.

Roles affected: Course Author (no visible change), Operator (deploy with the backend change).

Supporting information

Testing instructions

  1. Against a backend without the split: library home, collections and the search modal behave as before.
  2. Against the backend change, after ./manage.py cms reindex_studio --libraries-only: library home lists and searches components, collections and units; the Studio search modal finds course content; the course libraries Review tab lists out-of-sync blocks.

Automated: npm run types and npm run lint pass. Jest (with TZ=UTC, as the test scripts set it) over src/search-manager, src/search-modal, src/library-authoring, src/course-libraries, src/course-outline and src/course-unit: 133 suites, 1451 tests pass. New tests cover parsing both index names (including the index_name fallback), which index each hook queries, and the key-based choice in useContentFromSearchIndex.

Other information

Best Practices Checklist

  • Any new files are using TypeScript (.ts, .tsx).
  • Avoid propTypes and defaultProps in any new or modified code.
  • Tests should use the helpers in src/testUtils.tsx (specifically initializeMocks)
  • Do not add new fields to the Redux state/store. Use React Context to share state among multiple components.
  • Use React Query to load data from REST APIs. See any apiHooks.ts in this repo for examples.
  • All new i18n messages in messages.ts files have a description for translators to use.
  • Avoid using ../ in import paths. (library-authoring/data/apiHooks.ts only widens its existing '../../search-manager' import; no new relative imports.) To import from parent folders, use @src, e.g. import { initializeMocks } from '@src/testUtils'; instead of from '../../../../testUtils'

Studio's single studio_content index is being split into a course index
and a library index so library authoring isn't slowed down by indexing
cost that grows with course content (openedx/openedx-platform#38993).

The search config endpoint now returns course_index_name and
library_index_name. Each search surface picks the index for the content
it shows. When the backend only returns index_name, both fall back to it,
so this works against unsplit backends unchanged.

No surface searches courses and libraries together: the Studio search
modal has filtered to type = "course_block" since openedx#1148, so no
multi-index query is needed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J43t1WXNsmVV5mbdzxv6iT
@openedx-webhooks openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Sep 14, 2026
@openedx-webhooks

Copy link
Copy Markdown

Thanks for the pull request, @blarghmatey!

This repository is currently maintained by @bradenmacdonald.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

  • If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
    • This process (including the steps you'll need to take) is documented here.
  • If it doesn't, simply proceed with the next step.
🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

  • Dependencies

    This PR must be merged before / after / at the same time as ...

  • Blockers

    This PR is waiting for OEP-1234 to be accepted.

  • Timeline information

    This PR must be merged by XX date because ...

  • Partner information

    This is for a course on edx.org.

  • Supporting documentation
  • Relevant Open edX discussion forum threads
🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details
Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

  • The size and impact of the changes that it introduces
  • The need for product review
  • Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

@codecov

codecov Bot commented Sep 14, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.03%. Comparing base (87cba01) to head (94b2c27).

Additional details and impacted files
@@           Coverage Diff            @@
##           master    #3248    +/-   ##
========================================
  Coverage   96.03%   96.03%            
========================================
  Files        1407     1407            
  Lines       34287    34292     +5     
  Branches     7882     8140   +258     
========================================
+ Hits        32928    32933     +5     
+ Misses       1318     1303    -15     
- Partials       41       56    +15     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

Index routing and backward compatibility are correctly implemented and covered by focused tests; remaining feedback is non-blocking.

Pull request overview

Updates Studio search consumers for separate course and library Meilisearch indexes while preserving compatibility with older backends.

Changes:

  • Adds explicit course/library index selection across search hooks and providers.
  • Parses split index names with legacy index_name fallback.
  • Adds tests for configuration parsing and index routing.
File summaries
File Description
src/search-modal/SearchUI.tsx Routes modal searches to the course index.
src/search-manager/SearchManager.ts Requires and propagates the index type.
src/search-manager/index.ts Exports the index type.
src/search-manager/data/apiHooks.ts Selects the requested index in search hooks.
src/search-manager/data/apiHooks.test.tsx Tests hook routing for both indexes.
src/search-manager/data/api.ts Parses split and legacy configuration responses.
src/search-manager/data/api.test.ts Tests configuration parsing and fallback.
src/search-manager/data/api.mock.ts Updates shared search configuration mocks.
src/library-authoring/LibraryContent.tsx Queries course placeholders from the course index.
src/library-authoring/LibraryAuthoringPage.tsx Routes library browsing to the library index.
src/library-authoring/library-filters/CollectionDropdownFilter.tsx Queries collections from the library index.
src/library-authoring/import-course/ImportDetailsPage.tsx Queries imported course blocks from the course index.
src/library-authoring/generic/manage-collections/ManageCollections.tsx Searches collections in the library index.
src/library-authoring/data/apiHooks.ts Selects indexes based on content keys.
src/library-authoring/data/apiHooks.test.tsx Tests key-based index selection.
src/library-authoring/collections/LibraryCollectionPage.tsx Routes collection searches to the library index.
src/library-authoring/collections/CollectionDetails.tsx Loads collection facets from the library index.
src/course-outline/outline-sidebar/info-sidebar/InfoSection.tsx Loads course block counts from the course index.
src/course-outline/outline-sidebar/info-sidebar/CourseInfoSidebar.tsx Loads course summary counts from the course index.
src/course-libraries/ReviewTabContent.tsx Searches downstream course blocks in the course index.
Review details
  • Files reviewed: 20/20 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

useContentSearchResults,
buildSearchQueryKey,
type SearchIndexType,
} from '../../search-manager';
* Load the Meilisearch connection details from the CMS: the URL to use, the index name, and an API key specific
* to the current user that allows it to search all content he have permission to view.
* Load the Meilisearch connection details from the CMS: the URL to use, the name of the course or library index, and
* an API key specific to the current user that allows it to search all content he have permission to view.

@bradenmacdonald bradenmacdonald left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and works well. I just have one suggestion on top of the two very minor things that Copilot already pointed out.

export const useContentFromSearchIndex = (contentIds: string[]) => {
const { client, indexName } = useContentSearchConnection();
const extraFilter = [`usage_key IN ["${contentIds.join('","')}"]`];
let indexType: SearchIndexType = 'course';

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this code is in src/library-authoring/data/apiHooks.ts, it's primarily used for libraries and I think we should leave library as the default index. 4 of 5 usages are library-only.

The one exception is ComponentUsage.tsx where we use this to list the various places in courses that use these library blocks. In that case, I think it would be cleaner to explicitly pass index = 'library' as a parameter to this function to override the default rather than using the auto-detection.

The reason is that leaving it as auto-detection, it's hard to tell where and why this code would even deal with courses at all. With an explicit parameter, it's easier to trace the usage.

@mphilbrick211 mphilbrick211 moved this from Needs Triage to In Eng Review in Contributions Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

open-source-contribution PR author is not from Axim or 2U

Projects

Status: In Eng Review

Development

Successfully merging this pull request may close these issues.

5 participants