feat(ppl): Report completed PPL queries to Query Insights Top N - #5760
feat(ppl): Report completed PPL queries to Query Insights Top N#5760KishoreKicha14 wants to merge 1 commit into
Conversation
PR Code Analyzer ❗AI-powered 'Code-Diff-Analyzer' found issues on commit 393b517.
The table above displays the top 10 most important findings. Pull Requests Author(s): Please update your Pull Request according to the report above. Repository Maintainer(s): You can Thanks. |
aa97a0c to
c73c5a5
Compare
PR Reviewer Guide 🔍(Review updated until commit 393b517)Here are some key observations to aid the review process:
|
PR Code Suggestions ✨Latest suggestions up to 393b517 Explore these optional code suggestions:
Previous suggestionsSuggestions up to commit 5355c10
Suggestions up to commit 31fd289
Suggestions up to commit f4ba88d
Suggestions up to commit fae8fa0
Suggestions up to commit d05d99d
|
c73c5a5 to
d05d99d
Compare
|
Persistent review updated to latest commit d05d99d |
d05d99d to
fae8fa0
Compare
|
Persistent review updated to latest commit fae8fa0 |
fae8fa0 to
f4ba88d
Compare
|
Persistent review updated to latest commit f4ba88d |
f4ba88d to
31fd289
Compare
|
Persistent review updated to latest commit 31fd289 |
31fd289 to
5355c10
Compare
|
Persistent review updated to latest commit 5355c10 |
Emit a Query Insights record for every completed PPL query so it appears in the Top N overview alongside DSL queries. Each record carries the query source (PPL), the resolved indices, and query-total latency, CPU, and memory from the task resource-tracking framework. Child DSL searches spawned by a PPL query (e.g. a join's per-table scans) are tagged with a parent marker (X-Query-Insights-Parent) so Query Insights can associate them with their parent as sub-queries. The marker is carried on a ThreadLocal across the engine's worker and complex-worker thread hops and re-applied as a task header on the background scan pool. This is reporting only: no phase breakdown and no profiling changes. The profiling subsystem is left as-is for a future change. Known limitation: for multi-scan queries (joins), child capture can be partial under background-pool thread contention; parent reporting is unaffected. Signed-off-by: Kishore Kumaar Natarajan <kkumaarn@amazon.com>
5355c10 to
393b517
Compare
|
Persistent review updated to latest commit 393b517 |
Description
PPL queries don't currently show up in Query Insights (only DSL queries do). This wires PPL into the same Top N reporting so you can see PPL query latency/CPU/memory next to everything else.
When a PPL query finishes, we send a record to Query Insights with the source (PPL), the indices it read, and the total latency/CPU/memory (from the task resource tracking framework, same numbers OpenSearch already tracks per task).
A PPL query can fan out into multiple DSL searches under the hood — a join, for example, scans each table separately. We tag those child searches with a header (X-Query-Insights-Parent) pointing back at the PPL query that spawned them, so Query Insights can show them as sub-queries on the detail page. Getting the tag to survive was the fiddly part: the engine hops the query across a couple of thread pools (worker → complex-worker → background scan), and the header doesn't ride along automatically, so we carry it on a ThreadLocal and re-stamp it before each background search.
Screenshots:
Query Overview Page
Query Detail Page with DSL subqueries derived
Related Issues
Resolves #[Issue number to be closed when this PR is merged]
Check List
--signoffor-s.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.