Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/user/admin/settings.rst
Original file line number Diff line number Diff line change
Expand Up @@ -214,22 +214,22 @@ Version
Description
-----------

Prunes a wildcard index expression down to the concrete indices that can hold data in the query's ``@timestamp`` range, so fewer indices and shards are touched. The primary use currently is to avoid exhausting the open point-in-time (PIT) context limit when a query would otherwise open a reader context over many indices.
Prunes a wildcard index expression down to the concrete indices that can hold data in the query's ``@timestamp`` range, so fewer indices and shards are touched. The primary use currently is to avoid exhausting the open point-in-time (PIT) context limit when a query would otherwise open a reader context over many indices. Enabled by default.

Pruning only applies to a wildcard expression whose query filters on a ``@timestamp`` range; anything else is left untouched, and any failure while probing the cluster falls back to querying the full expression. Weigh these limitations before enabling it:
Pruning only applies to a wildcard expression whose query filters on a ``@timestamp`` range; anything else is left untouched, and any failure while probing the cluster falls back to querying the full expression. Weigh these limitations before turning it off:

1. An index whose shards are all unavailable is pruned rather than reported, because ``_field_caps`` does not surface per-index failures. Such a query returns fewer rows instead of an error.
2. Pruning fixes the list of index names, so an index created or deleted between pruning and PIT creation, by a rollover or retention policy for instance, is missed or fails the query. The interval between the two is short, so this is unlikely in practice.
3. An expression that matches an alias or a data stream is never pruned, because a filtered alias contributes a filter and routing that are resolved from the expression itself and so would be silently dropped.
4. Pruning probes the cluster with the ``indices:admin/resolve/index`` and ``indices:data/read/field_caps`` actions. A principal lacking either permission falls back to querying the full expression silently, so pruning simply never takes effect.
4. Pruning probes the cluster with the ``indices:admin/resolve/index`` and ``indices:data/read/field_caps*`` actions, both granted by the ``ppl_full_access`` role of the security plugin since 3.9. A principal lacking either permission falls back to querying the full expression silently, so pruning simply never takes effect.

Pruning is also skipped when it would not reduce the read, that is when no index is excluded. The query then uses the original wildcard expression and reads exactly the same indices.

Enable it with::
Disable it with::

>> curl -H 'Content-Type: application/json' -X PUT localhost:9200/_plugins/_query/settings -d '{
"transient" : {
"plugins.query.pruning.enabled" : true
"plugins.query.pruning.enabled" : false
}
}'

Expand All @@ -242,7 +242,7 @@ Result set::
"plugins" : {
"query" : {
"pruning" : {
"enabled" : "true"
"enabled" : "false"
}
}
}
Expand All @@ -251,7 +251,7 @@ Result set::

Settings:

1. The default value is false.
1. The default value is true.
2. This setting is node scope.
3. This setting can be updated dynamically.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ setup:
plugins.calcite.enabled: true
# Without this a Calcite failure is silently answered by the v2 engine, which never prunes.
plugins.calcite.fallback.allowed: false
plugins.query.pruning.enabled: true
- do:
indices.put_index_template:
name: pruning-it
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,7 @@ public class OpenSearchSettings extends Settings {
public static final Setting<?> QUERY_PRUNING_ENABLED_SETTING =
Setting.boolSetting(
Key.QUERY_PRUNING_ENABLED.getKeyValue(),
false,
true,
Setting.Property.NodeScope,
Setting.Property.Dynamic);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,18 @@ void testPplValuesMaxLimitSetting() {
assertEquals(5000, newLimit);
}

@Test
void testQueryPruningEnabledSetting() {
when(clusterSettings.get(ClusterName.CLUSTER_NAME_SETTING)).thenReturn(ClusterName.DEFAULT);
when(clusterSettings.get(not((eq(ClusterName.CLUSTER_NAME_SETTING))))).thenReturn(null);
OpenSearchSettings settings = new OpenSearchSettings(clusterSettings);

assertEquals(true, settings.getSettingValue(Settings.Key.QUERY_PRUNING_ENABLED));

settings.new Updater(Settings.Key.QUERY_PRUNING_ENABLED).accept(false);
assertEquals(false, settings.getSettingValue(Settings.Key.QUERY_PRUNING_ENABLED));
}

@Test
void testDeserializationStructuralLimitSettings() {
when(clusterSettings.get(ClusterName.CLUSTER_NAME_SETTING)).thenReturn(ClusterName.DEFAULT);
Expand Down
Loading