Field selection

[rgoldberg]

Unfinished rough draft.

Custom EBNF Grammar

Meta-Grammar

Within the definition of this custom EBNF grammar:

• All text is case-sensitive.
• Uppercase words represent placeholders for symbols, text, or production rules.
• Whitespace separating tokens in the grammar is insignificant.
• All other characters are used verbatim.

Grammar

When using this custom EBNF grammar to define a syntax:

• All text is case-sensitive.
• Whitespace separating tokens defined by this grammar is insignificant.
• Whitespace within tokens remains significant.
• Syntaxes defined by this grammar may have their own whitespace rules.

Multiplicity

A value's multiplicity is either:

• Scalar: a single value (e.g., an integer, a string, etc.).
• Repetition: a sequence of 0 or more elements. Empty or single-element
  repetitions remain repetitions, not scalars.

A construct's multiplicity is either:

• Inherently scalar: Every occurrence of the construct is a scalar.
• Inherently repetition: Every occurrence of the construct is a repetition.
• Content-dependent: The multiplicity of an occurrence of the construct is
  the same as its content.
• Context-dependent: The multiplicity of an occurrence of the construct is
  whatever is required by its context.

Convenience multiplicity categories for values include:

• Implicit scalars: Scalars whose construct is not inherently scalar.
• Implicit repetitions: Repetitions whose construct is not inherently
  repetition.

Null

null

Context-dependent multiplicity.

The absence of a value.

Bare Characters

BARE

Inherent scalar.

BARE is any character other than \.

BARE evaluates to BARE.

Escaped Characters

\ESCAPED

Inherent scalar.

ESCAPED is any character.

\ESCAPED evaluates to ESCAPED (e.g., \n evaluates to n).

Text Classification

Text Character Composition

• Text described as bare contains only bare characters.
• Text described as escaped contains at least one escaped character.
• Text not described as bare or escaped may contain any mixture of bare &
  escaped characters.

The underlying grammar may assign different semantics to bare text & escaped
text.

Text Function

Literal Text

Literal text functions as syntactic operands (i.e. data). Literal text may
consist of bare and/or escaped characters.

Syntax Text

Syntax text functions as syntactic operators. Syntax text consists strictly
of bare characters.

Escaping any character within a token that (taken in its entirety) would
otherwise be syntax text renders that token literal text.

The context of bare text may influence whether it is syntax text or literal
text.

Literal Text

"TEXT"

Inherent scalar.

"" is the escape sequence for an escaped " in TEXT.

Scalar Descriptions

{DESCRIPTION}

Inherent scalar.

DESCRIPTION describes valid literal scalars, e.g., {non-negative integer}.

DESCRIPTION must not begin with {.

Repetition Descriptions

{{DESCRIPTION}}

Inherent repetition.

DESCRIPTION describes valid literal elements for a repetition, e.g.,
{{non-negative integer}}.

Value Placeholders (Non-Terminal Symbols)

Value Placeholder Definitions

PLACEHOLDER = DEFINITION

Content-dependent multiplicity.

Binds a placeholder symbol to its definition.

A placeholder's default value, when absent, is its
implicit or explicit default.

Value Placeholder References

Defined via a modified meta-grammar supporting optionals:

<PLACEHOLDER[\INITIAL][\\ANYWHERE][\\\ENTIRE]>

Content-dependent multiplicity.

INITIAL, ANYWHERE & ENTIRE restrict values for PLACEHOLDER as
specified for Escaping Text Placeholders, except
\} is not supported as an escape sequence; the only supported escape sequence
is \>, which evaluates to >.

PLACEHOLDER must not contain any escape sequences or an unescaped \;
consecutive \ exclusively separate PLACEHOLDER, INITIAL, ANYWHERE &
ENTIRE.

Token Sets

A token set is a set of literal tokens.

Token Set Productions

[:NAME:] = EXPRESSION

Context-dependent multiplicity.

Token Set References

[:NAME:]

Context-dependent multiplicity.

References a token set NAME.

NAME is a bare token that must not contain :].

Placeholder Token Sets

[:<PLACEHOLDER>:]

Context-dependent multiplicity.

A token set containing the tokens permissible as values for placeholder
PLACEHOLDER.

Escaping Text Placeholders

Defined via a modified meta-grammar supporting optionals:

{text[\INITIAL][\\ANYWHERE][\\\ENTIRE]}

Inherent scalar.

Non-empty text that supports escaped characters.

INITIAL, ANYWHERE & ENTIRE specify bare tokens that are interpreted as
syntax text.

INITIAL, ANYWHERE & ENTIRE:

• Must not be empty.
• Must not contain any \ except as a member of \}.

Tokens in INITIAL, ANYWHERE & ENTIRE are specified via:

• [:NAME:] includes all tokens from token set NAME. Each [: must pair with
  a :].
• \} includes }.
• Any other character includes itself.

If the entire bare text matches a token in ENTIRE, the entire text is a syntax
token.

Otherwise, syntax tokens are found by iterating over characters from start to
end:

• On each iteration, the longest bare token starting at the current character
  that matches any of the following tokens, if any, is a syntax token:
  • Any token in ANYWHERE.
  • If the current character is the first character: any token in INITIAL.
• If the current character, or the last character of any found syntax token, is
  the last character, the iteration terminates.
• Otherwise, the iteration continues with the current character set to:
  • If a syntax token was found: the character after the syntax token.
  • Otherwise: the character after the current character.

Examples

{text\@.\\=:/,}

• \ must always be escaped, as it must be in any {text...}.
• @ & . must be escaped if first (from INITIAL).
• =, :, / & , must be escaped throughout (from ANYWHERE).

Groupings

( EXPRESSION )

Content-dependent multiplicity.

Choices

A | B

Content-dependent multiplicity.

A or B.

Optionals

[ EXPRESSION ]

Content-dependent multiplicity.

0 or 1 occurrence of EXPRESSION.

Optionality States

An expression has 3 mutually exclusive possible optionality states:

• Required: Neither the expression nor any of its ancestors are optional.
• Directly optional: The expression is, or is a child of, an optional.
• Transitively optional: Neither the expression nor its parent are an
  optional, but one of its other ancestors is an optional.

An expression is optional if it is either directly optional or transitively
optional.

Presence States

An optional expression has 3 mutually exclusive possible presence states:

• Present: A token is present.
• Directly absent: All ancestors of the expression are present, but the
  expression is itself absent.
• Transitively absent: At least one ancestor of the expression is absent.

An optional expression is absent if it is either directly absent or
transitively absent.

Repetitions (1 or more)

ELEMENT+

Inherent repetition.

ELEMENT must not match empty text.

Empty

empty

Inherent repetition.

A repetition containing zero elements. Distinct from null & from the empty
string ("").

Delimited Repetitions (1 or more)

ELEMENT … SEPARATOR

Inherent repetition.

e.g.:

• ( "a" | "b" | "c" ) … "," matches:
  • b
  • c,a,c,b,a,c,b
• ( "2" | "1" | "3" ) … ( " " | "/" ) matches:
  • 3
  • 1 3/1/2 2/3

Neither ELEMENT nor SEPARATOR may match empty text.

Comments

(* COMMENT *)

No multiplicity.

Semantic Comments

Semantic comments follow specific formats to concisely attach semantic
information to syntax constructs. They are not formally part of the custom EBNF
grammar; they are an adjunct custom grammar.

Default Value Comments

Default value comments specify explicit defaults applicable to optional
placeholders.

Self Default Value Comments

PLACEHOLDER "=" DEFINITION "(*" [ "direct" | "transitive" ] "default:" DEFAULT "*)"

DEFAULT is a default applicable for a PLACEHOLDER that is:

• If "direct" is present: directly absent.
• If "transitive" is present: transitively absent.
• Otherwise: absent.

Only 1 self default value comment may be attached to PLACEHOLDER's definition,
unless there are exactly 2, one of which is for "direct", the other of which
is for "transitive".

Descendant Default Value Comments

Defined via the custom EBNF grammar, instead of via the meta-grammar:

PLACEHOLDER "=" DEFINITION "(*" [ "direct" | "transitive" ] "default" DESCENDANT ":" DEFAULT "*)"

DEFAULT is a default applicable for a DESCENDANT that is a descendant of a
PLACEHOLDER that is:

• If "direct" is present: directly absent.
• If "transitive" is present: transitively absent.
• Otherwise: absent.

Only 1 descendant default value comment for DESCENDANT may be attached to
PLACEHOLDER's definition, unless there are exactly 2, one of which is for
"direct", the other of which is for "transitive".

Explicit & Implicit Defaults

• An explicit default is specified by any variant of a default value
  comment.
• An implicit default is the value applied when no explicit default value
  applies to a token.
  • The implicit default for scalars is null.
  • The implicit default for repetitions is:
    • If transitively absent: null.
    • If directly absent: empty.

Global Defaults

A token's global default for a given presence state is:

• The applicable self default attached to its definition, if any.
• Otherwise: its applicable implicit default.

Defaults for Absent Tokens

An absent TOKEN token's value is, in descending precedence:

• DEFAULT from the furthest ancestor from TOKEN that has an applicable
  descendant default value comment whose DESCENDANT is TOKEN.
  • Ancestors further from TOKEN take precedence because they have more
    context: PLACEHOLDER knows the defaults appropriate for each of its
    descendants, so overrides them if necessary.
  • Descendants of PLACEHOLDER that have the same type may be assigned
    different defaults by attaching different default value comments along
    divergent branches of DEFINITION.
• TOKEN's global default.

Field-Selection Options

Uses the custom EBNF grammar defined in ebnf.md.

Remove --price from search after field addition is implemented.

Positions & Indices

List elements are located at 1-based integer positions.

Positions are referenced via integer indices. Each index has an effective index,
computed as follows:

• If non-negative $index$: $index$.
• If negative $index$: $length + 1 + index$ (where $length$ is the list's
  length).

Index 0 represents a pseudo-position that precedes all list elements.

Accessing an invalid position causes an error to be reported. Invalid positions
are:

• Non-positive positions (despite non-positive indices being valid).
• Positions that are greater than a list's length.

A negative index may be used to reference an element, but the index of an
element is always positive.

Display Commands

A display command is a command whose primary function is to display data.

mas display commands are:

• config
• list
• lookup
• outdated
• search

Output Formats

Display commands support multiple output formats:

• Tabular: Row per item (normally an app), column per field.
• Key-Value: Key-value pair on its own row per field, blank line between
  items.
• JSON: JSON object per item, key-value pair per field.

Output configuration is universal across output formats unless otherwise
specified.

The default output format of each display command is either tabular or
key-value, never JSON. JSON is enabled via --json.

The default output formats are:

Command	Format
config	key-value
list	tabular
lookup	key-value
outdated	tabular
search	tabular

Field Specs

A field spec & its position define the configuration for a single output
field:

• Inclusion
• Order
• Label (tabular header; key-value / JSON key)
• Value format
• Item sorting

Fields Configs

A fields config specifies which fields are included in the output, along
with other output configuration.

A fields config may include:

• Rules that apply to all fields
• Field-specific field specs

Named Fields Configs

A fields config may be persisted with a name by which it may be referenced:

• On the command line
• By other persisted fields configs

A named fields config is found by returning the first fields config found with
that name while iterating through the context stack for the current command from
most to least specific context; if no match is found, an error is reported.

A hardcoded global none fields config includes no fields; it uses the global
defaults for all rules.

Each context has:

• A hardcoded all fields config that includes all fields
• A hardcoded standard fields config that includes default fields
• A user-configurable default fields config

All fields configs are ultimately derived transitively from none or the
contextual all.

If the user has not configured default for a context, it defaults to
standard.

For contexts that output JSON, standard is the same as all.

Effective Fields Config

The effective fields config is that which is used to generate output.

The effective fields config is always based on a single named fields config,
which is either explicitly referenced in, or implicitly selected by, a command
line:

• An absolute effective fields config is based on implicitly selected none
• A relative effective fields config is based on:
  • An explicitly named fields config.
  • default for the current context, if no fields config was explicitly named.

Semantically Significant Characters

Semantically significant characters are:

• Non-whitespace
• Escaped whitespace
• Bare whitespace between the first significant character & the last significant
  character in a token

Semantically Insignificant Characters

Semantically insignificant characters are:

• Bare whitespace before the first significant character in a token
• Bare whitespace after the last significant character in a token

Defaults for Absent Tokens

The following is inserted before all other cases in the
absent token value algorithm:

• The value for the given TOKEN token in the effective fields config.

Headers Option

Controls the visibility & format of the headers row.

headers-option = "--no-headers" | "--headers" [ "=" <headers-format> ] (* default: "--no-headers" *)

<headers-format> is reserved for future implementation. It will format color,
bold, underlined, etc.

Fields Option

Contexts

A context exists for each command in a command stack:

Command Stack	Most Specific Context	Least Specific Context
mas config	mas.config	mas
mas list	mas.list	mas
mas lookup	mas.lookup	mas
mas outdated	mas.outdated	mas
mas search	mas.search	mas

The context stack for each command is ordered from the most to the least
specific context (e.g., the context stack for mas outdated is mas.outdated,
mas).

Nonexistent Fields & Values

Nonexistent Fields

Nonexistent fields are guaranteed to not exist in any input, which is
determinable only for formats that specify all potential input fields up front,
e.g.: CSV, or JSON with a JSON schema.

Other formats cannot guarantee field nonexistence, e.g.: JSON without a JSON
schema, for which a field may be present for some JSON objects, but absent for
others.

A reference to a nonexistent field is an error.

Absent Values

Initial Version

• Tabular: A column is output for each field spec regardless of whether a
  value exists for a given item (absent values are output as empty strings).
• Key-Value: A key-value pair is output for the field label & value iff a
  non-null value exists.
• JSON: A key-value pair is output for the field label & value iff a value
  exists (which could be any value; e.g., for JSON input, null, true,
  false, 0, 1, "", "null", etc. are all values).

Future Versions

Configurable behavior per output format:

• error: Report an error for absent values.
• omit: Omit output for absent values.
• emit: Output a configurable default value for absent values.

Fields Option

fields-option = "--fields" {whitespace} <fields-config>
fields-config = <absolute-config> | <relative-config>

The effective fields config is:

• If <fields-option> is omitted: default for the current context.
• If <absolute-config> is supplied: sourced solely from <absolute-config>;
  this is the equivalent of the base fields config being none.
• If <relative-config> is supplied: sourced from the base fields config as
  modified by <relative-config>.

<absolute-config> & <relative-config> are overlaid on top of the base fields
config for the current command line only; they do not persistently affect named
fields configs.

Absolute Config

absolute-config = <first-absolute-field-spec> [ ( <field-spec-separator> <subsequent-absolute-field-spec> )+ ]

first-absolute-field-spec = <first-absolute-field-name> <field-modifiers>
first-absolute-field-name = <subsequent-absolute-field-name\[:<base-fields-config-section-prefix>:][:<field-order-section-prefix>:][:<item-sort-section-prefix>:][:<field-specs-section-prefix>:]>

subsequent-absolute-field-spec = <subsequent-absolute-field-name> <field-modifiers>
subsequent-absolute-field-name = {text\\[:<label-modifier-prefix>:][:<format-modifier-prefix>:][:<sort-modifier-prefix>:][:<field-spec-separator>:]}

Each field spec in an <absolute-config> appends a new output for its field.

Relative Config

relative-config = [ <base-fields-config-section> ] [ <field-order-section> ] [ <item-sort-section> ] [ <field-specs-section> ]

• Computes the effective fields config from the base fields config as adjusted
  by the command line.
• If <base-fields-config-section> is omitted, the base fields config is the
  contextual default.

Base Fields Config Selection

base-fields-config-section        = <base-fields-config-section-prefix> [ <base-fields-config-name> ]
base-fields-config-section-prefix = "@"

base-fields-config-name = {text\\[:<field-order-section-prefix>:][:<item-sort-section-prefix>:][:<field-specs-section-prefix>:]}

• If <base-fields-config-name> is omitted, the base fields config is all.
• If no fields config is named <base-fields-config-name> in the context stack,
  an error is reported.

Field Ordering

field-order-section        = <field-order-section-prefix> [ <field-order-option-set> ]
field-order-section-prefix = "/"

field-order-option-set = <original-order-option-set> | <sort-option-set>

original-order-option-set = [ <original-order-option>+ ] <original-order> [ <original-order-option>+ ]
original-order-option     = <original-order> | <direction>

original-order = "o" (* supported only for key-value & JSON, not for tabular *)

Fields are ordered:

• If <field-order-section> is:
  • Absent: per inherited field order.
  • Present: if <field-order-option-set> is:
    • Absent: per order in effective fields config; i.e. field order from
      base fields config as modified by <field-specs-section>.
    • Present, if <original-order-option-set> is:
      • Absent, <sort-option-set> is guaranteed present; if <source> is:
        • <input>, by sorting by name.
        • <output>, by sorting by label.
      • Present, per original order, e.g., the ordering of keys in a JSON
        object.

Item Sorting

item-sort-section        = <item-sort-section-prefix> [ <item-sort-option-set> ]
item-sort-section-prefix = "//"

item-sort-option-set = <item-sort-option>+
item-sort-option     = <reset> | <direction>

reset = <reset-to-contextual> | <reset-to-global>

reset-to-contextual = "r"
reset-to-global     = "R"

<reset-to-contextual> resets all inherited sort options to the contextual
defaults for the active output format from
Default Sort Options (not to the defaults from the base
fields config or global defaults).

<reset-to-global> resets all inherited sort options to the global defaults
(not to the defaults from the base fields config or
Default Sort Options).

<ascending> tiebreaks item sorting by input order, <descending> by reverse
input order.

Field Spec References

field-spec-reference         = <named-field-spec-reference> | <indexed-field-spec-reference>
named-field-spec-reference   = <reference-field-name> [ <index-prefix> <index> ] (* default index: "1" *)
indexed-field-spec-reference = <index-prefix> <index>

reference-field-name = {text\\[:<index-prefix>:][:<label-modifier-prefix>:][:<format-modifier-prefix>:][:<sort-modifier-prefix>:][:<field-spec-separator>:]}

index-prefix = "@"
index        = {integer}

The reference fields config consists of field specs in the input to the
current section, maintaining their input order (i.e. the order of field specs in
the reference fields config is unaffected by any inserts, moves, or removals
performed in the current section), with:

• <named-field-spec-reference> including only field specs for field
  <reference-field-name>.
• <indexed-field-spec-reference> including all field specs.

While the position of field specs in the reference fields config must not be
modified:

• Their <field-modifiers> may be modified.
• A null may replace them in their position, which does not modify any other
  field specs' positions.

When accessing a field spec position via an index:

• If the position access reports an error, or if a null exists at the effective
  index, an error is reported.
• Otherwise, the field spec at the effective index is the referenced field spec.

Relative Field Specs

field-specs-section        = <field-specs-section-prefix> [ <field-spec> … <field-spec-separator> ]
field-specs-section-prefix = "."

field-spec = <insert-field-spec> | <sourced-field-spec>

insert-field-spec = <insert> <insert-field-name> <field-modifiers>

sourced-field-spec = <overlay-field-spec> | <move-field-spec> | <remove-field-spec>

overlay-field-spec = <field-spec-reference\[:<insert>:][:<move>:][:<remove>:]> <field-modifiers>
move-field-spec    = <move> <field-spec-reference> <field-modifiers>
remove-field-spec  = <remove> <field-spec-reference>

insert = "+"
move   = "~"
remove = "-"

insert-field-name = {text\\[:<label-modifier-prefix>:][:<format-modifier-prefix>:][:<sort-modifier-prefix>:][:<field-spec-separator>:]}

field-spec-separator = ","

The source field spec of given <sourced-field-spec> fs is the field spec
from the reference fields config directly modified by fs, which is referenced
via fs's <field-spec-reference> (this prevents field specs that were
inserted in a section from being modified by other field specs in the same
section).

The $previous$ index is the index in the current effective fields config where
the direct results of the immediately preceding field spec were effected. It is
initially set to 0 (i.e. before the first field spec in the current effective
fields config).

Field specs perform actions as follows:

• <insert-field-spec>:
  • Inserts a new field spec immediately after $previous$.
  • Sets $previous$ to the new field spec's index in the effective fields
    config.
• <overlay-field-spec>:
  • Overlays its field modifiers onto source.
  • Sets $previous$ to source's index in the effective fields config.
• <move-field-spec>:
  • Moves source to immediately after $previous$.
  • Sets $previous$ to source's new index in the effective fields config.
• <remove-field-spec>:
  • Sets $previous$ to the index immediately before source's index in the
    effective fields config.
  • Removes source from the effective fields config.
  • Replaces source with null in the reference fields config.

Field Modifiers

field-modifiers = [ <label-modifier> ] [ <format-modifier> ] [ <sort-modifier> ]

Labeling

label-modifier        = <label-modifier-prefix> [ <label> ] (* transitive default label: {field name from the containing field-spec} *)
label-modifier-prefix = "="
label                 = {text\\[:<format-modifier-prefix>:][:<sort-modifier-prefix>:][:<field-spec-separator>:]} (* default: "" *)

• A field spec's <label> is its:
  • Header for tabular.
  • Key for key-value & JSON.

Formatting

format-modifier        = <format-modifier-prefix> [ <format> ]
format-modifier-prefix = ":"
format                 = {text\\[:<sort-modifier-prefix>:][:<field-spec-separator>:]} (* conceptualized; documentation pending *)

• A <format-modifier> without a <format> (:) clears inherited formatting
  for the field.

Sorting

sort-modifier        = <sort-modifier-prefix> [ <sort> ]
sort-modifier-prefix = "/"
sort                 = <sort-priority> [ <sort-option-set> ] | <sort-option-set>

• A <sort-modifier> without a <sort> (/) clears inherited sorting for the
  field.
• A field without an effective <sort> does not affect row sorting.

Sort Priority

sort-priority = {non-negative 64-bit integer}

A field with an effective <sort>:

• Requires a non-null effective <sort-priority>, otherwise an error is
  reported.
• Takes sort precedence over fields:
  • With a numerically higher <sort-priority> (i.e. lower numbers have higher
    priority).
  • Succeeding it with the same <sort-priority>.

Sort Options

sort-option-set = <sort-option>+
sort-option     = <source> | <direction> | <case-sensitivity> | <localization> | <grouping> | <interpretation> | <boundaries>

• Each sort option of the same type (e.g., each <source> value) is
  functionally mutually exclusive with all other values of the same type.
• If multiple sort options of the same type are supplied (e.g., <ascending> &
  <descending>), the last overrides the preceding.

source = <input> | <output> (* default: <input> *)
input  = "I"
output = "O"

If an input value is modified by a format for output, <input> sorts based on
the input, not the output. <output> is the opposite.

e.g., for an input timestamp being output as an ISO date (without time):

• <input> sorts by the input timestamp.
• <output> sorts by the output ISO date (without time).

direction  = <ascending> | <descending> (* default: <ascending> *)
ascending  = "a"
descending = "d"

case-sensitivity = <sensitive> | <insensitive> (* default: <sensitive> *)
sensitive        = "s"
insensitive      = "i"

localization = <canonical> | <localized> (* default: <canonical> *)
canonical    = "c"
localized    = <localized-prefix> [ <locale-name-fence> [ <locale-name> ] <locale-name-fence> ]

localized-prefix = "l"

locale-name-fence = "~"
locale-name       = {text\\[:<locale-name-fence>:]} (* default: {system default locale name} *)

<canonical> uses a locale-independent comparison, while <localized> uses a
locale-dependent comparison. This also affects numeric fractional part & integer
grouping separators.

grouping  = <ungrouped> | <grouped> (* default: <ungrouped> *)
ungrouped = "u"
grouped   = "g"

interpretation = <lexical> | <numeric> | <price> | <version> (* default: <lexical> *)
lexical        = "x"
numeric        = "n"
price          = "p"
version        = "v"

boundaries        = <boundaries-prefix> [ <boundaries-option-set> ] ( <grouped-boundaries> | <ungrouped-boundaries> ) (* default: "b~_~" *)
boundaries-prefix = "b"

boundaries-option-set = <boundaries-option>+
boundaries-option     = <collapse-contiguous>

collapse-contiguous = "%"

whitespace-boundary-modifier = <grouped-whitespace-boundary-modifier> | <ungrouped-whitespace-boundary-modifier>

grouped-boundaries                     = "~" ( ( <grouped-whitespace-boundary-suppressor> | <grouped-whitespace-boundary-modifier> ) [ <grouped-boundaries-list> ] | [ <grouped-whitespace-boundary-modifier> ] <grouped-boundaries-list> [ <grouped-whitespace-boundary-modifier> ] ) "~"
grouped-whitespace-boundary-suppressor = "~"
grouped-whitespace-boundary-modifier   = "_"
grouped-boundaries-list                = <boundary-list> … <group-separator>
group-separator                        = "_"

ungrouped-boundaries                     = "_" ( ( <ungrouped-whitespace-boundary-suppressor> | <ungrouped-whitespace-boundary-modifier> ) [ <ungrouped-boundaries-list> ] | [ <ungrouped-whitespace-boundary-modifier> ] <ungrouped-boundaries-list> [ <ungrouped-whitespace-boundary-modifier> ] ) "_"
ungrouped-whitespace-boundary-suppressor = "_"
ungrouped-whitespace-boundary-modifier   = "~"
ungrouped-boundaries-list                = <boundary-list> … <group-joiner>
group-joiner                             = "~"

boundary-list       = <boundary-characters> | <multi-character-boundary> | <character-class>
boundary-characters = {text\\[:<character-class-fence>:][:<multi-character-boundary-fence>:]~_}

multi-character-boundary       = <multi-character-boundary-fence> <multi-character-boundary-text> <multi-character-boundary-fence>
multi-character-boundary-fence = "%"
multi-character-boundary-text  = {text\\[:<multi-character-boundary-fence>:]}

character-class       = <character-class-fence> <character-class-name> <character-class-fence>
character-class-fence = ":"
character-class-name  = "alnum" | "alpha" | "ascii" | "blank" | "cntrl" | "digit" | "graph" | "lower" | "print" | "punct" | "space" | "upper" | "word" | "xdigit"

Input is tokenized by boundaries assigned to ordered sort precedence
boundary groups; each group's members share the same sort precedence.

Boundaries & boundary groups are either explicit or implicit.

Explicit boundaries are defined in <boundary-list>s:

• Each character in a <boundary-characters> is itself an explicit boundary.
• Each <multi-character-boundary-text> is itself an explicit boundary.
• Each character belonging to a POSIX character class named
  <character-class-name> is itself an explicit boundary.

The last occurrence of an explicit boundary for a value across all
<boundary-list>s overrides all other boundaries (explicit or implicit) for the
same value.

Boundaries take sort precedence over all boundaries in all groups succeeding
their group & over all non-boundary characters.

Explicit groups are specified in <boundaries>:

• In <grouped-boundaries>, all boundaries within a <boundary-list> belong to
  a single group.
• In <ungrouped-boundaries>:
  • By default:
    • All boundaries from a single <character-class> belong to a single group.
    • Each other boundary belongs to its own group.
  • A <group-joiner> merges the immediately succeeding group into the
    immediately preceding group.

By default, all whitespace characters are assigned to an implicit endmost group;
<grouped-whitespace-boundary-suppressor> &
<ungrouped-whitespace-boundary-suppressor> suppress it.

• A trailing <whitespace-boundary-modifier> (without a leading one)
  includes in the last explicit group all whitespace characters for which no
  explicit boundaries exist.
• A leading <whitespace-boundary-modifier> (without a trailing one)
  positions the whitespace implicit group before all explicit groups.
• Both a leading & trailing <whitespace-boundary-modifier> include in the
  first explicit group all whitespace characters for which no explicit
  boundaries exist.

By default, contiguous boundaries in input are preserved as separate characters.
<collapse-contiguous> collapses contiguous boundaries belonging to the same
boundary group into one.

Default Sort Options

Format	Type	Default
Tabular	Text	Iailgnb~_~
Tabular	Price	Iailgpb~_~
Tabular	Version	Iailuvb~_~
Tabular	Path	Iailgnb~/~
Key-Value	Text	Iailgnb~_~
Key-Value	Price	Iailgpb~_~
Key-Value	Version	Iailuvb~_~
Key-Value	Path	Iailgnb~/~
JSON	Text	Iascgnb~_~
JSON	Price	Iascgpb~_~
JSON	Version	Iascuvb~_~
JSON	Path	Iascgnb~/~