Skip to content

Fix four TLE reader issues in satTle2elem - #1677

Open
hneogy wants to merge 8 commits into
AVSLab:developfrom
hneogy:feature/bsk-1676--tle-reader-robustness
Open

hneogy wants to merge 8 commits into
AVSLab:developfrom
hneogy:feature/bsk-1676--tle-reader-robustness

Conversation

@hneogy

@hneogy hneogy commented Nov 10, 2026 •

Copy link
Copy Markdown

Description

Fix TLE reader failures involving blank launch metadata, unsupported ephemeris types, numeric satellite names, and malformed records. Preserve the existing fail-fast default while adding explicit recovery through on_error="skip".

Parsing fixes

  • Accept blank international designators in columns 10–17 of line 1. launchDate remains None; launchNo and pol retain their defaults of 1 and "A", which are placeholders rather than known launch metadata.
  • Accept ephemeris types 0, 2, and a blank field for SGP4 conversion. Reject other types, including 4 (SGP4-XP), instead of producing states with an incompatible propagator.
  • Recognize complete record layouts before counting element-line prefixes, preserving names such as 1KUNS-PF, 2026-195B, 1 TEST, and 2 TEST. Support these prefixes in optional 2LE file titles.
  • Ignore trailing blank lines.
  • Reject recognizable orphaned element lines in satellite-name and file-title positions, including truncated lines, malformed catalog fields, and shortened headers. Intentional element-like titles can be disambiguated with a 0 prefix; satellite names retain that prefix.

Optional error policy

satTle2elem(path)                   # Default: raise on the first record error
satTle2elem(path, on_error="raise")  # Explicitly request the default
satTle2elem(path, on_error="skip")   # Warn and return readable records

on_error is keyword-only. Unsupported values raise ValueError before opening the file.

Skip mode catches ValueError and ArithmeticError during complete-record parsing or orbital conversion, including checksum failures, unsupported ephemeris types, epoch overflow, and division by zero. Each skipped record produces a printed warning. If every attempted record fails, the first original exception is raised.

File access errors, file-level format errors, and unrelated exceptions propagate in either mode. The released default remains supported, so this addition requires no deprecation period.

Documentation

Update the API docstring, release notes, and known-issue entries to describe both policies, supported inputs, metadata defaults, and recovery limitations.

Validation

  • 1,296 TLE/OMM tests passed.
  • Constellation integration test passed using Matplotlib’s headless backend.
  • 8,135 additional adversarial checks passed, covering interacting fixes and shortened orphan lines.
  • Sphinx documentation built without warnings.
  • git diff --check passed.

The full repository test suite and a clean native rebuild were not rerun for the final revision.

CelesTrak's analyst-object TLEs, and two sets in Vallado's SGP4-VER.TLE, leave
the international designator (line 1, columns 10-17) blank. _parseTle() turned
it into a launch year and number with int(), so the whole file raised
"invalid literal for int() with base 10: '  '". The launch fields now keep
their defaults when the designator is blank.
A TLE with ephemeris type 4 is an SGP4-XP element set (CCSDS 502.0-B-3
section 4.2.4.7): its B* and second-derivative columns hold BTERM and AGOM.
satTle2elem() ran SGP4 on it and returned a plausible but wrong state, while
satOmm2elem() already refuses the same set. _parseTle() now accepts only types
0, 2 and a blank column, with the OMM reader's wording.
satTle2elem() told element lines from name lines by their first character
alone, so a 3LE name such as 1KUNS-PF or 2026-195B was counted as an element
line, the line-1/line-2 counts no longer matched, and the whole file raised
"... lines starting with '1' and ... lines starting with '2'". Element lines are
now recognized by their line number followed by the blank in column 2, which
every TLE element line has and such names do not.
@hneogy
hneogy requested a review from a team as a code owner November 10, 2026 20:21
satTle2elem() let any error from one set end the whole file, so one bad
checksum or malformed field lost every valid set around it. Each set is now
parsed and converted inside its own try block; an unreadable set is skipped
with a warning that names it, as satOmm2elem() does. If no set can be read the
first error is raised, so a file whose only set is bad still raises as before.

The up-front check that line-1 and line-2 counts match still fails a file as a
whole (for example a set missing its line 2).
@hneogy
hneogy force-pushed the feature/bsk-1676--tle-reader-robustness branch from f6e484c to f9e48c8 Compare November 10, 2026 20:27
@schaubh schaubh changed the title [#1676] Fix four TLE reader issues in satTle2elem Fix four TLE reader issues in satTle2elem Nov 10, 2026
@schaubh schaubh self-assigned this Nov 10, 2026
@schaubh schaubh added the enhancement New feature or request label Nov 10, 2026
@schaubh schaubh added this to Basilisk Nov 10, 2026
@schaubh schaubh moved this to 👀 In review in Basilisk Nov 10, 2026
Recognize complete 2LE/3LE layouts before counting line prefixes,
preserving names such as "1 TEST" and "2 TEST". Ignore trailing blanks.

Reject recognizable truncated or damaged element lines in title
positions, including malformed catalogs and shortened headers.
Document the "0 " prefix for intentional element-like titles.

Add regression coverage for record preservation and orphan detection.

Validation: 1,022 TLE/OMM tests passed.
…cord

Catch arithmetic errors during parsing and orbital conversion so epoch
overflow and invalid mean motion do not discard valid records in the file.

Preserve the first original error when every record fails and allow
unrelated exceptions to propagate. Update documentation and release notes.

Add 64 regression cases covering layouts, record positions, conversion
failures, and error propagation.

Validation: 1,086 TLE/OMM tests passed.
Clarify supported ephemeris values, per-record skip behavior, and
all-failed error handling. Replace the unavailable OMM workaround
for older releases with upgrade, backport, and filtering guidance.

Document blank-designator metadata defaults and the TleData return
structure. Explain that relabeling type-4 data does not convert it to SGP4.

Validation: 23 focused tests passed; Sphinx built without warnings.
Add keyword-only on_error="raise" to preserve existing fail-fast behavior.
Allow on_error="skip" to warn and return readable records, while still
raising the first error if every record fails.

Validate policy values before opening the file, expand regression tests
for both modes, and update API docs, release notes, and known issues.

Validation: 1,296 TLE/OMM tests passed; Sphinx built without warnings.

@schaubh schaubh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Record identification and satellite-name preservation

    The original prefix check needed additional handling for valid names such as 1 TEST and 2 TEST. The updated reader recognizes complete record layouts before counting element-line prefixes, preserving names and their associated satellites. It also ignores trailing blank lines and rejects recognizable orphaned element lines—including truncated lines, malformed catalog fields, and shortened headers—in both title positions. Intentional element-like titles can use the documented 0 prefix.

  2. Malformed-record recovery

    Catching only ValueError left numerical failures capable of aborting recovery, including epoch overflow and division by zero. Skip mode now handles both ValueError and ArithmeticError during record parsing and orbital conversion. Unrelated exceptions still propagate. Regression tests cover failures at the beginning, middle, and end of files, preserve valid records and their orbital elements, and verify that an entirely unreadable file raises its first original error.

  3. Backward compatibility through on_error

    Unconditionally skipping failed records would change the released API’s behavior. The new keyword-only on_error option preserves "raise" as the default and makes recovery explicitly opt-in with "skip". Existing callers continue to stop at the first record error without receiving partial results. Invalid policy values are rejected before opening the file, and file-level validation remains active in both modes. Because the existing default remains supported, this addition does not require a deprecation period.

  4. Documentation and final validation

    The API documentation, release notes, and known-issue entries now describe the supported ephemeris types, blank-designator placeholders, title disambiguation, both error policies, and recovery limitations. The workarounds also account for the OMM reader being unavailable in affected older releases.

    Final local validation passed: 1,296 TLE/OMM tests, the constellation integration test using a headless plotting backend, and 8,135 additional adversarial checks. Sphinx built without warnings. The full repository suite and a clean native rebuild were not rerun locally. I found no additional actionable defects in the complete updated diff.

@schaubh schaubh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hneogy , thanks for the support. I made a view review comments you can look over, and I pushed commits to address them. If you agree then please close my comments and let me know. I'll then approve.

@hneogy

hneogy commented Nov 11, 2026

Copy link
Copy Markdown
Author

thanks for going through it so carefully! I've looked over your four commits and they look good to me. making the skip opt-in with on_error makes sense for the released API, catching the arithmetic errors too is a good call, and thanks for fixing my known-issue workaround, I'd missed that the OMM reader isn't in those releases. the TLE tests and CelesTrak's analyst file (219 of 219) still check out on the updated branch. resolving your comments now.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

Status: 👀 In review

Development

Successfully merging this pull request may close these issues.

satTle2elem: type 4 sets run as SGP4, and three ways one set fails the whole file

2 participants