diff --git a/docs/source/Support/bskKnownIssuesSnippets/1676-tle-blank-designator.rst b/docs/source/Support/bskKnownIssuesSnippets/1676-tle-blank-designator.rst new file mode 100644 index 00000000000..32638816725 --- /dev/null +++ b/docs/source/Support/bskKnownIssuesSnippets/1676-tle-blank-designator.rst @@ -0,0 +1,9 @@ +- In versions 2.8.28 through 2.12.2, :ref:`tleHandling` ``satTle2elem()`` raised + ``ValueError`` for a whole file when any set had a blank international designator + (columns 10-17 of line 1), as CelesTrak's analyst-object TLEs do. Such sets now load with + ``launchDate`` left as ``None``; ``launchNo`` and ``pol`` retain their defaults of + ``1`` and ``"A"``, which do not represent known launch metadata for these objects. + Upgrade to a version containing this fix or backport the reader change to read + these sets. Until then, omit the complete affected sets to read the rest of the file. + The :ref:`ommHandling` reader was introduced during the 2.13 beta cycle and is + unavailable in the affected releases. diff --git a/docs/source/Support/bskKnownIssuesSnippets/1676-tle-ephemeris-type-4.rst b/docs/source/Support/bskKnownIssuesSnippets/1676-tle-ephemeris-type-4.rst new file mode 100644 index 00000000000..db3182357c1 --- /dev/null +++ b/docs/source/Support/bskKnownIssuesSnippets/1676-tle-ephemeris-type-4.rst @@ -0,0 +1,11 @@ +- In versions 2.8.28 through 2.12.2, :ref:`tleHandling` ``satTle2elem()`` converted + TLEs with ephemeris type 4 (SGP4-XP) using plain SGP4, + which can produce incorrect states because those sets carry BTERM and AGOM in + the B* and second-derivative columns. Unsupported sets now raise ``ValueError`` + by default. With ``on_error="skip"``, they are skipped with a printed warning, + while other readable sets are returned. If every attempted set fails in skip + mode, the first error is raised; a file containing only otherwise valid type-4 + sets raises ``ValueError`` in either mode. With an affected version, check + column 63 of line 1 and exclude complete sets unless that field is blank, + ``0`` or ``2``. + Changing a type-4 field to ``0`` or ``2`` does not convert SGP4-XP data to SGP4. diff --git a/docs/source/Support/bskKnownIssuesSnippets/1676-tle-name-lines.rst b/docs/source/Support/bskKnownIssuesSnippets/1676-tle-name-lines.rst new file mode 100644 index 00000000000..f821ab356ad --- /dev/null +++ b/docs/source/Support/bskKnownIssuesSnippets/1676-tle-name-lines.rst @@ -0,0 +1,15 @@ +- In versions 2.8.28 through 2.12.2, :ref:`tleHandling` ``satTle2elem()`` raised + ``ValueError`` for a whole 3LE file when any name line started with ``1`` or ``2`` + (for example ``1KUNS-PF``), because such lines were counted as element lines. Issue + #1676 is fixed by recognizing complete record layouts before counting element-line + prefixes, preserving names such as ``1 TEST`` and ``2 TEST`` and their associated + satellites, including files with trailing blank lines. These prefixes are also + accepted in optional 2LE file titles. Recognizable truncated element headers are + rejected as possible orphaned lines rather than consumed as names or file titles. + This includes longer fragments with malformed catalog fields when later field + structure still identifies them as element lines, and shortened headers when + intact sequences of element fields identify columns shifted by missing characters. + Prefix an intentional element-like title with ``0`` followed by a space to + disambiguate it; for a satellite name, that prefix is retained in ``satName``. + With an earlier version, rename satellites or file titles starting with ``1`` or + ``2`` before reading the file. diff --git a/docs/source/Support/bskKnownIssuesSnippets/1676-tle-skip-bad-set.rst b/docs/source/Support/bskKnownIssuesSnippets/1676-tle-skip-bad-set.rst new file mode 100644 index 00000000000..72c8dafa679 --- /dev/null +++ b/docs/source/Support/bskKnownIssuesSnippets/1676-tle-skip-bad-set.rst @@ -0,0 +1,9 @@ +- In versions 2.8.28 through 2.12.2, :ref:`tleHandling` ``satTle2elem()`` failed for + the whole file when any one set could not be read (for example a bad checksum or + a malformed field), so no valid sets were returned either. This remains the + default behavior for compatibility. Call ``satTle2elem(path, on_error="skip")`` + to recover readable sets: complete sets that raise validation or arithmetic + errors, including epoch overflow and division by zero, are skipped with a + printed warning. If every attempted set fails in skip mode, the first set's + original error is raised. File-level format checks, including orphaned element + lines, can still reject the whole file in either mode. diff --git a/docs/source/Support/bskReleaseNotesSnippets/1676-tle-blank-designator.rst b/docs/source/Support/bskReleaseNotesSnippets/1676-tle-blank-designator.rst new file mode 100644 index 00000000000..3e4265620b6 --- /dev/null +++ b/docs/source/Support/bskReleaseNotesSnippets/1676-tle-blank-designator.rst @@ -0,0 +1 @@ +- :ref:`tleHandling` ``satTle2elem()`` now reads element sets with a blank international designator, such as CelesTrak's analyst objects, leaving ``launchDate`` as ``None``. diff --git a/docs/source/Support/bskReleaseNotesSnippets/1676-tle-ephemeris-type.rst b/docs/source/Support/bskReleaseNotesSnippets/1676-tle-ephemeris-type.rst new file mode 100644 index 00000000000..900ea45b1d2 --- /dev/null +++ b/docs/source/Support/bskReleaseNotesSnippets/1676-tle-ephemeris-type.rst @@ -0,0 +1 @@ +- :ref:`tleHandling` ``satTle2elem()`` accepts a blank ephemeris-type field, ``0`` or ``2`` for SGP4 conversion. Other types, including ``4`` (SGP4-XP), raise ``ValueError`` by default. With ``on_error="skip"``, unsupported sets are skipped with a printed warning; if every attempted set fails, the first error is raised. diff --git a/docs/source/Support/bskReleaseNotesSnippets/1676-tle-name-lines.rst b/docs/source/Support/bskReleaseNotesSnippets/1676-tle-name-lines.rst new file mode 100644 index 00000000000..a2e3dbaa210 --- /dev/null +++ b/docs/source/Support/bskReleaseNotesSnippets/1676-tle-name-lines.rst @@ -0,0 +1,2 @@ +- :ref:`tleHandling` ``satTle2elem()`` now preserves satellite names and records in 3LE files whose names start with ``1`` or ``2``, including ``1 TEST`` and ``2 TEST``, accepts these prefixes in optional 2LE file titles, and ignores trailing blank lines. +- :ref:`tleHandling` rejects recognizable orphaned element lines, including truncated lines, shortened headers with missing characters, and longer fragments with malformed catalog fields, instead of consuming them as names or file titles. Prefix an intentional element-like title with ``0`` followed by a space to disambiguate it. diff --git a/docs/source/Support/bskReleaseNotesSnippets/1676-tle-skip-bad-set.rst b/docs/source/Support/bskReleaseNotesSnippets/1676-tle-skip-bad-set.rst new file mode 100644 index 00000000000..eb720e9aa7b --- /dev/null +++ b/docs/source/Support/bskReleaseNotesSnippets/1676-tle-skip-bad-set.rst @@ -0,0 +1 @@ +- :ref:`tleHandling` ``satTle2elem()`` now accepts the keyword-only option ``on_error="skip"`` to skip complete sets that fail validation or numerical conversion, including epoch overflow and division by zero, with a printed warning and return the readable sets. The default, ``on_error="raise"``, preserves the existing behavior of raising the first record error immediately. In skip mode, the first error is raised if every attempted set fails. File-level format checks can still reject the entire file in either mode. diff --git a/src/tests/test_readTLE.py b/src/tests/test_readTLE.py index 9dea194ca92..041a5d933ff 100644 --- a/src/tests/test_readTLE.py +++ b/src/tests/test_readTLE.py @@ -259,6 +259,581 @@ def test_read_tle_rejects_catalog_overflow(tmp_path): tleHandling.satTle2elem(tle_path) +def _with_checksum(line): + """Return a TLE line with its checksum column recomputed (digits, and 1 for each minus sign).""" + total = sum(int(c) if c.isdigit() else (1 if c == "-" else 0) for c in line[:68]) + return line[:68] + str(total % 10) + + +def test_read_tle_blank_international_designator(tmp_path): + """A set with a blank international designator (as for analyst objects) loads with default launch fields.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + lines = file.read().splitlines() + + line1 = _with_checksum(lines[1][:9] + " " * 8 + lines[1][17:]) + tle_path = _write_modified_hypso_tle(tmp_path, line1=line1) + reference = tleHandling.satTle2elem(DATA_DIR / "hypso1.tle")[0] + records = tleHandling.satTle2elem(tle_path) + + assert len(records) == 1 + assert records[0].launchDate is None + assert records[0].noradID == reference.noradID + for name in ("a", "e", "i", "Omega", "omega", "f"): + assert getattr(records[0].oe, name) == getattr(reference.oe, name) + + +@pytest.mark.parametrize("ephemeris_type", ["4", "1", "3"]) +def test_read_tle_rejects_non_sgp4_ephemeris_type(tmp_path, ephemeris_type): + """satTle2elem() refuses element sets that are not SGP4 (type 4 is SGP4-XP) instead of running SGP4 on them.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + lines = file.read().splitlines() + + line1 = _with_checksum(lines[1][:62] + ephemeris_type + lines[1][63:]) + tle_path = _write_modified_hypso_tle(tmp_path, line1=line1) + + with pytest.raises(ValueError, match="unsupported ephemeris type"): + tleHandling.satTle2elem(tle_path) + + +@pytest.mark.parametrize("ephemeris_type", ["0", "2", " "]) +def test_read_tle_accepts_sgp4_ephemeris_types(tmp_path, ephemeris_type): + """Types 0 and 2, and a blank column, give the same elements as the unmodified set.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + lines = file.read().splitlines() + + line1 = _with_checksum(lines[1][:62] + ephemeris_type + lines[1][63:]) + tle_path = _write_modified_hypso_tle(tmp_path, line1=line1) + reference = tleHandling.satTle2elem(DATA_DIR / "hypso1.tle")[0].oe + oe = tleHandling.satTle2elem(tle_path)[0].oe + + for name in ("a", "e", "i", "Omega", "omega", "f"): + assert getattr(oe, name) == getattr(reference, name) + + +@pytest.mark.parametrize("on_error", [None, "raise", "skip"]) +@pytest.mark.parametrize("names", [ + ["1KUNS-PF"], + ["HYPSO-1", "2026-195B"], + ["2026-195B", "1KUNS-PF"], + ["1 TEST"], + ["2 TEST"], + ["1 TEST", "2 TEST"], + ["HYPSO-1", "1 TEST", "2 TEST"], + ["1 12345U", "2 67890"], + ["1 OBJECT 12345", "2 PANEL 123.0000"], + ["1 OBJECT 12345678 PAYLOAD", "2 PANEL 123.0000 SIDE A"], + ["1 a0001", "2 12-53"], + ["1 5153U MODEL", "2 5153 CATALOG"], +]) +def test_read_tle_name_lines_starting_with_a_digit(tmp_path, capsys, names, on_error): + """Preserve every 3LE record's name and catalog ID, including names with element-line prefixes.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + _, line1, line2 = file.read().splitlines()[:3] + lines = [] + catalog_ids = [f"{90001 + index:05d}" for index in range(len(names))] + for name, catalog_id in zip(names, catalog_ids): + lines.extend([ + name, + _with_checksum(line1[:2] + catalog_id + line1[7:]), + _with_checksum(line2[:2] + catalog_id + line2[7:]), + ]) + tle_path = tmp_path / "digit-names.tle" + tle_path.write_text("\n".join(lines) + "\n", encoding="utf-8") + reference = tleHandling.satTle2elem(DATA_DIR / "hypso1.tle")[0] + + options = {} if on_error is None else {"on_error": on_error} + records = tleHandling.satTle2elem(tle_path, **options) + + assert [record.satName for record in records] == names + assert [record.noradID for record in records] == catalog_ids + assert capsys.readouterr().out == "" + for record in records: + for name in ("a", "e", "i", "Omega", "omega", "f"): + assert getattr(record.oe, name) == getattr(reference.oe, name) + + +@pytest.mark.parametrize("on_error", [None, "raise", "skip"]) +@pytest.mark.parametrize("title", [ + None, + "CATALOG", + "1 TEST", + "2 TEST", + "1 SATELLITE GROUP RECENT CATALOG 2026", + "2 SATELLITE LIST CATALOGS 2026", +]) +def test_read_tle_2le_optional_title(tmp_path, capsys, title, on_error): + """Preserve both unnamed 2LE records when an optional file title resembles an element line.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + _, line1, line2 = file.read().splitlines()[:3] + lines = [] if title is None else [title] + catalog_ids = ["90001", "90002"] + for catalog_id in catalog_ids: + lines.extend([ + _with_checksum(line1[:2] + catalog_id + line1[7:]), + _with_checksum(line2[:2] + catalog_id + line2[7:]), + ]) + tle_path = tmp_path / "optional-title.2le" + tle_path.write_text("\n".join(lines) + "\n", encoding="utf-8") + + options = {} if on_error is None else {"on_error": on_error} + records = tleHandling.satTle2elem(tle_path, **options) + + assert [record.noradID for record in records] == catalog_ids + assert [record.satName for record in records] == ["", ""] + assert capsys.readouterr().out == "" + + +@pytest.mark.parametrize("on_error", ["raise", "skip"]) +@pytest.mark.parametrize("missing_line", range(4)) +def test_read_tle_incomplete_2le_is_not_a_named_record(tmp_path, missing_line, on_error): + """Reject incomplete 2LE files instead of treating an orphaned element line as a satellite name.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + _, line1, line2 = file.read().splitlines()[:3] + lines = [line1, line2, line1, line2] + del lines[missing_line] + tle_path = tmp_path / "incomplete.2le" + tle_path.write_text("\n".join(lines) + "\n", encoding="utf-8") + + with pytest.raises(ValueError, match="orphan|must be equal"): + tleHandling.satTle2elem(tle_path, on_error=on_error) + + +@pytest.mark.parametrize("line_number, width", [ + (1, 14), + (1, 20), + (1, 32), + (1, 68), + (2, 12), + (2, 25), + (2, 52), + (2, 68), +]) +@pytest.mark.parametrize("good_sets", [1, 2]) +@pytest.mark.parametrize("catalog_field", ["51053", "a0001"]) +def test_read_tle_truncated_orphan_is_not_a_title(tmp_path, line_number, width, good_sets, catalog_field): + """Reject a truncated orphan before valid sets instead of using it as a name or file title.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + _, line1, line2 = file.read().splitlines()[:3] + element_line = line1 if line_number == 1 else line2 + lines = [element_line[:2] + catalog_field + element_line[7:width]] + for index in range(good_sets): + catalog_id = f"{90001 + index:05d}" + lines.extend([ + _with_checksum(line1[:2] + catalog_id + line1[7:]), + _with_checksum(line2[:2] + catalog_id + line2[7:]), + ]) + tle_path = tmp_path / "truncated-orphan.2le" + tle_path.write_text("\n".join(lines) + "\n", encoding="utf-8") + + with pytest.raises(ValueError, match="orphan|must be equal"): + tleHandling.satTle2elem(tle_path) + + +def test_read_tle_orphan_with_blank_designator_is_not_a_title(tmp_path): + """A truncated analyst-object header remains an element line when its designator is blank.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + _, line1, line2 = file.read().splitlines()[:3] + orphan = line1[:9] + " " * 8 + line1[17:20] + tle_path = tmp_path / "analyst-orphan.2le" + tle_path.write_text("\n".join([orphan, line1, line2]) + "\n", encoding="utf-8") + + with pytest.raises(ValueError, match="orphan"): + tleHandling.satTle2elem(tle_path) + + +@pytest.mark.parametrize("line_number, field_start, field_end", [(1, 9, 17), (2, 8, 16)]) +def test_read_tle_orphan_with_malformed_field_is_not_a_title(tmp_path, line_number, field_start, field_end): + """A malformed numeric field does not turn a truncated element header into a title.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + _, line1, line2 = file.read().splitlines()[:3] + orphan = (line1 if line_number == 1 else line2)[:68] + orphan = orphan[:field_start] + "X" * (field_end - field_start) + orphan[field_end:] + tle_path = tmp_path / "malformed-orphan.2le" + tle_path.write_text("\n".join([orphan, line1, line2]) + "\n", encoding="utf-8") + + with pytest.raises(ValueError, match="orphan"): + tleHandling.satTle2elem(tle_path) + + +@pytest.mark.parametrize("line_number, width", [(1, 35), (1, 68), (2, 27), (2, 68)]) +@pytest.mark.parametrize("catalog_field, field_end", [ + ("510531", 8), + ("5105312", 9), + ("510531", 7), + ("5105312", 7), + ("a0001", 7), + ("12-53", 7), +]) +@pytest.mark.parametrize("good_sets", [1, 2]) +def test_read_tle_orphan_with_malformed_catalog_is_not_a_title( + tmp_path, line_number, width, catalog_field, field_end, good_sets +): + """Reject truncated orphans with invalid catalogs in both satellite-name and file-title positions.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + _, line1, line2 = file.read().splitlines()[:3] + element_line = line1 if line_number == 1 else line2 + # Overlong catalogs can overwrite the next column or shift subsequent fields. + orphan = (element_line[:2] + catalog_field + element_line[field_end:])[:width] + lines = [orphan] + for index in range(good_sets): + catalog_id = f"{90001 + index:05d}" + lines.extend([ + _with_checksum(line1[:2] + catalog_id + line1[7:]), + _with_checksum(line2[:2] + catalog_id + line2[7:]), + ]) + tle_path = tmp_path / "malformed-catalog-orphan.2le" + tle_path.write_text("\n".join(lines) + "\n", encoding="utf-8") + title_position = "satellite-name" if good_sets == 1 else "file-title" + + with pytest.raises(ValueError, match=f"orphaned TLE element line in a {title_position} position"): + tleHandling.satTle2elem(tle_path) + + +@pytest.mark.parametrize("line_number, missing_start, missing_end", [ + *[(1, index, index + 1) for index in range(2, 18)], + *[(2, index, index + 1) for index in range(2, 17)], + (1, 2, 4), + (2, 2, 4), + (1, 2, 7), + (2, 2, 7), +]) +@pytest.mark.parametrize("good_sets", [1, 2]) +def test_read_tle_orphan_with_shortened_header_is_not_a_title( + tmp_path, line_number, missing_start, missing_end, good_sets +): + """Reject orphans with deleted header characters in satellite-name and file-title positions.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + _, line1, line2 = file.read().splitlines()[:3] + element_line = line1 if line_number == 1 else line2 + orphan = element_line[:missing_start] + element_line[missing_end:] + lines = [orphan] + for index in range(good_sets): + catalog_id = f"{90001 + index:05d}" + lines.extend([ + _with_checksum(line1[:2] + catalog_id + line1[7:]), + _with_checksum(line2[:2] + catalog_id + line2[7:]), + ]) + tle_path = tmp_path / "shortened-header-orphan.2le" + tle_path.write_text("\n".join(lines) + "\n", encoding="utf-8") + title_position = "satellite-name" if good_sets == 1 else "file-title" + + with pytest.raises(ValueError, match=f"orphaned TLE element line in a {title_position} position"): + tleHandling.satTle2elem(tle_path) + + +@pytest.mark.parametrize("prefix", ["", "0 "]) +def test_read_tle_element_like_titles_are_explicit(tmp_path, capsys, prefix): + """Reject balanced element-like names unless a zero prefix explicitly marks them as titles.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + _, line1, line2 = file.read().splitlines()[:3] + names = [prefix + line1[:20], prefix + line2[:25]] + catalog_ids = ["90001", "90002"] + lines = [] + for name, catalog_id in zip(names, catalog_ids): + lines.extend([ + name, + _with_checksum(line1[:2] + catalog_id + line1[7:]), + _with_checksum(line2[:2] + catalog_id + line2[7:]), + ]) + tle_path = tmp_path / "element-like-titles.tle" + tle_path.write_text("\n".join(lines) + "\n", encoding="utf-8") + + if prefix: + records = tleHandling.satTle2elem(tle_path) + assert [record.satName for record in records] == names + assert [record.noradID for record in records] == catalog_ids + assert capsys.readouterr().out == "" + else: + with pytest.raises(ValueError, match="orphan"): + tleHandling.satTle2elem(tle_path) + + +@pytest.mark.parametrize("layout", ["3le", "2le", "titled-2le"]) +@pytest.mark.parametrize("newline, trailing", [ + ("\n", "\n"), + ("\r\n", " \t\r\n\r\n"), +]) +def test_read_tle_trailing_blank_lines(tmp_path, capsys, layout, newline, trailing): + """Trailing blank lines preserve all names, catalog IDs and orbital elements across layouts.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + _, line1, line2 = file.read().splitlines()[:3] + names = ["1 TEST", "2 TEST"] + catalog_ids = ["90001", "90002"] + lines = ["1 CATALOG"] if layout == "titled-2le" else [] + for name, catalog_id in zip(names, catalog_ids): + if layout == "3le": + lines.append(name) + lines.extend([ + _with_checksum(line1[:2] + catalog_id + line1[7:]), + _with_checksum(line2[:2] + catalog_id + line2[7:]), + ]) + tle_path = tmp_path / "trailing-blanks.tle" + text = newline.join(lines) + newline + tle_path.write_bytes(text.encode("utf-8")) + reference = tleHandling.satTle2elem(tle_path) + tle_path.write_bytes((text + trailing).encode("utf-8")) + + records = tleHandling.satTle2elem(tle_path) + + assert [record.noradID for record in records] == catalog_ids + assert [record.satName for record in records] == (names if layout == "3le" else ["", ""]) + assert capsys.readouterr().out == "" + for record, expected in zip(records, reference): + for element in ("a", "e", "i", "Omega", "omega", "f"): + assert getattr(record.oe, element) == getattr(expected.oe, element) + + +@pytest.mark.parametrize("text", ["", "\n\n", " \t\r\n\r\n"]) +def test_read_tle_blank_file_raises(tmp_path, text): + """Blank-file normalization raises the normal input error instead of returning an empty result.""" + tle_path = tmp_path / "blank.tle" + tle_path.write_bytes(text.encode("utf-8")) + + with pytest.raises(ValueError, match="at least two lines"): + tleHandling.satTle2elem(tle_path) + + +@pytest.mark.parametrize("on_error", [None, "raise", "skip"]) +@pytest.mark.parametrize("bad_set", [0, 1, 2]) +@pytest.mark.parametrize("corruption", ["checksum", "epoch", "alpha5", "truncated", "ephemeris"]) +def test_read_tle_validation_error_policy(tmp_path, capsys, bad_set, corruption, on_error): + """Raise by default; only skip mode returns valid records around a validation failure.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + name, line1, line2 = file.read().splitlines()[:3] + sets = [] + names = ["1 SAT-0", "2 SAT-1", "SAT-2"] + for setNo in range(3): + number = f"{90001 + setNo:05d}" + l1 = _with_checksum(line1[:2] + number + line1[7:]) + l2 = _with_checksum(line2[:2] + number + line2[7:]) + if setNo == bad_set: + if corruption == "checksum": + l1 = l1[:68] + str((int(l1[68]) + 1) % 10) + elif corruption == "epoch": + l1 = _with_checksum(l1[:25] + "O" + l1[26:]) + elif corruption == "alpha5": + l1 = l1[:2] + "A0001" + l1[7:] + l2 = l2[:2] + "A0001" + l2[7:] + elif corruption == "ephemeris": + l1 = _with_checksum(l1[:62] + "4" + l1[63:]) + else: + l1 = l1[:20] + sets.extend([names[setNo], l1, l2]) + tle_path = tmp_path / "three-sets.tle" + tle_path.write_text("\n".join(sets) + "\n", encoding="utf-8") + + options = {} if on_error is None else {"on_error": on_error} + if on_error != "skip": + with pytest.raises(ValueError): + tleHandling.satTle2elem(tle_path, **options) + assert capsys.readouterr().out == "" + return + + records = tleHandling.satTle2elem(tle_path, **options) + + assert [record.noradID for record in records] == [f"{90001 + n:05d}" for n in range(3) if n != bad_set] + assert [record.satName for record in records] == [name for index, name in enumerate(names) if index != bad_set] + warning = capsys.readouterr().out + assert f"skipped TLE set {bad_set + 1}" in warning + + +@pytest.mark.parametrize("on_error", [None, "raise", "skip"]) +@pytest.mark.parametrize("layout", ["2le", "titled-2le", "3le"]) +@pytest.mark.parametrize("bad_set", [0, 1, 2]) +@pytest.mark.parametrize("line_number, field_start, field_end, value, expected_error", [ + # Epoch day fields use [day]; mean-motion fields use [rev/day]. + pytest.param(1, 20, 32, "9999999.0000", OverflowError, id="epoch-date-overflow"), + pytest.param(1, 20, 32, "999999999999", OverflowError, id="epoch-timedelta-overflow"), + pytest.param(1, 20, 32, " inf", OverflowError, id="epoch-infinite"), + pytest.param(2, 52, 63, " 0.00000000", ZeroDivisionError, id="mean-motion-zero"), + pytest.param(2, 52, 63, " 1e-300", ZeroDivisionError, id="mean-motion-underflow"), + pytest.param(2, 52, 63, " 1e300", OverflowError, id="mean-motion-overflow"), +]) +def test_read_tle_arithmetic_error_policy( + tmp_path, monkeypatch, capsys, layout, bad_set, line_number, field_start, field_end, + value, expected_error, on_error +): + """Stop at the first numerical error by default and preserve valid records when skipping is requested.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + _, line1, line2 = file.read().splitlines()[:3] + names = ["1 SAT-0", "2 SAT-1", "SAT-2"] + catalog_ids = ["90001", "90002", "90003"] + lines = ["1 CATALOG"] if layout == "titled-2le" else [] + for index, catalog_id in enumerate(catalog_ids): + record = [ + _with_checksum(line1[:2] + catalog_id + line1[7:]), + _with_checksum(line2[:2] + catalog_id + line2[7:]), + ] + if index == bad_set: + element_line = record[line_number - 1] + record[line_number - 1] = _with_checksum( + element_line[:field_start] + value + element_line[field_end:] + ) + if layout == "3le": + lines.append(names[index]) + lines.extend(record) + tle_path = tmp_path / "arithmetic-failure.tle" + tle_path.write_text("\n".join(lines) + "\n", encoding="utf-8") + reference = tleHandling.satTle2elem(DATA_DIR / "hypso1.tle")[0] + parse = tleHandling._parseTle + attempted = [] + + def track_parse(record): + attempted.append(record[-2][2:7]) + return parse(record) + + monkeypatch.setattr(tleHandling, "_parseTle", track_parse) + options = {} if on_error is None else {"on_error": on_error} + if on_error != "skip": + with pytest.raises(expected_error): + tleHandling.satTle2elem(tle_path, **options) + assert attempted == catalog_ids[:bad_set + 1] + assert capsys.readouterr().out == "" + return + + records = tleHandling.satTle2elem(tle_path, **options) + + assert attempted == catalog_ids + assert [record.noradID for record in records] == [ + catalog_id for index, catalog_id in enumerate(catalog_ids) if index != bad_set + ] + expected_names = [name for index, name in enumerate(names) if index != bad_set] + assert [record.satName for record in records] == (expected_names if layout == "3le" else ["", ""]) + for record in records: + for element in ("a", "e", "i", "Omega", "omega", "f"): + assert getattr(record.oe, element) == getattr(reference.oe, element) + warning = capsys.readouterr().out + assert warning.count("skipped TLE set") == 1 + assert f"skipped TLE set {bad_set + 1}" in warning + + +@pytest.mark.parametrize("on_error", [None, "raise", "skip"]) +@pytest.mark.parametrize("first_record, expected_error", [ + (0, OverflowError), + (1, ZeroDivisionError), + (2, ValueError), +]) +def test_read_tle_all_numeric_failures_raise_first_error(tmp_path, capsys, first_record, expected_error, on_error): + """Every policy raises the first original error when all records are invalid.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + _, line1, line2 = file.read().splitlines()[:3] + overflow_line = _with_checksum(line1[:20] + "9999999.0000" + line1[32:]) # Epoch [day] + zero_motion_line = _with_checksum(line2[:52] + " 0.00000000" + line2[63:]) # Mean motion [rev/day] + checksum_line = line1[:68] + str((int(line1[68]) + 1) % 10) + bad_records = [[overflow_line, line2], [line1, zero_motion_line], [checksum_line, line2]] + ordered_records = bad_records[first_record:] + bad_records[:first_record] + tle_path = tmp_path / "all-numeric-failures.2le" + tle_path.write_text("\n".join(line for record in ordered_records for line in record) + "\n", encoding="utf-8") + options = {} if on_error is None else {"on_error": on_error} + + with pytest.raises(expected_error): + tleHandling.satTle2elem(tle_path, **options) + + warning = capsys.readouterr().out + if on_error == "skip": + assert warning.count("skipped TLE set") == 3 + for set_number in range(1, 4): + assert f"skipped TLE set {set_number}" in warning + else: + assert warning == "" + + +@pytest.mark.parametrize("on_error", ["raise", "skip"]) +@pytest.mark.parametrize("stage", ["_parseTle", "_convertMean2osculating"]) +@pytest.mark.parametrize("error_type", [TypeError, RuntimeError]) +def test_read_tle_unexpected_failure_propagates(monkeypatch, capsys, stage, error_type, on_error): + """Unexpected implementation failures must not be reported as bad input and silently skipped.""" + error = error_type("unexpected internal failure") + + def fail(*args): + raise error + + monkeypatch.setattr(tleHandling, stage, fail) + + with pytest.raises(error_type) as caught: + tleHandling.satTle2elem(DATA_DIR / "hypso1.tle", on_error=on_error) + + assert caught.value is error + assert capsys.readouterr().out == "" + + +@pytest.mark.parametrize("on_error", [None, "raise", "skip"]) +@pytest.mark.parametrize("error_type", [OverflowError, ZeroDivisionError, FloatingPointError]) +def test_read_tle_conversion_arithmetic_error_policy(tmp_path, monkeypatch, capsys, error_type, on_error): + """The policy also controls whether conversion errors stop the file or skip the failed record.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + _, line1, line2 = file.read().splitlines()[:3] + catalog_ids = ["90001", "90002", "90003"] + lines = [] + for catalog_id in catalog_ids: + lines.extend([ + _with_checksum(line1[:2] + catalog_id + line1[7:]), + _with_checksum(line2[:2] + catalog_id + line2[7:]), + ]) + tle_path = tmp_path / "conversion-failure.2le" + tle_path.write_text("\n".join(lines) + "\n", encoding="utf-8") + convert = tleHandling._convertMean2osculating + attempted = [] + + def convert_with_failure(line1, line2, tle_data): + attempted.append(tle_data.noradID) + if tle_data.noradID == "90002": + raise error_type("numerical conversion failure") + return convert(line1, line2, tle_data) + + monkeypatch.setattr(tleHandling, "_convertMean2osculating", convert_with_failure) + options = {} if on_error is None else {"on_error": on_error} + if on_error != "skip": + with pytest.raises(error_type, match="numerical conversion failure"): + tleHandling.satTle2elem(tle_path, **options) + assert attempted == catalog_ids[:2] + assert capsys.readouterr().out == "" + return + + records = tleHandling.satTle2elem(tle_path, **options) + + assert attempted == catalog_ids + assert [record.noradID for record in records] == ["90001", "90003"] + warning = capsys.readouterr().out + assert warning.count("skipped TLE set") == 1 + assert "skipped TLE set 2" in warning + assert "numerical conversion failure" in warning + + +@pytest.mark.parametrize("on_error", [None, "raise", "skip"]) +def test_read_tle_all_sets_bad_raises(tmp_path, capsys, on_error): + """A single invalid set raises its error with either policy, including the default.""" + with open(DATA_DIR / "hypso1.tle", 'r') as file: + lines = file.read().splitlines() + line1 = lines[1][:68] + str((int(lines[1][68]) + 1) % 10) + tle_path = _write_modified_hypso_tle(tmp_path, line1=line1) + options = {} if on_error is None else {"on_error": on_error} + + with pytest.raises(ValueError, match="checksum mismatch"): + tleHandling.satTle2elem(tle_path, **options) + + warning = capsys.readouterr().out + if on_error == "skip": + assert warning.count("skipped TLE set") == 1 + else: + assert warning == "" + + +@pytest.mark.parametrize("on_error", ["", "ignore", "SKIP", None, True, False, 0, [], {}]) +def test_read_tle_invalid_error_policy(tmp_path, on_error): + """Reject unsupported error policies before accessing the file.""" + with pytest.raises(ValueError, match="on_error must be 'raise' or 'skip'"): + tleHandling.satTle2elem(tmp_path / "missing.tle", on_error=on_error) + + +@pytest.mark.parametrize("on_error", ["raise", "skip"]) +def test_read_tle_file_error_propagates(tmp_path, capsys, on_error): + """Skipping individual records never suppresses file access failures.""" + with pytest.raises(FileNotFoundError): + tleHandling.satTle2elem(tmp_path / "missing.tle", on_error=on_error) + assert capsys.readouterr().out == "" + + def test_write_tle_6digit_norad_raises(): """generateTle() must raise ValueError for catalog numbers >= 100000 (6-digit NORAD IDs assigned after ~July 2026 cannot be represented in TLE format).""" tleData = tleHandling.TleData(oe=oeHypso1, tleEpoch=hypso1tleEpoch) diff --git a/src/utilities/tleHandling.py b/src/utilities/tleHandling.py index b9049353399..314415f8628 100644 --- a/src/utilities/tleHandling.py +++ b/src/utilities/tleHandling.py @@ -433,6 +433,66 @@ def _norad_id_to_tle_str(norad_id) -> str: ) return f"{norad_id_int:05d}" +def _isTleElementLine(line: str, lineNumber: int) -> bool: + """ + Check whether a line has the prefix of a TLE element line with the given number. + + An element line starts with its line number in column 1 and a blank in column 2. + A name can also have this prefix, so complete record layouts take precedence when + identifying names in :func:`satTle2elem`. Field validation is left to the parser. + + :param line: one line of a TLE file + :param lineNumber: 1 or 2 + :return: True if the line starts with ``lineNumber`` followed by a blank + """ + return line.startswith(f"{lineNumber} ") + +def _isTleTitleLine(line: str) -> bool: + """ + Check whether a line can occupy a satellite-name or file-title position. + + Names can start with a line number, but full-width element lines and recognizable + truncated headers must not be consumed as titles when a companion line is missing. + Short fragments require a numeric or Alpha-5-like catalog field, including + overlong numeric IDs that shift subsequent fields. Line-1 headers + include the classification and launch number, or the designator/epoch field + boundaries. Line-2 headers include the catalog and start of the inclination + field, or the inclination/ascending-node field boundaries. Longer fragments + also identify themselves by field boundaries and decimal-point placement in + the epoch (line 1) or ascending node (line 2), even when the catalog is malformed. + Intact sequences of epoch/rate fields or orbital fields identify element lines + whose later columns have shifted because header characters are missing. + Ignore trailing whitespace so padded names are recognized as well. + + :param line: one line of a TLE file + :return: True if the line can be a title without matching an element-line header + """ + line = line.rstrip() + if not (_isTleElementLine(line, 1) or _isTleElementLine(line, 2)): + return True + if len(line) >= 69: + return False + # Deleted header characters shift later columns. Recognize intact sequences + # of numeric fields without relying on their starting column: epoch and mean + # motion derivative on line 1; ascending node, eccentricity and perigee on line 2. + line1Body = r"[0-9]{5}\.[0-9]{8} [ +-]\.[0-9]{8}(?: |$)" + line2Body = r"[ 0-9]{3}\.[0-9]{4} [0-9]{7} [ 0-9]{3}\.[0-9]{4}(?: |$)" + bodyPattern = line1Body if line[0] == "1" else line2Body + if re.search(bodyPattern, line[2:]): + return False + # Later field structure identifies longer fragments independently of catalog + # validity, including overflow into the classification or separator column. + # Decimal points distinguish these fields from words separated by spaces. + line1FieldBoundaries = r"1 .{15} .{5}\..{8} ." + line2FieldBoundaries = r"2 .{14} .{3}\..{4} ." + if re.match(line1FieldBoundaries, line) or re.match(line2FieldBoundaries, line): + return False + # Short fragments need a recognizable catalog to distinguish ordinary names. + catalogField = r"(?:[ 0-9]{5}|[A-Za-z][0-9]{4}|[0-9]{6,})" + line1Header = "1 " + catalogField + r"[A-Z ] (?:[0-9]{5}|.{8} .)" + line2Header = "2 " + catalogField + r" (?:[ 0-9]{3}\.|.{8} .)" + return re.match(line1Header, line) is None and re.match(line2Header, line) is None + def _calcTleChecksum(stringArray: str) -> int: """ Calculate TLE checksum per line. @@ -565,6 +625,14 @@ def _parseTle(tle: str) -> TleData: nDot = _firstDeriv2ballistCoef(nDot_str) # dn/dt in [rev/day^2] nDotDot = _tleStr2physVal(nDotDot_str) # d^2n/dt^2 in [rev/day^3] bStar = _tleStr2physVal(bStar_str) + # Only SGP4 element sets can be converted here. Distributed TLEs carry type 0; CCSDS 502.0-B-3 + # section 4.2.4.7 codes 2 = SGP4 and 4 = SGP4-XP, whose drag and second-derivative fields hold BTERM + # and AGOM, so SGP4 would return a plausible but wrong state. This matches satOmm2elem(). + if ephemType_str.strip() not in ('', '0', '2'): + raise ValueError( + f"_parseTle() received an unsupported ephemeris type '{ephemType_str}' for NORAD ID " + f"{sat_id_1_clean}; expected 0 (default) or 2 (SGP4). Type 4 element sets need SGP4-XP." + ) propagator = {'1': 'SGP4', '2': 'SDP4', '3': 'SGP8', '4': 'SDP8', '0': 'Unknown'}.get(ephemType_str, 'Unknown') elemSetNo = int(elemSetNo_str) @@ -590,9 +658,12 @@ def _parseTle(tle: str) -> TleData: tleData.nDot = nDot tleData.nDotDot = nDotDot tleData.bStar = bStar - tleData.launchDate = dt.datetime(_parseTleYear(launchYear_str), 1, 1) - tleData.launchNo = int(launchNo_str) - tleData.pol = pol_str + # Analyst objects and some unidentified objects have a blank international designator + # (columns 10-17). Leave the launch fields at their defaults instead of failing the set. + if (launchYear_str + launchNo_str + pol_str).strip(): + tleData.launchDate = dt.datetime(_parseTleYear(launchYear_str), 1, 1) + tleData.launchNo = int(launchNo_str) + tleData.pol = pol_str return tleData @@ -764,14 +835,50 @@ def _generateTleFromMean(oe_mean: om.ClassicElements, tleData: TleData) -> str: return f"{tleData.satName}\n{line1}\n{line2}" -def satTle2elem(tle_path: str): +def satTle2elem(tle_path: str, *, on_error: str = "raise"): """ - Convert the TLEs of a constellation to classical orbital elements for each satellite. + Convert SGP4 TLE records to osculating orbital elements and satellite metadata. + + The ephemeris-type field (column 63 of line 1) may be blank, ``0`` or ``2``. + Other types, including ``4`` (SGP4-XP), are unsupported and follow the per-set + error handling below. + + A blank international designator (columns 10-17 of line 1) is accepted. + In that case, ``launchDate`` is ``None``, while ``launchNo`` and ``pol`` retain + their :class:`TleData` defaults of ``1`` and ``"A"``. These defaults do not + represent known launch metadata for the object. + + By default, the first error during parsing or orbital conversion is raised + immediately, and no partial result is returned. Set ``on_error="skip"`` to + skip complete sets that raise ``ValueError`` or ``ArithmeticError`` and return + the readable sets. Each skipped set produces a printed warning. These errors + include bad checksums, malformed fields, unsupported catalog numbers, SGP4 + errors, overflow and division by zero. If every attempted set fails in skip + mode, the first set's original error is raised. File access errors, file-level + format errors and exceptions outside these two classes propagate in both modes. + + Trailing blank lines are ignored. Lines with a 69-column element-line prefix or + a recognizable truncated element header are rejected in title positions as + possible orphaned lines. Longer fragments with recognizable field structure + are rejected even if their catalog fields are malformed. + Intact sequences of element fields also identify shortened headers whose + missing characters have shifted later columns. + Prefix an intentional element-like title with ``0`` + followed by a space to disambiguate it. For a satellite-name line, the prefix + is retained in ``satName``. :param tle_path: path to a TLE with one or multiple satellites in either 2LE or 3LE format - :return: elementsList: list of classical orbital elements for each satellite - metadataList: list of dictionaries containing TLE metadata for each satellite + :param on_error: keyword-only error policy: ``"raise"`` (default) stops at the + first record error; ``"skip"`` warns and skips complete sets + with validation or arithmetic errors. Other values raise + ``ValueError`` before the file is opened. + :return: list of :class:`TleData` objects for successfully converted records, + with osculating orbital elements in ``oe`` and satellite metadata + in the other attributes """ + if not isinstance(on_error, str) or on_error not in ("raise", "skip"): + raise ValueError("satTle2elem() on_error must be 'raise' or 'skip'.") + def resetTleStr() -> None: """ Reset the TLE strings and flags for reading a new satellite. @@ -790,32 +897,61 @@ def resetTleStr() -> None: tle_text = f.read() tle_lines = tle_text.splitlines() + while tle_lines and not tle_lines[-1].strip(): + tle_lines.pop() noLines = len(tle_lines) if noLines < 2: raise ValueError(f"constTLE2Elem() received a TLE with {noLines} lines. A valid TLE must have at least two lines.") - noOfLinesStartingWith1 = sum(1 for line in tle_lines if line.startswith('1')) - noOfLinesStartingWith2 = sum(1 for line in tle_lines if line.startswith('2')) - noOfLinesStartingWithOther = len(tle_lines) - noOfLinesStartingWith1 - noOfLinesStartingWith2 - # Initialize variables tleDataList = [] readingOrderIndex = 0 - - # Basic checks on the TLE input format (Robustness) - if noOfLinesStartingWith1 != noOfLinesStartingWith2: - raise ValueError(f"constTLE2Elem() received a TLE with {noOfLinesStartingWith1} lines starting with '1' and {noOfLinesStartingWith2} lines starting with '2'. The number of lines starting with '1' and '2' must be equal.") - if noOfLinesStartingWithOther >= noOfLinesStartingWith1: - #TLE contains at least as many titles as lines starting with '1' or '2'=> 3LE datatype - tleType = 3 # 3LE - elif noOfLinesStartingWithOther == 0 and noLines >= 2: - # TLE has no title lines at least two or more lines => 2LE - tleType = 2 # 2LE - elif noOfLinesStartingWithOther == 1 and noLines > 2: - # TLE has one title line and more than two lines => 2LE (first line likely title for 2LE-file) - tleType = 2 # 2LE - if tle_lines[0].startswith('1') or tle_lines[0].startswith('2'): - raise ValueError("constTLE2Elem() received a TLE with one title line but the first line starts with '1' or '2'. --- TLE not valid ---.") + firstRecordLine = 0 + + # Recognize complete layouts before counting prefixes: names and file titles + # may themselves start with '1 ' or '2 '. Check only element-line prefixes so + # malformed fields and truncated lines still reach the per-set parser. + if noLines % 3 == 0 and all( + _isTleElementLine(tle_lines[lineNo], 1) + and _isTleElementLine(tle_lines[lineNo + 1], 2) + for lineNo in range(1, noLines, 3) + ): + for lineNo in range(0, noLines, 3): + if not _isTleTitleLine(tle_lines[lineNo]): + raise ValueError( + f"satTle2elem() found a possible orphaned TLE element line in a satellite-name " + f"position on line {lineNo + 1} in {tle_path}." + ) + tleType = 3 + elif noLines % 2 == 1 and all( + _isTleElementLine(tle_lines[lineNo], 1) and _isTleElementLine(tle_lines[lineNo + 1], 2) + for lineNo in range(1, noLines, 2) + ): + if not _isTleTitleLine(tle_lines[0]): + raise ValueError( + f"satTle2elem() found a possible orphaned TLE element line in a file-title " + f"position on line 1 in {tle_path}." + ) + tleType = 2 + firstRecordLine = 1 + else: + # Preserve the existing format checks for files without a complete layout. + noOfLinesStartingWith1 = sum(1 for line in tle_lines if _isTleElementLine(line, 1)) + noOfLinesStartingWith2 = sum(1 for line in tle_lines if _isTleElementLine(line, 2)) + noOfLinesStartingWithOther = noLines - noOfLinesStartingWith1 - noOfLinesStartingWith2 + if noOfLinesStartingWith1 != noOfLinesStartingWith2: + raise ValueError(f"constTLE2Elem() received a TLE with {noOfLinesStartingWith1} lines starting with '1' and {noOfLinesStartingWith2} lines starting with '2'. The number of lines starting with '1' and '2' must be equal.") + if noOfLinesStartingWithOther >= noOfLinesStartingWith1: + #TLE contains at least as many titles as lines starting with '1' or '2'=> 3LE datatype + tleType = 3 # 3LE + elif noOfLinesStartingWithOther == 0 and noLines >= 2: + # TLE has no title lines at least two or more lines => 2LE + tleType = 2 # 2LE + elif noOfLinesStartingWithOther == 1 and noLines > 2: + # TLE has one title line and more than two lines => 2LE (first line likely title for 2LE-file) + tleType = 2 # 2LE + if _isTleElementLine(tle_lines[0], 1) or _isTleElementLine(tle_lines[0], 2): + raise ValueError("constTLE2Elem() received a TLE with one title line but the first line starts with '1' or '2'. --- TLE not valid ---.") # Define the reading order based on the TLE tleType if tleType == 2: @@ -831,10 +967,12 @@ def resetTleStr() -> None: raise ValueError(f"constTLE2Elem() received an invalid TLE type '{tleType}'. The type must be '2LE' or '3LE'.") resetTleStr() + setNo = 0 # number of complete sets met so far, for warnings + skipped = [] # errors of the sets that were skipped # Read through all lines and extract TLEs - for lineNo, line in enumerate(tle_lines): + for lineNo, line in enumerate(tle_lines[firstRecordLine:], start=firstRecordLine): expected = readingOrder[readingOrderIndex] if readingOrderIndex < len(readingOrder) else None - if line.startswith('1') and expected == 'line1': + if _isTleElementLine(line, 1) and expected == 'line1': if satTleReady[1]: print(f"WARNING: constTLE2Elem() found a new TLE line1 before completing the previous TLE (line number {lineNo}). The previous TLE will be discarded.") # Reset for next TLE @@ -844,7 +982,7 @@ def resetTleStr() -> None: satTleReady[tleType-2] = True readingOrderIndex += 1 line1 = line - elif line.startswith('2') and expected == 'line2': + elif _isTleElementLine(line, 2) and expected == 'line2': if satTleReady[tleType-1]: print(f"WARNING: constTLE2Elem() found a new TLE line2 before completing the previous TLE (line number {lineNo}). The previous TLE will be discarded.") # Reset for next TLE @@ -868,12 +1006,22 @@ def resetTleStr() -> None: continue if all(satTleReady): - tleDataClass = _parseTle(satTLE) - oscOE = _convertMean2osculating(line1, line2, tleDataClass) - tleDataClass.oe = oscOE - tleDataList.append(tleDataClass) + setNo += 1 + try: + tleDataClass = _parseTle(satTLE) + oscOE = _convertMean2osculating(line1, line2, tleDataClass) + except (ValueError, ArithmeticError) as error: + if on_error == "raise": + raise + print(f"WARNING: satTle2elem() skipped TLE set {setNo} (ending on line {lineNo}) in {tle_path}: {error}") + skipped.append(error) + else: + tleDataClass.oe = oscOE + tleDataList.append(tleDataClass) # Reset for next TLE resetTleStr() + if not tleDataList and skipped: + raise skipped[0] return tleDataList def _str2tleFormat(value: float) -> str: