[SPARK-58458][SQL][TESTS] Add coverage for the record split when multiLine is disabled

[Joorgem]

What changes were proposed in this pull request?

Adds a test to CSVSuite pinning what the CSV reader does with a line break inside a
quoted value when multiLine is left at its default of false: the record is cut at
the break into a NULL-tailed head plus a fragment, and the row count rises by one.

The test also pins the three consequences that follow from it — the multiLine=true
contrast, DROPMALFORMED discarding both halves, and the raw text captured when
columnNameOfCorruptRecord is declared.

Why are the changes needed?

The behaviour is not covered today. CSVSuite contains exactly one input literal with
a newline inside quotes, in the lineSep with 2 chars when multiLine set to ... test,
and it is confined to the multiLine = true branch:

val inputData = if (multiLine) {
  s"""name,"i am the${newLine} column1"${newLine}jack,30${newLine}tom,18"""
} else {
  s"name,age${newLine}jack,30${newLine}tom,18"
}

The multiLine = false branch deliberately drops the quoted break, so the default
path is never exercised with the input that distinguishes it. UnivocityParserSuite
does not cover it either.

That leaves an untested behaviour that users do meet: a line break inside a quoted
value is valid CSV (RFC 4180 §2.6) and appears in published datasets, so reading such
a file with default options is an ordinary thing to do. The split is a consequence of
the non-multiLine read path being line-oriented by design — HadoopFileLinesReader
splits the input on the line separator before the tokenizer sees it, and
UnivocityParser.parseLine then receives one physical line at a time — which is what
makes the file splittable. This PR does not propose changing that; it pins what the
current design produces, so a future change to that path cannot alter it silently.

DROPMALFORMED losing the whole source record rather than the damaged half is the
part most worth having locked down, since it is the opposite of what the mode's name
suggests for one bad line.

Does this PR introduce any user-facing change?

No. Test-only.

How was this patch tested?

This PR is the test. It was run in CI against this base before the PR was opened:
Build modules: sql - slow tests, sql - extended tests and sql - other tests all
passed, so the test holds in CSVv1Suite, CSVv2Suite and CSVLegacyTimeParserSuite
alike.

The expected rows were derived before the Scala was written, from pyspark 3.5.9 with
the same three-record input and the same explicit three-column schema. 3.5.9 is two
majors behind this base, so those values were treated as a hypothesis to be confirmed
by that CI run rather than as evidence in themselves:

read	rows
default (multiLine=false)	4 — ("1","ACME LTDA","SP"), ("2","EMPRESA COM",null), ("QUEBRA DE LINHA\"","RJ",null), ("3","OUTRA EMPRESA","MG")
multiLine=true	3, with the break preserved inside the value
mode=DROPMALFORMED	2 — records 1 and 3; record 2 lost entirely
columnNameOfCorruptRecord declared	4, both halves carrying raw text 2,"EMPRESA COM and QUEBRA DE LINHA",RJ

Reproduction script: docs/evidence/c5_expected_rows.py in the authoring repo (not
part of this patch).

Was this patch authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Opus 5)

Related

This is the third of three separate concerns in the same area, filed in order:

1. #57608 — documents what multiLine=false
   does to a line break inside a quoted value (open, reviewed, revision applied).
2. #57658 — corrects the PERMISSIVE
   "drops corrupt records" sentence in the CSV and JSON docs.
3. This PR — pins the behaviour [MINOR][DOCS] Document CSV read behaviour when multiLine is disabled #57608 documents, so it cannot change silently.

Flagging the overlap explicitly since all three touch CSV: this one adds a test and
changes no docs and no main code.

[uros-b]

The test covers PERMISSIVE and DROPMALFORMED but omits FAILFAST. Please expand test coverage accordingly.