Repository navigation
feat(core,ooxml,cli): tatweel-padding rule — M4 4.4 - #35
Merged
Merged
Conversation
U+0640 is legitimate — it is the kashida, and a font inserts it during layout, which is why it never reaches the stored string. So the rule judges what surrounds each run of typed tatweel rather than the character, and `crate::tatweel` holds the argument. Reported: two or more in a row joined to a letter (a word stretched to a width — no typography needs a second tatweel, so the repetition is the evidence), and one wedged between two characters that already join (width, and no change to any letter's form). Silent: one carrying a harakat, which is the base that mark is written on; one at a joining edge, which is how a letter's contextual form is written alone; and a run joined to nothing, which is a rule somebody drew. All three are committed as cases a rule regressing to "any tatweel is a defect" fails. A warning, not an error: the text renders as arranged, and what is wrong is the string behind it. `repair --strip-tatweel` deletes exactly the runs the finding listed. Opt-in, for `--convert-bullets`' reason and more so — it deletes characters somebody typed. Safety is the same question as the threshold, and is asserted against `joining::forms`: deleting a run a join crosses changes no letter's form. The adapter now deletes `RemoveControls` and `RemoveTatweel` from one merged, descending sequence — both carry offsets into the text as scanned, so applying them in turn would delete the second set from a string the first had already shortened.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The last item in M4. PLAN §4.4.
The hard part is that tatweel is legitimate
U+0640 is the kashida — the elongation Arabic typography has justified text with for a thousand years — and a font inserts it during layout, which is why it never reaches the stored string. A justified paragraph has none of it in the file. Every tatweel this tool can see was typed by somebody, and only some of those are a defect. A rule that concluded one from the character alone would be ADR 0004's first failure mode for the third time.
So
crate::tatweelgroups them into maximal runs and asks what each run is joined to.Reported
Silent, and each one is a real document
All three are committed as cases that a rule regressing to "any tatweel is a defect" fails.
A warning, not an error
ADR 0004's severity table read honestly: the text renders exactly as the author arranged it. What is wrong is the string behind it, and every cost is one a screenshot cannot show — a search for the heading no longer matches the heading, a spell-checker does not know the word, a screen reader reads the padding aloud, and the width it was measured against is gone the moment the box, the font or the point size changes.
evidence.offendersgives each run asU+0640 ARABIC TATWEEL ×5 @14— the length and a byte offset intoevidence.logical.repair --strip-tatweelThe second repair ever to edit the characters an author typed rather than the properties around them, after
literal-bullet, and opt-in for that rule's reason and more so.Whether stripping is safe turned out to be the same question as the threshold, and it is asserted against
joining::formsrather than restated: deleting a run a join crosses changes no letter's form, however long the run is. A stretched run at a word edge is not form-preserving — a noon padded to a width reverts to a plain noon — and that is the repair rather than a loss, because a plain noon is what the unpadded word says. TheCarrierandDisplayedcases are safe by never being touched.One thing the adapter learned
RemoveControlsandRemoveTatweelboth carry byte offsets into the text as scanned. A paragraph with a bidi control in a padded heading would have had the second set applied to a string the first had already shortened, and the second deletion would have landed on a letter.rewrite.rsnow deletes from one merged, descending sequence, and the case is committed in both fix orders — the planner's emission order is not a contract the rewriter may lean on.Acceptance
Met in the conformance suite, so it holds for
.pptxand.docxalike:tatweel-paddingwarning naming the run's offset and length in both formatsThrough the writer, a padded heading in a run of its own is repaired to the word it was and re-audits clean; a tatweel carrying a fatha is neither reported nor planned against.
Corpus
The five repairable reports gained one
optionskey each (strip_tatweel) and zero findings changed — the claim that nothing else moved.--strip-tatweeljoins the golden repair, so every opt-in fix is exercised.make verifyis green: fmt, clippy-D warnings, all tests, docs-D warnings, release build.Bookkeeping
M4 is now
[x].ROADMAP.md's M4 bullets are rewritten as landed items pointing at their plan sections.