Skip to content

Restore the -k option (keep horizontal whitespace) - #49

Merged
externl merged 5 commits into
mainfrom
restore-keep-spaces-option
Nov 17, 2026
Merged

externl merged 5 commits into
mainfrom
restore-keep-spaces-option

Conversation

@externl

@externl externl commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

Restore the -k option of mcpp 2.7.2, so that a token in the preprocessed output keeps the column it has in the source file. The source spans in zeroc-ice/ice#7007 need this to be exact.

This repository is a fork of mcpp 2.7.2, which is no longer maintained. In April 2015 (42be792), the fork dropped the code the Slice compilers didn't use, and -k was part of it.

The Slice compilers use this fork only where we build mcpp ourselves, such as Windows and RHEL. Elsewhere, such as macOS and Debian, they use the platform's mcpp 2.7.2, which accepts -k. None of the restored code is new: it is the 2.7.2 code from the import in e6e749e, apart from a check for a preprocessing mode this fork no longer has.

  • Accept -k again, and list it in the usage text.
  • Write spaces and tabs out as they are in the input when -k is given, and replace each comment with spaces of the same length (parse_line and read_a_comment in support.c).
  • Leave the output unchanged when -k isn't given.

With -k, spaces and tabs are written out as they are in the input, and each comment becomes spaces of the
same length, so a token keeps the column it has in the source file.
Copilot AI balanced review requested due to automatic review settings November 10, 2026 15:11

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The -k comment-padding path can write past its allocation, while multiline comments and non-invoked function-like macros still shift token columns.

3 open issues
What changed in this PR

Adds the upstream mcpp -k option to preserve horizontal whitespace, supporting accurate source-column spans in the Slice compiler.

Changes:

  • Registers and documents -k.
  • Adds whitespace-preservation state and updates it during directive processing.
  • Preserves spaces and tabs and pads block comments based on their length.
File Description
system.c Parses -k and adds usage text.
support.c Preserves horizontal whitespace and calculates comment padding.
mcpp_main.c Initializes and enables whitespace-preservation mode.
internal.H Declares the new option and shared state.
directive.c Updates preservation state after directives.

Review effort: Balanced

Comment thread support.c Resolved
Comment thread mcpp_main.c Resolved
Comment thread support.c Resolved
With -k, a comment that follows a truncated comment and a token on the same line had its padding length
computed as a negative number, which became a huge count.

@pepone pepone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Claude (Opus 5.5):

  • With -k -M, a function-like macro call that spans lines and has tokens after it sets src_col, but only the output path in mcpp_main clears it, so every later line skips directive(). With the input below, mcpp -k -M prints dep2.o: dep2.ice without dep.h, while -M alone lists it. Please clear src_col next to wrong_line = FALSE in the no_output branch (mcpp_main.c:511). I checked that this fixes -k -M and leaves the -k and -C -k output unchanged.

    #define F(x) x
    F(
    1) z
    #include "dep.h"

With -k and -M, a function-like macro call that spans lines and has
tokens after it sets 'src_col', and every later directive is then
skipped, so the dependency line misses the files included after it.
@externl

externl commented Nov 12, 2026

Copy link
Copy Markdown
Member Author

Fixed in 0133d03: src_col is cleared in the no_output branch, and -k -M lists dep.h for your input. Homebrew's mcpp has the same bug, so it comes with the 2.7.2 code.

🤖 Addressed by Claude Code

@externl

externl commented Nov 12, 2026

Copy link
Copy Markdown
Member Author

Reverted in ef1e5eb; my earlier comment said this was fixed. -k here has to behave as it does in the mcpp 2.7.2 that Homebrew and the Linux distributions package, so that it is the same on every platform, and that mcpp has this -k -M behavior too.

@externl
externl requested a review from pepone November 12, 2026 16:28

@InsertCreativityHere InsertCreativityHere left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve of this PR because it brings our fork closer to what the other MCPPs do.
And I would rather be faithful to the original, than go around fixing minor bugs.

But, -k is not sufficient to the goal of preserving column counts, at least not with the logic of this fork. I didn't check everything, but at the very least, MCPP expands/moves macros and comments in ways that -k can't handle.

So even with this PR, we won't always have accurate column counts in our Slice files.

@pepone

pepone commented Nov 12, 2026

Copy link
Copy Markdown
Member

🤖 Claude (Opus 5.5): Agreed: Ice links the Homebrew mcpp on macOS and the distribution's libmcpp on Linux, so fixing this only here wouldn't help those platforms. The Ice change that adds -k should then pass it only when preprocessing, not together with -M for --depend.

@pepone pepone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Claude (Opus 5.5): LGTM.

@externl

externl commented Nov 12, 2026

Copy link
Copy Markdown
Member Author

Agreed, -k won't give exact columns everywhere: a macro expansion or a block comment that spans lines still moves what follows it. So the language server will treat the span's columns as a hint and match each name against the editor's text, falling back to the whole line when the text doesn't match. With -k that hint should be exact for nearly all real Slice files.

@bernardnormier bernardnormier left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The restore is correct and faithful to 2.7.2, with one exception: the change to the comment padding in parse_line. Please keep this PR a pure restore and move that fix to a follow-up PR (see the inline comment).

Title and description. The description doesn't give a reader the background needed to judge the change, and the title says "Add" for code that is restored. Suggested title: Restore the -k option (keep horizontal whitespace). Suggested opening for the description, ahead of the existing bullet list:

Background

This repository is a fork of mcpp 2.7.2. The upstream project is no longer maintained. In April 2015 (42be792) we removed from this fork the code the Slice compilers didn't use, and the -k option was part of what was removed.

The Slice compilers use this fork only on the platforms where we build mcpp ourselves, such as Windows and RHEL. On the other platforms, such as macOS, they use the mcpp 2.7.2 that the platform provides, which is complete and accepts -k.

zeroc-ice/ice#7007 needs -k on every platform, so this PR restores it here. None of this code is new: it is the 2.7.2 code from the import in e6e749e.

The bullet about a comment that starts when the line buffer is already full then moves to the follow-up PR.

Comment thread support.c Outdated
Comment on lines +1112 to +1121
/*
* Truncate a comment too long for the rest of the buffer.
* The buffer may already be full: an earlier comment of
* this line was truncated to end at 'limit - 1', and a
* token after it moved 'tp' to 'limit'. 'limit - tp - 1'
* is then negative, and would become a huge 'com_size',
* so the comment gets no padding instead.
*/
if (tp + com_size >= limit - 1) /* Too long comment */
com_size = (tp < limit - 1) ? limit - tp - 1 : 0;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a fix to the 2.7.2 code, and the mcpp that Homebrew and the Linux distributions package doesn't have it. Please restore the 2.7.2 lines here and propose the fix in a follow-up PR:

Suggested change
/*
* Truncate a comment too long for the rest of the buffer.
* The buffer may already be full: an earlier comment of
* this line was truncated to end at 'limit - 1', and a
* token after it moved 'tp' to 'limit'. 'limit - tp - 1'
* is then negative, and would become a huge 'com_size',
* so the comment gets no padding instead.
*/
if (tp + com_size >= limit - 1) /* Too long comment */
com_size = (tp < limit - 1) ? limit - tp - 1 : 0;
if (tp + com_size >= limit - 1) /* Too long comment */
com_size = limit - tp - 1; /* Truncate */

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restored the 2.7.2 lines in 5f6aa94, and took the suggested title. The description now opens with the background on the fork and on which platforms use it.

@externl externl changed the title Add the -k option to keep horizontal whitespace Restore the -k option (keep horizontal whitespace) Nov 16, 2026
@externl
externl merged commit 0e0c521 into main Nov 17, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants