Skip to content

Fix char issues - #240

Merged
Sebbo94BY merged 9 commits into
planetteamspeak:masterfrom
MajorOli:patch/char-issues
Sep 22, 2026
Merged

Sebbo94BY merged 9 commits into
planetteamspeak:masterfrom
MajorOli:patch/char-issues

Conversation

@MajorOli

Copy link
Copy Markdown
Contributor

Description

Add Tests to CharTest.php and fix related errors.

Changes Made

public function toUnicode(): int
    {
        $h = ord($this->char[0]);

        if ($h <= 0x7F) {
            return $h;
        } elseif ($h < 0xC2) {
            return false; <<<< changed to -1
        } elseif ($h <= 0xDF) {
            return ($h & 0x1F) << 6 | (ord($this->char[1]) & 0x3F);
        } elseif ($h <= 0xEF) {
            return ($h & 0x0F) << 12 | (ord($this->char[1]) & 0x3F) << 6 | (ord($this->char[2]) & 0x3F);
        } elseif ($h <= 0xF4) {
            return ($h & 0x0F) << 18 | (ord($this->char[1]) & 0x3F) << 12 | (ord($this->char[2]) & 0x3F) << 6 | (ord($this->char[3]) & 0x3F);
        } else {
            return -1;
        }
    }

Fix issues like this

        //
        // 3-BYTE UTF-8 (U+0800 – U+FFFF)
        // Example: '€' (U+20AC) → E2 82 AC
        //
        $this->assertEquals(
            static::calculateUTF8Ordinal("\xE2\x82\xAC"),
            Char::fromHex('E282AC')->toUnicode()
        );

Changes covered in public function testUnicode1Byte()

Additional Notes

Maybe separate 2Byte, 3Byte and 4Byte test in separate functions.

Merge Request Checklists

  • Documentation reflects the changes made.
  • I have already covered the unit testing.

@Sebbo94BY

Copy link
Copy Markdown
Collaborator

Thanks for your pull request!

From my perspective it looks good, but the AI requests one change:

fromHex('C2') is now accepted because mb_strlen("\xC2", 'UTF-8') === 1, but it is a truncated UTF‑8 lead byte. Then toUnicode() reads a missing continuation byte as zero and returns 128 (U+0080)—silently turning malformed input into a valid, different character.

The new constructor/fromHex behavior needs either strict UTF‑8 validation or length/continuation-byte validation in toUnicode() that returns -1 for truncated sequences. A regression test for Char::fromHex('C2')->toUnicode() is important.

Otherwise, the core direction looks good: permitting a single multibyte UTF‑8 character fixes the previous byte-length limitation, and changing the invalid-leading-byte return from false to -1 matches the declared int return type. The added valid 2-, 3-, and 4-byte coverage is useful.

Could you please double-check this and especially add this regression test?

@Sebbo94BY

Copy link
Copy Markdown
Collaborator

Follow-up AI feedback:

The prior blocker is resolved. C2 now correctly returns -1, and the regression test covers it.

I would still request one change if this PR intends UTF‑8 Char support:

toHex() still delegates to toAscii(), which returns only the first byte. Consequently:

Char::fromHex('C2A2')->toHex(); // "C2", not "C2A2"
new Char('€')->toHex(); // "E2", not "E282AC"

That makes the newly supported multibyte fromHex() non-round-trippable and contradicts “hexadecimal value of the char.” toHex() should encode the complete stored byte string (e.g. uppercase bin2hex($this->char)), with a multibyte round-trip test.

Aside from that, the updated toUnicode() validation looks good: it checks expected byte length, validates continuation bytes, fixes the four-byte mask, and preserves the -1 invalid-input convention.

@Sebbo94BY

Copy link
Copy Markdown
Collaborator

One last thing, then we should be good to go:

Rechecked the latest head (82880f7)—both prior issues are fixed.

  • Truncated UTF‑8 sequences return -1, with a regression test.
  • toHex() now encodes all bytes, and multibyte UTF‑8 round-trips are tested.
  • The updated source passed syntax validation and focused checks for ASCII, valid 2/3/4-byte code points, invalid leads, truncation, and hex round-trip.

I don’t see any remaining merge-blocking issue. The only tiny nit is an old test comment saying invalid leading bytes “should return false” while assertions correctly expect -1; that can be cleaned up separately.

@Sebbo94BY
Sebbo94BY changed the base branch from dev to master September 22, 2026 15:01
@Sebbo94BY
Sebbo94BY merged commit 72718f6 into planetteamspeak:master Sep 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants