Skip to content

Internals::SvREADONLY loses UTF-8 metadata for wide scalars #1970

Closed

Description

@fglock

Summary

Internals::SvREADONLY($scalar, 1) loses the scalar's UTF-8-on metadata for wide strings. A subsequent interpolation or string operation therefore treats the unchanged Unicode value as byte-backed. This breaks Readonly::Scalar users: the pure-Perl Readonly module marks values by calling Internals::SvREADONLY.

Minimal reproducer

my $wide = "\x{2122}";
Internals::SvREADONLY($wide, 1);
my $copy = "$wide";
printf "ord=%X utf8=%d\n", ord($copy), utf8::is_utf8($copy) ? 1 : 0;

System Perl 5.42.2 prints ord=2122 utf8=1. PerlOnJava prints ord=2122 utf8=0 on both the JVM and interpreter backends. The mismatch is already observable on $wide immediately after SvREADONLY, before interpolation.

CPAN evidence

  • Distribution: Encode-Wide 0.07
  • Run: 20261104-151051-20617
  • System-Perl oracle: all 343 tests pass.
  • PerlOnJava: t/integration.t fails the “No shared state: interleaved calls produce correct independent results” subtest; eight conversions for U+017E, U+017D, U+010D and U+2122 return the original wide character instead of an entity. The failure repeats after dependency retry.
  • Environment: macOS 26.6 ARM64, OpenJDK 24.0.2, system Perl 5.42.2.

The test's calls are sequential; this is not a threading failure. The common cause is that read-only scalar state obscures the underlying string type/UTF-8 flag, which string interpolation and utf8::is_utf8 use to choose byte versus Unicode behavior.

Activity

  1. added
    bugSomething isn't working
    area:runtimeCore Perl runtime semantics
    on Nov 4, 2026
  2. fglock commented on Nov 4, 2026

    @fglock
    OwnerAuthor

    State: released
    Agent/session: Codex-utf8-1970-20261104
    Scope: #1970; preserve scalar string/UTF-8 metadata through runtime read-only wrapping and add permanent system-Perl-validated coverage for both backends.
    Work: fix/utf8-on-readonly-captured-scalar, delivered in PR #1973, merge commit 9db2a7d790388f1a36e790642eca0fff67e47154.
    Checkpoint: 2026-11-04 UTC; PR #1973 merged after Windows and Ubuntu CI passed; issue #1970 auto-closed by the merge.
    Evidence: nice -n 19 make passed; the issue regression passed under system Perl 5.42.2 and both PerlOnJava backends; Encode-Wide 0.07 t/integration.t passed on JVM and interpreter; nice -n 19 make check-links passed.
    Blocked by: none.
    Next: none; the confirmed issue is resolved.

  3. changed the title [-]Preserve UTF-8 flag when interpolating captured read-only wide scalars[/-] [+]Internals::SvREADONLY loses UTF-8 metadata for wide scalars[/+] on Nov 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:runtimeCore Perl runtime semanticsbugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions