certain f/t-string expressions not copied correctly to debug str or interpolations

[tom-pytel]

Bug report

Bug description:

The function set_ftstring_expr() in Parser/lexer/lexer.c does not currently copy certain f/t-string expressions correctly. It has problems with non-coding quotes (escaped or in comments) and nested f/t-strings. This applies to both self-documenting debug expressions and more importantly to t-string interpolation expressions.

For example, the # comment below should not print:

>>> print(f'''{'\'' # comment
... =}''')
'\'' # comment
="'"

Copied expression terminates early:

>>> print(f'{(f'''
... # not comment A
... { # comment B '
... 3 # comment C
... * 2}''', '\n# not comment D\n6')=}')
(f'''
# not comment A
{ # comment B '
3
* 2}''', '\n('\n# not comment A\n6', '\n# not comment D\n6')

It does not appear the function was written to handle nested f/t-strings and the string quote handling does not take into account escaped quotes.

CPython versions tested on:

3.14, CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-141271: fix f/t-str set expr quotes and nested comments #141274

[tom-pytel]

Ping @pablogsal I have a fix if these cases are not too degenerate if you are interested:

>>> print(f'''{'\'' # comment
... =}''')
'\''
="'"

>>> print(f'{(f'''
... # not comment A
... { # comment B '
... 3 # comment C
... * 2}''', '\n# not comment D\n6')=}')
(f'''
# not comment A
{
3
* 2}''', '\n# not comment D\n6')=('\n# not comment A\n6', '\n# not comment D\n6')

[pablogsal]

Go ahead and create a PR! I think also @lysnikolaou probably wants to get a look