Skip to content
Open
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
5cd7ade
GH-126910 jit_unwind: refactor EH frame generation
diegorusso Mar 12, 2026
669dfb9
GH-126910 jit_unwind: add GDB JIT interface and test
diegorusso Mar 12, 2026
255c0b3
Fix make smelly
diegorusso Mar 17, 2026
ac018d6
Add __jit_debug_descriptor to ignored.tsv
diegorusso Mar 17, 2026
b0bab8c
📜🤖 Added by blurb_it.
blurb-it[bot] Mar 17, 2026
a0dff1f
Fix check C globals
diegorusso Mar 17, 2026
2b52588
Merge branch 'main' into add-gdb-support
diegorusso Mar 25, 2026
e44170e
Address Pablo's feedback
diegorusso Mar 27, 2026
2e40f1d
Fix smelly
diegorusso Mar 25, 2026
d890add
Merge branch 'main' into add-gdb-support
diegorusso Mar 30, 2026
965a543
Strengthen JIT GDB backtrace tests
diegorusso Mar 30, 2026
17be0a2
Fix x86_64 unwind
diegorusso Mar 30, 2026
f47d763
Add comment for inviariant and fix CFI instructions
diegorusso Mar 31, 2026
67ae6cb
Rename jit_executor/jit_shim to just jit_entry
diegorusso Apr 1, 2026
a18cb96
Address Pablo's feedback
diegorusso Apr 8, 2026
bdc8d12
Make the mutex private
diegorusso Apr 8, 2026
6357698
Address Pablo's feedback
diegorusso Apr 16, 2026
0b07c57
Fix CFI prologue mismatch in GDB JIT unwind info
pablogsal Apr 18, 2026
93bbf99
Tighten skip guards on test_gdb.test_jit
pablogsal Apr 18, 2026
3eeddb9
Cross-check stepping stays inside JIT region in test_gdb.test_jit
pablogsal Apr 18, 2026
a9c6315
Assert backtrace shape in test_gdb.test_jit
pablogsal Apr 18, 2026
6ee9fbc
Fix issues
diegorusso Apr 20, 2026
0193cb6
Fix GDB JIT unwind info and harden the tests
pablogsal Apr 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Include/internal/pycore_ceval.h
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,10 @@ extern PyStatus _PyPerfTrampoline_AfterFork_Child(void);
#ifdef PY_HAVE_PERF_TRAMPOLINE
extern _PyPerf_Callbacks _Py_perfmap_callbacks;
extern _PyPerf_Callbacks _Py_perfmap_jit_callbacks;
extern void _PyPerfJit_WriteNamedCode(const void *code_addr,
unsigned int code_size,
const char *entry,
const char *filename);
#endif

static inline PyObject*
Expand Down
28 changes: 28 additions & 0 deletions Include/internal/pycore_jit_unwind.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
#ifndef Py_CORE_JIT_UNWIND_H
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be Py_INTERNAL_JIT_UNWIND_H

#define Py_CORE_JIT_UNWIND_H

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is missing Py_BUILD_CORE guard no?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I've seen now the other headers files.

#ifdef PY_HAVE_PERF_TRAMPOLINE
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The entire file is gated on PY_HAVE_PERF_TRAMPOLINE, but the GDB JIT interface is conceptually independent of perf no?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops yea you're right.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for now, I'll add the bare minimum to address this but I already in mind some refactoring to do with another PR. Let's land this first and then I will refactor the code in light of adding libcc (for gnu backtrace)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this solution won't be the best, but it will be improved in subsequent PRs. I don't want to keep changing this PR.


#include <stddef.h>

/* Return the size of the generated .eh_frame data for the given encoding. */
size_t _PyJitUnwind_EhFrameSize(int absolute_addr);

/*
* Build DWARF .eh_frame data for JIT code; returns size written or 0 on error.
* absolute_addr selects the FDE address encoding:
* - 0: PC-relative offsets (perf jitdump synthesized DSO).
* - nonzero: absolute addresses (GDB JIT in-memory ELF).
*/
size_t _PyJitUnwind_BuildEhFrame(uint8_t *buffer, size_t buffer_size,
const void *code_addr, size_t code_size,
int absolute_addr);

void _PyJitUnwind_GdbRegisterCode(const void *code_addr,
unsigned int code_size,
const char *entry,
const char *filename);

#endif // PY_HAVE_PERF_TRAMPOLINE

#endif // Py_CORE_JIT_UNWIND_H
20 changes: 20 additions & 0 deletions Lib/test/test_gdb/gdb_jit_sample.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Sample script for use by test_gdb.test_jit

import operator
import sys


def jit_bt_hot(depth, warming_up_caller=False):
if warming_up_caller:
return
if depth == 0:
id(42)
return

warming_up = True
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this loop hang? When warming_up=True, the call passes warming_up_caller=True which returns immediately at line 8, so the recursive body never actually executes. If the JIT does not activate via some other path, would this not spin forever until the timeout kills it? Should there be a max iteration count as a safety net?

Also, line 16 uses bitwise & instead of and. Was that intentional? It means is_active() is always evaluated even when is_enabled() is False.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've simplified the test, the loop is not more controlled and deterministic.

while warming_up:
warming_up = sys._jit.is_enabled() & (not sys._jit.is_active())
operator.call(jit_bt_hot, depth - 1, warming_up)


jit_bt_hot(10)
35 changes: 35 additions & 0 deletions Lib/test/test_gdb/test_jit.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
import os
import re
import sys
import unittest

from .util import setup_module, DebuggerTests


JIT_SAMPLE_SCRIPT = os.path.join(os.path.dirname(__file__), "gdb_jit_sample.py")


def setUpModule():
setup_module()


@unittest.skipUnless(
hasattr(sys, "_jit") and sys._jit.is_available(),
"requires a JIT-enabled build",
)
class JitBacktraceTests(DebuggerTests):
def test_bt_unwinds_through_jit_frames(self):
gdb_output = self.get_stack_trace(
script=JIT_SAMPLE_SCRIPT,
cmds_after_breakpoint=["bt"],
PYTHON_JIT="1",
)
self.assertIn("py::jit_executor:<jit>", gdb_output)
self.assertIn("py::jit_shim:<jit>", gdb_output)
self.assertRegex(
gdb_output,
re.compile(
r"py::jit_executor:<jit>.*(_PyEval_EvalFrameDefault|_PyEval_Vector)",
re.DOTALL,
),
)
5 changes: 3 additions & 2 deletions Lib/test/test_gdb/util.py
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,8 @@ def get_stack_trace(self, source=None, script=None,
breakpoint=BREAKPOINT_FN,
cmds_after_breakpoint=None,
import_site=False,
ignore_stderr=False):
ignore_stderr=False,
**env_vars):
'''
Run 'python -c SOURCE' under gdb with a breakpoint.
Expand Down Expand Up @@ -239,7 +240,7 @@ def get_stack_trace(self, source=None, script=None,
args += [script]

# Use "args" to invoke gdb, capturing stdout, stderr:
out, err = run_gdb(*args, PYTHONHASHSEED=PYTHONHASHSEED)
out, err = run_gdb(*args, PYTHONHASHSEED=PYTHONHASHSEED, **env_vars)

if not ignore_stderr:
for line in err.splitlines():
Expand Down
1 change: 1 addition & 0 deletions Makefile.pre.in
Original file line number Diff line number Diff line change
Expand Up @@ -510,6 +510,7 @@ PYTHON_OBJS= \
Python/suggestions.o \
Python/perf_trampoline.o \
Python/perf_jit_trampoline.o \
Python/jit_unwind.o \
Python/remote_debugging.o \
Python/$(DYNLOADFILE) \
$(LIBOBJS) \
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add support for unwinding JIT frames using GDB. Patch by Diego Russo
31 changes: 31 additions & 0 deletions Python/jit.c
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
#include "pycore_interpframe.h"
#include "pycore_interpolation.h"
#include "pycore_intrinsics.h"
#include "pycore_jit_unwind.h"
#include "pycore_lazyimportobject.h"
#include "pycore_list.h"
#include "pycore_long.h"
Expand Down Expand Up @@ -60,6 +61,28 @@ jit_error(const char *message)
PyErr_Format(PyExc_RuntimeWarning, "JIT %s (%d)", message, hint);
}

static void
jit_record_code(const void *code_addr, size_t code_size,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will leave this for the future but as this is unconditionally active I assume will have a perf cost we probably want top measure

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm measuring it.. it might take some time.

const char *entry, const char *filename)
{
#ifdef PY_HAVE_PERF_TRAMPOLINE
_PyPerf_Callbacks callbacks;
_PyPerfTrampoline_GetCallbacks(&callbacks);
if (callbacks.write_state == _Py_perfmap_jit_callbacks.write_state) {
_PyPerfJit_WriteNamedCode(
code_addr, (unsigned int)code_size, entry, filename);
return;
}
_PyJitUnwind_GdbRegisterCode(
code_addr, (unsigned int)code_size, entry, filename);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code_size comes in as size_t but gets cast to unsigned int here. I know JIT regions will not be 4GB, but should the API just take size_t throughout for consistency?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is now done.

#else
(void)code_addr;
(void)code_size;
(void)entry;
(void)filename;
#endif
}

static size_t _Py_jit_shim_size = 0;

static int
Expand Down Expand Up @@ -731,6 +754,10 @@ _PyJIT_Compile(_PyExecutorObject *executor, const _PyUOpInstruction trace[], siz
}
executor->jit_code = memory;
executor->jit_size = total_size;
jit_record_code(memory,
code_size + state.trampolines.size,
"jit_executor",
"<jit>");
return 0;
}

Expand Down Expand Up @@ -781,6 +808,10 @@ compile_shim(void)
return NULL;
}
_Py_jit_shim_size = total_size;
jit_record_code(memory,
code_size + state.trampolines.size,
"jit_shim",
"<jit>");
return (_PyJitEntryFuncPtr)memory;
}

Expand Down
Loading
Loading