Skip to content

Commit 3df5c13

Browse files
[3.10] pythongh-137973: Add a non-parallel test plan to the iOS testbed project (pythonGH-138018) (python#138039)
Modifies the iOS testbed project to add a test plan. This simplifies the iOS test runner, as we can now use the built-in log streaming to see test results. It also allows for some other affordances, like providing a default LLDB config, and using a standardized mechanism for specifying test arguments. (cherry picked from commit 2ba2287) Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
1 parent 1f43a06 commit 3df5c13

10 files changed

Lines changed: 260 additions & 298 deletions

File tree

.gitignore

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,6 @@ iOS/testbed/Python.xcframework/ios-*/lib
7171
iOS/testbed/Python.xcframework/ios-*/Python.framework
7272
iOS/testbed/iOSTestbed.xcodeproj/project.xcworkspace
7373
iOS/testbed/iOSTestbed.xcodeproj/xcuserdata
74-
iOS/testbed/iOSTestbed.xcodeproj/xcshareddata
7574
Mac/Makefile
7675
Mac/PythonLauncher/Info.plist
7776
Mac/PythonLauncher/Makefile
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
The iOS test runner has been simplified, resolving some issues that have
2+
been observed using the runner in GitHub Actions and Azure Pipelines test
3+
environments.

iOS/README.rst

Lines changed: 15 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -295,7 +295,7 @@ project, and then boot and prepare the iOS simulator.
295295
Debugging test failures
296296
-----------------------
297297

298-
Running ``make test`` generates a standalone version of the ``iOS/testbed``
298+
Running ``make testios`` generates a standalone version of the ``iOS/testbed``
299299
project, and runs the full test suite. It does this using ``iOS/testbed``
300300
itself - the folder is an executable module that can be used to create and run
301301
a clone of the testbed project.
@@ -318,12 +318,26 @@ This is the equivalent of running ``python -m test -W test_os`` on a desktop
318318
Python build. Any arguments after the ``--`` will be passed to testbed as if
319319
they were arguments to ``python -m`` on a desktop machine.
320320

321+
Testing in Xcode
322+
^^^^^^^^^^^^^^^^
323+
321324
You can also open the testbed project in Xcode by running::
322325

323326
$ open my-testbed/iOSTestbed.xcodeproj
324327

325328
This will allow you to use the full Xcode suite of tools for debugging.
326329

330+
The arguments used to run the test suite are defined as part of the test plan.
331+
To modify the test plan, select the test plan node of the project tree (it
332+
should be the first child of the root node), and select the "Configurations"
333+
tab. Modify the "Arguments Passed On Launch" value to change the testing
334+
arguments.
335+
336+
The test plan also disables parallel testing, and specifies the use of the
337+
``iOSTestbed.lldbinit`` file for providing configuration of the debugger. The
338+
default debugger configuration disables automatic breakpoints on the
339+
``SIGINT``, ``SIGUSR1``, ``SIGUSR2``, and ``SIGXFSZ`` signals.
340+
327341
Testing on an iOS device
328342
^^^^^^^^^^^^^^^^^^^^^^^^
329343

@@ -338,40 +352,3 @@ select the root node of the project tree (labeled "iOSTestbed"), then the
338352
(this will likely be your own name), and plug in a physical device to your
339353
macOS machine with a USB cable. You should then be able to select your physical
340354
device from the list of targets in the pulldown in the Xcode titlebar.
341-
342-
Running specific tests
343-
^^^^^^^^^^^^^^^^^^^^^^
344-
345-
As the test suite is being executed on an iOS simulator, it is not possible to
346-
pass in command line arguments to configure test suite operation. To work
347-
around this limitation, the arguments that would normally be passed as command
348-
line arguments are configured as part of the ``iOSTestbed-Info.plist`` file
349-
that is used to configure the iOS testbed app. In this file, the ``TestArgs``
350-
key is an array containing the arguments that would be passed to ``python -m``
351-
on the command line (including ``test`` in position 0, the name of the test
352-
module to be executed).
353-
354-
Disabling automated breakpoints
355-
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
356-
357-
By default, Xcode will inserts an automatic breakpoint whenever a signal is
358-
raised. The Python test suite raises many of these signals as part of normal
359-
operation; unless you are trying to diagnose an issue with signals, the
360-
automatic breakpoints can be inconvenient. However, they can be disabled by
361-
creating a symbolic breakpoint that is triggered at the start of the test run.
362-
363-
Select "Debug > Breakpoints > Create Symbolic Breakpoint" from the Xcode menu, and
364-
populate the new brewpoint with the following details:
365-
366-
* **Name**: IgnoreSignals
367-
* **Symbol**: UIApplicationMain
368-
* **Action**: Add debugger commands for:
369-
- ``process handle SIGINT -n true -p true -s false``
370-
- ``process handle SIGUSR1 -n true -p true -s false``
371-
- ``process handle SIGUSR2 -n true -p true -s false``
372-
- ``process handle SIGXFSZ -n true -p true -s false``
373-
* Check the "Automatically continue after evaluating" box.
374-
375-
All other details can be left blank. When the process executes the
376-
``UIApplicationMain`` entry point, the breakpoint will trigger, run the debugger
377-
commands to disable the automatic breakpoints, and automatically resume.

0 commit comments

Comments
 (0)