From 408f9e0d59323c877845948d7e72aa2b3b426ddd Mon Sep 17 00:00:00 2001 From: Brandon Page Date: Thu, 24 Sep 2026 15:44:13 -0700 Subject: [PATCH 1/2] Fix Android headless test hangs --- androidTests/README.md | 56 +++-- androidTests/android/app/build.gradle.kts | 1 - .../reactnative/BaseReactNativeTest.kt | 205 +++++++++++------- .../reactnative/ReactMobileSyncTest.kt | 2 - .../androidsdk/reactnative/ReactNetTest.kt | 2 - .../reactnative/HeadlessLogcatParser.kt | 73 +++++++ .../util/SalesforceReactTestApp.kt | 6 + test/HeadlessTestApp.js | 15 +- 8 files changed, 231 insertions(+), 129 deletions(-) create mode 100644 androidTests/android/app/src/main/java/com/salesforce/androidsdk/reactnative/HeadlessLogcatParser.kt diff --git a/androidTests/README.md b/androidTests/README.md index 02432933..8d37cf3e 100644 --- a/androidTests/README.md +++ b/androidTests/README.md @@ -1,6 +1,6 @@ # Android Tests -UI-driven tests for the React Native bridge using UIAutomator. +Headless instrumentation tests for the React Native bridge. ## Test Credentials @@ -10,28 +10,24 @@ Tests require Salesforce org credentials. Copy `shared/test/test_credentials.jso ### From Android Studio 1. Set up the test app: `./prepareandroid.js` -2. Start the Metro bundler: `npm start` -3. Open the project in Android Studio: `androidTests/android/` -4. Right-click on a test class or method → Run +2. Open the project in Android Studio: `androidTests/android/` +3. Right-click on a test class or method → Run ### From Command Line ```bash -cd androidTests -./gradlew connectedAndroidTest +cd androidTests/android +./gradlew connectedDebugAndroidTest ``` ## Test Structure -Each test class extends `BaseReactNativeTest` and provides: - -- `testTimeoutMs`: Timeout for each test (optional, default: 15s) +Each test class extends `BaseReactNativeTest` and maps its JUnit methods to the +shared JavaScript test names. Example: ```kotlin class ReactNetTest : BaseReactNativeTest() { - override val testTimeoutMs: Long = 30_000 - @Test fun testGetApiVersion() = runTest("testGetApiVersion") @Test fun testVersions() = runTest("testVersions") // ... @@ -40,36 +36,34 @@ class ReactNetTest : BaseReactNativeTest() { ## Timeouts -Default timeouts: -- **OAuth, Harness, SmartStore**: 15 seconds (default) -- **Net**: 30 seconds -- **MobileSync**: 60 seconds - -Override `testTimeoutMs` to customize per test class. +Every headless test has a 30-second JavaScript timeout. The Android collector +independently fails after 30 seconds without a new result, even if a blocking +native call prevents the JavaScript timeout from running. Its 45-minute overall +ceiling remains as a final guard for a suite that continues to make progress. +The collector values can be overridden with the `progressTimeoutMs` and +`maxRunMs` instrumentation arguments. ## How It Works -1. Each test method triggers the activity to launch via `@Rule` -2. `runTest()` waits for the React Native app to load -3. Taps the individual test button in the UI (testID: `run_{testName}`) -4. Waits for the test result element to appear (testID: `result_{testName}_pass` or `result_{testName}_fail`) -5. Asserts the result +1. The first JUnit test launches the app and authenticates from `test_credentials.json`. +2. `HeadlessTestApp` runs the shared JavaScript tests sequentially. +3. Each JavaScript result and the final summary are written to logcat sentinels. +4. The Android collector reads finite logcat snapshots and caches all results. +5. Each JUnit method asserts its corresponding cached result. ## Authentication Tests use instant authentication via `TestAuthenticationActivity`: - Credentials are loaded from `assets/test_credentials.json` -- `ActivityScenarioRule` launches `TestAuthenticationActivity` with credentials as an intent extra +- The headless collector launches `TestAuthenticationActivity` with credentials as an intent extra - Activity authenticates and launches the main React Native activity -- Tests interact with the running React Native app via UIAutomator +- The JavaScript suite reports results through logcat; no UI interaction is required -## Test IDs +## Result Protocol -The JavaScript test app must expose these accessibility identifiers: -- `testList`: The ScrollView containing all test buttons -- `run_{testName}`: Button to run individual test -- `result_{testName}_pass`: Success indicator -- `result_{testName}_fail`: Failure indicator -- `error_{testName}`: Error message text (if test fails) +The JavaScript test app emits `SFTESTBEGIN::`, one `SFTESTRESULT::` JSON line per +test, and `SFTESTDONE::` when the suite completes. The instrumentation collector +uses finite `logcat -d` snapshots so Android 12L cannot retain the final buffered +line in a long-lived logcat pipe. See `docs/android-tests/README.md` for detailed documentation. diff --git a/androidTests/android/app/build.gradle.kts b/androidTests/android/app/build.gradle.kts index d6296b96..da01ed56 100644 --- a/androidTests/android/app/build.gradle.kts +++ b/androidTests/android/app/build.gradle.kts @@ -69,7 +69,6 @@ dependencies { implementation("com.facebook.react:hermes-android:0.86.0") androidTestImplementation("androidx.test:runner:1.6.2") - androidTestImplementation("androidx.test:rules:1.6.1") androidTestImplementation("androidx.test.ext:junit:1.2.1") androidTestImplementation("androidx.test.uiautomator:uiautomator:2.3.0") } diff --git a/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/BaseReactNativeTest.kt b/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/BaseReactNativeTest.kt index 2a2a5aa8..582bbd2f 100644 --- a/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/BaseReactNativeTest.kt +++ b/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/BaseReactNativeTest.kt @@ -26,22 +26,14 @@ */ package com.salesforce.androidsdk.reactnative -import android.Manifest import android.content.Intent -import android.os.Build -import android.os.ParcelFileDescriptor +import android.os.SystemClock import androidx.test.platform.app.InstrumentationRegistry -import androidx.test.rule.GrantPermissionRule import androidx.test.uiautomator.UiDevice import com.salesforce.androidsdk.util.test.TestAuthenticationActivity import org.json.JSONObject import org.junit.Assert.assertTrue -import org.junit.Rule -import java.io.BufferedReader -import java.io.InputStreamReader import java.util.concurrent.ConcurrentHashMap -import java.util.concurrent.CountDownLatch -import java.util.concurrent.TimeUnit data class TestResult(val success: Boolean, val message: String?) @@ -56,27 +48,14 @@ data class TestResult(val success: Boolean, val message: String?) * * This version drives NO UI. It launches the app ONCE; the app mounts * HeadlessTestApp (see androidTests/index.js), which runs the whole shared suite - * and emits one logcat line per result. [HeadlessResults] streams logcat, parses - * those lines, and each @Test simply asserts on its parsed result. Because the run - * happens once for the whole process, the ~70min (35 cold starts) runtime collapses - * to a single launch while every @Test still reports independently in the JUnit XML. + * and emits one logcat line per result. [HeadlessResults] reads finite logcat + * snapshots, parses those lines, and each @Test simply asserts on its parsed result. + * Because the run happens once for the whole process, the ~70min (35 cold starts) + * runtime collapses to a single launch while every @Test still reports independently + * in the JUnit XML. */ abstract class BaseReactNativeTest { - // Pre-grant POST_NOTIFICATIONS so no permission dialog can interrupt the run on - // API 33+. (The app manifest removes the permission; granting is a no-op if absent.) - @get:Rule - val permissionRule: GrantPermissionRule = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU) { - GrantPermissionRule.grant(Manifest.permission.POST_NOTIFICATIONS) - } else { - GrantPermissionRule.grant() - } - - // Vestigial: kept only so the two subclass overrides (ReactNetTest, ReactMobileSyncTest) - // still compile. Real per-test timeouts now live in HeadlessTestApp.js (SUITE_TIMEOUTS). - open val testTimeoutMs: Long - get() = 60_000 - fun runTest(name: String) { val result = HeadlessResults.resultFor(name) assertTrue(result.message ?: "Test '$name' failed", result.success) @@ -91,13 +70,17 @@ abstract class BaseReactNativeTest { */ object HeadlessResults { - private const val BEGIN_PREFIX = "SFTESTBEGIN::" private const val RESULT_PREFIX = "SFTESTRESULT::" - private const val DONE_PREFIX = "SFTESTDONE::" private const val DEFAULT_MAX_RUN_MS = 45L * 60 * 1000 // < Firebase --timeout 60m // App must launch and mount HeadlessTestApp (emit BEGIN) within this, else fail // fast with a real cause instead of blocking the whole run on a silent no-mount. private const val DEFAULT_BEGIN_TIMEOUT_MS = 3L * 60 * 1000 + // Match HeadlessTestApp's per-test cap, but enforce it outside the JS event + // loop so a blocking native call cannot suppress the JavaScript timer. + private const val DEFAULT_PROGRESS_TIMEOUT_MS = 30_000L + private const val LOGCAT_POLL_INTERVAL_MS = 1_000L + private const val LOGCAT_SNAPSHOT_COMMAND = + "logcat -d -v raw -s ReactNativeJS:I AndroidRuntime:E" private val results = ConcurrentHashMap() private val lock = Object() @@ -135,56 +118,16 @@ object HeadlessResults { val maxRunMs = InstrumentationRegistry.getArguments() .getString("maxRunMs")?.toLongOrNull() ?: DEFAULT_MAX_RUN_MS - // Clear logcat so we only read this run's output. - UiDevice.getInstance(instrumentation).executeShellCommand("logcat -c") + val device = UiDevice.getInstance(instrumentation) + // Clear logcat so every finite snapshot only contains this run's output. + device.executeShellCommand("logcat -c") val beginTimeoutMs = InstrumentationRegistry.getArguments() .getString("beginTimeoutMs")?.toLongOrNull() ?: DEFAULT_BEGIN_TIMEOUT_MS + val progressTimeoutMs = InstrumentationRegistry.getArguments() + .getString("progressTimeoutMs")?.toLongOrNull() ?: DEFAULT_PROGRESS_TIMEOUT_MS val targetPackage = context.packageName - val begun = CountDownLatch(1) - val done = CountDownLatch(1) - // Stream logcat from the shell uid (which holds READ_LOGS). Start reading - // BEFORE launching so no early sentinel is missed. AndroidRuntime:E is - // included so a FATAL crash in the app can be captured as the real cause. - val pfd = instrumentation.uiAutomation - .executeShellCommand("logcat -v raw -s ReactNativeJS:I AndroidRuntime:E") - val reader = BufferedReader(InputStreamReader(ParcelFileDescriptor.AutoCloseInputStream(pfd))) - val readerThread = Thread { - // Capture the FATAL EXCEPTION block only when it belongs to our process. - val fatalBuf = StringBuilder() - var fatalLinesLeft = 0 - try { - reader.forEachLine { line -> - when { - line.contains(BEGIN_PREFIX) -> begun.countDown() - // One malformed line must not kill the reader (the sole DONE - // consumer) — guard the parse. - line.contains(RESULT_PREFIX) -> runCatching { parseResult(line) } - line.contains(DONE_PREFIX) -> done.countDown() - line.contains("FATAL EXCEPTION") -> { - fatalBuf.setLength(0) - fatalBuf.append(line).append('\n') - fatalLinesLeft = 25 - } - fatalLinesLeft > 0 -> { - fatalBuf.append(line).append('\n') - fatalLinesLeft-- - // The "Process:" line tells us whose crash this is. - if (line.contains("Process:") && line.contains(targetPackage)) { - crashHint = fatalBuf.toString().take(1500) - fatalLinesLeft = 0 - } - } - } - } - } catch (_: Throwable) { - // Stream closed after the run finished — expected. - } - } - readerThread.isDaemon = true - readerThread.start() - // Launch once: TestAuthenticationActivity authenticates from the creds asset, // then starts MainActivity, which mounts HeadlessTestApp and runs the suite. context.startActivity( @@ -198,22 +141,120 @@ object HeadlessResults { // the whole maxRunMs and then report a misleading "no DONE". (A crash that // kills the shared instrumentation process is already reported quickly by // `am instrument` as "Process crashed"; this covers the alive-but-silent case.) - if (!begun.await(beginTimeoutMs, TimeUnit.MILLISECONDS)) { + var events = awaitEvents(device, targetPackage, beginTimeoutMs) { it.began || it.done } + if (!events.began) { runError = "Headless run did not emit BEGIN within ${beginTimeoutMs}ms — " + "app launched but HeadlessTestApp never mounted " + "(check test_credentials.json and the JS bundle)." + (crashHint?.let { "\nApp FATAL:\n$it" } ?: "") - runCatching { pfd.close() } return } - // Condition-wait on the DONE sentinel — no Thread.sleep / polling. - val finished = done.await(maxRunMs, TimeUnit.MILLISECONDS) - if (!finished) { - runError = "Headless run did not emit DONE within ${maxRunMs}ms" + - (crashHint?.let { " — app FATAL:\n$it" } ?: "") + // Android 12L's long-running `logcat` pipe can retain the final buffered + // lines indefinitely. Read finite `logcat -d` snapshots instead: the + // command exits and flushes, so a tail-position DONE is observable. + if (!events.done) { + val completion = awaitCompletion( + device, + targetPackage, + events, + maxRunMs, + progressTimeoutMs + ) + events = completion.events + if (!events.done) { + runError = completion.timeoutMessage + + (crashHint?.let { " — app FATAL:\n$it" } ?: "") + } + } + } + + private data class CompletionWait( + val events: HeadlessLogcatEvents, + val timeoutMessage: String + ) + + private fun awaitCompletion( + device: UiDevice, + targetPackage: String, + initialEvents: HeadlessLogcatEvents, + maxRunMs: Long, + progressTimeoutMs: Long + ): CompletionWait { + val startedAt = SystemClock.elapsedRealtime() + val overallDeadline = startedAt + maxRunMs + var lastProgressAt = startedAt + var lastResultCount = initialEvents.resultLines.size + var events = initialEvents + + while (true) { + if (events.done) return CompletionWait(events, "") + + val now = SystemClock.elapsedRealtime() + if (events.resultLines.size > lastResultCount) { + lastResultCount = events.resultLines.size + lastProgressAt = now + } + + val idleMs = now - lastProgressAt + if (idleMs >= progressTimeoutMs) { + return CompletionWait( + events, + "Headless run made no progress for ${progressTimeoutMs}ms " + + "after ${lastResultCount} result(s)" + ) + } + if (now >= overallDeadline) { + return CompletionWait( + events, + "Headless run did not emit DONE within ${maxRunMs}ms" + ) + } + + SystemClock.sleep( + minOf( + LOGCAT_POLL_INTERVAL_MS, + overallDeadline - now, + progressTimeoutMs - idleMs + ) + ) + events = readEvents(device, targetPackage) } - runCatching { pfd.close() } + } + + private fun awaitEvents( + device: UiDevice, + targetPackage: String, + timeoutMs: Long, + finished: (HeadlessLogcatEvents) -> Boolean + ): HeadlessLogcatEvents { + val deadline = SystemClock.elapsedRealtime() + timeoutMs + var events: HeadlessLogcatEvents + do { + events = readEvents(device, targetPackage) + if (finished(events)) return events + + val remainingMs = deadline - SystemClock.elapsedRealtime() + if (remainingMs > 0) { + SystemClock.sleep(minOf(LOGCAT_POLL_INTERVAL_MS, remainingMs)) + } + } while (SystemClock.elapsedRealtime() < deadline) + + // One last finite dump closes the race where DONE arrives at the deadline. + return readEvents(device, targetPackage) + } + + private fun readEvents( + device: UiDevice, + targetPackage: String + ): HeadlessLogcatEvents { + val events = HeadlessLogcatParser.parse( + device.executeShellCommand(LOGCAT_SNAPSHOT_COMMAND), + targetPackage + ) + events.resultLines.forEach { line -> runCatching { parseResult(line) } } + events.crashHint?.let { crashHint = it } + return events } private fun parseResult(line: String) { diff --git a/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/ReactMobileSyncTest.kt b/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/ReactMobileSyncTest.kt index e3682c12..629f0e42 100644 --- a/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/ReactMobileSyncTest.kt +++ b/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/ReactMobileSyncTest.kt @@ -30,8 +30,6 @@ package com.salesforce.androidsdk.reactnative import org.junit.Test class ReactMobileSyncTest : BaseReactNativeTest() { - override val testTimeoutMs: Long = 180_000 - @Test fun testSyncDown() = runTest("testSyncDown") @Test fun testSyncUp() = runTest("testSyncUp") @Test fun testReSync() = runTest("testReSync") diff --git a/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/ReactNetTest.kt b/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/ReactNetTest.kt index f392eaa1..e8240abf 100644 --- a/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/ReactNetTest.kt +++ b/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/ReactNetTest.kt @@ -30,8 +30,6 @@ package com.salesforce.androidsdk.reactnative import org.junit.Test class ReactNetTest : BaseReactNativeTest() { - override val testTimeoutMs: Long = 90_000 - @Test fun testGetApiVersion() = runTest("testGetApiVersion") @Test fun testVersions() = runTest("testVersions") @Test fun testResources() = runTest("testResources") diff --git a/androidTests/android/app/src/main/java/com/salesforce/androidsdk/reactnative/HeadlessLogcatParser.kt b/androidTests/android/app/src/main/java/com/salesforce/androidsdk/reactnative/HeadlessLogcatParser.kt new file mode 100644 index 00000000..ce61620a --- /dev/null +++ b/androidTests/android/app/src/main/java/com/salesforce/androidsdk/reactnative/HeadlessLogcatParser.kt @@ -0,0 +1,73 @@ +/* + * Copyright (c) 2026-present, salesforce.com, inc. + * All rights reserved. + * Redistribution and use of this software in source and binary forms, with or + * without modification, are permitted provided that the following conditions + * are met: + * - Redistributions of source code must retain the above copyright notice, this + * list of conditions and the following disclaimer. + * - Redistributions in binary form must reproduce the above copyright notice, + * this list of conditions and the following disclaimer in the documentation + * and/or other materials provided with the distribution. + * - Neither the name of salesforce.com, inc. nor the names of its contributors + * may be used to endorse or promote products derived from this software without + * specific prior written permission. + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" + * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + * POSSIBILITY OF SUCH DAMAGE. + */ +package com.salesforce.androidsdk.reactnative + +data class HeadlessLogcatEvents( + val began: Boolean, + val done: Boolean, + val resultLines: List, + val crashHint: String? +) + +object HeadlessLogcatParser { + + private const val BEGIN_PREFIX = "SFTESTBEGIN::" + private const val RESULT_PREFIX = "SFTESTRESULT::" + private const val DONE_PREFIX = "SFTESTDONE::" + + fun parse(snapshot: String, targetPackage: String): HeadlessLogcatEvents { + var began = false + var done = false + val resultLines = mutableListOf() + var crashHint: String? = null + val fatalBuffer = StringBuilder() + var fatalLinesLeft = 0 + + snapshot.lineSequence().forEach { line -> + when { + line.contains(BEGIN_PREFIX) -> began = true + line.contains(RESULT_PREFIX) -> resultLines += line + line.contains(DONE_PREFIX) -> done = true + line.contains("FATAL EXCEPTION") -> { + fatalBuffer.setLength(0) + fatalBuffer.append(line).append('\n') + fatalLinesLeft = 25 + } + fatalLinesLeft > 0 -> { + fatalBuffer.append(line).append('\n') + fatalLinesLeft-- + if (line.contains("Process:") && line.contains(targetPackage)) { + crashHint = fatalBuffer.toString().take(1500) + fatalLinesLeft = 0 + } + } + } + } + + return HeadlessLogcatEvents(began, done, resultLines, crashHint) + } +} diff --git a/androidTests/android/app/src/main/java/com/salesforce/androidsdk/reactnative/util/SalesforceReactTestApp.kt b/androidTests/android/app/src/main/java/com/salesforce/androidsdk/reactnative/util/SalesforceReactTestApp.kt index b32ffb0f..d4d68586 100644 --- a/androidTests/android/app/src/main/java/com/salesforce/androidsdk/reactnative/util/SalesforceReactTestApp.kt +++ b/androidTests/android/app/src/main/java/com/salesforce/androidsdk/reactnative/util/SalesforceReactTestApp.kt @@ -63,5 +63,11 @@ class SalesforceReactTestApp : Application(), ReactApplication { SoLoader.init(this, OpenSourceMergedSoMapping) load() SalesforceReactSDKManager.initReactNative(applicationContext, MainActivity::class.java) + // This debuggable test app removes POST_NOTIFICATIONS so Firebase runs do + // not require UI interaction. Disable SDK developer support as well: its + // foreground notification otherwise requests the removed permission on + // every activity resume, creating a permission-activity resume loop on + // API 33+ that repeatedly pauses React Native and can stall the test run. + SalesforceReactSDKManager.getInstance().setIsDevSupportEnabled(false) } } diff --git a/test/HeadlessTestApp.js b/test/HeadlessTestApp.js index d6121c8b..ac423ecd 100644 --- a/test/HeadlessTestApp.js +++ b/test/HeadlessTestApp.js @@ -56,16 +56,9 @@ import './net.test'; import './smartstore.test'; import './mobilesync.test'; -// Per-suite hard caps (ms). testRunner has no internal timeout, so each test MUST -// be bounded here or a single hung test would stall the whole run. -const SUITE_TIMEOUTS = { - Harness: 30000, - OAuth: 90000, - Net: 120000, - SmartStore: 90000, - MobileSync: 240000, -}; -const DEFAULT_TIMEOUT = 60000; +// Every test gets the same hard cap. These tests normally complete in seconds; +// letting one test run longer only hides a stalled bridge or network callback. +const TEST_TIMEOUT = 30000; // Module-level guard so a StrictMode double-mount / remount runs the suite once. let started = false; @@ -76,7 +69,7 @@ function emit(line) { } async function runOne(suiteName, testName) { - const cap = SUITE_TIMEOUTS[suiteName] || DEFAULT_TIMEOUT; + const cap = TEST_TIMEOUT; lastUnhandledRejection = null; // IMPORTANT: the timer MUST be cleared once the race settles. Promise.race does // not cancel the loser, so a timer left running after the test wins would fire From cd78b66c2d54c12d29a1592289521330ff3cfb00 Mon Sep 17 00:00:00 2001 From: Brandon Page Date: Thu, 24 Sep 2026 17:11:57 -0700 Subject: [PATCH 2/2] Address Android test review feedback --- androidTests/README.md | 11 +- .../reactnative/BaseReactNativeTest.kt | 19 +- docs/android-tests/PREPAREANDROID_DETAILED.md | 6 +- docs/android-tests/README.md | 562 ++++++------------ test/HeadlessTestApp.js | 8 +- 5 files changed, 215 insertions(+), 391 deletions(-) diff --git a/androidTests/README.md b/androidTests/README.md index 8d37cf3e..6b881df3 100644 --- a/androidTests/README.md +++ b/androidTests/README.md @@ -37,11 +37,12 @@ class ReactNetTest : BaseReactNativeTest() { ## Timeouts Every headless test has a 30-second JavaScript timeout. The Android collector -independently fails after 30 seconds without a new result, even if a blocking -native call prevents the JavaScript timeout from running. Its 45-minute overall -ceiling remains as a final guard for a suite that continues to make progress. -The collector values can be overridden with the `progressTimeoutMs` and -`maxRunMs` instrumentation arguments. +independently fails after 35 seconds without a new result, including a final +logcat snapshot at the timeout boundary. This provides polling and scheduling +slack while still catching a blocking native call that suppresses the JavaScript +timer. Its 45-minute overall ceiling remains as a final guard for a suite that +continues to make progress. The collector values can be overridden with the +`progressTimeoutMs` and `maxRunMs` instrumentation arguments. ## How It Works diff --git a/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/BaseReactNativeTest.kt b/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/BaseReactNativeTest.kt index 582bbd2f..46b1f9b6 100644 --- a/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/BaseReactNativeTest.kt +++ b/androidTests/android/app/src/androidTest/java/com/salesforce/androidsdk/reactnative/BaseReactNativeTest.kt @@ -75,9 +75,10 @@ object HeadlessResults { // App must launch and mount HeadlessTestApp (emit BEGIN) within this, else fail // fast with a real cause instead of blocking the whole run on a silent no-mount. private const val DEFAULT_BEGIN_TIMEOUT_MS = 3L * 60 * 1000 - // Match HeadlessTestApp's per-test cap, but enforce it outside the JS event - // loop so a blocking native call cannot suppress the JavaScript timer. - private const val DEFAULT_PROGRESS_TIMEOUT_MS = 30_000L + // HeadlessTestApp caps each test at 30 seconds. Enforce a slightly longer + // watchdog outside the JS event loop so polling and scheduling at the timeout + // boundary cannot discard the JavaScript timeout result. + private const val DEFAULT_PROGRESS_TIMEOUT_MS = 35_000L private const val LOGCAT_POLL_INTERVAL_MS = 1_000L private const val LOGCAT_SNAPSHOT_COMMAND = "logcat -d -v raw -s ReactNativeJS:I AndroidRuntime:E" @@ -198,8 +199,18 @@ object HeadlessResults { val idleMs = now - lastProgressAt if (idleMs >= progressTimeoutMs) { + // Close the boundary race where the JavaScript timeout/result is + // emitted just after the snapshot used to calculate idleMs. + val finalEvents = readEvents(device, targetPackage) + if (finalEvents.done) return CompletionWait(finalEvents, "") + if (finalEvents.resultLines.size > lastResultCount) { + events = finalEvents + lastResultCount = finalEvents.resultLines.size + lastProgressAt = SystemClock.elapsedRealtime() + continue + } return CompletionWait( - events, + finalEvents, "Headless run made no progress for ${progressTimeoutMs}ms " + "after ${lastResultCount} result(s)" ) diff --git a/docs/android-tests/PREPAREANDROID_DETAILED.md b/docs/android-tests/PREPAREANDROID_DETAILED.md index c546f26c..c0d97314 100644 --- a/docs/android-tests/PREPAREANDROID_DETAILED.md +++ b/docs/android-tests/PREPAREANDROID_DETAILED.md @@ -39,7 +39,8 @@ It then patches the SDK's build files to work as a composite build: Copies `shared/test/test_credentials.json` to `android/app/src/main/assets/test_credentials.json` where the Android `TestCredentials.java` class reads it at runtime. -If the file is not found, a warning is printed and an empty `{}` placeholder is written (tests will fail at runtime with a clear error). +If the file is not found, preparation exits with an error. This prevents a +credential problem from being reported later as a headless test timeout. **Note:** The Gradle `copyTestCredentials` task in `app/build.gradle.kts` also copies this file at build time as a safety net, so credentials are picked up even if `prepareandroid.js` ran before the file was created. @@ -82,7 +83,8 @@ The `app/build.gradle.kts` includes a `copyTestCredentials` task that runs befor | Issue | Cause | Fix | |-------|-------|-----| -| `Failed to read test_credentials.json` at runtime | Missing or empty credentials file | Place valid credentials at `shared/test/test_credentials.json` | +| Preparation reports that `test_credentials.json` is missing | The shared credentials file was not created | Copy `shared/test/test_credentials.json.sample` to `shared/test/test_credentials.json` and populate it | +| Authentication fails at runtime | Credentials are invalid or incomplete | Verify the values in `shared/test/test_credentials.json`, then rebuild so the Gradle copy task refreshes the app asset | | `yarn install` fails | Network issue or incompatible Node version | Ensure Node 22+, check network | | Gradle build fails with dependency errors | Stale `mobile_sdk/` clone | Delete `mobile_sdk/` and re-run | | Bundle fails with Metro error | Incompatible babel config | Delete `node_modules/` and re-run | diff --git a/docs/android-tests/README.md b/docs/android-tests/README.md index ba029222..49addeac 100644 --- a/docs/android-tests/README.md +++ b/docs/android-tests/README.md @@ -1,472 +1,282 @@ # Android Test App Documentation -This document describes the Android test application structure and how to run tests for the Salesforce Mobile SDK React Native bridge. +The Android test app runs the shared React Native bridge tests through an +AndroidX instrumentation suite. Android uses a headless JavaScript driver and +logcat result protocol; it does not navigate the interactive test UI with +UIAutomator. -## Table of Contents +## Architecture -- [Overview](#overview) -- [Test Architecture](#test-architecture) -- [Directory Structure](#directory-structure) -- [Setup and Running Tests](#setup-and-running-tests) -- [Writing Tests](#writing-tests) -- [Test Utilities](#test-utilities) -- [Troubleshooting](#troubleshooting) - -## Overview +```mermaid +flowchart TD + A[AndroidJUnitRunner] --> B[BaseReactNativeTest] + B --> C[TestAuthenticationActivity] + C --> D[MainActivity] + D --> E[HeadlessTestApp] + E --> F[testRunner] + F --> G[Shared JavaScript suites] + G --> H[React Native bridge modules] + H --> I[Salesforce Android SDK] + E --> J[SFTEST logcat sentinels] + J --> K[HeadlessLogcatParser] + K --> B +``` -The Android test app is a React Native application that runs JavaScript tests through the AndroidX Test framework (instrumentation tests). This approach allows testing the complete bridge from JavaScript → React Native → Android Native → Android SDK. +The first JUnit method that requests a result launches the app and authenticates +from `test_credentials.json`. `HeadlessTestApp` then runs every registered +JavaScript test sequentially. The instrumentation collector reads the complete +result set once and caches it; each JUnit method asserts the cached result with +the matching JavaScript test name. -### Key Components +Android intentionally differs from iOS here. `androidTests/index.js` mounts +`HeadlessTestApp`, while the iOS test app keeps the interactive `TestApp` used by +XCUITest. -1. **JavaScript Test Suite** (`test/`) - Shared test files for all platforms -2. **Android Test App** (`androidTests/`) - React Native app that loads tests -3. **AndroidX Test Suite** (`androidTests/android/app/src/androidTest/`) - Instrumentation test runner -4. **Test Harness** (`src/react.force.test.tsx`) - Bridge between JS and native tests +## Result Protocol -## Test Architecture +The JavaScript driver writes one physical logcat line for each event: -```mermaid -graph TB - subgraph "AndroidX Test (Kotlin/JUnit)" - A[ReactTestCase] - B[ReactOauthTests] - C[ReactNetTests] - D[ReactSmartStoreTests] - E[ReactMobileSyncTests] - end - - subgraph "React Native Runtime" - F[SalesforceReactActivity - bridgeless mode] - end - - subgraph "JavaScript Test Suite" - G[test/alltests.js] - H[test/oauth.test.js] - I[test/net.test.js] - J[test/smartstore.test.js] - K[test/mobilesync.test.js] - end - - subgraph "SDK Bridge Modules (TurboModules)" - L[SFOauthReactBridge] - M[SFNetReactBridge] - N[SFSmartStoreReactBridge] - O[SFMobileSyncReactBridge] - end - - subgraph "Android SDK" - P[SalesforceSDK] - Q[SmartStore] - R[MobileSync] - end - - B --> F - C --> F - D --> F - E --> F - - F --> H - F --> I - F --> J - F --> K - - H --> L - I --> M - J --> N - K --> O - - L --> P - M --> P - N --> Q - O --> R +```text +SFTESTBEGIN:: +SFTESTRESULT::{"s":"","n":"","ok":true|false,"e":""} +SFTESTDONE::{"total":35,"passed":35,"failed":0} ``` -## Directory Structure +`HeadlessResults` clears logcat before launch and polls finite `logcat -d` +snapshots. Finite snapshots are important on Android 12L, where a long-lived +logcat pipe can retain the final buffered `SFTESTDONE` line. -``` -androidTests/ -├── android/ # Android native project -│ ├── app/ -│ │ ├── build.gradle.kts # Gradle build config (includes copyTestCredentials task) -│ │ └── src/ -│ │ ├── main/ -│ │ │ ├── AndroidManifest.xml -│ │ │ ├── assets/ -│ │ │ │ └── test_credentials.json # Copied at build time (gitignored) -│ │ │ ├── java/.../ -│ │ │ │ └── MainApplication.kt # App entry point -│ │ │ └── res/ -│ │ └── androidTest/ -│ │ └── java/.../ -│ │ ├── ReactTestCase.kt # Base test class -│ │ ├── ReactOauthTests.kt # OAuth tests -│ │ ├── ReactNetTests.kt # REST API tests -│ │ ├── ReactSmartStoreTests.kt # SmartStore tests -│ │ └── ReactMobileSyncTests.kt # MobileSync tests -│ ├── build.gradle # Root Gradle config -│ ├── settings.gradle # Project settings -│ ├── gradle.properties -│ └── gradlew # Gradle wrapper -│ -├── mobile_sdk/ # Cloned Android SDK (from updatesdk.js) -│ └── SalesforceMobileSDK-Android/ -│ -├── index.js # React Native entry point -├── package.json # npm dependencies -├── metro.config.js # Metro bundler config -├── babel.config.js # Babel config -├── prepareandroid.js # Setup script -├── updatebundle.js # Bundle update script -├── updatesdk.js # SDK update script -└── create_test_credentials_from_env.js # CI credential generation -``` +If the app crashes, the parser also captures an `AndroidRuntime` fatal exception +for the test package and adds it to the failure message when available. -## Setup and Running Tests +## Timeouts -### Prerequisites +- Every JavaScript test has a 30-second timeout in `HeadlessTestApp.js`. +- The instrumentation collector allows 35 seconds without a new result. The + extra five seconds covers polling and scheduling at the JavaScript timeout + boundary. It takes one final logcat snapshot before declaring no progress. +- A 45-minute overall ceiling remains as a final guard while results continue to + arrive. Firebase Test Lab has its own 60-minute execution ceiling. -- **Android Studio**: Latest stable version -- **Java**: JDK 17+ -- **Node.js**: 22 or later -- **Android SDK**: API 31+ (compileSdk 36) -- **Emulator or device**: API 31+ for running tests -- **Salesforce Org**: For authentication tests +The collector thresholds can be overridden with the `progressTimeoutMs`, +`beginTimeoutMs`, and `maxRunMs` instrumentation arguments. -### Step 1: Setup Test Workspace +## Key Files -From the `androidTests` directory: +```text +test/ +├── HeadlessTestApp.js # Android-only headless driver and per-test timeout +├── testRunner.js # Suite registration and test lifecycle +├── harness.test.js +├── oauth.test.js +├── net.test.js +├── smartstore.test.js +└── mobilesync.test.js -```bash -cd androidTests -./prepareandroid.js +androidTests/ +├── index.js # Mounts HeadlessTestApp for Android +├── prepareandroid.js # Installs dependencies, SDK, credentials, bundle +├── updatebundle.js # Creates index.android.bundle +├── updatesdk.js # Installs SalesforceMobileSDK-Android +└── android/app/src/ + ├── main/java/com/salesforce/androidsdk/reactnative/ + │ ├── HeadlessLogcatParser.kt + │ └── util/SalesforceReactTestApp.kt + └── androidTest/java/com/salesforce/androidsdk/reactnative/ + ├── BaseReactNativeTest.kt + ├── ReactHarnessTest.kt + ├── ReactOAuthTest.kt + ├── ReactNetTest.kt + ├── ReactSmartStoreTest.kt + └── ReactMobileSyncTest.kt ``` -**What it does** (4 phases): -1. **Phase 1**: Installs npm dependencies (React Native, SDK, build tools) -2. **Phase 2**: Clones Android SDK from configured repository branch (`updatesdk.js`) -3. **Phase 3**: Copies `shared/test/test_credentials.json` into the app assets directory -4. **Phase 4**: Bundles JavaScript tests into `index.android.bundle` +The debuggable test app disables Salesforce SDK developer support. Its manifest +removes `POST_NOTIFICATIONS`, so leaving developer support enabled would request +an undeclared permission on every activity resume on API 33 and later. -**For detailed explanation of each phase**, see [PREPAREANDROID_DETAILED.md](./PREPAREANDROID_DETAILED.md). +## Setup -**Key files created**: -- `node_modules/` - npm dependencies -- `mobile_sdk/SalesforceMobileSDK-Android/` - Cloned Android SDK -- `android/app/src/main/assets/test_credentials.json` - Test credentials (copied from shared) -- `android/app/src/main/assets/index.android.bundle` - Bundled JavaScript tests +### Prerequisites -### Step 2: Configure Test Credentials +- Node.js 22 or later +- JDK 17 or later +- Android SDK with API 31 or later +- A connected emulator/device for local execution, or Firebase Test Lab access +- Salesforce test-org credentials -Both iOS and Android tests share a single credentials source at `shared/test/test_credentials.json` (relative to the repo root). Copy the sample template and fill in your values: +### Credentials + +Copy the shared sample and populate it: ```bash cp shared/test/test_credentials.json.sample shared/test/test_credentials.json ``` -See the [sample template](../../shared/test/test_credentials.json.sample) for the expected fields. +Do not commit `test_credentials.json`. In CI, +`create_test_credentials_from_env.js` writes the `TEST_CREDENTIALS` secret to +the shared location. -**Note**: The `prepareandroid.js` script copies this file into `android/app/src/main/assets/test_credentials.json`. Additionally, the Gradle `copyTestCredentials` task re-copies it before each build, so credentials stay up to date even if you edit the source file later. +### Prepare the Test App -**Alternative** (using environment variables in CI): +From the repository root: ```bash cd androidTests -node create_test_credentials_from_env.js +./prepareandroid.js ``` -### Step 3: Run Tests +The script: -#### Via Gradle (command line) +1. Reinstalls the Android test app's JavaScript dependencies. +2. Installs the configured Salesforce Android SDK dependency. +3. Copies `shared/test/test_credentials.json` into the app assets. Preparation + fails immediately if the credential file is absent. +4. Bundles the JavaScript tests into + `android/app/src/main/assets/index.android.bundle`. -Ensure an emulator is running or a device is connected: +See [PREPAREANDROID_DETAILED.md](./PREPAREANDROID_DETAILED.md) for preparation +details. -```bash -cd androidTests/android -./gradlew :app:connectedDebugAndroidTest -``` +## Running Tests -#### Via Android Studio - -1. Open the `androidTests/android` project in Android Studio -2. Wait for Gradle sync to complete -3. Navigate to `app/src/androidTest/java/` -4. Right-click on the test class you want to run, then "Run" - -#### Via Firebase Test Lab - -For CI environments, tests can be run on Firebase Test Lab: +### Connected Device or Emulator ```bash cd androidTests/android - -# Build the app APK and test APK -./gradlew :app:assembleDebug :app:assembleDebugAndroidTest - -# Run on Firebase Test Lab (requires gcloud CLI configured) -gcloud firebase test android run \ - --type instrumentation \ - --app app/build/outputs/apk/debug/app-debug.apk \ - --test app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk \ - --device model=Pixel2,version=30 +./gradlew :app:connectedDebugAndroidTest ``` -## The `copyTestCredentials` Gradle Task +Running one JUnit method still starts the shared headless suite once; the selected +method then asserts only its named result. -The `build.gradle.kts` for the test app includes a custom Gradle task that ensures test credentials are always available at build time: +### Android Studio -```kotlin -// Copy test_credentials.json from shared/test/ into assets before each build -tasks.register("copyTestCredentials") { - from("${rootProject.projectDir}/../../shared/test/test_credentials.json") - into("src/main/assets") - duplicatesStrategy = DuplicatesStrategy.INCLUDE -} -tasks.matching { it.name.startsWith("merge") && it.name.contains("Assets") }.configureEach { - dependsOn("copyTestCredentials") -} -``` +1. Open `androidTests/android`. +2. Wait for Gradle sync. +3. Run a class or method under `app/src/androidTest/java`. -This task: -- Runs automatically before any asset-merging step (debug and androidTest builds) -- Copies from the canonical location `shared/test/test_credentials.json` -- Silently succeeds even if `prepareandroid.js` has not been run (the file must exist though) +Metro is not required because `prepareandroid.js` produces the bundled test +application. -## The `prepareandroid.js` Script +### Firebase Test Lab -The setup script automates all steps needed to prepare the Android test environment. - -### Phase 1: Install npm Dependencies +Build both APKs: ```bash -rm -rf node_modules -rm -f yarn.lock -yarn install -``` - -Installs React Native, the SDK package (`react-native-force` via `file:../`), and build tools. - -### Phase 2: Clone Android SDK - -```bash -node ./updatesdk.js +cd androidTests/android +./gradlew :app:assembleDebug :app:assembleDebugAndroidTest ``` -Reads `sdkDependencies` from `package.json` and shallow-clones the Android SDK into `mobile_sdk/SalesforceMobileSDK-Android/`. The Gradle build uses a composite build to include SDK libraries from this clone. - -### Phase 3: Copy Test Credentials - -Copies `../shared/test/test_credentials.json` into `android/app/src/main/assets/`. If the source file does not exist, writes an empty JSON object and prints a warning. - -### Phase 4: Bundle JavaScript Tests +Then run the instrumentation APK, for example: ```bash -node ./updatebundle.js +gcloud --quiet beta firebase test android run \ + --project mobile-apps-firebase-test \ + --type instrumentation \ + --app app/build/outputs/apk/debug/app-debug.apk \ + --test app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk \ + --device model=MediumPhone.arm,version=34,locale=en,orientation=portrait \ + --timeout 60m \ + --no-auto-google-login \ + --no-performance-metrics \ + --no-record-video ``` -Runs Metro to create `android/app/src/main/assets/index.android.bundle` containing all JavaScript test code. +The reusable Android workflow tests API 36 for pull requests and APIs 31–37 for +full nightly runs. API 36 and later use the 16 KB page-size device model. ## Writing Tests -### JavaScript Test Structure +### JavaScript Suite -Tests are shared between iOS and Android. They live in `test/` at the repo root and use a lightweight custom assert module (`test/assert.js`) plus the `registerTest`/`testDone` harness. +Register a suite before its tests. Tests finish by calling `testDone()` with no +argument on success or with an error on failure. ```javascript -// test/oauth.test.js -import { assert } from './assert'; -import * as oauth from '../src/react.force.oauth'; -import { registerTest, testDone } from '../src/react.force.test'; - -testGetAuthCredentials = () => { - oauth.getAuthCredentials( - (creds) => { - assert.containsAllKeys( - creds, - ["accessToken","instanceUrl","loginUrl","orgId","refreshToken","userAgent","userId"], - 'Wrong keys in credentials' - ); - testDone(); - }, - (error) => { throw error; } - ); - return false; // not done (async) -}; - -registerTest(testGetAuthCredentials); -``` - -### Adding a New Test - -1. **Add JavaScript test function** in `test/.test.js` -2. **Register it** with `registerTest(testFunctionName);` -3. **Add it to the Android test class** (Kotlin parameterized test list) -4. **Run tests** (see [Setup and Running Tests](#setup-and-running-tests)) - -### Test Naming Convention +import { registerSuite, registerTest, testDone } from './testRunner'; -JavaScript test function names start with `test` followed by the name in camelCase. The Android test runner extracts the name (without `test` prefix) and uses it as the React Native component name to mount. +registerSuite('Example'); -Example mapping: -| JavaScript Function | Registered Component | Android Parameterized Entry | -|---------------------|---------------------|----------------------------| -| `testGetAuthCredentials` | `GetAuthCredentials` | `"GetAuthCredentials"` | -| `testRegisterSoup` | `RegisterSoup` | `"RegisterSoup"` | - -## Test Utilities - -### Test Credentials Loading - -At runtime, the test app loads `test_credentials.json` from the Android assets directory. This file is placed there by: -1. `prepareandroid.js` Phase 3 (initial setup) -2. The Gradle `copyTestCredentials` task (every subsequent build) +function testExample() { + someAsyncOperation( + () => testDone(), + (error) => testDone(error), + ); +} -### Test Harness (`react.force.test.tsx`) +registerTest(testExample); +``` -The `testDone()` function signals test completion to the native side via `NativeModules.SalesforceTestBridge.markTestCompleted()`. This is the Android-specific counterpart to iOS's `TestModule.markTestCompleted()`. +Suites may provide asynchronous setup and teardown functions: -## Test Categories +```javascript +registerSuite('Example', { + setUp: async () => resetState(), + tearDown: async () => resetState(), +}); +``` -### 1. OAuth Tests (`test/oauth.test.js`) -- `testGetAuthCredentials` - Get current user credentials +Avoid throwing inside asynchronous callbacks; pass the exception to +`testDone(error)` or reject a promise that the test chain handles. The headless +driver records unhandled Hermes promise rejections in timeout diagnostics. -### 2. Net Tests (`test/net.test.js`) -- `testGetApiVersion`, `testVersions`, `testResources` -- `testDescribeGlobal`, `testMetaData`, `testDescribe`, `testDescribeLayout` -- `testCreateRetrieve`, `testUpsertUpdateRetrieve`, `testCreateDelRetrieve` -- `testQuery`, `testSearch`, `testPublicApiCall` -- `testCollectionCreateRetrieve`, `testCollectionUpsertUpdateRetrieve`, `testCollectionCreateDeleteRetrieve` +### JUnit Mapping -### 3. SmartStore Tests (`test/smartstore.test.js`) -- `testGetDatabaseSize`, `testRegisterExistsRemoveExists` -- `testGetSoupIndexSpecs`, `testUpsertRetrieve` -- `testQuerySoup`, `testMoveCursor`, `testSmartQuerySoup` -- `testRemoveFromSoup`, `testClearSoup` -- `testGetRemoveStores`, `testGetRemoveGlobalStores` +Add a JUnit method with exactly the same JavaScript function name: -### 4. MobileSync Tests (`test/mobilesync.test.js`) -- `testSyncUp`, `testSyncDown`, `testReSync` -- `testCleanResyncGhosts`, `testGetSyncStatusDeleteSync` +```kotlin +class ReactExampleTest : BaseReactNativeTest() { + @Test fun testExample() = runTest("testExample") +} +``` -### 5. Harness Tests (`test/harness.test.js`) -- `testPassing`, `testAsyncPassing` +Tests registered with `excludeFromRunAll: true` are not executed by the Android +headless suite and should not have an Android JUnit mapping. ## Troubleshooting -### Tests Don't Run - -**Problem**: Instrumentation tests fail to start or timeout - -**Solutions**: -1. Ensure an emulator is running (`adb devices` should show a device) -2. Check that the JavaScript bundle was created: `ls android/app/src/main/assets/index.android.bundle` -3. Verify credentials exist: `ls android/app/src/main/assets/test_credentials.json` -4. Clean and rebuild: `cd android && ./gradlew clean :app:assembleDebug` - -### Authentication Failures +### No `SFTESTBEGIN` -**Problem**: OAuth tests fail with "Not authenticated" +Check that: -**Solutions**: -1. Verify `shared/test/test_credentials.json` is valid and populated -2. Check that the credentials were copied: `cat android/app/src/main/assets/test_credentials.json` -3. Ensure the Connected App allows the configured redirect URI -4. Check logcat for detailed error messages: `adb logcat | grep -i salesforce` +- `shared/test/test_credentials.json` exists and is populated. +- `prepareandroid.js` completed successfully. +- `android/app/src/main/assets/index.android.bundle` exists. +- The app did not crash during authentication or bundle loading. -### Build Errors +### No Progress or Missing `SFTESTDONE` -**Problem**: Gradle build fails +Search logcat for the sentinel sequence and fatal exceptions: -**Solutions**: ```bash -cd androidTests/android -./gradlew clean -./gradlew :app:assembleDebug --info -``` - -If SDK dependencies are missing: -```bash -cd androidTests -node updatesdk.js +adb logcat -d -v raw -s ReactNativeJS:I AndroidRuntime:E ``` -### Metro Bundler Issues +A no-progress failure reports how many results arrived before the 35-second +watchdog expired. A JavaScript test timeout should instead produce a failed +`SFTESTRESULT` and allow the suite to continue to `SFTESTDONE`. -**Problem**: JavaScript bundle is outdated or missing +### Stale JavaScript Bundle -**Solutions**: ```bash cd androidTests node updatebundle.js ``` -### Emulator Issues - -**Problem**: `connectedDebugAndroidTest` fails with "No connected devices" - -**Solutions**: -1. Start an emulator from Android Studio or command line -2. Verify connection: `adb devices` -3. For headless CI, create an emulator: - ```bash - sdkmanager "system-images;android-30;google_apis;x86_64" - avdmanager create avd -n test -k "system-images;android-30;google_apis;x86_64" - emulator -avd test -no-window & - adb wait-for-device - ``` - -## CI/CD Integration - -### GitHub Actions Example - -```yaml -name: Android Tests - -on: [push, pull_request] - -jobs: - test: - runs-on: ubuntu-latest - - steps: - - uses: actions/checkout@v3 - - - name: Setup Node - uses: actions/setup-node@v3 - with: - node-version: '22' - - - name: Setup Java - uses: actions/setup-java@v3 - with: - distribution: 'temurin' - java-version: '17' - - - name: Setup Test Credentials - env: - SFDC_TEST_CLIENT_ID: ${{ secrets.TEST_CLIENT_ID }} - SFDC_TEST_USERNAME: ${{ secrets.TEST_USERNAME }} - SFDC_TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }} - run: | - cd androidTests - node create_test_credentials_from_env.js - - - name: Prepare Android Tests - run: | - cd androidTests - ./prepareandroid.js - - - name: Start Emulator - uses: reactivecircus/android-emulator-runner@v2 - with: - api-level: 30 - script: | - cd androidTests/android - ./gradlew :app:connectedDebugAndroidTest +### Build Failure + +```bash +cd androidTests/android +./gradlew :app:assembleDebug :app:assembleDebugAndroidTest --info ``` +If the composite Android SDK dependency is absent, rerun `androidTests/updatesdk.js` +or the full `prepareandroid.js` setup. + ## Further Reading -- [JavaScript API Reference](../javascript/API_REFERENCE.md) - Complete API documentation -- [Architecture Guide](../ARCHITECTURE.md) - Overall architecture -- [iOS Test Documentation](../ios-tests/README.md) - iOS testing (mirrors this structure) -- [Main README](../../README.md) - Getting started guide +- [Preparation details](./PREPAREANDROID_DETAILED.md) +- [JavaScript API reference](../javascript/API_REFERENCE.md) +- [Repository architecture](../ARCHITECTURE.md) +- [iOS test documentation](../ios-tests/README.md) diff --git a/test/HeadlessTestApp.js b/test/HeadlessTestApp.js index ac423ecd..06ec16e8 100644 --- a/test/HeadlessTestApp.js +++ b/test/HeadlessTestApp.js @@ -36,10 +36,10 @@ * SFTESTRESULT::{"s":"","n":"","ok":true|false,"e":""} * SFTESTDONE::{"total":N,"passed":N,"failed":N} * - * The Kotlin harness (BaseReactNativeTest.kt) streams logcat (tag ReactNativeJS), - * parses these lines into per-test results, and asserts. No UIAutomator, no - * scrolling, no gesture guessing — which is what made the old harness flaky on - * Firebase Test Lab's slow ARM emulators. + * The Kotlin harness (BaseReactNativeTest.kt) reads finite logcat snapshots (tag + * ReactNativeJS), parses these lines into per-test results, and asserts. No + * UIAutomator, no scrolling, no gesture guessing — which is what made the old + * harness flaky on Firebase Test Lab's slow ARM emulators. * * This file is imported ONLY by androidTests/index.js, so it is never bundled for * iOS. Do NOT import it from iosTests/index.js.