From 3883bf50754c618e91ebb08675806c551b63884d Mon Sep 17 00:00:00 2001 From: frathe Date: Tue, 29 Sep 2026 20:58:44 +0200 Subject: [PATCH 01/63] Prepare Mac App Store distribution foundations --- ARCHITECTURE.md | 17 +- Makefile | 4 + PRIVACY.md | 9 +- THIRD-PARTY-NOTICES.md | 7 + ...apple-app-store-requirements-2026-09-29.md | 111 +++++++++ internal/distribution/distribution.go | 7 +- internal/distribution/distribution_apple.go | 10 + .../distribution/distribution_apple_test.go | 23 ++ internal/distribution/distribution_store.go | 5 +- .../distribution/distribution_store_test.go | 9 +- internal/distribution/distribution_test.go | 5 +- internal/similarity/assets_install.go | 2 +- internal/similarity/assets_package.go | 56 ++++- internal/similarity/assets_test.go | 69 +++++- internal/ui/autoupdate.go | 4 + internal/ui/autoupdate_test.go | 9 +- internal/ui/explorer/setup.go | 7 +- internal/ui/explorer/setup_test.go | 6 +- internal/ui/launchpolicy_test.go | 7 +- internal/ui/settingswin/settingswin.go | 7 +- internal/ui/settingswin/settingswin_test.go | 17 +- packaging/apple-app-store/README.md | 89 +++++++ plans/2026-09-29-apple-app-store.md | 234 ++++++++++++++++++ qodana.yaml | 1 + scripts/apple-store-preflight.sh | 73 ++++++ todos.md | 12 +- translations/de.json | 2 + translations/en.json | 2 + 28 files changed, 769 insertions(+), 35 deletions(-) create mode 100644 docs/apple-app-store-requirements-2026-09-29.md create mode 100644 internal/distribution/distribution_apple.go create mode 100644 internal/distribution/distribution_apple_test.go create mode 100644 packaging/apple-app-store/README.md create mode 100644 plans/2026-09-29-apple-app-store.md create mode 100644 scripts/apple-store-preflight.sh diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 99cecd55..1c87f53a 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -133,6 +133,11 @@ skip debug symbols. Store builds verify/load DLLs beside the executable, ignore model-cache runtime overrides, and download only model data. `assets_package.go` verifies architecture-specific archives for MSIX staging and extracts both DLLs and the upstream license, third-party notices and privacy document. +`StageMacRuntime` stages architecture-pinned macOS native code and upstream +notices only after archive and extracted-library verification, for later bundle +signing. Both Windows and macOS packaging verify in private staging before +copying allowlisted files into the package. Signed Mac runtime validation is +still pending and must not compare signed Mach-O bytes with upstream hashes. Asset availability is separate from analysis admission; analysis never starts a download. `offline.go` verifies actual TCP/UDP OS denial on macOS/Linux; `files.go` registers the driverless read-only @@ -290,6 +295,11 @@ reviewed WASM payload, build recipe and retained source license texts against `manifest.json`; it generates the AVIF section of `THIRD-PARTY-NOTICES.md` offline. Its README records the libyuv/WASI source-provenance limits. +`scripts/apple-store-preflight.sh`, reached through `make apple-store-preflight`, +checks full Xcode and caller-supplied Store signing prerequisites without changing +credentials or uploading an artifact. `packaging/apple-app-store/README.md` +records the experimental channel and outstanding signed qualification. + `packaging/tools.mk` owns reviewed CLI versions and multiarchitecture image digests consumed by Makefile and the release/Store workflows. `docs/packaging-inputs.md` describes provenance and native artifact validation. @@ -935,8 +945,11 @@ switching automatically. ### `internal/distribution` Compile-time distribution policy. `StoreManaged` is false for ordinary builds -and true only with the `microsoftstore` build tag. Production startup captures -it in `launch.Policy`; UI composition receives that explicit decision. +and true with either the `microsoftstore` or macOS-only `appleappstore` build +tag. `AppleAppStore` distinguishes the Apple channel for localized repair and +update explanations. Apple builds remain experimental until signed sandbox and +packaging qualification is recorded in the active Apple Store plan. Production +startup captures it in `launch.Policy`; UI composition receives that decision. ### `internal/wingesture` diff --git a/Makefile b/Makefile index aea28147..f4413968 100644 --- a/Makefile +++ b/Makefile @@ -66,6 +66,7 @@ COVERAGE_HTML := $(COVERAGE_DIR)/coverage.html .PHONY: verify-build --skip-local-tests .PHONY: generate-updater-notices check-updater-notices generate-avif-notices check-avif-notices .PHONY: fossa-findings +.PHONY: apple-store-preflight .PHONY: loc all: build @@ -102,6 +103,9 @@ loc: ## Show tracked source files and physical lines, split into Go production/t fossa-findings: ## Retrieve current FOSSA licensing findings (PR=61 or FOSSA_REVISION=; key in .env.local) go run ./scripts/fossafindings +apple-store-preflight: ## Check native Xcode and Mac App Store signing inputs (see packaging/apple-app-store/README.md) + bash scripts/apple-store-preflight.sh + generate-updater-notices: ## Regenerate updater notices from the reviewed source/license manifest go run ./scripts/updaternotices -write diff --git a/PRIVACY.md b/PRIVACY.md index cfc0f3a2..f530e472 100644 --- a/PRIVACY.md +++ b/PRIVACY.md @@ -1,6 +1,6 @@ # PicFetch privacy policy -Effective date: September 25, 2026 +Effective date: September 29, 2026 PicFetch is a free and open-source desktop image viewer. It has no accounts, advertising, analytics, or telemetry. The PicFetch developer does not collect, @@ -56,8 +56,8 @@ cancels that map's pending tile work; requests already sent cannot be recalled. ## Similarity Explorer downloads On first use, the Explorer explains its local analysis and offers to download -the required model data. Standalone builds also download the runtime; the -Microsoft Store version includes it, as described below. Downloads begin only +the required model data. Standalone builds also download the runtime; +Store-managed builds require an included runtime, as described below. Downloads begin only when the user chooses Download. PicFetch obtains the pinned SigLIP 2 model from Hugging Face and, in standalone builds, the ONNX Runtime package from Microsoft's ONNX Runtime releases on GitHub. These @@ -103,7 +103,8 @@ privacy statement; public posts can be read by other people. ## Updates -The Microsoft Store build uses Microsoft Store delivery and updates. In other +Store-managed builds use their app store for delivery and updates: Microsoft +Store on Windows and Mac App Store on macOS. In direct-download distributions, users can optionally enable PicFetch's update checker; when enabled, it contacts GitHub to check and download PicFetch releases. GitHub processes those requests under its own diff --git a/THIRD-PARTY-NOTICES.md b/THIRD-PARTY-NOTICES.md index 2d3617c7..a3021a74 100644 --- a/THIRD-PARTY-NOTICES.md +++ b/THIRD-PARTY-NOTICES.md @@ -5735,6 +5735,13 @@ Explorer setup, except Intel macOS, which obtains ONNX Runtime 1.23.2, Microsoft's last official Intel binary. Microsoft Store setup downloads only the model and its processor configuration; runtime DLL installation and updates belong to the Store package. +Mac App Store preparation stages the same architecture-pinned macOS releases: +ONNX Runtime 1.23.2 for Intel and 1.29.0 for Apple Silicon. Each staged runtime +retains the complete upstream `LICENSE`, `ThirdPartyNotices.txt`, and +`Privacy.md` in its versioned directory. These notices must accompany the native +library in the eventual Store bundle; PicFetch's source license does not replace +the runtime's component licenses. Signed Store packaging remains under qualification. + Intel macOS setup retains the complete upstream `LICENSE`, `ThirdPartyNotices.txt` and `Privacy.md` beside `onnxruntime-osx-x86_64-1.23.2/lib/libonnxruntime.1.23.2.dylib`. diff --git a/docs/apple-app-store-requirements-2026-09-29.md b/docs/apple-app-store-requirements-2026-09-29.md new file mode 100644 index 00000000..367b974d --- /dev/null +++ b/docs/apple-app-store-requirements-2026-09-29.md @@ -0,0 +1,111 @@ +# Mac App Store requirements: Apple-source research + +Research date: 2026-09-29. Scope: a native macOS app submitted to the Mac App Store. Sources are Apple documentation, App Store Connect help, and the developer agreement. This document does not assess PicFetch's implementation or certify submission readiness. Unchecked items require evidence from the final signed store artifact, runtime tests, or App Store Connect. Checklist scenarios below are proposed verification work, not claims that Apple prescribes those exact tests. + +## Distribution and runtime constraints + +[App Review Guidelines](https://developer.apple.com/app-store/review/guidelines/) §2.4.5 requires sandboxing; an Xcode-based, self-contained single-app installation; no installation in shared locations; no root escalation/setuid; consent for startup or surviving processes; no launch licensing/copy protection; and store-delivered updates. It prohibits downloading additional code or resources that add functionality or substantially change the reviewed app. All localizations belong in the bundle. §2.5.1 requires public APIs and current-OS compatibility. §2.5.2 also restricts downloaded executable functionality. §5.1.1 requires an accessible in-app privacy-policy link covering collection, uses, third parties, retention/deletion and consent withdrawal. + +Verification proposals: + +- [ ] Exercise all startup, quit, installation and update paths in the store build; record policy and process outcomes. +- [ ] Inventory downloadable assets and explain their reviewed purpose, distinguishing content from code and functionality-changing resources. +- [ ] Inventory APIs used by every shipped component, including native dependencies. +- [ ] Check all bundled locales and the privacy-policy entry point. + +The [developer agreement](https://developer.apple.com/support/terms/apple-developer-program-license-agreement/) §3.3.1 further limits macOS store apps to documented APIs supplied with the default macOS installation, Xcode/Mac SDK, or Swift Playground; it excludes deprecated technologies such as Java. An API's presence on a development machine is insufficient evidence that it is supported for store distribution. + +## App Sandbox, files and durable access + +[App Sandbox](https://developer.apple.com/documentation/security/app-sandbox) is mandatory for Mac App Store distribution. Apple's [entitlement reference](https://developer.apple.com/library/archive/documentation/Miscellaneous/Reference/EntitlementKeyReference/Chapters/EnablingAppSandbox.html) defines `com.apple.security.files.user-selected.read-write` for user-selected file read/write access. Request only capabilities justified by actual operations; read-only access cannot support modifying originals or writing outputs. + +[Accessing files from the macOS App Sandbox](https://developer.apple.com/documentation/security/accessing-files-from-the-macos-app-sandbox) explains that open/save panels grant access to selected URLs; selected directories include descendants. Protected or otherwise inaccessible items can still fail. Stored security-scoped bookmarks preserve access across relaunch. Resolve with security scope, refresh stale bookmark data, call `startAccessingSecurityScopedResource()` before use, and balance with `stopAccessingSecurityScopedResource()` after use. User-selected file entitlements do not authorize executing programs in arbitrary selected locations. + +[Bookmark entitlements](https://developer.apple.com/documentation/professional-video-applications/enabling-security-scoped-bookmark-and-url-access) distinguish app-scoped and document-scoped persistence. [The access API reference](https://developer.apple.com/documentation/foundation/nsurl/startaccessingsecurityscopedresource()) warns that leaked scope resources can prevent further file access until relaunch. + +Verification proposals: + +- [ ] Inspect final entitlements for sandbox, required user-selected access and bookmark scope. +- [ ] Test native selection, Finder/Dock opening and drag/drop; record which URL authority each entry path receives. +- [ ] On a fresh account, exercise nested folders, external volumes and protected locations; failures must remain understandable. +- [ ] Close and relaunch after saving a collection/session; verify authority comes from resolved bookmarks rather than persisted path strings. +- [ ] Test moved files/folders, stale bookmarks, missing volumes and rejected scope acquisition. +- [ ] Exercise export, metadata edits and trash operations under their actual granted scope; audit overlapping worker scope lifetimes and balanced releases. + +The last two scenarios require implementation evidence. This research establishes no blanket entitlement that grants access to every file referenced by a collection. + +## Helpers and native dependencies + +Apple's [helper embedding guide](https://developer.apple.com/documentation/xcode/embedding-a-helper-tool-in-a-sandboxed-app) expressly supports externally built command-line helpers. Its example embeds them in `Contents/MacOS`, signs them, and uses `com.apple.security.app-sandbox` plus `com.apple.security.inherit`. It warns against additional helper entitlements and incompatible `get-task-allow`. Hardened Runtime is recommended there, but explicitly not required for App Store distribution. + +The [inheritance entitlement reference](https://developer.apple.com/library/archive/documentation/Miscellaneous/Reference/EntitlementKeyReference/Chapters/EnablingAppSandbox.html) says `posix_spawn`/`NSTask` inheritance carries static parent rights, not later Powerbox grants. It documents passing data or a bookmark to authorize files selected after launch. Main applications must not enable `inherit`. Other launch mechanisms need separate evaluation. + +[Technical Note TN2206](https://developer.apple.com/library/archive/technotes/tn2206/) describes nested code placement and signing: sign nested components before enclosing bundles and place code in recognized bundle locations. A native library being bundled does not exempt its code or dependencies from signing and API requirements. + +Verification proposals: + +- [ ] Enumerate every executable, dynamic library, framework and executable runtime in the submitted bundle; record architecture, minimum OS, provenance and dependent-library paths. +- [ ] Confirm dependencies resolve to bundled components or supported OS libraries on a clean Mac without developer tooling. +- [ ] Inspect signatures/entitlements for app and each helper independently; verify nested code and outer bundle after packaging. +- [ ] Test each child-process route with a newly selected external file, cancellation and app quit; demonstrate the authority passed to the child. +- [ ] Establish whether same-binary worker modes require distinct signed embedded helpers: the standard helper recipe alone does not prove that re-executing the main app is compatible. + +## Signing, package and submission + +[Certificates overview](https://developer.apple.com/help/account/certificates/certificates-overview) distinguishes store app/installer certificates from Developer ID certificates. Apple's helper guide shows Apple Distribution for the app and tool; certificate selection must match the chosen distribution workflow. [Provisioning profiles](https://developer.apple.com/help/account/provisioning-profiles/create-an-app-store-provisioning-profile) require an app record with an explicit App ID and a Mac App Store Connect distribution profile; Xcode can manage profiles automatically. + +[Packaging Mac software](https://developer.apple.com/documentation/xcode/packaging-mac-software-for-distribution) specifies a signed `.pkg` for store submission and Mac Installer Distribution signing, whose identity is named `3rd Party Mac Developer Installer`. Upload using Transporter or Apple's documented tooling. [Upload builds](https://developer.apple.com/help/app-store-connect/manage-builds/upload-builds) explains that bundle ID/version associate the uploaded artifact with its record and build string identifies the build. [Notarization](https://developer.apple.com/documentation/security/notarizing-macos-software-before-distribution) is a separate direct-distribution process; store submission includes equivalent checks. + +Verification proposals: + +- [ ] Confirm valid developer membership, app record, explicit identifier, store profile and certificates; archive their identifiers without secrets. +- [ ] Inspect the package's extracted app, nested signatures, profile and entitlements; validate the installer signature and ensure no post-sign mutations. +- [ ] Record Transporter validation/upload logs, processing results, artifact checksum, version and build. +- [ ] Install the actual store/TestFlight distribution and exercise fresh installation, upgrade and another user account. + +[Upcoming requirements](https://developer.apple.com/news/upcoming-requirements/) requires removing `com.apple.quarantine` from all macOS app files before upload. It also requires updated age-rating answers and verified EU trader status where applicable. Its April 2026 Xcode 26 SDK rule lists iOS, iPadOS, tvOS, visionOS and watchOS, not macOS. The upload-help page currently has inconsistent broad Xcode-14 guidance and an older macOS table; do not infer a precise macOS minimum from the mobile-platform rule. Recheck live requirements and validate with the current release tooling before submission. + +- [ ] Record Xcode/SDK/deployment-target versions and successful current upload validation. +- [ ] Check the final bundle for quarantine attributes, complete age-rating answers, and resolve EU distribution/trader status. + +## Privacy manifests and labels + +[Privacy manifest files](https://developer.apple.com/documentation/bundleresources/privacy-manifest-files) explicitly says data-collection information applies on **all platforms**. Its required-reason API information applies to **iOS, iPadOS, tvOS, visionOS and watchOS**; macOS is absent. Do not import the iOS required-reason API obligation into a native macOS target solely because the same API exists there. + +[Third-party SDK requirements](https://developer.apple.com/support/third-party-SDK-requirements/) requires manifests for listed SDKs in new apps, or updates adding listed SDKs, including repackaged SDKs and any version. Binary dependencies also require signatures. Its wording is broadly App Store based, so this research does not establish a macOS exemption for that separate requirement. Check the complete native/transitive closure against the live list and retain applicable vendor manifests/signatures. Unlisted dependencies still need accurate privacy assessment. + +[App privacy details](https://developer.apple.com/app-store/app-privacy-details/) applies across Apple platforms. It defines collection in terms of off-device transmission retained beyond servicing a real-time request; processing confined to the device is not collected for these labels. Third-party practices count. [Manage app privacy](https://developer.apple.com/help/app-store-connect/manage-app-information/manage-app-privacy) requires accurate App Store Connect answers and a privacy-policy URL for macOS, including an explicit no-collection answer when supported by evidence. + +Verification proposals: + +- [ ] Inventory app and dependency network traffic, payloads, recipients, server retention, logs and tracking; include optional paths and third-party tile/image services. +- [ ] Match manifests, published privacy policy and store answers to that inventory; do not equate local photo processing with collection. +- [ ] Check listed/repackaged SDKs and preserve required manifests and binary signatures in the actual bundle. +- [ ] Keep a native-macOS applicability record; treat upload validator feedback as evidence requiring investigation rather than assuming mobile API rules. + +## Encryption export determination + +[Export compliance overview](https://developer.apple.com/help/app-store-connect/manage-app-information/overview-of-export-compliance) requires a determination for apps incorporating or accessing encryption, including OS cryptography. [Encryption documentation](https://developer.apple.com/help/app-store-connect/reference/app-information/export-compliance-documentation-for-encryption) distinguishes OS-only encryption (no documentation), standard algorithms supplied outside the OS (French declaration when distributing in France), and proprietary algorithms (CCATS and applicable French declaration). [The submission workflow](https://developer.apple.com/help/app-store-connect/manage-app-information/determine-and-upload-app-encryption-documentation) uses a questionnaire and allows a supported exemption to be recorded in `Info.plist`. + +Verification proposals: + +- [ ] Inventory TLS, signature verification, cryptographic libraries and other crypto implementations throughout the shipped dependency closure. +- [ ] Record the implementation provider and questionnaire rationale; obtain any required documents before review. +- [ ] Verify the final property-list declaration matches the determination. A bundled TLS implementation cannot be classified as Apple-OS-only merely because it makes HTTPS requests. + +This is an Apple submission summary, not a completed legal classification for the app; the implementation and intended countries remain unassessed. + +## Listing and review metadata + +[App information](https://developer.apple.com/help/app-store-connect/reference/app-information/app-information/) specifies identity, name, categories and privacy link. [Platform version information](https://developer.apple.com/help/app-store-connect/reference/app-information/platform-version-information) requires screenshots, description, keywords, support URL and copyright. The support URL must reach actual contact details. Review information includes a reachable contact, useful testing notes and a non-expiring demo login when login is required. Subsequent releases need release notes. [Prepare for distribution](https://help.apple.com/xcode/mac/current/en.lproj/dev91fe7130a.html) also calls for macOS category and copyright in the bundle's property list. + +Verification proposals: + +- [ ] Prepare honest screenshots, localized descriptions/keywords, categories, icon, copyright, rating, availability and price. +- [ ] Open support/privacy URLs as an unauthenticated visitor; verify contact and policy content. +- [ ] Provide review notes covering file authorization, helper/dependency behavior and optional networking; include test assets and accounts as applicable. +- [ ] Confirm listing capabilities against the actual store artifact on every advertised architecture and minimum supported OS. + +## Remaining uncertainty and evidence boundaries + +Apple's current platform-specific privacy page is more precise than generic announcements about approved API reasons. SDK-list requirements and upload-tool version guidance still need live artifact validation. Archived entitlement and signing references are identified above and supported by the current helper guide where available. No sandbox tests, dependency inventory, signing validation, developer-account checks or submission were performed for this research. A compliant design and successful upload are useful evidence, but final App Review approval remains Apple's decision. diff --git a/internal/distribution/distribution.go b/internal/distribution/distribution.go index bf8572d6..674164c8 100644 --- a/internal/distribution/distribution.go +++ b/internal/distribution/distribution.go @@ -1,8 +1,11 @@ -//go:build !microsoftstore +//go:build !microsoftstore && !appleappstore // Package distribution exposes immutable build-channel facts. package distribution -// StoreManaged reports whether Microsoft Store owns delivery and updates for +// StoreManaged reports whether an app store owns delivery and updates for // this build. Ordinary direct-download builds keep PicFetch's GitHub updater. const StoreManaged = false + +// AppleAppStore identifies the Mac App Store channel at compile time. +const AppleAppStore = false diff --git a/internal/distribution/distribution_apple.go b/internal/distribution/distribution_apple.go new file mode 100644 index 00000000..bbfc5b97 --- /dev/null +++ b/internal/distribution/distribution_apple.go @@ -0,0 +1,10 @@ +//go:build darwin && appleappstore && !microsoftstore + +// Package distribution exposes immutable build-channel facts. +package distribution + +// StoreManaged reports that the Mac App Store owns delivery and updates. +const StoreManaged = true + +// AppleAppStore identifies the Mac App Store channel at compile time. +const AppleAppStore = true diff --git a/internal/distribution/distribution_apple_test.go b/internal/distribution/distribution_apple_test.go new file mode 100644 index 00000000..2f4873e3 --- /dev/null +++ b/internal/distribution/distribution_apple_test.go @@ -0,0 +1,23 @@ +//go:build darwin && appleappstore && !microsoftstore + +package distribution_test + +import ( + "testing" + + "github.com/frathe/picfetch/internal/distribution" + "github.com/frathe/picfetch/internal/launch" +) + +func TestAppleStoreLaunchRefusesSelfUpdates(t *testing.T) { + if !distribution.AppleAppStore { + t.Fatal("Apple build does not identify its store") + } + policy, err := launch.NewPolicy(launch.Options{}, "io.github.frathe.picfetch", distribution.StoreManaged) + if err != nil { + t.Fatal(err) + } + if policy.Updates().Allowed() { + t.Fatal("Mac App Store launch admits GitHub self-updates") + } +} diff --git a/internal/distribution/distribution_store.go b/internal/distribution/distribution_store.go index 6bafdd6c..cea2ca65 100644 --- a/internal/distribution/distribution_store.go +++ b/internal/distribution/distribution_store.go @@ -1,4 +1,4 @@ -//go:build microsoftstore +//go:build microsoftstore && !appleappstore // Package distribution exposes immutable build-channel facts. package distribution @@ -6,3 +6,6 @@ package distribution // StoreManaged reports whether Microsoft Store owns delivery and updates for // this build. const StoreManaged = true + +// AppleAppStore identifies the Mac App Store channel at compile time. +const AppleAppStore = false diff --git a/internal/distribution/distribution_store_test.go b/internal/distribution/distribution_store_test.go index 5f3a025c..d2afc00b 100644 --- a/internal/distribution/distribution_store_test.go +++ b/internal/distribution/distribution_store_test.go @@ -1,11 +1,14 @@ -//go:build microsoftstore +//go:build microsoftstore && !appleappstore package distribution import "testing" -func TestStoreManaged_MicrosoftStoreBuildIsTrue(t *testing.T) { +func TestStoreManaged_StoreBuildIsTrue(t *testing.T) { if !StoreManaged { - t.Error("microsoftstore build does not report Store-managed updates") + t.Error("Store build does not report Store-managed updates") + } + if AppleAppStore { + t.Error("Microsoft Store build reports Apple distribution") } } diff --git a/internal/distribution/distribution_test.go b/internal/distribution/distribution_test.go index f3d1c3bc..5776258c 100644 --- a/internal/distribution/distribution_test.go +++ b/internal/distribution/distribution_test.go @@ -1,4 +1,4 @@ -//go:build !microsoftstore +//go:build !microsoftstore && !appleappstore package distribution @@ -8,4 +8,7 @@ func TestStoreManaged_DefaultBuildIsFalse(t *testing.T) { if StoreManaged { t.Error("ordinary build reports Store-managed updates") } + if AppleAppStore { + t.Error("ordinary build reports Apple distribution") + } } diff --git a/internal/similarity/assets_install.go b/internal/similarity/assets_install.go index 48a1241b..b5b6c8e6 100644 --- a/internal/similarity/assets_install.go +++ b/internal/similarity/assets_install.go @@ -25,7 +25,7 @@ type DownloadProgress struct { } // ErrBundledRuntimeUnavailable requires package repair, never a code download. -var ErrBundledRuntimeUnavailable = errors.New("bundled runtime is unavailable; repair or update PicFetch through Microsoft Store") +var ErrBundledRuntimeUnavailable = errors.New("bundled runtime is unavailable; repair or update PicFetch through the app store") type assetDownload struct { name, address, digest string diff --git a/internal/similarity/assets_package.go b/internal/similarity/assets_package.go index 0a7b967d..17b4fb00 100644 --- a/internal/similarity/assets_package.go +++ b/internal/similarity/assets_package.go @@ -20,6 +20,27 @@ func StageWindowsRuntime(ctx context.Context, arch, archivePath, root string) er if !ok { return fmt.Errorf("unsupported Windows runtime architecture %q", arch) } + return stageRuntime(ctx, asset, archivePath, root) +} + +// StageMacRuntime verifies the architecture-pinned upstream archive and stages +// its native library and notices before signing. Signing changes Mach-O bytes; +// the signed bundle needs code-signature validation instead of this checksum. +func StageMacRuntime(ctx context.Context, arch, archivePath, root string) error { + if err := ctx.Err(); err != nil { + return err + } + asset, ok := platformRuntime("darwin", arch) + if !ok { + return fmt.Errorf("unsupported macOS runtime architecture %q", arch) + } + // Both pinned macOS releases provide this notice. Keep it with the native + // payload even where ordinary asset installation does not require it. + asset.privacyNotice = true + return stageRuntime(ctx, asset, archivePath, root) +} + +func stageRuntime(ctx context.Context, asset runtimeAsset, archivePath, root string) error { archive, err := os.Open(archivePath) if err != nil { return err @@ -30,16 +51,14 @@ func StageWindowsRuntime(ctx context.Context, arch, archivePath, root string) er return err } if !info.Mode().IsRegular() || info.Size() != asset.size { - // Windows is a proper name. - //goland:noinspection GoErrorStringFormat - return fmt.Errorf("Windows runtime archive has an unexpected size or type") + return fmt.Errorf("runtime archive has an unexpected size or type") } staging, err := os.MkdirTemp("", "picfetch-package-runtime-") if err != nil { return err } defer func() { _ = os.RemoveAll(staging) }() - verifiedPath := filepath.Join(staging, "runtime.zip") + verifiedPath := filepath.Join(staging, "runtime."+asset.archiveExtension) file, err := os.OpenFile(verifiedPath, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0600) if err != nil { return err @@ -54,12 +73,31 @@ func StageWindowsRuntime(ctx context.Context, arch, archivePath, root string) er return closeErr } if n != asset.size || fmt.Sprintf("%x", hash.Sum(nil)) != asset.digest { - // Windows is a proper name. - //goland:noinspection GoErrorStringFormat - return fmt.Errorf("Windows runtime archive checksum mismatch") + return fmt.Errorf("runtime archive checksum mismatch") + } + if _, err := unpackRuntimeAsset(ctx, staging, asset); err != nil { + return err + } + if err := verifyRuntime(ctx, staging, asset); err != nil { + return err + } + for _, name := range asset.files() { + if err := stageRuntimeFile(ctx, staging, root, name); err != nil { + return err + } + } + return ctx.Err() +} + +func stageRuntimeFile(ctx context.Context, staging, root, name string) error { + file, err := os.Open(filepath.Join(staging, name)) + if err != nil { + return err } - if _, err := unpackRuntimeZIPFile(ctx, root, asset, verifiedPath); err != nil { + defer func() { _ = file.Close() }() + info, err := file.Stat() + if err != nil { return err } - return verifyRuntime(ctx, root, asset) + return writeRuntimeFile(ctx, root, name, file, info.Size()) } diff --git a/internal/similarity/assets_test.go b/internal/similarity/assets_test.go index 368ca347..7735d8b7 100644 --- a/internal/similarity/assets_test.go +++ b/internal/similarity/assets_test.go @@ -58,13 +58,80 @@ func TestStoreDownloadPolicy(t *testing.T) { requests++ return nil, errors.New("unexpected download") })}} - if _, err := client.InstallAssets(context.Background(), nil); err == nil || !strings.Contains(err.Error(), "Microsoft Store") || requests != 0 { + if _, err := client.InstallAssets(context.Background(), nil); err == nil || !errors.Is(err, ErrBundledRuntimeUnavailable) || requests != 0 { t.Fatalf("missing bundled runtime triggered a download: requests=%d err=%v", requests, err) } } type similarityAssetTransport func(*http.Request) (*http.Response, error) +func TestStageMacRuntime(t *testing.T) { + for _, arch := range []string{"amd64", "arm64", "386"} { + t.Run(arch, func(t *testing.T) { + root := t.TempDir() + archive := filepath.Join(root, "runtime.tgz") + if err := os.WriteFile(archive, []byte("not a release archive"), 0600); err != nil { + t.Fatal(err) + } + out := filepath.Join(root, "package") + if err := StageMacRuntime(t.Context(), arch, archive, out); err == nil { + t.Fatal("unverified native code entered Mac App Store staging") + } + if _, err := os.Stat(out); !os.IsNotExist(err) { + t.Fatalf("rejected archive changed package output: %v", err) + } + }) + } + t.Run("cancelled", func(t *testing.T) { + ctx, cancel := context.WithCancel(t.Context()) + cancel() + if err := StageMacRuntime(ctx, "arm64", "missing.tgz", t.TempDir()); !errors.Is(err, context.Canceled) { + t.Fatalf("cancelled package staging: %v", err) + } + }) + t.Run("checksum", func(t *testing.T) { + root := t.TempDir() + archive := filepath.Join(root, "runtime.tgz") + file, err := os.Create(archive) + if err != nil { + t.Fatal(err) + } + asset, _ := platformRuntime("darwin", "arm64") + if err := file.Truncate(asset.size); err != nil { + _ = file.Close() + t.Fatal(err) + } + if err := file.Close(); err != nil { + t.Fatal(err) + } + out := filepath.Join(root, "package") + if err := StageMacRuntime(t.Context(), "arm64", archive, out); err == nil || !strings.Contains(err.Error(), "checksum") { + t.Fatalf("runtime archive hash was not checked: %v", err) + } + if _, err := os.Stat(out); !os.IsNotExist(err) { + t.Fatalf("rejected archive changed package output: %v", err) + } + }) +} + +func TestStageMacRuntimePinnedArchive(t *testing.T) { + archive := os.Getenv("PICFETCH_TEST_MAC_RUNTIME_ARCHIVE") + if archive == "" { + t.Skip("explicit pinned macOS arm64 archive required") + } + root := t.TempDir() + if err := StageMacRuntime(t.Context(), "arm64", archive, root); err != nil { + t.Fatal(err) + } + asset, _ := platformRuntime("darwin", "arm64") + asset.privacyNotice = true + for _, name := range asset.files() { + if info, err := os.Stat(filepath.Join(root, name)); err != nil || !info.Mode().IsRegular() || info.Size() == 0 { + t.Fatalf("missing packaged runtime or notice %s: %v", name, err) + } + } +} + func (f similarityAssetTransport) RoundTrip(request *http.Request) (*http.Response, error) { return f(request) } diff --git a/internal/ui/autoupdate.go b/internal/ui/autoupdate.go index 05b03223..a3673c5e 100644 --- a/internal/ui/autoupdate.go +++ b/internal/ui/autoupdate.go @@ -13,6 +13,7 @@ import ( "fyne.io/fyne/v2/lang" "fyne.io/fyne/v2/widget" + "github.com/frathe/picfetch/internal/distribution" "github.com/frathe/picfetch/internal/launch" "github.com/frathe/picfetch/internal/requestlife" "github.com/frathe/picfetch/internal/ui/autoupdate" @@ -173,6 +174,9 @@ func (v *viewer) CheckForUpdatesNow(callbacks settingswin.UpdateCallbacks) { func (v *viewer) updateRestrictionError() error { if v.launchPolicy.Valid() && v.launchPolicy.Purpose() == launch.Ordinary && v.launchPolicy.StoreManaged() { + if distribution.AppleAppStore { + return errors.New(lang.L("Updates are managed by Mac App Store.")) + } return errors.New(lang.L("Updates are managed by Microsoft Store.")) } return errors.New(lang.L("Updates are unavailable in this session")) diff --git a/internal/ui/autoupdate_test.go b/internal/ui/autoupdate_test.go index 83a2d2aa..9f1387f2 100644 --- a/internal/ui/autoupdate_test.go +++ b/internal/ui/autoupdate_test.go @@ -29,6 +29,7 @@ import ( "fyne.io/fyne/v2/lang" "fyne.io/fyne/v2/widget" + "github.com/frathe/picfetch/internal/distribution" "github.com/frathe/picfetch/internal/launch" "github.com/frathe/picfetch/internal/preferences" "github.com/frathe/picfetch/internal/ui/autoupdate" @@ -291,12 +292,16 @@ func TestMicrosoftStoreUpdateActionsAreRefused(t *testing.T) { t.Error("Store-managed build retained the GitHub update preference") } + want := lang.L("Updates are managed by Microsoft Store.") + if distribution.AppleAppStore { + want = lang.L("Updates are managed by Mac App Store.") + } var manualErr error v.CheckForUpdatesNow(settingswin.UpdateCallbacks{Failed: func(err error) { manualErr = err }}) - if manualErr == nil || manualErr.Error() != lang.L("Updates are managed by Microsoft Store.") { + if manualErr == nil || manualErr.Error() != want { t.Fatalf("manual update error = %v", manualErr) } - if err := v.PerformUpdate(); err == nil || err.Error() != lang.L("Updates are managed by Microsoft Store.") { + if err := v.PerformUpdate(); err == nil || err.Error() != want { t.Fatalf("PerformUpdate error = %v", err) } if v.updater.Done().Begun() { diff --git a/internal/ui/explorer/setup.go b/internal/ui/explorer/setup.go index 2a810930..17f6892d 100644 --- a/internal/ui/explorer/setup.go +++ b/internal/ui/explorer/setup.go @@ -13,6 +13,7 @@ import ( "fyne.io/fyne/v2/layout" "fyne.io/fyne/v2/widget" + "github.com/frathe/picfetch/internal/distribution" "github.com/frathe/picfetch/internal/similarity" "github.com/frathe/picfetch/internal/ui/assets" ) @@ -174,7 +175,11 @@ func (f *Feature) downloadExplorerAssets(s *explorerSetup) { fyne.LogError("install similarity assets", err) s.status.SetText(lang.L("Setup could not finish. Check your connection and available disk space, then retry.")) if errors.Is(err, similarity.ErrBundledRuntimeUnavailable) { - s.status.SetText(lang.L("The bundled analysis runtime is missing or damaged. Repair or update PicFetch through Microsoft Store, then retry.")) + if distribution.AppleAppStore { + s.status.SetText(lang.L("The bundled analysis runtime is missing or damaged. Repair or update PicFetch through Mac App Store, then retry.")) + } else { + s.status.SetText(lang.L("The bundled analysis runtime is missing or damaged. Repair or update PicFetch through Microsoft Store, then retry.")) + } } s.primary.SetText(lang.L("Retry")) s.primary.Enable() diff --git a/internal/ui/explorer/setup_test.go b/internal/ui/explorer/setup_test.go index f704b45d..f5ff6dc0 100644 --- a/internal/ui/explorer/setup_test.go +++ b/internal/ui/explorer/setup_test.go @@ -73,9 +73,13 @@ func TestFeatureSetupDownloadRetryCancel(t *testing.T) { // Store binaries preserve their bundled-runtime repair path. //goland:noinspection GoBoolExpressions if distribution.StoreManaged && requests == 0 { + want := lang.L("The bundled analysis runtime is missing or damaged. Repair or update PicFetch through Microsoft Store, then retry.") + if distribution.AppleAppStore { + want = lang.L("The bundled analysis runtime is missing or damaged. Repair or update PicFetch through Mac App Store, then retry.") + } explained := false walkFeature(host.win.Canvas().Overlays().Top(), func(o fyne.CanvasObject) { - if l, ok := o.(*widget.Label); ok && l.Text == lang.L("The bundled analysis runtime is missing or damaged. Repair or update PicFetch through Microsoft Store, then retry.") { + if l, ok := o.(*widget.Label); ok && l.Text == want { explained = true } }) diff --git a/internal/ui/launchpolicy_test.go b/internal/ui/launchpolicy_test.go index d8205be8..0c4e01bd 100644 --- a/internal/ui/launchpolicy_test.go +++ b/internal/ui/launchpolicy_test.go @@ -24,6 +24,7 @@ import ( fynetest "fyne.io/fyne/v2/test" "fyne.io/fyne/v2/widget" + "github.com/frathe/picfetch/internal/distribution" "github.com/frathe/picfetch/internal/launch" "github.com/frathe/picfetch/internal/locationtrial" "github.com/frathe/picfetch/internal/preferences" @@ -402,7 +403,11 @@ func TestLaunchPolicyIntegration(t *testing.T) { meta := v.app.Metadata() want := []string{fmt.Sprintf(lang.L("Version %s (Build %d)"), meta.Version, meta.Build)} if tc.policy.StoreManaged() { - want = append(want, lang.L("Updates are managed by Microsoft Store.")) + if distribution.AppleAppStore { + want = append(want, lang.L("Updates are managed by Mac App Store.")) + } else { + want = append(want, lang.L("Updates are managed by Microsoft Store.")) + } } if tc.policy.Purpose() != launch.Ordinary { want = append(want, lang.L("Updates are unavailable in this session")) diff --git a/internal/ui/settingswin/settingswin.go b/internal/ui/settingswin/settingswin.go index bb59d834..34ac39dc 100644 --- a/internal/ui/settingswin/settingswin.go +++ b/internal/ui/settingswin/settingswin.go @@ -25,6 +25,7 @@ import ( "fyne.io/fyne/v2/widget" "github.com/frathe/picfetch/internal/appearance" + "github.com/frathe/picfetch/internal/distribution" "github.com/frathe/picfetch/internal/filesort" "github.com/frathe/picfetch/internal/heic" "github.com/frathe/picfetch/internal/launch" @@ -444,7 +445,11 @@ func (w *Window) build() fyne.CanvasObject { var explanation string switch reason { case launch.StoreManagedUpdates: - explanation = lang.L("Updates are managed by Microsoft Store.") + if distribution.AppleAppStore { + explanation = lang.L("Updates are managed by Mac App Store.") + } else { + explanation = lang.L("Updates are managed by Microsoft Store.") + } case launch.TrialUpdates, launch.MissingPolicy: explanation = lang.L("Updates are unavailable in this session") } diff --git a/internal/ui/settingswin/settingswin_test.go b/internal/ui/settingswin/settingswin_test.go index e09527c8..b9a1a68b 100644 --- a/internal/ui/settingswin/settingswin_test.go +++ b/internal/ui/settingswin/settingswin_test.go @@ -13,6 +13,7 @@ import ( "fyne.io/fyne/v2/widget" "github.com/frathe/picfetch/internal/appearance" + "github.com/frathe/picfetch/internal/distribution" "github.com/frathe/picfetch/internal/filesort" "github.com/frathe/picfetch/internal/launch" "github.com/frathe/picfetch/internal/preferences" @@ -938,7 +939,11 @@ func TestUpdatesTab_MicrosoftStoreOwnsUpdates(t *testing.T) { if w.updateManaged == nil { t.Fatal("Microsoft Store update explanation is missing") } - if got, want := w.updateManaged.Text, "Updates are managed by Microsoft Store."; got != want { + want := "Updates are managed by Microsoft Store." + if distribution.AppleAppStore { + want = "Updates are managed by Mac App Store." + } + if got := w.updateManaged.Text; got != want { t.Errorf("update explanation = %q, want %q", got, want) } if !containsCanvasObject(updates, w.updateManaged) { @@ -950,6 +955,10 @@ func TestUpdatesTab_MicrosoftStoreOwnsUpdates(t *testing.T) { } func TestUpdatesTabLaunchPolicy(t *testing.T) { + storeExplanation := "Updates are managed by Microsoft Store." + if distribution.AppleAppStore { + storeExplanation = "Updates are managed by Mac App Store." + } for _, tc := range []struct { name string purpose launch.Purpose @@ -957,11 +966,11 @@ func TestUpdatesTabLaunchPolicy(t *testing.T) { want []string }{ {"ordinary_portable", launch.Ordinary, false, false, nil}, - {"ordinary_store", launch.Ordinary, true, false, []string{"Updates are managed by Microsoft Store."}}, + {"ordinary_store", launch.Ordinary, true, false, []string{storeExplanation}}, {"explorer_portable", launch.ExplorerTrial, false, false, []string{"Updates are unavailable in this session"}}, - {"explorer_store", launch.ExplorerTrial, true, false, []string{"Updates are managed by Microsoft Store.", "Updates are unavailable in this session"}}, + {"explorer_store", launch.ExplorerTrial, true, false, []string{storeExplanation, "Updates are unavailable in this session"}}, {"location_map_portable", launch.LocationMapTrial, false, false, []string{"Updates are unavailable in this session"}}, - {"location_map_store", launch.LocationMapTrial, true, false, []string{"Updates are managed by Microsoft Store.", "Updates are unavailable in this session"}}, + {"location_map_store", launch.LocationMapTrial, true, false, []string{storeExplanation, "Updates are unavailable in this session"}}, {"missing_policy", launch.Ordinary, false, true, []string{"Updates are unavailable in this session"}}, } { t.Run(tc.name, func(t *testing.T) { diff --git a/packaging/apple-app-store/README.md b/packaging/apple-app-store/README.md new file mode 100644 index 00000000..4010baa3 --- /dev/null +++ b/packaging/apple-app-store/README.md @@ -0,0 +1,89 @@ +# Mac App Store preparation + +The Apple channel is experimental. No submission package or signed sandbox +qualification has been completed. Retain all PicFetch features; do not use +temporary filesystem exceptions or weaken the analysis worker's network denial +to pass a packaging check. + +The [active plan](../../plans/2026-09-29-apple-app-store.md) owns implementation +and evidence. [Apple requirements](../../docs/apple-app-store-requirements-2026-09-29.md) +records current primary sources and distinguishes native macOS requirements +from mobile-platform privacy rules. + +## Build policy + +Use `no_emoji,nodynamic,appleappstore` for this macOS channel. Do not combine +`appleappstore` with `microsoftstore`. Apple builds capture Store-managed update +permission at launch, never download a replacement native runtime, and explain +missing bundled code as an App Store repair. This tag alone does not enable +App Sandbox or produce a submission artifact. `make package-mac` continues to +produce the direct-download distribution and must not be submitted to the Store. + +`similarity.StageMacRuntime` accepts a pinned macOS architecture's upstream +archive and stages only its verified native library, LICENSE, +ThirdPartyNotices.txt and Privacy.md. The complete archive is verified in a +temporary directory before any payload enters the package. It is a pre-sign +step; signed Mach-O validation is separate work. The same library's SHA-256 +changes when signed, so checking it against the original upstream digest after +signing is incorrect. + +## Developer inputs + +Run the read-only prerequisite check: + +```sh +APPLE_STORE_TEAM_ID=YOURTEAMID \ +APPLE_STORE_PROFILE=/absolute/path/PicFetch.provisionprofile \ +make apple-store-preflight +``` + +It uses `/Applications/Xcode.app/Contents/Developer` without changing the system +developer-directory selection. Override `APPLE_STORE_DEVELOPER_DIR` for another +full Xcode installation. It checks for Store application and installer signing +identities with private keys and confirms that a profile file was supplied. +It does not install credentials, access an Apple account, upload a package or +certify the profile or app. Supplying a profile does not establish its signature, +expiry, registered identifier or entitlement compatibility. + +Required inputs: + +- Active Apple Developer Program team and a registered explicit bundle ID + `io.github.frathe.picfetch` available to that team. +- Apple Distribution / Mac App Store application signing certificate with its + private key, plus a Mac Installer Distribution identity. +- Mac App Store Connect provisioning profile for that identifier and signing + certificate; exact helpers and their profiles depend on the sandbox design. +- App Store Connect app record, public support/contact and privacy-policy URLs, + localized listing, screenshots, category, rating and intended countries. +- Encryption questionnaire determination covering Go's bundled TLS/cryptography; + do not assume the Apple-OS-only exemption applies. + +## Qualification before submission + +1. Preserve native open/drop/Dock access as real URL authority and retain + security-scoped bookmarks for complete sessions and Favorites. Test moved, + missing, protected and external-volume sources, parent-folder browsing, + original edits, exports, deletion, clipboard and wallpaper behavior. +2. Use a public sandbox-compatible helper architecture. Verify actual TCP/UDP + denial in image-analysis workers, current source grants, HEIC decoding, + inference and worker exit after cancellation, app quit and parent failure. +3. Stage architecture-pinned runtimes with their upstream notices; preserve the + app's complete dependency/license closure. Bundle models if Apple's review + requirements or the selected first-run design call for it. Inspect every + native dependent-library path and minimum OS version. +4. Validate nested code before the outer app signature and provisioning profile; + finish all plist/resource edits before signing. Inspect sandbox entitlements, + library signatures, quarantine attributes and the signed installer's payload. +5. Qualify the final Store artifact on Apple Silicon and Intel, with a fresh + account and without developer codecs/model caches. Retain the exact artifact + hash, build number and verification output. +6. Reconcile optional tile/download requests with published privacy disclosures + and App Store privacy answers. Check the entire dependency closure against + Apple's SDK manifest list; do not copy iOS API-reason requirements onto macOS. +7. Validate and upload with Apple's current tools, then test the actual + TestFlight/Store installation and upgrade. Submission/upload needs Ronin's + authorization separately; this preparation does not publish anything. + +No `package-mac-store` target is advertised until these runtime and signing +contracts are implemented and exercised. A successful prerequisite check cannot +substitute for signed sandbox or submission evidence. diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md new file mode 100644 index 00000000..d863958b --- /dev/null +++ b/plans/2026-09-29-apple-app-store.md @@ -0,0 +1,234 @@ +# Mac App Store preparation + +Status: active; submission readiness is not established. +Route: Deep (macOS distribution and cross-package permission lifetimes). + +## Deliverable + +Prepare a separate Mac App Store distribution while retaining PicFetch's +features and preserving ordinary and Microsoft Store distribution behavior. +Ronin authorized committing the preparation patch after reviewing the initial +handoff. No upload, release, or change to signing credentials is authorized. + +## Observed gaps + +- `package-mac` produces the ordinary, unsandboxed distribution. +- `internal/distribution` recognizes only ordinary and Microsoft Store builds. +- Session and Favorite file lists retain paths, without security-scoped grants. +- Darwin HEIC and similarity launch through `/usr/bin/sandbox-exec`. +- Ordinary similarity setup downloads native ONNX libraries. +- The runtime verifier compares complete upstream library bytes; signing a + Mach-O library changes those bytes, so this verifier cannot simply be reused + after Mac App Store signing. +- The development host selects Command Line Tools; full Xcode 27 is available through a per-command + `DEVELOPER_DIR`, and + `security find-identity -v -p codesigning` reports no valid signing identities. + +Apple requirements and sources are recorded in +`docs/apple-app-store-requirements-2026-09-29.md`. + +## Decisions + +| Decision | Reason | +| --- | --- | +| Separate `appleappstore` build channel | A Store package must never activate GitHub binary updates or download native code. | +| Preserve current application ID | Changing it would change preferences/session identity; confirm it can be registered with the developer account. | +| Preserve full feature set (Ronin confirmed) | Do not silently remove Explorer, deletion, or wallpaper support. | +| No broad filesystem or temporary exception entitlements | User-selected grants must carry actual authority; saved paths do not. | +| No weaker offline guarantee | Replacing `sandbox-exec` with an ordinary network-enabled child is not sufficient. | + +## Acceptance criteria + +AC1: Ordinary, Microsoft Store, and Apple Store builds select their intended +update policy; Apple builds refuse native runtime download fallback. +`go test -tags no_emoji,nodynamic ./internal/distribution ./internal/launch` +`go test -tags no_emoji,nodynamic,microsoftstore ./internal/distribution ./internal/similarity -run 'TestStore|TestRuntimeDownloadSelection'` +`go test -tags no_emoji,nodynamic,appleappstore ./internal/distribution ./internal/similarity -run 'TestAppleStore|TestStore|TestRuntimeDownloadSelection'` + +AC2: Store explanations name the correct store, with locale parity. +`go test -tags no_emoji,nodynamic,appleappstore ./internal/ui/settingswin ./internal/ui/explorer ./internal/ui -run 'TestUpdatesTab|TestFeatureSetupDownloadRetryCancel|TestUpdateCheck_StoreManaged|TestLaunchPolicyIntegration'` +`go test -tags no_emoji,nodynamic . -run 'TestTranslations|TestLaunchStartupContract'` + +AC3: Grants survive relaunch and are retained through all workers using them; +cancel/close/shutdown do not release grants while producers still use them. +Verification: native signed sandbox qualification on both architectures, +including picker, Dock/Open With, dropped directory, session restoration, +Favorites, renamed/missing files, exports, committed writes, and shutdown. +The test runner and commands must be pinned before implementation starts. + +AC4: Public sandboxed workers preserve actual TCP/UDP denial and tracked exit. +Verification: signed native helper qualification, including real HEIC and ONNX +analysis, cancelled runs, worker shutdown, and parent crash. Exact executable +layout and IPC grant transport remain design work. + +AC5: Native dependencies are pinned before signing, bundled with notices, +validated by signature after signing, and cannot be replaced by model-cache +overrides. Verification: final signed bundle validation, tampered-library +rejection, real inference, and pre-sign archive checksum rejection. + +AC6: Submission package and metadata pass Apple's validation; privacy labels, +export compliance, licensing, screenshots, support and policy URLs are accurate. +Verification: final bundle inspection, `codesign --verify --strict`, +`pkgutil --check-signature`, Apple's current upload validation and TestFlight. +Credentials and account details are needed for these external checks. + +## Task graph and routing + +`T1 -> T2 -> T3 -> T4 -> T5; pre-sign staging in T4a is independent of T2/T3`; T6 research informs every task. +All implementation, architecture, review and fixes belong to T0 inline. +One research delegation covers public Apple documentation only; no platform +implementation or review is delegated. No model overrides. + +### T1 — Distribution policy and accurate Store messages +Owner: T0 inline. +Files: internal/distribution, similarity/assets_install.go and assets_test.go, +UI update/settings/setup code and existing tests, translations, architecture. +Contract: `distribution.StoreManaged` remains immutable; a new +`distribution.AppleAppStore` identifies the Apple channel. Existing launch +permission remains the authority for update effects. +Test: captured policy disables updates; missing bundled runtime fails before +any HTTP request; UI names the correct store. +Verify: AC1/AC2 commands. +Budget: 0 spawns; 2 review rounds; full suite at final gate only. + +### T2 — Persistent user-selected filesystem access +Owner: T0 inline. +Files: new permission owner package; filepicker, openwith, UI input/startup/ +shutdown, session and Favorite authority integration. +Depends: T1. +Contract: design a bounded owner for grants and bookmarks before coding; keep +native URL authority through admission and retain grants through worker exit. +Test: denied sibling access never silently broadens authority; stale or absent +grants offer reselection; complete Favorite/session sources retain valid grants. +Verify: AC3 qualification and focused lifetime tests. +Budget: 0 spawns; 2 review rounds; no extra complete suite. + +### T3 — Public sandbox worker architecture +Owner: T0 inline. +Files: main private dispatch, HEIC and similarity Darwin launchers/protocols, +packaged helper entitlements and native qualification tooling. +Depends: T2. +Contract: a signed worker with no networking, documented grant transfer, and +existing stop/done and offline-verification contracts. +Test: worker cannot network, accepts only current source grants, and exits after +cancellation/shutdown. Do not introduce temporary sandbox exceptions. +Verify: AC4 native qualification. +Budget: 0 spawns; 2 review rounds; no extra complete suite. + +### T4 — Bundle runtime and build/sign package +Owner: T0 inline. +Files: similarity assets/package verification, Apple packaging scripts, +Makefile, entitlements and metadata templates, third-party notices. +Depends: T3. +Contract: Mac runtime loaded exclusively from the signed application bundle; +release archive digest verified before signing, signature after signing. +Test: tampering, wrong architecture, bad archive and missing notices fail closed. +Verify: AC5 plus signed package inspection. +Budget: 0 spawns; 2 review rounds; no extra complete suite. + +### T4a — Pre-sign native staging and prerequisite tooling +Owner: T0 inline. +Files: similarity/assets_package.go and existing assets_test.go; +Makefile, scripts/apple-store-preflight.sh, packaging/apple-app-store/README.md. +Depends: no runtime permission contract; can precede T2/T3. +Contract: `StageMacRuntime(ctx, arch, archivePath, root)` verifies pinned archives +and stages only library/notices before signing; prerequisites are read-only. +Test: invalid/unsupported/cancelled archives cannot change package output; +real pinned arm64 archive yields verified runtime plus upstream notices. +Verify: targeted StageMacRuntime/StageWindowsRuntime tests with the explicit +existing arm64 archive; `bash -n scripts/apple-store-preflight.sh`; +`make apple-store-preflight` reports actual missing inputs. +Budget: 0 spawns; 2 review rounds; no additional complete suite. + +### T5 — Submission evidence and final gate +Owner: T0 inline. +Files: packaging/apple-app-store/listing.md, privacy and submission records, +todos.md, plan evidence. +Depends: T4. +Contract: every claim binds to an actual final artifact; report unverified gates. +Test: metadata mirrors shipped behavior and has no placeholders at submission. +Verify: AC6, GoLand inspections, `make verify`, latest CI where available. +Budget: 0 spawns; 1 review round; complete suite once. + +### T6 — Apple primary-source research +Owner: delegated research agent. +Files: docs/apple-app-store-requirements-2026-09-29.md only. +Contract: Apple citations, scope distinctions and explicit uncertainty. +Verify: T0 checks links/claims against fetched Apple documentation. +Budget: 1 spawn; 1 review round; no suite. +Delegation gate: G1 yes (bounded public research), G2 no (judgment required), +G3 yes, G4 yes, G5 yes. This delegation is specifically required by the invoked +research skill; that instruction is the reason for the G2 exception. The +agent supplies evidence only; T0 owns assessment and review. + +## Dependencies and licensing + +No dependency upgrades. Retain the existing six-target notice checks, AVIF WASM +source/license manifest, Fyne/GLFW licensing and SigLIP model notices. Apple +packages must include architecture-pinned ONNX 1.23.2 (Intel) / 1.29.0 (arm64), +their exact upstream LICENSE, ThirdPartyNotices.txt and available Privacy.md. +Record the complete shipped closure and notice delivery before release readiness. + +## Honest limit + +A build tag is preparation, not evidence of sandbox compliance. Until T2–T5 +are complete, Apple builds are experimental and must not be submitted. No +signed sandbox test, Apple validation or App Review result exists yet. + +## Evidence and cost ledger + +Initial reconnaissance: branch `feature/apple-appstore-preparations`, clean +working tree; host arm64; selected developer directory +`/Library/Developer/CommandLineTools`; zero valid code-signing identities. + +| Task | Spawns budget/actual | Review rounds | Complete suite | Status | +| --- | --- | --- | --- | --- | +| T1 | 0/0 | 2 | no | verified foundation | +| T2–T5 | 0/0 | pending | unverified | open | +| T4a | 0/0 | 2 | no | verified pre-sign staging | +| T6 | 1/1 | 1 | no | research recorded; T0 checked key requirements | + +### Preparation evidence (2026-09-29) + +Analyzed revision: HEAD `09dea07` plus the uncommitted preparation patch. +Changed code scope digest (sorted path/NUL/content/NUL): SHA-256 `379807100409ec6246a78142d7f8802d55c1ee1528c72e9bd01b07a785b4a061` (18 files, Go source/tests, shell and Makefile). + +- `make verify-build` passed: formatting, TUF freshness, exact Qodana exclusions, + generated assets, dependency notice checks, vet and ordinary build. +- Complete affected-package regressions passed for distribution, launch, + similarity, Settings, Explorer and MSIX staging with ordinary build tags. +- Focused Apple-channel tests passed in main, distribution, similarity, + Settings, Explorer and root UI: startup policy, translation parity, update + refusal, Store runtime-download refusal and correct Store explanations. +- Focused Microsoft Store policy, download-selection and setup/Settings tests + passed. No native Windows Store artifact was newly qualified. +- Real pre-sign arm64 staging passed against the existing pinned + `onnxruntime-osx-arm64-1.29.0.tgz`; library checksum and all upstream + notices including Privacy.md were verified. Intel pinned-archive staging + was not exercised against a real archive. +- Guards were deliberately violated: false Apple channel, allowed GitHub + updates and skipped runtime archive digest. All three tests failed for the + intended behavioral reason; original sources were restored and tests passed. +- GoLand `get_file_problems(errorsOnly=false)` inspected every changed Go file + and the shell preflight, including weak warnings; no timeouts were reported. + Four duplicate-fragment weak warnings in `internal/ui/autoupdate_test.go` + describe existing unchanged test blocks. They are intentional test + duplication covered by that file's existing exact Qodana exclusion; no + refactoring or new suppression was added. No actionable finding remained. + New Apple-tagged test and runtime-staging files were inspected after final edits. +- GoLand build reported success but limited diagnostic collection. The actual + `go build` in `make verify-build` supplies command-level build evidence. +- `bash -n scripts/apple-store-preflight.sh` passed. Native preflight found Xcode + 27.0 (27A266a) through its per-command developer directory and reports missing + Team ID/profile. An independent keychain check found zero valid code-signing + identities. The profile itself is not validated by this prerequisite script. +- `make verify` stopped at the required platform guard: the Docker daemon is + Linux/aarch64. Complete Linux/amd64 race verification remains unverified; + no isolation test was skipped and no sandbox filter was relaxed. +- Fresh Qodana CI/SARIF, CodeQL, signed sandbox qualification, signed bundle/ + installer validation and Apple processing/review have not run for this patch. + No commit, push, submission or release was performed. + +Remaining implementation is T2/T3 and the signed portion of T4/T5. Signing +identity/profile/account inputs have been requested from Ronin. The foundation +is reviewable; it does not establish a full-feature Store app's readiness. diff --git a/qodana.yaml b/qodana.yaml index cd1c4450..71ad77e8 100644 --- a/qodana.yaml +++ b/qodana.yaml @@ -109,6 +109,7 @@ exclude: - "internal/decodepool/decodepool_test.go" - "internal/displays/displays_test.go" - "internal/displays/xrandr_test.go" + - "internal/distribution/distribution_apple_test.go" - "internal/distribution/distribution_store_test.go" - "internal/distribution/distribution_test.go" - "internal/dupes/dupes_test.go" diff --git a/scripts/apple-store-preflight.sh b/scripts/apple-store-preflight.sh new file mode 100644 index 00000000..350f2c4e --- /dev/null +++ b/scripts/apple-store-preflight.sh @@ -0,0 +1,73 @@ +#!/bin/bash +# Read-only developer-tool and signing-input checks. This is not a validation +# of a signed app, its sandbox behavior, or its App Store submission. +set -eu + +export DEVELOPER_DIR="${APPLE_STORE_DEVELOPER_DIR:-/Applications/Xcode.app/Contents/Developer}" +failures=0 +fail() { + printf 'NEEDED: %s\n' "$1" >&2 + failures=$((failures + 1)) +} + +if [ "$(uname -s)" != Darwin ]; then + printf 'Mac App Store preflight requires macOS.\n' >&2 + exit 1 +fi + +if /usr/bin/xcodebuild -version; then + printf 'Xcode developer directory: %s\n' "$DEVELOPER_DIR" +else + fail 'Install full Xcode, complete its first launch, or set APPLE_STORE_DEVELOPER_DIR.' +fi + +for tool in /usr/bin/codesign /usr/bin/productbuild /usr/sbin/pkgutil /usr/bin/security; do + if [ ! -x "$tool" ]; then + fail "Apple tool $tool is unavailable." + fi +done + +team="${APPLE_STORE_TEAM_ID:-}" +if [[ ! "$team" =~ ^[A-Z0-9]{10}$ ]]; then + fail 'Set APPLE_STORE_TEAM_ID to the 10-character developer Team ID.' +else + work=$(mktemp -d) + trap 'rm -rf "$work"' EXIT + # Check for identities with private keys. Do not print the keychain's + # account names, certificate fingerprints, or provisioning-profile data. + if /usr/bin/security find-identity -v -p codesigning > "$work/app-identities"; then + if ! awk -v team="$team" ' + /"(Apple Distribution:|3rd Party Mac Developer Application:)/ && + index($0, "(" team ")") { found = 1 } + END { exit !found } + ' "$work/app-identities"; then + fail 'Install a valid Mac App Store application signing identity for this Team ID.' + fi + else + fail 'Unable to inspect application signing identities.' + fi + if /usr/bin/security find-identity -v > "$work/all-identities"; then + if ! awk -v team="$team" ' + /"3rd Party Mac Developer Installer:/ && + index($0, "(" team ")") { found = 1 } + END { exit !found } + ' "$work/all-identities"; then + fail 'Install a valid Mac Installer Distribution signing identity for this Team ID.' + fi + else + fail 'Unable to inspect installer signing identities.' + fi +fi + +profile="${APPLE_STORE_PROFILE:-}" +if [ -z "$profile" ] || [ ! -f "$profile" ]; then + fail 'Set APPLE_STORE_PROFILE to the Mac App Store Connect provisioning profile for io.github.frathe.picfetch.' +else + printf 'Provisioning profile supplied; its signature, identity, expiry and entitlements still require artifact validation.\n' +fi + +if [ "$failures" -ne 0 ]; then + printf '%s preparation inputs remain unresolved.\n' "$failures" >&2 + exit 1 +fi +printf 'Preparation inputs found. Signed sandbox qualification and final package validation are still required.\n' diff --git a/todos.md b/todos.md index 49bd4dd7..288f149d 100644 --- a/todos.md +++ b/todos.md @@ -12,7 +12,17 @@ ## Open -No open items. +### Mac App Store preparation + +Preserve the full feature set. Active plan: [Mac App Store preparation](plans/2026-09-29-apple-app-store.md). + +- [x] Add an immutable Apple Store channel, localized update/repair messages, verified pre-sign runtime staging and a read-only developer-input preflight. +- [ ] Qualify persistent security-scoped permissions for opening, saved sessions, Favorites, writes and workers. +- [ ] Replace the Store worker launch with public sandbox-compatible helpers while retaining verified network denial. +- [ ] Bundle pinned native runtimes and notices, and validate signed Mach-O code without comparing its post-sign bytes with upstream archive hashes. +- [ ] Add the signed Store packaging route and validate the final artifact on Intel and Apple Silicon. +- [ ] Complete privacy/dependency/export assessment, listing metadata and Apple submission validation. +- [ ] Obtain full Xcode, developer Team ID, Store application/installer certificates and provisioning profile for signed qualification. ## Deferred diff --git a/translations/de.json b/translations/de.json index 990bb654..fc49b20a 100644 --- a/translations/de.json +++ b/translations/de.json @@ -28,6 +28,7 @@ "Image size: %.2fx": "Bildgröße: %.2fx", "Image gap: %.2f s": "Bildabstand: %.2f s", "The bundled analysis runtime is missing or damaged. Repair or update PicFetch through Microsoft Store, then retry.": "Die enthaltene Analyse-Laufzeit fehlt oder ist beschädigt. Reparieren oder aktualisieren Sie PicFetch über den Microsoft Store und versuchen Sie es erneut.", + "The bundled analysis runtime is missing or damaged. Repair or update PicFetch through Mac App Store, then retry.": "Die enthaltene Analyse-Laufzeit fehlt oder ist beschädigt. Reparieren oder aktualisieren Sie PicFetch über den Mac App Store und versuchen Sie es erneut.", "Finis": "Finis", "please hypnotize me": "please hypnotize me", "(search for it)": "(suche danach)", @@ -40,6 +41,7 @@ "About %s": "Über %s", "Version %s (Build %d)": "Version %s (Build %d)", "Updates are managed by Microsoft Store.": "Updates werden vom Microsoft Store verwaltet.", + "Updates are managed by Mac App Store.": "Updates werden vom Mac App Store verwaltet.", "Open the manual": "Handbuch öffnen", "Search for...": "Suchen nach...", "Scanning... 0 images": "Scannen... 0 Bilder", diff --git a/translations/en.json b/translations/en.json index 424ba627..c9ac7ade 100644 --- a/translations/en.json +++ b/translations/en.json @@ -28,6 +28,7 @@ "Image size: %.2fx": "Image size: %.2fx", "Image gap: %.2f s": "Image gap: %.2f s", "The bundled analysis runtime is missing or damaged. Repair or update PicFetch through Microsoft Store, then retry.": "The bundled analysis runtime is missing or damaged. Repair or update PicFetch through Microsoft Store, then retry.", + "The bundled analysis runtime is missing or damaged. Repair or update PicFetch through Mac App Store, then retry.": "The bundled analysis runtime is missing or damaged. Repair or update PicFetch through Mac App Store, then retry.", "Finis": "Finis", "please hypnotize me": "please hypnotize me", "(search for it)": "(search for it)", @@ -40,6 +41,7 @@ "About %s": "About %s", "Version %s (Build %d)": "Version %s (Build %d)", "Updates are managed by Microsoft Store.": "Updates are managed by Microsoft Store.", + "Updates are managed by Mac App Store.": "Updates are managed by Mac App Store.", "Open the manual": "Open the manual", "Search for...": "Search for...", "Scanning... 0 images": "Scanning... 0 images", From 2ec93e7890a9c16949c2f5d84ea99d37363eb77d Mon Sep 17 00:00:00 2001 From: frathe Date: Tue, 29 Sep 2026 21:21:46 +0200 Subject: [PATCH 02/63] Add sandboxed XPC workers for Apple Store builds --- ARCHITECTURE.md | 21 ++- Makefile | 5 +- internal/heic/process_darwin.go | 2 + internal/heic/process_darwin_store.go | 41 +++++ internal/heic/process_unix.go | 2 +- internal/heic/process_unix_test.go | 2 + internal/macworker/command.go | 26 ++++ internal/macworker/command_test.go | 21 +++ internal/macworker/native/client.m | 63 ++++++++ internal/macworker/native/protocol.h | 10 ++ internal/macworker/native/service.m | 142 ++++++++++++++++++ internal/similarity/client.go | 16 +- internal/similarity/heic_capture_test.go | 17 ++- internal/similarity/search_client.go | 6 +- .../similarity/worker_cancel_unix_test.go | 4 +- internal/similarity/worker_darwin_store.go | 19 +++ internal/similarity/worker_other.go | 2 +- internal/similarity/worker_stop_other.go | 7 + packaging/apple-app-store/README.md | 33 +++- plans/2026-09-29-apple-app-store.md | 51 ++++++- qodana.yaml | 1 + scripts/apple-store-preflight.sh | 4 + scripts/macworkerqualify/check.py | 79 ++++++++++ scripts/macworkerqualify/fixture.c | 50 ++++++ scripts/macworkerqualify/run.sh | 42 ++++++ todos.md | 6 +- 26 files changed, 648 insertions(+), 24 deletions(-) create mode 100644 internal/heic/process_darwin_store.go create mode 100644 internal/macworker/command.go create mode 100644 internal/macworker/command_test.go create mode 100644 internal/macworker/native/client.m create mode 100644 internal/macworker/native/protocol.h create mode 100644 internal/macworker/native/service.m create mode 100644 internal/similarity/worker_darwin_store.go create mode 100644 internal/similarity/worker_stop_other.go create mode 100644 scripts/macworkerqualify/check.py create mode 100644 scripts/macworkerqualify/fixture.c create mode 100755 scripts/macworkerqualify/run.sh diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 1c87f53a..67c17d94 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -78,7 +78,9 @@ rule compatibility. and publication counts as immutable values; grouping includes its named sub-stages. `control_unix.go`/`control_other.go` own the pollable worker input descriptor. `worker_linux.go` installs worker-only seccomp denial on x64/ARM64 synchronized across all -threads before reading requests; `worker_other.go` retains the macOS sandbox launcher. +threads before reading requests; `worker_other.go` retains the ordinary macOS sandbox launcher. +`worker_darwin_store.go` launches the bundled `internal/macworker` broker; +`worker_stop_other.go` preserves immediate retirement outside the Apple channel. `worker_windows.go` launches a hidden ordinary subprocess with closeable control pipes; Windows does not install OS network denial and events keep `OfflineVerified` false. `offline.go` exposes that distinction through `EnforcesNetworkIsolation` @@ -771,6 +773,18 @@ Help's `privacy.go` uses the same document-injection pattern through policy opens a separate, word-wrapped Markdown singleton with vertical scrolling, Escape-to-close, and the same command-admission and shutdown guards. +### `internal/macworker` + +Mac App Store worker broker factory and native XPC service. `command.go` selects +only the broker beside the main executable and cancels with SIGTERM so process +Wait includes the service's child retirement. `native/protocol.h` transfers +stdin/stdout/stderr handles and a fixed HEIC/similarity mode; `client.m` forwards +cancellation; `service.m` owns a process group and reaps its leader after killing +descendants. The image worker lives inside the XPC service's `Contents/MacOS`. +`scripts/macworkerqualify` builds an ad-hoc signed native fixture to exercise +actual TCP/UDP denial, pipe transfer, cancellation, broker crash and group exit. +Source grant transport and final Store signing remain qualification work. + ### `internal/heic` Instance-owned system HEIC boundary. `Capability` coalesces representative checks, @@ -785,8 +799,9 @@ including repeated URIs, independently of displayed files. `Client` bounds native children (two slots, deadlines, framed output, Stop/Wait). Analysis producers use `NewInheritedSandboxClient` after establishing their own sandbox and verify OS network denial before reading images. macOS children -inherit that policy without a second `sandbox-exec`; desktop children still -install their own sandbox, and all client cancellation/resource bounds remain. +inherit that policy without a second `sandbox-exec`; ordinary desktop children +still install their own sandbox. Apple Store desktop children use the XPC +service and nested decoders retain its process group, and all client cancellation/resource bounds remain. `WorkerMain` dispatches before desktop startup. `container.go` performs checked primary-item/property and EXIF-transform interpretation inside that child; `native_linux.go/.c` dynamically loads system libheif with an HEVC provider. diff --git a/Makefile b/Makefile index f4413968..b795f319 100644 --- a/Makefile +++ b/Makefile @@ -66,7 +66,7 @@ COVERAGE_HTML := $(COVERAGE_DIR)/coverage.html .PHONY: verify-build --skip-local-tests .PHONY: generate-updater-notices check-updater-notices generate-avif-notices check-avif-notices .PHONY: fossa-findings -.PHONY: apple-store-preflight +.PHONY: apple-store-preflight apple-worker-test .PHONY: loc all: build @@ -103,6 +103,9 @@ loc: ## Show tracked source files and physical lines, split into Go production/t fossa-findings: ## Retrieve current FOSSA licensing findings (PR=61 or FOSSA_REVISION=; key in .env.local) go run ./scripts/fossafindings +apple-worker-test: ## Qualify signed Mac XPC worker isolation and retirement with native fixtures + bash scripts/macworkerqualify/run.sh + apple-store-preflight: ## Check native Xcode and Mac App Store signing inputs (see packaging/apple-app-store/README.md) bash scripts/apple-store-preflight.sh diff --git a/internal/heic/process_darwin.go b/internal/heic/process_darwin.go index 67ae40a4..e740418e 100644 --- a/internal/heic/process_darwin.go +++ b/internal/heic/process_darwin.go @@ -1,3 +1,5 @@ +//go:build darwin && !appleappstore + package heic import ( diff --git a/internal/heic/process_darwin_store.go b/internal/heic/process_darwin_store.go new file mode 100644 index 00000000..feb6f9ac --- /dev/null +++ b/internal/heic/process_darwin_store.go @@ -0,0 +1,41 @@ +//go:build darwin && appleappstore + +package heic + +import ( + "context" + "errors" + "os" + "os/exec" + "syscall" + + "github.com/frathe/picfetch/internal/macworker" +) + +func workerCommand(ctx context.Context, executable string) *exec.Cmd { + return macworker.Command(ctx, executable, macworker.HEIC) +} + +func inheritedWorkerCommand(ctx context.Context, executable string) *exec.Cmd { + return exec.CommandContext(ctx, executable) +} + +// The XPC service owns the group. Nested decoders must stay in that group so +// service cancellation also retires analysis's native decoding descendants. +func prepareWorker(_ *exec.Cmd) {} + +func retireWorker(cmd *exec.Cmd) error { + if cmd.Process == nil { + return os.ErrProcessDone + } + // A caller explicitly owning a group (including isolated test fixtures) + // remains responsible for retiring it. Production commands inherit XPC's group. + if cmd.SysProcAttr != nil && cmd.SysProcAttr.Setpgid { + err := syscall.Kill(-cmd.Process.Pid, syscall.SIGKILL) + if errors.Is(err, syscall.ESRCH) { + return os.ErrProcessDone + } + return err + } + return cmd.Process.Signal(syscall.SIGTERM) +} diff --git a/internal/heic/process_unix.go b/internal/heic/process_unix.go index f32c3049..1461c5b9 100644 --- a/internal/heic/process_unix.go +++ b/internal/heic/process_unix.go @@ -1,4 +1,4 @@ -//go:build linux || darwin +//go:build linux || (darwin && !appleappstore) package heic diff --git a/internal/heic/process_unix_test.go b/internal/heic/process_unix_test.go index c5f6623e..3449dfb9 100644 --- a/internal/heic/process_unix_test.go +++ b/internal/heic/process_unix_test.go @@ -24,6 +24,8 @@ func TestHEICWorkerRetiresDescendants(t *testing.T) { command := client.command client.command = func(ctx context.Context, executable string) *exec.Cmd { cmd := command(ctx, executable) + // This fixture owns its process group, as the Store XPC service does. + cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true} cmd.ExtraFiles = []*os.File{writer} return cmd } diff --git a/internal/macworker/command.go b/internal/macworker/command.go new file mode 100644 index 00000000..0d7356cb --- /dev/null +++ b/internal/macworker/command.go @@ -0,0 +1,26 @@ +// Package macworker launches the bundled broker for the Mac App Store's +// network-free XPC service. Packaging supplies both signed executables. +package macworker + +import ( + "context" + "os/exec" + "path/filepath" + "syscall" +) + +// Mode selects one private worker protocol; the native service validates it. +type Mode string + +const ( + HEIC Mode = "heic" + Similarity Mode = "similarity" +) + +// Command selects the broker beside the main application executable. It sends +// termination rather than killing the broker so Wait includes service cleanup. +func Command(ctx context.Context, executable string, mode Mode) *exec.Cmd { + cmd := exec.CommandContext(ctx, filepath.Join(filepath.Dir(executable), "picfetch-worker-client"), string(mode)) + cmd.Cancel = func() error { return cmd.Process.Signal(syscall.SIGTERM) } + return cmd +} diff --git a/internal/macworker/command_test.go b/internal/macworker/command_test.go new file mode 100644 index 00000000..e01a0309 --- /dev/null +++ b/internal/macworker/command_test.go @@ -0,0 +1,21 @@ +package macworker + +import ( + "context" + "path/filepath" + "reflect" + "testing" +) + +func TestCommandUsesOnlyBundledBrokerAndMode(t *testing.T) { + for _, mode := range []Mode{HEIC, Similarity} { + cmd := Command(context.Background(), "/Applications/PicFetch.app/Contents/MacOS/picfetch", mode) + want := []string{"/Applications/PicFetch.app/Contents/MacOS/picfetch-worker-client", string(mode)} + if !reflect.DeepEqual(cmd.Args, want) { + t.Fatalf("arguments = %v, want %v", cmd.Args, want) + } + if cmd.Path != filepath.Join("/Applications/PicFetch.app/Contents/MacOS", "picfetch-worker-client") { + t.Fatal(cmd.Path) + } + } +} diff --git a/internal/macworker/native/client.m b/internal/macworker/native/client.m new file mode 100644 index 00000000..383fe6c3 --- /dev/null +++ b/internal/macworker/native/client.m @@ -0,0 +1,63 @@ +// The app's inherited helper transfers its existing pipes to a network-free +// XPC service. No image paths or executable locations cross this interface. +#import "protocol.h" +#include +#include +#include + +int main(int argc, const char **argv) { + @autoreleasepool { + if (argc != 2 || (strcmp(argv[1], "heic") && strcmp(argv[1], "similarity"))) { + fprintf(stderr, "usage: picfetch-worker-client heic|similarity\n"); + return 2; + } + NSXPCConnection *connection = [[NSXPCConnection alloc] + initWithServiceName:@"io.github.frathe.picfetch.worker"]; + connection.remoteObjectInterface = [NSXPCInterface interfaceWithProtocol:@protocol(PicFetchWorker)]; + dispatch_semaphore_t done = dispatch_semaphore_create(0); + NSLock *completionLock = [NSLock new]; + __block BOOL finished = NO; + __block int status = 1; + void (^finish)(int) = ^(int result) { + [completionLock lock]; + if (!finished) { + status = result; + finished = YES; + dispatch_semaphore_signal(done); + } + [completionLock unlock]; + }; + connection.interruptionHandler = ^{ finish(1); }; + connection.invalidationHandler = ^{ finish(1); }; + + // Cancellation must allow the service to kill and reap its children + // before the caller's Wait returns. A killed broker also invalidates + // the connection; the service independently retires its process group. + signal(SIGTERM, SIG_IGN); + dispatch_source_t cancellation = dispatch_source_create(DISPATCH_SOURCE_TYPE_SIGNAL, + SIGTERM, 0, dispatch_get_global_queue(QOS_CLASS_DEFAULT, 0)); + dispatch_source_set_event_handler(cancellation, ^{ + [[connection remoteObjectProxyWithErrorHandler:^(NSError *error) { + fprintf(stderr, "worker cancellation: %s\n", error.description.UTF8String); + finish(1); + }] cancel]; + }); + dispatch_resume(cancellation); + [connection resume]; + [[connection remoteObjectProxyWithErrorHandler:^(NSError *error) { + fprintf(stderr, "worker service: %s\n", error.description.UTF8String); + finish(1); + }] runMode:[NSString stringWithUTF8String:argv[1]] + input:[NSFileHandle fileHandleWithStandardInput] + output:[NSFileHandle fileHandleWithStandardOutput] + error:[NSFileHandle fileHandleWithStandardError] + reply:^(int result) { finish(result); }]; + dispatch_semaphore_wait(done, DISPATCH_TIME_FOREVER); + dispatch_source_cancel(cancellation); + [connection invalidate]; + [completionLock lock]; + int result = status; + [completionLock unlock]; + return result; + } +} diff --git a/internal/macworker/native/protocol.h b/internal/macworker/native/protocol.h new file mode 100644 index 00000000..43ac3995 --- /dev/null +++ b/internal/macworker/native/protocol.h @@ -0,0 +1,10 @@ +#import + +@protocol PicFetchWorker +- (void)runMode:(NSString *)mode + input:(NSFileHandle *)input + output:(NSFileHandle *)output + error:(NSFileHandle *)error + reply:(void (^)(int))reply; +- (void)cancel; +@end diff --git a/internal/macworker/native/service.m b/internal/macworker/native/service.m new file mode 100644 index 00000000..3fb5af3d --- /dev/null +++ b/internal/macworker/native/service.m @@ -0,0 +1,142 @@ +// Each connection owns one child process group. Its leader is not reaped until +// remaining descendants have been killed, so cancellation never signals a PID +// that the kernel could already have reused for an unrelated process. +#import "protocol.h" +#include +#include +#include +#include +#include +#include +#include +#include +#include + +extern char **environ; + +@interface PicFetchJob : NSObject +@property(nonatomic, strong) NSLock *lock; +@property(nonatomic) pid_t child; +@property(nonatomic) BOOL submitted; +@property(nonatomic) BOOL cancelled; +@end + +@implementation PicFetchJob +- (instancetype)init { + self = [super init]; + if (self) _lock = [NSLock new]; + return self; +} +- (void)cancel { + [self.lock lock]; + self.cancelled = YES; + if (self.child > 0) kill(-self.child, SIGKILL); + [self.lock unlock]; +} +- (void)runMode:(NSString *)mode input:(NSFileHandle *)input + output:(NSFileHandle *)output error:(NSFileHandle *)error + reply:(void (^)(int))reply { + [self.lock lock]; + if (self.submitted || self.cancelled || + (![mode isEqualToString:@"heic"] && ![mode isEqualToString:@"similarity"])) { + [self.lock unlock]; reply(2); return; + } + self.submitted = YES; + // Duplicating above stderr prevents dup2 ordering from clobbering another + // transferred descriptor, including when the service began with closed stdio. + int descriptors[] = {-1, -1, -1}; + NSFileHandle *handles[] = {input, output, error}; + for (int i = 0; i < 3; i++) { + if (handles[i]) descriptors[i] = fcntl(handles[i].fileDescriptor, F_DUPFD_CLOEXEC, 3); + if (descriptors[i] < 0) { + for (int j = 0; j < i; j++) close(descriptors[j]); + [self.lock unlock]; reply(1); return; + } + } + NSURL *contents = [NSBundle.mainBundle.bundleURL URLByAppendingPathComponent:@"Contents"]; + NSString *executable = [contents URLByAppendingPathComponent:@"MacOS/picfetch-image-worker"].path; + posix_spawn_file_actions_t actions; + posix_spawnattr_t attributes; + int failure = posix_spawn_file_actions_init(&actions); + if (failure) { for (int i = 0; i < 3; i++) close(descriptors[i]); [self.lock unlock]; reply(1); return; } + failure = posix_spawnattr_init(&attributes); + if (failure) { + posix_spawn_file_actions_destroy(&actions); + for (int i = 0; i < 3; i++) close(descriptors[i]); + [self.lock unlock]; reply(1); return; + } + if (!failure) failure = posix_spawn_file_actions_adddup2(&actions, descriptors[0], STDIN_FILENO); + if (!failure) failure = posix_spawn_file_actions_adddup2(&actions, descriptors[1], STDOUT_FILENO); + if (!failure) failure = posix_spawn_file_actions_adddup2(&actions, descriptors[2], STDERR_FILENO); + if (!failure) failure = posix_spawnattr_setflags(&attributes, POSIX_SPAWN_SETPGROUP | POSIX_SPAWN_CLOEXEC_DEFAULT); + if (!failure) failure = posix_spawnattr_setpgroup(&attributes, 0); + char *arguments[] = {(char *)executable.fileSystemRepresentation, NULL}; + // The worker mode is selected by this trusted service, not caller-supplied + // environment or arguments. Source/model/cache inputs stay on the pipe. + NSString *workerFlag = [mode isEqualToString:@"heic"] ? @"PICFETCH_HEIC_WORKER=1" : @"PICFETCH_SIMILARITY_WORKER=1"; + NSUInteger count = 0; + while (environ[count]) count++; + char **environment = calloc(count + 2, sizeof(char *)); + if (!environment) failure = ENOMEM; + if (!failure) { + NSUInteger used = 0; + for (NSUInteger i = 0; i < count; i++) { + if (strncmp(environ[i], "PICFETCH_HEIC_WORKER=", 20) && + strncmp(environ[i], "PICFETCH_SIMILARITY_WORKER=", 26)) environment[used++] = environ[i]; + } + environment[used] = (char *)workerFlag.UTF8String; + failure = posix_spawn(&_child, executable.fileSystemRepresentation, &actions, + &attributes, arguments, environment); + } + free(environment); + posix_spawnattr_destroy(&attributes); + posix_spawn_file_actions_destroy(&actions); + for (int i = 0; i < 3; i++) close(descriptors[i]); + if (failure) { dprintf(error.fileDescriptor, "worker spawn: %s (%s)\n", strerror(failure), executable.UTF8String); [self.lock unlock]; reply(1); return; } + pid_t child = self.child; + [self.lock unlock]; + dispatch_async(dispatch_get_global_queue(QOS_CLASS_DEFAULT, 0), ^{ + siginfo_t info = {0}; + int result; + do { result = waitid(P_PID, child, &info, WEXITED | WNOWAIT); } while (result < 0 && errno == EINTR); + [self.lock lock]; + // The leader still belongs to us. Retire descendants before reaping. + kill(-child, SIGKILL); + int status = 0; + pid_t reaped; + do { reaped = waitpid(child, &status, 0); } while (reaped < 0 && errno == EINTR); + self.child = 0; + [self.lock unlock]; + if (result < 0 || reaped != child) dprintf(error.fileDescriptor, "worker wait: result=%d reaped=%d errno=%d\n", result, reaped, errno); + int exitStatus = 1; + if (result == 0 && reaped == child) { + if (WIFEXITED(status)) exitStatus = WEXITSTATUS(status); + else if (WIFSIGNALED(status)) exitStatus = 128 + WTERMSIG(status); + } + reply(exitStatus); + }); +} +@end + +@interface PicFetchListener : NSObject +@end +@implementation PicFetchListener +- (BOOL)listener:(NSXPCListener *)listener shouldAcceptNewConnection:(NSXPCConnection *)connection { + PicFetchJob *job = [PicFetchJob new]; + connection.exportedInterface = [NSXPCInterface interfaceWithProtocol:@protocol(PicFetchWorker)]; + connection.exportedObject = job; + connection.invalidationHandler = ^{ [job cancel]; }; + [connection resume]; + return YES; +} +@end + +int main(void) { + @autoreleasepool { + PicFetchListener *delegate = [PicFetchListener new]; + NSXPCListener *listener = NSXPCListener.serviceListener; + listener.delegate = delegate; + [listener resume]; + } + return 0; +} diff --git a/internal/similarity/client.go b/internal/similarity/client.go index 8d0146ce..d36cba59 100644 --- a/internal/similarity/client.go +++ b/internal/similarity/client.go @@ -83,6 +83,7 @@ func (d *workerDecoder) Decode(value any) error { // Client runs native inference and batch algorithms outside the viewer process. // Assets may override the installed assets directory for a local trial. type Client struct { + command func(context.Context, string) *exec.Cmd Assets string AnalysisLimits AnalysisLimits // HTTPClient configures asset downloads; analysis never uses it. @@ -136,7 +137,7 @@ func (c Client) Analyze(ctx context.Context, paths []string, controls <-chan Con } req := request{Assets: assets, FavoritesDir: c.FavoritesDir, GeneralAnalysisDir: c.GeneralAnalysisDir, GeneralAnalysisLimitBytes: c.GeneralAnalysisLimitBytes, DisableFavoriteCache: c.DisableFavoriteCache, Paths: paths, MaxEncodedBytes: imaging.MaxEncodedBytes(), AnalysisLimits: limits} req.captureHEIC(heic.FromContext(ctx)) - cmd := workerCommand(ctx, executable) + cmd := c.workerCommand(ctx, executable) cmd.Env = append(os.Environ(), workerEnvironment+"=1") return analyzeCommand(ctx, cmd, req, controls, emit) } @@ -187,7 +188,7 @@ func analyzeCommand(ctx context.Context, cmd *exec.Cmd, req request, controls <- _ = input.Close() <-writerDone if cmd.ProcessState == nil { - _ = cmd.Process.Kill() + _ = stopWorkerProcess(cmd) _ = cmd.Wait() } }() @@ -213,7 +214,7 @@ func analyzeCommand(ctx context.Context, cmd *exec.Cmd, req request, controls <- emit(event) } if err != nil && !errors.Is(err, io.EOF) { - _ = cmd.Process.Kill() + _ = stopWorkerProcess(cmd) } waitErr := cmd.Wait() if ctx.Err() != nil { @@ -332,3 +333,12 @@ func runWorker() error { } return err } + +// workerCommand keeps launch configuration on the client instance. Protocol +// tests can supply a process fixture without requiring a packaged application. +func (c Client) workerCommand(ctx context.Context, executable string) *exec.Cmd { + if c.command != nil { + return c.command(ctx, executable) + } + return workerCommand(ctx, executable) +} diff --git a/internal/similarity/heic_capture_test.go b/internal/similarity/heic_capture_test.go index 2aa8fbcf..9b5d8a5a 100644 --- a/internal/similarity/heic_capture_test.go +++ b/internal/similarity/heic_capture_test.go @@ -6,6 +6,7 @@ import ( "errors" "io" "os" + "os/exec" "os/signal" "syscall" "testing" @@ -101,11 +102,11 @@ func TestHEICAnalysisWorkerReportsBackendLoss(t *testing.T) { defer cancel() run := func() error { if kind == "finite" { - return (Client{}).Analyze(ctx, nil, nil, func(_ Event) {}) + return heicCaptureClient().Analyze(ctx, nil, nil, func(_ Event) {}) } queries := make(chan SearchQuery) close(queries) - return (Client{}).Search(ctx, SearchRequest{SessionID: 1}, queries, func(_ SearchEvent) {}) + return heicCaptureClient().Search(ctx, SearchRequest{SessionID: 1}, queries, func(_ SearchEvent) {}) } if err := run(); err != nil { t.Fatal(err) @@ -140,7 +141,7 @@ func TestHEICAnalysisWorkerCapturedPolicy(t *testing.T) { for _, snapshot := range []heic.Snapshot{{}, {Available: true, Generation: 2}, {Backend: backend, Available: false, Generation: 3}, {Backend: backend, Available: true, Generation: 4}} { ctx, cancel := context.WithTimeout(heic.WithSnapshot(context.Background(), snapshot), 10*time.Second) var actual heicCaptureReport - err := (Client{}).Analyze(ctx, nil, nil, func(event Event) { + err := heicCaptureClient().Analyze(ctx, nil, nil, func(event Event) { if err := json.Unmarshal([]byte(event.CacheWarning), &actual); err != nil { t.Error(err) } @@ -159,7 +160,7 @@ func TestHEICAnalysisWorkerCapturedPolicy(t *testing.T) { defer cancel() queries := make(chan SearchQuery, 1) var finals int - err := (Client{}).Search(ctx, SearchRequest{SessionID: 77}, queries, func(event SearchEvent) { + err := heicCaptureClient().Search(ctx, SearchRequest{SessionID: 77}, queries, func(event SearchEvent) { var actual heicCaptureReport if err := json.Unmarshal([]byte(event.CacheWarning), &actual); err != nil { t.Error(err) @@ -185,3 +186,11 @@ func TestHEICAnalysisWorkerCapturedPolicy(t *testing.T) { t.Fatalf("retained request did not settle both references: finals=%d error=%v", finals, err) } } + +func heicCaptureClient() Client { + return Client{command: func(ctx context.Context, executable string) *exec.Cmd { + cmd := exec.CommandContext(ctx, executable) + cmd.Cancel = func() error { return cmd.Process.Signal(syscall.SIGTERM) } + return cmd + }} +} diff --git a/internal/similarity/search_client.go b/internal/similarity/search_client.go index d9340321..549eb3ad 100644 --- a/internal/similarity/search_client.go +++ b/internal/similarity/search_client.go @@ -36,7 +36,7 @@ func (c Client) Search(ctx context.Context, search SearchRequest, queries <-chan search.Paths = slices.Clone(search.Paths) req := request{Assets: assets, MaxEncodedBytes: imaging.MaxEncodedBytes(), Search: &search} req.captureHEIC(heic.FromContext(ctx)) - cmd := workerCommand(ctx, executable) + cmd := c.workerCommand(ctx, executable) cmd.Env = append(os.Environ(), workerEnvironment+"=1") return searchCommand(ctx, cmd, req, queries, emit) } @@ -105,7 +105,7 @@ func searchCommand(ctx context.Context, cmd *exec.Cmd, req request, queries <-ch _ = input.Close() <-done if cmd.ProcessState == nil { - _ = cmd.Process.Kill() + _ = stopWorkerProcess(cmd) _ = cmd.Wait() } }() @@ -141,7 +141,7 @@ func searchCommand(ctx context.Context, cmd *exec.Cmd, req request, queries <-ch emit(event) } if err != nil && !errors.Is(err, io.EOF) { - _ = cmd.Process.Kill() + _ = stopWorkerProcess(cmd) } waitErr := cmd.Wait() if ctx.Err() != nil { diff --git a/internal/similarity/worker_cancel_unix_test.go b/internal/similarity/worker_cancel_unix_test.go index b7eccf73..3bfb96f1 100644 --- a/internal/similarity/worker_cancel_unix_test.go +++ b/internal/similarity/worker_cancel_unix_test.go @@ -20,9 +20,9 @@ func TestHEICAnalysisWorkerCancellationJoinsRetirement(t *testing.T) { defer cancel() var err error if kind == "finite" { - err = (Client{}).Analyze(ctx, nil, nil, func(_ Event) { cancel() }) + err = heicCaptureClient().Analyze(ctx, nil, nil, func(_ Event) { cancel() }) } else { - err = (Client{}).Search(ctx, SearchRequest{SessionID: 1}, nil, func(_ SearchEvent) { cancel() }) + err = heicCaptureClient().Search(ctx, SearchRequest{SessionID: 1}, nil, func(_ SearchEvent) { cancel() }) } if !errors.Is(err, context.Canceled) { t.Fatalf("producer cancellation: %v", err) diff --git a/internal/similarity/worker_darwin_store.go b/internal/similarity/worker_darwin_store.go new file mode 100644 index 00000000..4aa7cd5a --- /dev/null +++ b/internal/similarity/worker_darwin_store.go @@ -0,0 +1,19 @@ +//go:build darwin && appleappstore + +package similarity + +import ( + "context" + "os/exec" + "syscall" + + "github.com/frathe/picfetch/internal/macworker" +) + +func workerCommand(ctx context.Context, executable string) *exec.Cmd { + return macworker.Command(ctx, executable, macworker.Similarity) +} + +func stopWorkerProcess(cmd *exec.Cmd) error { return cmd.Process.Signal(syscall.SIGTERM) } + +func isolateWorker() error { return nil } diff --git a/internal/similarity/worker_other.go b/internal/similarity/worker_other.go index 39f29c92..333d0f77 100644 --- a/internal/similarity/worker_other.go +++ b/internal/similarity/worker_other.go @@ -1,4 +1,4 @@ -//go:build !windows && !(linux && (amd64 || arm64)) +//go:build !windows && !(linux && (amd64 || arm64)) && !(darwin && appleappstore) package similarity diff --git a/internal/similarity/worker_stop_other.go b/internal/similarity/worker_stop_other.go new file mode 100644 index 00000000..e952d5fe --- /dev/null +++ b/internal/similarity/worker_stop_other.go @@ -0,0 +1,7 @@ +//go:build !(darwin && appleappstore) + +package similarity + +import "os/exec" + +func stopWorkerProcess(cmd *exec.Cmd) error { return cmd.Process.Kill() } diff --git a/packaging/apple-app-store/README.md b/packaging/apple-app-store/README.md index 4010baa3..ee158b4f 100644 --- a/packaging/apple-app-store/README.md +++ b/packaging/apple-app-store/README.md @@ -1,7 +1,8 @@ # Mac App Store preparation The Apple channel is experimental. No submission package or signed sandbox -qualification has been completed. Retain all PicFetch features; do not use +qualification of the complete app has been completed. The native worker +fixture is qualified on this Apple Silicon host. Retain all PicFetch features; do not use temporary filesystem exceptions or weaken the analysis worker's network denial to pass a packaging check. @@ -27,6 +28,36 @@ step; signed Mach-O validation is separate work. The same library's SHA-256 changes when signed, so checking it against the original upstream digest after signing is incorrect. +## Native worker qualification + +```sh +DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer make apple-worker-test +``` + +This creates a temporary ad-hoc signed app with network permission and an +embedded XPC service without it. The service launches its own signed inherited +fixture executable from `Contents/MacOS/picfetch-image-worker`. Native sources +compile with warnings treated as errors. The checks exercise both modes, +transferred pipes, TCP/UDP denial, cancellation, leader exit with descendants, +and broker crash. No image/model/user files are used. Fixture container IDs are +separate from PicFetch's production identifier. + +Apple-tagged Go launchers use this broker. The production layout is: + +- App `Contents/MacOS/picfetch-worker-client`: inherited app sandbox helper. +- App `Contents/XPCServices/io.github.frathe.picfetch.worker.xpc`: independently + sandboxed service with no network entitlements. +- Service `Contents/MacOS/picfetch-image-worker`: signed sandbox/inherit helper. + +The service owns the process group; nested HEIC decoders must not create a new +one. Cancellation terminates the broker with SIGTERM so it can wait for child +retirement. The service kills remaining descendants before reaping the leader, +and connection invalidation independently cancels its child. + +This fixture establishes the process boundary on arm64. It does not establish +real ONNX/HEIC execution, security-scoped source grants, cache access, Intel +behavior, provisioning compatibility or acceptance by Apple. + ## Developer inputs Run the read-only prerequisite check: diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index d863958b..ef7b2634 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -58,8 +58,8 @@ The test runner and commands must be pinned before implementation starts. AC4: Public sandboxed workers preserve actual TCP/UDP denial and tracked exit. Verification: signed native helper qualification, including real HEIC and ONNX -analysis, cancelled runs, worker shutdown, and parent crash. Exact executable -layout and IPC grant transport remain design work. +analysis, cancelled runs, worker shutdown, and parent crash. The native fixture command is `DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer make apple-worker-test`. +Real image/model execution, IPC grant transport and Intel qualification remain open. AC5: Native dependencies are pinned before signing, bundled with notices, validated by signature after signing, and cannot be replaced by model-cache @@ -107,7 +107,7 @@ Budget: 0 spawns; 2 review rounds; no extra complete suite. Owner: T0 inline. Files: main private dispatch, HEIC and similarity Darwin launchers/protocols, packaged helper entitlements and native qualification tooling. -Depends: T2. +Depends: T2 for source authority; the process-boundary fixture is independent. Contract: a signed worker with no networking, documented grant transfer, and existing stop/done and offline-verification contracts. Test: worker cannot network, accepts only current source grants, and exits after @@ -232,3 +232,48 @@ Changed code scope digest (sorted path/NUL/content/NUL): SHA-256 `379807100409ec Remaining implementation is T2/T3 and the signed portion of T4/T5. Signing identity/profile/account inputs have been requested from Ronin. The foundation is reviewable; it does not establish a full-feature Store app's readiness. + +## Continuation evidence — XPC worker boundary + +Base revision: `3883bf5`; work below is the next implementation slice. +T3 process boundary is implemented; full AC3/AC4/AC5 are still open. + +- Full Xcode compiler now works after Ronin accepted its license locally. + Preflight checks `xcrun clang --version` as well as `xcodebuild -version`. +- A separately sandboxed XPC service owns the image-worker process group. + The inherited image helper must be inside the service bundle: the first + fixture failed `posix_spawn` with EPERM from the parent app's MacOS directory; + moving it into service Contents/MacOS passed without extra entitlements. +- `make apple-worker-test` on macOS arm64 passes both modes, pipe transfer, + TCP/UDP EPERM, invalid mode refusal, SIGTERM cancellation, SIGKILL broker + crash, and normal leader exit retiring descendants. Leader reaping is checked + after broker completion; descendant pipe EOF is observed with bounded waits. +- Negative verification added network.client to the temporary service and saw + the guard fail on TCP EINPROGRESS/UDP success; restored entitlements pass. + The first negative run timed out on blocking TCP, so the fixture now uses + nonblocking sockets to discriminate policy denial without external responses. +- Apple-tagged HEIC/similarity launchers select the fixed bundled broker. + Native completion is synchronized; descriptor duplication prevents dup2 + collisions; unreaped leaders prevent PID reuse during group retirement. + Nested HEIC retains the XPC-owned group. Existing process-group fixtures + explicitly own their group and retain descendant-retirement coverage. +- Protocol tests use an instance-owned command factory so ordinary Go test + executables need no signed app bundle. No isolation test is skipped. +- `go test -tags no_emoji,nodynamic,appleappstore ./internal/macworker + ./internal/heic ./internal/similarity` passes. The same ordinary-tag command + passes. Apple-tagged focused `go vet` passes. +- GoLand get_file_problems, errorsOnly=false, reviewed all 20 changed code files + in this slice. One shell warning about empty CDPATH assignment was corrected + and re-inspected clean. Objective-C/C also compile with -Wall -Wextra -Werror; + GoLand native-language inspection depth is limited by installed language support. + IDE build reports success with limited diagnostics, so CLI tests/vet remain + the build evidence. Qodana CI/profile scan is not established by these checks. +- New command_test.go is exactly excluded from Qodana duplicate inspection. + No top-level root UI test or new dependency was added. + +Honest limit: the fixture establishes native containment and lifecycle, not a +complete sandboxed PicFetch app. Security-scoped source/cache authority, signed +runtime validation, production packaging, Intel and final Store qualification +remain required. No upload or release has been performed. + +T3 budget so far: 0 spawns; 2 inline review rounds; no complete suite rerun. diff --git a/qodana.yaml b/qodana.yaml index 71ad77e8..451275df 100644 --- a/qodana.yaml +++ b/qodana.yaml @@ -146,6 +146,7 @@ exclude: - "internal/heic/metadata_test.go" - "internal/heic/native_darwin_test.go" - "internal/heic/primaryfixture_test.go" + - "internal/macworker/command_test.go" - "internal/heic/process_linux_test.go" - "internal/heic/process_unix_test.go" - "internal/heic/restrict_linux_test.go" diff --git a/scripts/apple-store-preflight.sh b/scripts/apple-store-preflight.sh index 350f2c4e..84408f0f 100644 --- a/scripts/apple-store-preflight.sh +++ b/scripts/apple-store-preflight.sh @@ -21,6 +21,10 @@ else fail 'Install full Xcode, complete its first launch, or set APPLE_STORE_DEVELOPER_DIR.' fi +if ! /usr/bin/xcrun clang --version >/dev/null; then + fail 'Complete full Xcode first launch and review/accept its license locally with xcodebuild -license.' +fi + for tool in /usr/bin/codesign /usr/bin/productbuild /usr/sbin/pkgutil /usr/bin/security; do if [ ! -x "$tool" ]; then fail "Apple tool $tool is unavailable." diff --git a/scripts/macworkerqualify/check.py b/scripts/macworkerqualify/check.py new file mode 100644 index 00000000..2d348cdc --- /dev/null +++ b/scripts/macworkerqualify/check.py @@ -0,0 +1,79 @@ +"""Exercise the actual signed broker/service and transferred process pipes.""" +import errno +import json +import os +from pathlib import Path +import selectors +import signal +import subprocess +import sys + +broker = str(Path(sys.argv[1]).resolve()) + + +def command(mode): + return [broker, mode] + + +def verify_worker(record, mode): + assert record["tcp"] in (errno.EPERM, errno.EACCES), record + assert record["udp"] in (errno.EPERM, errno.EACCES), record + assert record["heic"] == (mode == "heic"), record + assert record["similarity"] == (mode == "similarity"), record + + +def assert_reaped(pid): + try: + os.kill(pid, 0) + except ProcessLookupError: + return + raise AssertionError(f"worker {pid} survived broker completion") + + +for mode in ("heic", "similarity"): + result = subprocess.run(command(mode), input="echo\n", capture_output=True, + text=True, timeout=15) + assert result.returncode == 0, (result.returncode, result.stdout, result.stderr) + record = json.loads(result.stdout) + verify_worker(record, mode) + assert_reaped(record["pid"]) + print(f"PASS {mode}: transferred pipes, mode, TCP/UDP denial and reaped exit") + +result = subprocess.run(command("invalid"), input="echo\n", capture_output=True, + text=True, timeout=15) +assert result.returncode == 2, result +assert not result.stdout, result +print("PASS invalid mode refused before worker launch") + +for action, termination in (("hold", signal.SIGTERM), ("spawn", signal.SIGTERM), + ("spawn", signal.SIGKILL)): + process = subprocess.Popen(command("similarity"), stdin=subprocess.PIPE, + stdout=subprocess.PIPE, stderr=subprocess.PIPE, + text=True) + try: + process.stdin.write(action + "\n") + process.stdin.flush() + with selectors.DefaultSelector() as ready: + ready.register(process.stdout, selectors.EVENT_READ) + assert ready.select(timeout=15), "worker did not publish readiness" + record = json.loads(process.stdout.readline()) + verify_worker(record, "similarity") + process.send_signal(termination) + output, error = process.communicate(timeout=15) + expected = -signal.SIGKILL if termination == signal.SIGKILL else 128 + signal.SIGKILL + assert process.returncode == expected, (output, error, process.returncode) + assert_reaped(record["pid"]) + # communicate observes EOF on both inherited pipes as well as broker + # exit; a live descendant holding those handles would time out. + print(f"PASS {action}/{termination.name}: leader reaped and descendant pipes closed") + finally: + if process.poll() is None: + process.kill() + process.communicate(timeout=15) + +result = subprocess.run(command("similarity"), input="spawnexit\n", + capture_output=True, text=True, timeout=15, check=True) +record = json.loads(result.stdout) +verify_worker(record, "similarity") +assert_reaped(record["pid"]) +print("PASS normal leader exit retires descendants before broker completion") diff --git a/scripts/macworkerqualify/fixture.c b/scripts/macworkerqualify/fixture.c new file mode 100644 index 00000000..f527cd85 --- /dev/null +++ b/scripts/macworkerqualify/fixture.c @@ -0,0 +1,50 @@ +// Qualification-only worker: no application data or model is used. +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static int denial(int type) { + int fd = socket(AF_INET, type, 0); + if (fd < 0) return errno; + if (fcntl(fd, F_SETFL, O_NONBLOCK) < 0) { + int failure = errno; close(fd); return failure; + } + struct sockaddr_in address = {0}; + address.sin_family = AF_INET; + address.sin_port = htons(443); + inet_pton(AF_INET, "192.0.2.1", &address.sin_addr); + int result = connect(fd, (struct sockaddr *)&address, sizeof address); + int failure = result < 0 ? errno : 0; + close(fd); + return failure; +} + +int main(void) { + char command[128]; + if (!fgets(command, sizeof command, stdin)) return 3; + int tcp = denial(SOCK_STREAM), udp = denial(SOCK_DGRAM); + pid_t descendant = 0; + if (!strcmp(command, "spawn\n") || !strcmp(command, "spawnexit\n")) { + descendant = fork(); + if (descendant < 0) return 4; + if (descendant == 0) { + signal(SIGTERM, SIG_IGN); + for (;;) pause(); + } + } + printf("{\"pid\":%d,\"descendant\":%d,\"tcp\":%d,\"udp\":%d,\"heic\":%d,\"similarity\":%d}\n", + getpid(), descendant, tcp, udp, getenv("PICFETCH_HEIC_WORKER") != NULL, + getenv("PICFETCH_SIMILARITY_WORKER") != NULL); + fflush(stdout); + if (!strcmp(command, "hold\n") || !strcmp(command, "spawn\n")) { + signal(SIGTERM, SIG_IGN); + for (;;) pause(); + } + return 0; +} diff --git a/scripts/macworkerqualify/run.sh b/scripts/macworkerqualify/run.sh new file mode 100755 index 00000000..9fd80183 --- /dev/null +++ b/scripts/macworkerqualify/run.sh @@ -0,0 +1,42 @@ +#!/bin/sh +# Qualify the native XPC process boundary with an ad-hoc signed fixture. +set -eu +repo=$(CDPATH='' cd -- "$(dirname -- "$0")/../.." && pwd) +stage=$(mktemp -d /private/tmp/picfetch-worker.XXXXXX) +trap 'rm -rf "$stage"' EXIT HUP INT TERM +app="$stage/Qualification.app" +service="$app/Contents/XPCServices/io.github.frathe.picfetch.worker.xpc" +mkdir -p "$app/Contents/MacOS" "$service/Contents/MacOS" +cat > "$app/Contents/Info.plist" <<'PLIST' + +CFBundleIdentifierio.github.frathe.picfetch.workerqualification +CFBundleExecutablepicfetch-worker-client +CFBundlePackageTypeAPPL + +PLIST +cat > "$service/Contents/Info.plist" <<'PLIST' + +CFBundleIdentifierio.github.frathe.picfetch.worker +CFBundleExecutableworker +CFBundlePackageTypeXPC! +XPCServiceServiceTypeApplication + +PLIST +for kind in app service helper; do + cat > "$stage/$kind.plist" <<'PLIST' + +com.apple.security.app-sandbox +PLIST + case "$kind" in + app) echo 'com.apple.security.network.client' >> "$stage/$kind.plist" ;; + helper) echo 'com.apple.security.inherit' >> "$stage/$kind.plist" ;; + esac + echo '' >> "$stage/$kind.plist" +done +xcrun clang -fobjc-arc -Wall -Wextra -Werror -framework Foundation "$repo/internal/macworker/native/client.m" -o "$app/Contents/MacOS/picfetch-worker-client" +xcrun clang -fobjc-arc -Wall -Wextra -Werror -framework Foundation "$repo/internal/macworker/native/service.m" -o "$service/Contents/MacOS/worker" +xcrun clang -Wall -Wextra -Werror "$repo/scripts/macworkerqualify/fixture.c" -o "$service/Contents/MacOS/picfetch-image-worker" +codesign --force --sign - --entitlements "$stage/helper.plist" "$service/Contents/MacOS/picfetch-image-worker" +codesign --force --sign - --entitlements "$stage/service.plist" "$service" +codesign --force --sign - --entitlements "$stage/app.plist" "$app" +python3 "$repo/scripts/macworkerqualify/check.py" "$app/Contents/MacOS/picfetch-worker-client" diff --git a/todos.md b/todos.md index 288f149d..6bb3e63b 100644 --- a/todos.md +++ b/todos.md @@ -18,11 +18,13 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Add an immutable Apple Store channel, localized update/repair messages, verified pre-sign runtime staging and a read-only developer-input preflight. - [ ] Qualify persistent security-scoped permissions for opening, saved sessions, Favorites, writes and workers. -- [ ] Replace the Store worker launch with public sandbox-compatible helpers while retaining verified network denial. +- [x] Implement the Apple Store XPC worker boundary and qualify native fixture TCP/UDP denial, cancellation and broker crash on Apple Silicon. +- [ ] Qualify real HEIC/ONNX execution and granted source/cache access through that worker boundary on both Mac architectures. - [ ] Bundle pinned native runtimes and notices, and validate signed Mach-O code without comparing its post-sign bytes with upstream archive hashes. - [ ] Add the signed Store packaging route and validate the final artifact on Intel and Apple Silicon. - [ ] Complete privacy/dependency/export assessment, listing metadata and Apple submission validation. -- [ ] Obtain full Xcode, developer Team ID, Store application/installer certificates and provisioning profile for signed qualification. +- [x] Enable the full Xcode compiler after local license acceptance. +- [ ] Obtain developer Team ID, Store application/installer certificates and provisioning profile for final signed qualification. ## Deferred From a29313757f4fe1d16a2160452e3ab159ecc81674 Mon Sep 17 00:00:00 2001 From: frathe Date: Tue, 29 Sep 2026 21:31:20 +0200 Subject: [PATCH 03/63] Qualify sandbox bookmark transfer to image workers --- ARCHITECTURE.md | 3 +- packaging/apple-app-store/README.md | 4 +- plans/2026-09-29-apple-app-store.md | 53 +++++++++++++++++++++ scripts/macworkerqualify/check.py | 11 +++++ scripts/macworkerqualify/driver.m | 46 +++++++++++++++++++ scripts/macworkerqualify/fixture.c | 50 -------------------- scripts/macworkerqualify/fixture.m | 71 +++++++++++++++++++++++++++++ scripts/macworkerqualify/run.sh | 9 ++-- 8 files changed, 192 insertions(+), 55 deletions(-) create mode 100644 scripts/macworkerqualify/driver.m delete mode 100644 scripts/macworkerqualify/fixture.c create mode 100644 scripts/macworkerqualify/fixture.m diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 67c17d94..858c4351 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -782,7 +782,8 @@ stdin/stdout/stderr handles and a fixed HEIC/similarity mode; `client.m` forward cancellation; `service.m` owns a process group and reaps its leader after killing descendants. The image worker lives inside the XPC service's `Contents/MacOS`. `scripts/macworkerqualify` builds an ad-hoc signed native fixture to exercise -actual TCP/UDP denial, pipe transfer, cancellation, broker crash and group exit. +actual TCP/UDP denial, pipe transfer, cancellation, broker crash, group exit +and implicit bookmark transfer from a separate app container. Source grant transport and final Store signing remain qualification work. ### `internal/heic` diff --git a/packaging/apple-app-store/README.md b/packaging/apple-app-store/README.md index ee158b4f..31e2ec81 100644 --- a/packaging/apple-app-store/README.md +++ b/packaging/apple-app-store/README.md @@ -39,7 +39,9 @@ embedded XPC service without it. The service launches its own signed inherited fixture executable from `Contents/MacOS/picfetch-image-worker`. Native sources compile with warnings treated as errors. The checks exercise both modes, transferred pipes, TCP/UDP denial, cancellation, leader exit with descendants, -and broker crash. No image/model/user files are used. Fixture container IDs are +and broker crash. A generated file in the fixture app container is unreadable +from the worker before bookmark resolution, readable afterward, while an +ungranted sibling remains denied. No image/model/user files are used. Fixture container IDs are separate from PicFetch's production identifier. Apple-tagged Go launchers use this broker. The production layout is: diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index ef7b2634..fcb1843f 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -277,3 +277,56 @@ runtime validation, production packaging, Intel and final Store qualification remain required. No upload or release has been performed. T3 budget so far: 0 spawns; 2 inline review rounds; no complete suite rerun. + +### Next authority design and native feasibility evidence + +Base revision: `2ec93e7`. T2/T3 source authority remains open. + +Apple's current [file-access guidance](https://developer.apple.com/documentation/security/accessing-files-from-the-macos-app-sandbox) +distinguishes app-scoped bookmarks stored for relaunch from bookmarks created +with options 0 for interprocess access. Resolve temporary bookmarks in the +process doing the actual file I/O and balance its implicit access with Stop. +Persistent bookmarks cannot simply be sent to a differently signed XPC helper. + +The native qualification now includes the production layout's inherited broker. +A fixture app creates one file and an unrelated sibling in its own container, +then sends an options-0 bookmark through the existing input pipe. The separately +sandboxed image worker is denied the file before resolving the bookmark, +reads its exact content after resolution, and remains denied the sibling. +Both TCP/UDP checks and retirement checks still pass. Native sources compile +with -Wall -Wextra -Werror; strict ad-hoc bundle signature verification passes. +GoLand inspections of driver.m, fixture.m, check.py and run.sh return no findings. +Negative verification replaced the transferred bookmark with invalid bytes; +only the source-access guard failed (granted=0); the fixture was restored. + +Planned contract before source implementation: + +- An immutable source URI carries persistent bookmark data and the relative + child identity when its authority is a selected directory. Paths never + manufacture authority. Folder scanning propagates only that captured grant. +- Each file I/O operation resolves/acquires its source scope, retains it through + actual read/write/native completion, and releases it exactly once. The URI + carries no indefinitely retained native scope, and features share no global + mutable permission registry. Capture/resolution must happen on tracked workers. +- Session and Favorite records retain grants aligned with the complete recorded + membership and occurrences. Missing, stale or inaccessible authority needs + reselection; it must not be interpreted as an empty successful collection. +- Analysis holds its parent source/cache leases for the whole existing Analyze/ + Search call. Add bounded temporary bookmark bytes to the existing request + protocol; the image worker resolves and holds them through all worker exit + and nested decoder retirement. No new broker/XPC side channel is needed. +- File-only grants do not imply parent-directory rights. Sibling browsing and + atomic source replacement need verified folder authority or a deliberate + native authorization flow; do not silently broaden the selected scope. + +Focused lifetime tests must cover acquisition failure rollback, cancelled reads, +exact-once release, directory-child inheritance/traversal rejection, restored +membership/occurrences, and delayed native completion. Their package-specific +commands and source boundary map will be pinned before those edits. + +A scratch native picker probe is open for user-assisted qualification because +both computer-use accessibility requests timed out. It uses separate identifier +io.github.frathe.picfetch.grantqualification and only the generated +.scratch/apple-store-grant-probe/selected folder. User selection/relaunch/moved +folder evidence is pending. No production bookmark readiness is inferred from +the successful temporary interprocess grant fixture. diff --git a/scripts/macworkerqualify/check.py b/scripts/macworkerqualify/check.py index 2d348cdc..b391145d 100644 --- a/scripts/macworkerqualify/check.py +++ b/scripts/macworkerqualify/check.py @@ -77,3 +77,14 @@ def assert_reaped(pid): verify_worker(record, "similarity") assert_reaped(record["pid"]) print("PASS normal leader exit retires descendants before broker completion") + +result = subprocess.run(command("grant"), input="", capture_output=True, + text=True, timeout=15) +assert result.returncode == 0, (result.returncode, result.stdout, result.stderr) +record = json.loads(result.stdout) +verify_worker(record, "similarity") +assert record["before"] in (errno.EPERM, errno.EACCES), record +assert record["granted"] == 1, record +assert record["sibling"] in (errno.EPERM, errno.EACCES), record +assert_reaped(record["pid"]) +print("PASS bookmark transfer: private source denied before resolution, allowed after; sibling denied") diff --git a/scripts/macworkerqualify/driver.m b/scripts/macworkerqualify/driver.m new file mode 100644 index 00000000..4b419cc7 --- /dev/null +++ b/scripts/macworkerqualify/driver.m @@ -0,0 +1,46 @@ +// Qualification app: its own container is private from the worker service. +// A temporary bookmark transfers only one fixture file, without a file picker. +#import +#include +#include +#include + +int main(int argc, char **argv) { + @autoreleasepool { + if (argc != 2) return 2; + NSString *broker = [NSBundle.mainBundle.bundlePath + stringByAppendingPathComponent:@"Contents/MacOS/picfetch-worker-client"]; + const char *mode = argv[1]; + if (!strcmp(mode, "grant")) { + NSURL *directory = [NSFileManager.defaultManager URLsForDirectory:NSDocumentDirectory + inDomains:NSUserDomainMask].firstObject; + if (![NSFileManager.defaultManager createDirectoryAtURL:directory + withIntermediateDirectories:YES attributes:nil error:NULL]) return 4; + NSURL *source = [directory URLByAppendingPathComponent:@"picfetch-grant-fixture.dat"]; + NSURL *sibling = [directory URLByAppendingPathComponent:@"picfetch-ungranted-fixture.dat"]; + NSData *content = [@"fixture" dataUsingEncoding:NSUTF8StringEncoding]; + if (![content writeToURL:source options:NSDataWritingAtomic error:NULL] || + ![content writeToURL:sibling options:NSDataWritingAtomic error:NULL]) return 4; + NSData *bookmark = [source bookmarkDataWithOptions:0 includingResourceValuesForKeys:nil + relativeToURL:nil error:NULL]; + if (!bookmark) return 4; + NSData *payload = [NSJSONSerialization dataWithJSONObject:@{ + @"path": source.path, @"sibling": sibling.path, + @"bookmark": [bookmark base64EncodedStringWithOptions:0]} + options:0 error:NULL]; + // Keep this fixture write within PIPE_BUF; production uses its + // existing bounded request stream rather than a pre-filled pipe. + if (!payload || payload.length + 1 > 4096) return 4; + int input[2]; + if (pipe(input)) return 4; + if (write(input[1], payload.bytes, payload.length) != (ssize_t)payload.length || + write(input[1], "\n", 1) != 1 || dup2(input[0], STDIN_FILENO) < 0) return 4; + close(input[0]); close(input[1]); + mode = "similarity"; + } + char *arguments[] = {(char *)broker.fileSystemRepresentation, (char *)mode, NULL}; + execv(broker.fileSystemRepresentation, arguments); + perror("qualification broker"); + return 4; + } +} diff --git a/scripts/macworkerqualify/fixture.c b/scripts/macworkerqualify/fixture.c deleted file mode 100644 index f527cd85..00000000 --- a/scripts/macworkerqualify/fixture.c +++ /dev/null @@ -1,50 +0,0 @@ -// Qualification-only worker: no application data or model is used. -#include -#include -#include -#include -#include -#include -#include -#include -#include - -static int denial(int type) { - int fd = socket(AF_INET, type, 0); - if (fd < 0) return errno; - if (fcntl(fd, F_SETFL, O_NONBLOCK) < 0) { - int failure = errno; close(fd); return failure; - } - struct sockaddr_in address = {0}; - address.sin_family = AF_INET; - address.sin_port = htons(443); - inet_pton(AF_INET, "192.0.2.1", &address.sin_addr); - int result = connect(fd, (struct sockaddr *)&address, sizeof address); - int failure = result < 0 ? errno : 0; - close(fd); - return failure; -} - -int main(void) { - char command[128]; - if (!fgets(command, sizeof command, stdin)) return 3; - int tcp = denial(SOCK_STREAM), udp = denial(SOCK_DGRAM); - pid_t descendant = 0; - if (!strcmp(command, "spawn\n") || !strcmp(command, "spawnexit\n")) { - descendant = fork(); - if (descendant < 0) return 4; - if (descendant == 0) { - signal(SIGTERM, SIG_IGN); - for (;;) pause(); - } - } - printf("{\"pid\":%d,\"descendant\":%d,\"tcp\":%d,\"udp\":%d,\"heic\":%d,\"similarity\":%d}\n", - getpid(), descendant, tcp, udp, getenv("PICFETCH_HEIC_WORKER") != NULL, - getenv("PICFETCH_SIMILARITY_WORKER") != NULL); - fflush(stdout); - if (!strcmp(command, "hold\n") || !strcmp(command, "spawn\n")) { - signal(SIGTERM, SIG_IGN); - for (;;) pause(); - } - return 0; -} diff --git a/scripts/macworkerqualify/fixture.m b/scripts/macworkerqualify/fixture.m new file mode 100644 index 00000000..ea9bae0d --- /dev/null +++ b/scripts/macworkerqualify/fixture.m @@ -0,0 +1,71 @@ +// Qualification-only worker: no application data or model is used. +#import +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static int denial(int type) { + int fd = socket(AF_INET, type, 0); + if (fd < 0) return errno; + if (fcntl(fd, F_SETFL, O_NONBLOCK) < 0) { + int failure = errno; close(fd); return failure; + } + struct sockaddr_in address = {0}; + address.sin_family = AF_INET; + address.sin_port = htons(443); + inet_pton(AF_INET, "192.0.2.1", &address.sin_addr); + int result = connect(fd, (struct sockaddr *)&address, sizeof address); + int failure = result < 0 ? errno : 0; + close(fd); + return failure; +} + +int main(void) { + @autoreleasepool { + char command[4096]; + if (!fgets(command, sizeof command, stdin)) return 3; + int before = 0, granted = 0, siblingDenied = 0; + NSURL *location = nil; + if (command[0] == '{') { + NSData *json = [[NSString stringWithUTF8String:command] dataUsingEncoding:NSUTF8StringEncoding]; + NSDictionary *record = [NSJSONSerialization JSONObjectWithData:json options:0 error:NULL]; + int fd = open([record[@"path"] fileSystemRepresentation], O_RDONLY); + before = fd < 0 ? errno : 0; + if (fd >= 0) close(fd); + NSData *bookmark = [[NSData alloc] initWithBase64EncodedString:record[@"bookmark"] options:0]; + location = [NSURL URLByResolvingBookmarkData:bookmark options:NSURLBookmarkResolutionWithoutUI + relativeToURL:nil bookmarkDataIsStale:NULL error:NULL]; + NSData *data = location ? [NSData dataWithContentsOfURL:location] : nil; + granted = [data isEqualToData:[@"fixture" dataUsingEncoding:NSUTF8StringEncoding]]; + fd = open([record[@"sibling"] fileSystemRepresentation], O_RDONLY); + siblingDenied = fd < 0 ? errno : 0; + if (fd >= 0) close(fd); + } + int tcp = denial(SOCK_STREAM), udp = denial(SOCK_DGRAM); + pid_t descendant = 0; + if (!strcmp(command, "spawn\n") || !strcmp(command, "spawnexit\n")) { + descendant = fork(); + if (descendant < 0) return 4; + if (descendant == 0) { + signal(SIGTERM, SIG_IGN); + for (;;) pause(); + } + } + printf("{\"pid\":%d,\"descendant\":%d,\"tcp\":%d,\"udp\":%d,\"heic\":%d,\"similarity\":%d,\"before\":%d,\"granted\":%d,\"sibling\":%d}\n", + getpid(), descendant, tcp, udp, getenv("PICFETCH_HEIC_WORKER") != NULL, + getenv("PICFETCH_SIMILARITY_WORKER") != NULL, before, granted, siblingDenied); + fflush(stdout); + if (!strcmp(command, "hold\n") || !strcmp(command, "spawn\n")) { + signal(SIGTERM, SIG_IGN); + for (;;) pause(); + } + if (location) [location stopAccessingSecurityScopedResource]; + return 0; + } +} diff --git a/scripts/macworkerqualify/run.sh b/scripts/macworkerqualify/run.sh index 9fd80183..7f1f66e1 100755 --- a/scripts/macworkerqualify/run.sh +++ b/scripts/macworkerqualify/run.sh @@ -10,7 +10,7 @@ mkdir -p "$app/Contents/MacOS" "$service/Contents/MacOS" cat > "$app/Contents/Info.plist" <<'PLIST' CFBundleIdentifierio.github.frathe.picfetch.workerqualification -CFBundleExecutablepicfetch-worker-client +CFBundleExecutablequalification CFBundlePackageTypeAPPL PLIST @@ -35,8 +35,11 @@ PLIST done xcrun clang -fobjc-arc -Wall -Wextra -Werror -framework Foundation "$repo/internal/macworker/native/client.m" -o "$app/Contents/MacOS/picfetch-worker-client" xcrun clang -fobjc-arc -Wall -Wextra -Werror -framework Foundation "$repo/internal/macworker/native/service.m" -o "$service/Contents/MacOS/worker" -xcrun clang -Wall -Wextra -Werror "$repo/scripts/macworkerqualify/fixture.c" -o "$service/Contents/MacOS/picfetch-image-worker" +xcrun clang -fobjc-arc -Wall -Wextra -Werror -framework Foundation "$repo/scripts/macworkerqualify/driver.m" -o "$app/Contents/MacOS/qualification" +xcrun clang -fobjc-arc -Wall -Wextra -Werror -framework Foundation "$repo/scripts/macworkerqualify/fixture.m" -o "$service/Contents/MacOS/picfetch-image-worker" +codesign --force --sign - --entitlements "$stage/helper.plist" "$app/Contents/MacOS/picfetch-worker-client" codesign --force --sign - --entitlements "$stage/helper.plist" "$service/Contents/MacOS/picfetch-image-worker" codesign --force --sign - --entitlements "$stage/service.plist" "$service" codesign --force --sign - --entitlements "$stage/app.plist" "$app" -python3 "$repo/scripts/macworkerqualify/check.py" "$app/Contents/MacOS/picfetch-worker-client" +codesign --verify --strict "$app" +python3 "$repo/scripts/macworkerqualify/check.py" "$app/Contents/MacOS/qualification" From 03a2eaf47439fec04e09eb8feda730cc49e87812 Mon Sep 17 00:00:00 2001 From: frathe Date: Tue, 29 Sep 2026 22:37:38 +0200 Subject: [PATCH 04/63] Preserve scoped file access through scans and saved collections --- ARCHITECTURE.md | 22 ++- internal/favstore/membership.go | 84 +++++++++-- internal/favstore/membership_test.go | 9 ++ internal/favstore/ownership.go | 14 +- internal/favstore/saving.go | 10 +- internal/favstore/saving_test.go | 67 +++++++++ internal/fileaccess/manifest.go | 114 ++++++++++++++ internal/fileaccess/manifest_test.go | 66 ++++++++ internal/fileaccess/native_darwin.go | 68 +++++++++ internal/fileaccess/native_other.go | 25 ++++ internal/fileaccess/reader.go | 36 +++++ internal/fileaccess/source.go | 157 +++++++++++++++++++ internal/fileaccess/source_test.go | 200 +++++++++++++++++++++++++ internal/filepicker/darwin.go | 119 +++++++++++---- internal/filepicker/filepicker.go | 32 +++- internal/filepicker/filepicker_test.go | 29 ++++ internal/filescan/authority_test.go | 89 +++++++++++ internal/filescan/filescan.go | 50 +++++-- internal/imaging/loader.go | 4 +- internal/macworker/command_test.go | 8 +- internal/session/session.go | 33 +++- internal/session/session_test.go | 80 +++++++++- plans/2026-09-29-apple-app-store.md | 132 ++++++++++++++++ qodana.yaml | 3 + todos.md | 1 + 25 files changed, 1382 insertions(+), 70 deletions(-) create mode 100644 internal/fileaccess/manifest.go create mode 100644 internal/fileaccess/manifest_test.go create mode 100644 internal/fileaccess/native_darwin.go create mode 100644 internal/fileaccess/native_other.go create mode 100644 internal/fileaccess/reader.go create mode 100644 internal/fileaccess/source.go create mode 100644 internal/fileaccess/source_test.go create mode 100644 internal/filescan/authority_test.go diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 858c4351..5d58ee6d 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -773,6 +773,22 @@ Help's `privacy.go` uses the same document-injection pattern through policy opens a separate, word-wrapped Markdown singleton with vertical scrolling, Escape-to-close, and the same command-admission and shutdown guards. +### `internal/fileaccess` + +Immutable source URI authority and bounded native acquisition. `source.go` +restores/snapshots bookmark records, propagates selected directory grants to +validated children and resolves an independent scope for each operation. +`reader.go` holds the scope through the actual ReadCloser.Close; imaging's +canonical read path uses it. `native_darwin.go` resolves explicit scopes in +Apple Store builds; `native_other.go` preserves ordinary URI behavior. +`manifest.go` stores each distinct scope once while retaining every ordered +source occurrence. Session and Favorite persistence validate the complete +manifest against their saved membership before publishing restored sources. +The Darwin open picker captures native URL bookmarks on its tracked chooser +worker; folder scans pass captured directory authority to children. Native +save/Open With/drop capture, permission reselection, writes, source versions +and analysis transfer remain open in the active Apple Store plan. + ### `internal/macworker` Mac App Store worker broker factory and native XPC service. `command.go` selects @@ -889,7 +905,7 @@ unavailable. | File | Responsibility | |------|----------------| | `favstore.go` | `Save` / `Load` / `Count` / `DefaultDir`; trash-backed remove. | -| `membership.go` | One strict numeric-position decoder, cancellable reads, validated encoding and the exact 64 MiB read/write definition limit. | +| `membership.go` | Strict numeric-position membership with optional `$access` scope manifest, cancellable reads, complete authority/membership validation and the exact 64 MiB read/write definition limit. | | `ownership.go` | `Open` / `Definition`, captured directory/list `Owner`, short-lived `Access`, permanent observed retirement, full-list fingerprint and captured relative-path interpretation. `Observe` supports unknown-membership maintenance. | | `inventory.go` | Cancellable 64-entry enumeration, complete validation with scoped retained membership, healthy/unknown outcomes and enumeration completeness. | | `listing.go` | Batched Favorite discovery and complete validated counts, retaining bare names for unreadable definitions; no partial cancelled listing. | @@ -917,7 +933,7 @@ Last-open file set via Fyne’s app-scoped cache. | File | Responsibility | |------|----------------| -| `session.go` | `Save`, `Load`. | +| `session.go` | `Save`, `Load`; legacy path lists plus an optional deduplicated scope manifest aligned to complete occurrence order. | ### `internal/update` @@ -1134,6 +1150,8 @@ viewer's handler and flushes in the same critical section. Recursive image gather for drop/open, plus a non-recursive sibling listing when the user opened a single file. Recorded Favorite/session replay shares the bounded admission loop without discovering directories or collapsing occurrences. +Listings acquire operation-bound scopes, preserve directory grants in descendants, +and refuse sibling discovery beyond a selected file or directory root. | File | Responsibility | |------|----------------| diff --git a/internal/favstore/membership.go b/internal/favstore/membership.go index 3fa84125..027c6e8f 100644 --- a/internal/favstore/membership.go +++ b/internal/favstore/membership.go @@ -12,6 +12,8 @@ import ( "strings" "fyne.io/fyne/v2" + + "github.com/frathe/picfetch/internal/fileaccess" ) // MaxDefinitionBytes bounds the complete encoded Favorite definition. @@ -22,7 +24,7 @@ var ErrDefinitionTooLarge = errors.New("favorite definition exceeds 64 MiB") func encodeMembership(ctx context.Context, files []fyne.URI) ([]byte, []string, error) { paths := make([]string, len(files)) - list := make(map[string]string, len(files)) + list := make(map[string]any, len(files)) rawBytes := 0 for i, file := range files { if err := ctx.Err(); err != nil { @@ -41,6 +43,24 @@ func encodeMembership(ctx context.Context, files []fyne.URI) ([]byte, []string, rawBytes += len(source) paths[i], list[strconv.Itoa(i)] = source, source } + var scoped bool + for _, file := range files { + scoped = scoped || fileaccess.HasScope(file) + } + if scoped { + manifest, err := fileaccess.Pack(ctx, files) + if err != nil { + return nil, nil, err + } + for _, scope := range manifest.Scopes { + if len(scope.Bookmark) > MaxDefinitionBytes-rawBytes { + return nil, nil, ErrDefinitionTooLarge + } + rawBytes += len(scope.Bookmark) + } + list["$access"] = manifest + } + data, err := json.Marshal(list) if err != nil { return nil, nil, err @@ -76,47 +96,64 @@ func (r contextReader) Read(p []byte) (int, error) { // decodeMembership preserves occurrences while rejecting ambiguous positions. // Token decoding matters: unmarshalling a map silently accepts duplicate keys. -func decodeMembership(ctx context.Context, data []byte) ([]string, error) { +func decodeMembership(ctx context.Context, data []byte) ([]string, []fyne.URI, error) { decoder := json.NewDecoder(bytes.NewReader(data)) token, err := decoder.Token() if err != nil || token != json.Delim('{') { - return nil, errors.New("favorite definition must be an object") + return nil, nil, errors.New("favorite definition must be an object") } positions := map[int]string{} + var access *fileaccess.Manifest for decoder.More() { if err := ctx.Err(); err != nil { - return nil, err + return nil, nil, err } token, err := decoder.Token() if err != nil { - return nil, err + return nil, nil, err } key, ok := token.(string) if !ok { - return nil, errors.New("invalid favorite index") + return nil, nil, errors.New("invalid favorite index") + } + if key == "$access" { + if access != nil { + return nil, nil, errors.New("duplicate favorite authority") + } + var manifest fileaccess.Manifest + if err := decoder.Decode(&manifest); err != nil { + return nil, nil, err + } + if manifest.Sources == nil { + return nil, nil, errors.New("favorite authority has no membership") + } + access = &manifest + continue } index, err := strconv.Atoi(key) if err != nil || index < 0 { - return nil, fmt.Errorf("invalid file index %q", key) + return nil, nil, fmt.Errorf("invalid file index %q", key) } if _, exists := positions[index]; exists { - return nil, fmt.Errorf("duplicate file position %q", key) + return nil, nil, fmt.Errorf("duplicate file position %q", key) } + token, err = decoder.Token() if err != nil { - return nil, err + return nil, nil, err } path, ok := token.(string) if !ok || path == "" || strings.ContainsRune(path, 0) { - return nil, fmt.Errorf("invalid path at file index %q", key) + return nil, nil, fmt.Errorf("invalid path at file index %q", key) } + positions[index] = path } if _, err := decoder.Token(); err != nil { - return nil, err + return nil, nil, err } if _, err := decoder.Token(); !errors.Is(err, io.EOF) { - return nil, errors.New("trailing favorite document") + return nil, nil, errors.New("trailing favorite document") } indexes := make([]int, 0, len(positions)) for index := range positions { @@ -128,7 +165,26 @@ func decodeMembership(ctx context.Context, data []byte) ([]string, error) { paths[i] = positions[index] } if err := ctx.Err(); err != nil { - return nil, err + return nil, nil, err + } + var sources []fyne.URI + if access != nil { + if len(access.Sources) != len(paths) { + return nil, nil, errors.New("favorite authority membership mismatch") + } + var err error + sources, err = fileaccess.Unpack(ctx, *access) + if err != nil { + return nil, nil, err + } + for i, source := range sources { + if source.Path() != paths[i] { + return nil, nil, errors.New("favorite authority path mismatch") + } + if !fileaccess.HasScope(source) { + sources[i] = nil + } + } } - return paths, nil + return paths, sources, nil } diff --git a/internal/favstore/membership_test.go b/internal/favstore/membership_test.go index 125575b2..aa2de676 100644 --- a/internal/favstore/membership_test.go +++ b/internal/favstore/membership_test.go @@ -148,8 +148,17 @@ func TestFavoriteMembership(t *testing.T) { `{"0":"/a","00":"/b"}`, `{"+1":"/a","1":"/b"}`, `{"-1":"/a"}`, `{"1.0":"/a"}`, `{"999999999999999999999999":"/a"}`, `{"one":"/a"}`, `{" 1":"/a"}`, `[]`, `null`, `"path"`, `{} {}`, + `{"0":"/a","$access":null}`, + `{"0":"/a","$access":{"sources":[]}}`, + `{"0":"/a","$access":{"sources":[{"uri":"file:///b","scope":-1}]}}`, + `{"0":"/a","$access":{"sources":[{"uri":"file:///a","scope":2}]}}`, + `{"$access":{"sources":[]},"$access":{"sources":[]}}`, `{"0":null}`, `{"0":1}`, `{"0":true}`, `{"0":[]}`, `{"0":{}}`, `{"0":""}`, `{"0":"/a\u0000b"}`, `{"0":"/valid","99":""}`, + `{"0":"/valid","1":{"uri":"file:///photo.jpg","directory":true}}`, + `{"0":{"uri":"file:///photo.jpg","bookmark":"eA==","directory":true,"relative":"../photo.jpg"}}`, + `{"0":{"uri":"https://example.com/photo.jpg","bookmark":"eA=="}}`, + `{"0":{"uri":"file:///photo.jpg","bookmark":"not base64"}}`, } { t.Run(data, func(t *testing.T) { dir, name := putDefinition(t, data) diff --git a/internal/favstore/ownership.go b/internal/favstore/ownership.go index a244adf6..0af69b21 100644 --- a/internal/favstore/ownership.go +++ b/internal/favstore/ownership.go @@ -19,9 +19,10 @@ var ErrRetired = errors.New("favorite owner retired") // Definition is the complete ordered saved list and its captured owner. type Definition struct { - Owner *Owner - Paths []string - base string + Owner *Owner + Paths []string + base string + sources []fyne.URI } // Files interprets the saved occurrences as native file URIs. @@ -29,6 +30,9 @@ func (d Definition) Files() []fyne.URI { files := make([]fyne.URI, len(d.Paths)) for i, source := range d.Paths { files[i] = storage.NewFileURI(interpretedPath(d.base, source)) + if i < len(d.sources) && d.sources[i] != nil { + files[i] = d.sources[i] + } } return files } @@ -217,7 +221,7 @@ func (s *Store) open(ctx context.Context, dir, base string) (Definition, error) if err != nil { return Definition{}, err } - paths, err := decodeMembership(ctx, data) + paths, sources, err := decodeMembership(ctx, data) if err != nil { return Definition{}, err } @@ -226,7 +230,7 @@ func (s *Store) open(ctx context.Context, dir, base string) (Definition, error) if err := access.Current(ctx); err != nil { return Definition{}, err } - return Definition{Owner: owner, Paths: paths, base: base}, nil + return Definition{Owner: owner, Paths: paths, base: base, sources: sources}, nil } func definitionVersion(data []byte, list os.FileInfo) string { diff --git a/internal/favstore/saving.go b/internal/favstore/saving.go index 0a92f060..13e48025 100644 --- a/internal/favstore/saving.go +++ b/internal/favstore/saving.go @@ -10,6 +10,8 @@ import ( "sync/atomic" "fyne.io/fyne/v2" + + "github.com/frathe/picfetch/internal/fileaccess" ) // ErrConflict requires a fresh target observation and user decision. @@ -247,7 +249,13 @@ func (s *Store) Save(ctx context.Context, target *Target, files []fyne.URI) (res // if cancellation or external replacement follows the rename. owner := &Owner{dir: access.owner.dir, base: target.base, directory: access.owner.directory, list: list} owner.version = definitionVersion(data, list) - result = SaveResult{Committed: true, Definition: Definition{Owner: owner, Paths: paths, base: target.base}} + sources := make([]fyne.URI, len(files)) + for i, file := range files { + if fileaccess.HasScope(file) { + sources[i] = file + } + } + result = SaveResult{Committed: true, Definition: Definition{Owner: owner, Paths: paths, base: target.base, sources: sources}} if s.afterPublish != nil { s.afterPublish() } diff --git a/internal/favstore/saving_test.go b/internal/favstore/saving_test.go index 4df5c20a..8bfea9c2 100644 --- a/internal/favstore/saving_test.go +++ b/internal/favstore/saving_test.go @@ -1,12 +1,16 @@ package favstore import ( + "bytes" "context" "errors" "os" "path/filepath" + "reflect" "testing" + "github.com/frathe/picfetch/internal/fileaccess" + "fyne.io/fyne/v2" "fyne.io/fyne/v2/storage" @@ -235,3 +239,66 @@ func testFavoriteSaveCancellation(t *testing.T) { }) } } + +func TestFavoriteSavePreservesAuthorityForEachOccurrence(t *testing.T) { + var files []fyne.URI + for _, bookmark := range []string{"first-folder", "second-folder"} { + source, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///photos/a.jpg", Bookmark: []byte(bookmark), Directory: true, Relative: "a.jpg"}) + if err != nil { + t.Fatal(err) + } + files = append(files, source) + } + files = append(files, storage.NewFileURI("/photos/plain.jpg"), files[0]) + ctx := context.Background() + store := &Store{} + target, err := store.Capture(ctx, t.TempDir(), "Scoped") + if err != nil { + t.Fatal(err) + } + result, err := store.Save(ctx, target, files) + if err != nil || !result.Committed { + t.Fatalf("save: %+v, %v", result, err) + } + reopened, err := store.Open(ctx, result.Definition.Owner.Path()) + if err != nil { + t.Fatal(err) + } + for _, definition := range []Definition{result.Definition, reopened} { + got := definition.Files() + if len(got) != 4 { + t.Fatalf("restored %d occurrences, want 4", len(got)) + } + for i := range files { + if !reflect.DeepEqual(fileaccess.Snapshot(got[i]), fileaccess.Snapshot(files[i])) { + t.Fatalf("occurrence %d lost its authority: %+v", i, fileaccess.Snapshot(got[i])) + } + } + } +} + +func TestFavoriteSharedFolderGrantFitsDefinitionBudget(t *testing.T) { + root, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///photos", Bookmark: bytes.Repeat([]byte("x"), 128*1024), Directory: true}) + if err != nil { + t.Fatal(err) + } + source, err := fileaccess.Child(root, storage.NewFileURI("/photos/a.jpg")) + if err != nil { + t.Fatal(err) + } + files := make([]fyne.URI, 1024) + for i := range files { + files[i] = source + } + dir := t.TempDir() + if err := Save(dir, "Folder", files); err != nil { + t.Fatal(err) + } + restored, err := Load(dir, "Folder") + if err != nil || len(restored) != 1024 { + t.Fatalf("shared grant round trip: %d, %v", len(restored), err) + } + if got := fileaccess.Snapshot(restored[1023]); len(got.Bookmark) != 128*1024 || got.Relative != "a.jpg" { + t.Fatal("last occurrence lost the shared grant") + } +} diff --git a/internal/fileaccess/manifest.go b/internal/fileaccess/manifest.go new file mode 100644 index 00000000..2a38b41f --- /dev/null +++ b/internal/fileaccess/manifest.go @@ -0,0 +1,114 @@ +package fileaccess + +import ( + "context" + "crypto/sha256" + "encoding/json" + "errors" + + "fyne.io/fyne/v2" +) + +// Manifest stores each selected scope once and the complete ordered source set. +// Sources without a captured scope use -1. Occurrences are never deduplicated. +type Manifest struct { + Scopes []Scope `json:"scopes,omitempty"` + Sources []Source `json:"sources"` +} + +type Scope struct { + Bookmark []byte `json:"bookmark"` + Directory bool `json:"directory,omitempty"` +} + +type Source struct { + URI string `json:"uri"` + Scope int `json:"scope"` + Relative string `json:"relative,omitempty"` +} + +type scopeKey struct { + digest [32]byte + directory bool +} + +// Pack captures immutable metadata without resolving scopes or inspecting disk. +func Pack(ctx context.Context, files []fyne.URI) (Manifest, error) { + result := Manifest{Sources: make([]Source, len(files))} + indexes := make(map[scopeKey]int) + for i, file := range files { + if err := ctx.Err(); err != nil { + return Manifest{}, err + } + if file == nil { + return Manifest{}, errors.New("source manifest contains nil URI") + } + entry := Source{URI: file.String(), Scope: -1} + if source, ok := file.(*sourceURI); ok { + record := source.record + key := scopeKey{digest: sha256.Sum256(record.Bookmark), directory: record.Directory} + index, found := indexes[key] + if !found { + index = len(result.Scopes) + indexes[key] = index + result.Scopes = append(result.Scopes, Scope{Bookmark: append([]byte(nil), record.Bookmark...), Directory: record.Directory}) + } + entry.Scope, entry.Relative = index, record.Relative + } + result.Sources[i] = entry + } + return result, ctx.Err() +} + +// Unpack validates a complete captured set before publishing any of its URIs. +func Unpack(ctx context.Context, manifest Manifest) ([]fyne.URI, error) { + scopes := make([]Scope, len(manifest.Scopes)) + for i, scope := range manifest.Scopes { + if err := ctx.Err(); err != nil { + return nil, err + } + if len(scope.Bookmark) == 0 { + return nil, errors.New("source manifest has an empty scope") + } + scopes[i] = Scope{Bookmark: append([]byte(nil), scope.Bookmark...), Directory: scope.Directory} + } + files := make([]fyne.URI, len(manifest.Sources)) + for i, source := range manifest.Sources { + if err := ctx.Err(); err != nil { + return nil, err + } + record := Record{URI: source.URI, Relative: source.Relative} + if source.Scope < -1 || source.Scope >= len(scopes) { + return nil, errors.New("source manifest references a missing scope") + } + if source.Scope >= 0 { + scope := scopes[source.Scope] + record.Bookmark, record.Directory = scope.Bookmark, scope.Directory + } + // The private constructor shares only this function's own immutable copy. + uri, err := fromRecord(record, false) + if err != nil { + return nil, err + } + files[i] = uri + } + return files, ctx.Err() +} + +// UnmarshalJSON requires an explicit scope reference. Missing metadata cannot +// silently assign the first captured grant to a different source occurrence. +func (s *Source) UnmarshalJSON(data []byte) error { + var value struct { + URI string `json:"uri"` + Scope *int `json:"scope"` + Relative string `json:"relative,omitempty"` + } + if err := json.Unmarshal(data, &value); err != nil { + return err + } + if value.Scope == nil { + return errors.New("source manifest omitted scope reference") + } + *s = Source{URI: value.URI, Scope: *value.Scope, Relative: value.Relative} + return nil +} diff --git a/internal/fileaccess/manifest_test.go b/internal/fileaccess/manifest_test.go new file mode 100644 index 00000000..80257bb1 --- /dev/null +++ b/internal/fileaccess/manifest_test.go @@ -0,0 +1,66 @@ +package fileaccess + +import ( + "context" + "encoding/json" + "testing" + + "fyne.io/fyne/v2" + "fyne.io/fyne/v2/storage" +) + +func TestManifestDeduplicatesScopesAndPreservesOccurrences(t *testing.T) { + root, err := FromRecord(Record{URI: "file:///selected", Bookmark: []byte("scope"), Directory: true}) + if err != nil { + t.Fatal(err) + } + first, err := Child(root, storage.NewFileURI("/selected/first.jpg")) + if err != nil { + t.Fatal(err) + } + second, err := Child(root, storage.NewFileURI("/selected/second.jpg")) + if err != nil { + t.Fatal(err) + } + files := []fyne.URI{first, second, first, storage.NewFileURI("/plain.jpg")} + manifest, err := Pack(context.Background(), files) + if err != nil { + t.Fatal(err) + } + if len(manifest.Scopes) != 1 || len(manifest.Sources) != 4 { + t.Fatalf("manifest: %+v", manifest) + } + restored, err := Unpack(context.Background(), manifest) + if err != nil { + t.Fatal(err) + } + manifest.Scopes[0].Bookmark[0] = 'X' + for i, source := range restored { + if source.String() != files[i].String() { + t.Fatalf("occurrence %d changed", i) + } + if i < 3 && string(Snapshot(source).Bookmark) != "scope" { + t.Fatal("mutable manifest escaped") + } + } +} + +func TestManifestRejectsMissingScopeAndTraversal(t *testing.T) { + for _, manifest := range []Manifest{ + {Scopes: []Scope{{Bookmark: []byte("scope"), Directory: true}}, Sources: []Source{{URI: "file:///selected/a.jpg", Scope: 1}}}, + {Scopes: []Scope{{Bookmark: []byte("scope"), Directory: true}}, Sources: []Source{{URI: "file:///selected/a.jpg", Scope: 0, Relative: "../outside"}}}, + {Scopes: []Scope{{}}, Sources: []Source{{URI: "file:///selected/a.jpg", Scope: 0}}}, + {Sources: []Source{{URI: "file:///selected/a.jpg", Scope: -1, Relative: "nested/a.jpg"}}}, + } { + if _, err := Unpack(context.Background(), manifest); err == nil { + t.Fatalf("accepted invalid manifest %+v", manifest) + } + } +} + +func TestManifestJSONRequiresScopeReference(t *testing.T) { + var manifest Manifest + if err := json.Unmarshal([]byte(`{"scopes":[{"bookmark":"c2NvcGU="}],"sources":[{"uri":"file:///selected/a.jpg"}]}`), &manifest); err == nil { + t.Fatal("missing reference silently selected first scope") + } +} diff --git a/internal/fileaccess/native_darwin.go b/internal/fileaccess/native_darwin.go new file mode 100644 index 00000000..8fea6c35 --- /dev/null +++ b/internal/fileaccess/native_darwin.go @@ -0,0 +1,68 @@ +//go:build darwin && appleappstore + +package fileaccess + +/* +#cgo CFLAGS: -x objective-c -fobjc-arc +#cgo LDFLAGS: -framework Foundation +#import +#include +#include + +static char *acquireBookmark(const void *bytes, size_t length, int directory, void **owner, char **failure) { + @autoreleasepool { + NSData *bookmark = [NSData dataWithBytes:bytes length:length]; + NSError *error = nil; + NSURL *url = [NSURL URLByResolvingBookmarkData:bookmark + options:NSURLBookmarkResolutionWithSecurityScope | NSURLBookmarkResolutionWithoutUI + relativeToURL:nil bookmarkDataIsStale:NULL error:&error]; + if (!url || ![url startAccessingSecurityScopedResource]) { + *failure = strdup(error ? error.description.UTF8String : "security-scoped source access denied"); + return NULL; + } + NSNumber *isDirectory = nil; + if (![url getResourceValue:&isDirectory forKey:NSURLIsDirectoryKey error:&error] || + !isDirectory || isDirectory.boolValue != (directory != 0)) { + [url stopAccessingSecurityScopedResource]; + *failure = strdup("security-scoped source type changed"); + return NULL; + } + char *path = strdup(url.fileSystemRepresentation); + if (!path) { [url stopAccessingSecurityScopedResource]; return NULL; } + *owner = (__bridge_retained void *)url; + return path; + } +} +static void releaseBookmark(void *owner) { + @autoreleasepool { + NSURL *url = (__bridge_transfer NSURL *)owner; + [url stopAccessingSecurityScopedResource]; + } +} +*/ +import "C" + +import ( + "context" + "errors" + "unsafe" +) + +func resolveBookmark(_ context.Context, record Record) (string, func(), error) { + var owner unsafe.Pointer + var failure *C.char + directory := C.int(0) + if record.Directory { + directory = 1 + } + path := C.acquireBookmark(unsafe.Pointer(&record.Bookmark[0]), C.size_t(len(record.Bookmark)), directory, &owner, &failure) + defer C.free(unsafe.Pointer(path)) + defer C.free(unsafe.Pointer(failure)) + if path == nil { + if failure != nil { + return "", nil, errors.New(C.GoString(failure)) + } + return "", nil, errors.New("unable to acquire security-scoped source") + } + return C.GoString(path), func() { C.releaseBookmark(owner) }, nil +} diff --git a/internal/fileaccess/native_other.go b/internal/fileaccess/native_other.go new file mode 100644 index 00000000..088d9077 --- /dev/null +++ b/internal/fileaccess/native_other.go @@ -0,0 +1,25 @@ +//go:build !(darwin && appleappstore) + +package fileaccess + +import ( + "context" + "path/filepath" + "strings" + + "fyne.io/fyne/v2/storage" +) + +func resolveBookmark(_ context.Context, record Record) (string, func(), error) { + uri, err := storage.ParseURI(record.URI) + if err != nil { + return "", nil, err + } + root := uri.Path() + if record.Relative != "" { + for range strings.Split(record.Relative, "/") { + root = filepath.Dir(root) + } + } + return root, func() {}, nil +} diff --git a/internal/fileaccess/reader.go b/internal/fileaccess/reader.go new file mode 100644 index 00000000..fa3853d6 --- /dev/null +++ b/internal/fileaccess/reader.go @@ -0,0 +1,36 @@ +package fileaccess + +import ( + "context" + "sync" + + "fyne.io/fyne/v2" + "fyne.io/fyne/v2/storage" +) + +// Reader retains source authority until the returned reader closes. Failed +// opens release immediately, and concurrent or repeated Close calls release once. +func Reader(ctx context.Context, source fyne.URI) (fyne.URIReadCloser, error) { + uri, release, err := Acquire(ctx, source) + if err != nil { + return nil, err + } + reader, err := storage.Reader(uri) + if err != nil { + release() + return nil, err + } + return &scopedReader{URIReadCloser: reader, release: release}, nil +} + +type scopedReader struct { + fyne.URIReadCloser + release func() + once sync.Once + err error +} + +func (r *scopedReader) Close() error { + r.once.Do(func() { r.err = r.URIReadCloser.Close(); r.release() }) + return r.err +} diff --git a/internal/fileaccess/source.go b/internal/fileaccess/source.go new file mode 100644 index 00000000..0ae4aaa1 --- /dev/null +++ b/internal/fileaccess/source.go @@ -0,0 +1,157 @@ +// Package fileaccess carries immutable filesystem authority with source URIs. +// Native scopes belong to bounded operations, rather than to viewer lifetime. +package fileaccess + +import ( + "context" + "errors" + "path" + "path/filepath" + "strings" + "sync" + + "fyne.io/fyne/v2" + "fyne.io/fyne/v2/storage" +) + +// Record persists one occurrence's URI and its selected scope. Relative names +// refer only to children of a selected directory, never to its parent. +// Directory describes the selected scope, including when URI names its child. +type Record struct { + URI string `json:"uri"` + Bookmark []byte `json:"bookmark,omitempty"` + Directory bool `json:"directory,omitempty"` + Relative string `json:"relative,omitempty"` +} + +type sourceURI struct { + fyne.URI + record Record + resolve func(context.Context, Record) (string, func(), error) +} + +// FromRecord restores metadata without starting native access or inspecting disk. +func FromRecord(record Record) (fyne.URI, error) { + return fromRecord(record, true) +} + +func fromRecord(record Record, copyBookmark bool) (fyne.URI, error) { + uri, err := storage.ParseURI(record.URI) + if err != nil { + return nil, err + } + if len(record.Bookmark) == 0 { + if record.Relative != "" || record.Directory { + return nil, errors.New("source scope missing bookmark") + } + return uri, nil + } + if uri.Scheme() != "file" || !(path.IsAbs(uri.Path()) || filepath.IsAbs(uri.Path())) || strings.ContainsRune(uri.Path(), 0) { + return nil, errors.New("bookmarked source requires an absolute file URI") + } + if record.Relative != "" && (!record.Directory || !validRelative(record.Relative)) { + return nil, errors.New("source escapes its selected scope") + } + record.URI = uri.String() + if copyBookmark { + record.Bookmark = append([]byte(nil), record.Bookmark...) + } + return &sourceURI{URI: uri, record: record, resolve: resolveBookmark}, nil +} + +func validRelative(name string) bool { + return name != "." && !path.IsAbs(name) && !filepath.IsAbs(name) && path.Clean(name) == name && + name != ".." && !strings.HasPrefix(name, "../") && !strings.ContainsRune(name, 0) +} + +// HasScope reports whether a URI carries captured authority, without native I/O. +func HasScope(uri fyne.URI) bool { + _, ok := uri.(*sourceURI) + return ok +} + +// Snapshot returns independent persistent metadata; it acquires no native scope. +func Snapshot(uri fyne.URI) Record { + if source, ok := uri.(*sourceURI); ok { + record := source.record + record.Bookmark = append([]byte(nil), record.Bookmark...) + return record + } + if uri == nil { + return Record{} + } + return Record{URI: uri.String()} +} + +// Child propagates only captured directory authority to a discovered child. +func Child(parent, child fyne.URI) (fyne.URI, error) { + source, ok := parent.(*sourceURI) + if !ok { + return child, nil + } + if !source.record.Directory || child == nil || child.Scheme() != "file" { + return nil, errors.New("source does not grant directory access") + } + relative, err := filepath.Rel(parent.Path(), child.Path()) + relative = filepath.ToSlash(relative) + if err != nil || !validRelative(relative) { + return nil, errors.New("child escapes selected directory") + } + record := source.record + record.URI = child.String() + record.Relative = path.Join(record.Relative, relative) + return &sourceURI{URI: child, record: record, resolve: source.resolve}, nil +} + +// Acquire resolves one operation's authority. Call release after all I/O and +// native completion using the returned URI; release is safe to call repeatedly. +func Acquire(ctx context.Context, uri fyne.URI) (fyne.URI, func(), error) { + if err := ctx.Err(); err != nil { + return nil, nil, err + } + source, ok := uri.(*sourceURI) + if !ok { + return uri, func() {}, nil + } + root, stop, err := source.resolve(ctx, Snapshot(source)) + if err != nil { + return nil, nil, err + } + var once sync.Once + release := func() { once.Do(stop) } + if err := ctx.Err(); err != nil { + release() + return nil, nil, err + } + resolved := root + if source.record.Relative != "" { + resolved = filepath.Join(root, filepath.FromSlash(source.record.Relative)) + } + resolvedURI := storage.NewFileURI(resolved) + record := source.record + record.URI = resolvedURI.String() + return &sourceURI{URI: resolvedURI, record: record, resolve: source.resolve}, release, nil +} + +// Parent retains only already selected directory authority. A selected file or +// a selected directory's own parent requires another explicit user grant. +func Parent(uri fyne.URI) (fyne.URI, error) { + source, ok := uri.(*sourceURI) + if !ok { + return storage.Parent(uri) + } + if !source.record.Directory || source.record.Relative == "" { + return nil, errors.New("parent is outside selected scope") + } + parent, err := storage.Parent(uri) + if err != nil { + return nil, err + } + record := source.record + record.URI = parent.String() + record.Relative = path.Dir(record.Relative) + if record.Relative == "." { + record.Relative = "" + } + return &sourceURI{URI: parent, record: record, resolve: source.resolve}, nil +} diff --git a/internal/fileaccess/source_test.go b/internal/fileaccess/source_test.go new file mode 100644 index 00000000..ab9a81d1 --- /dev/null +++ b/internal/fileaccess/source_test.go @@ -0,0 +1,200 @@ +package fileaccess + +import ( + "context" + "errors" + "io" + "os" + "path/filepath" + "sync/atomic" + "testing" + + "fyne.io/fyne/v2/storage" + "fyne.io/fyne/v2/test" +) + +func TestSourceAcquireKeepsIndependentOperationLifetimes(t *testing.T) { + var active atomic.Int32 + source, err := FromRecord(Record{URI: "file:///original/photo.jpg", Bookmark: []byte("bookmark")}) + if err != nil { + t.Fatal(err) + } + source.(*sourceURI).resolve = func(_ context.Context, _ Record) (string, func(), error) { + active.Add(1) + return "/moved/photo.jpg", func() { active.Add(-1) }, nil + } + first, releaseFirst, err := Acquire(context.Background(), source) + if err != nil { + t.Fatal(err) + } + second, releaseSecond, err := Acquire(context.Background(), source) + if err != nil { + t.Fatal(err) + } + if first.Path() != "/moved/photo.jpg" || second.Path() != first.Path() || source.Path() != "/original/photo.jpg" { + t.Fatal("lost resolved path or changed captured identity") + } + releaseFirst() + releaseFirst() + if active.Load() != 1 { + t.Fatal("release retired another operation") + } + releaseSecond() + if active.Load() != 0 { + t.Fatal("access leaked") + } + ctx, cancel := context.WithCancel(context.Background()) + cancel() + if _, _, err := Acquire(ctx, source); !errors.Is(err, context.Canceled) || active.Load() != 0 { + t.Fatalf("cancelled acquire: %v", err) + } +} + +func TestChildCannotBroadenCapturedDirectory(t *testing.T) { + directory, err := FromRecord(Record{URI: "file:///selected", Bookmark: []byte("folder"), Directory: true}) + if err != nil { + t.Fatal(err) + } + child, err := Child(directory, storage.NewFileURI("/selected/sub/photo.jpg")) + if err != nil { + t.Fatal(err) + } + record := Snapshot(child) + if record.Relative != "sub/photo.jpg" { + t.Fatal(record.Relative) + } + record.Bookmark[0] = 'X' + if string(Snapshot(directory).Bookmark) != "folder" || string(Snapshot(child).Bookmark) != "folder" { + t.Fatal("mutable bookmark escaped") + } + for _, path := range []string{"/other/photo.jpg", "/selected/../other.jpg", "/selected-sibling/photo.jpg"} { + if _, err := Child(directory, storage.NewFileURI(path)); err == nil { + t.Fatalf("broadened to %q", path) + } + } + file, err := FromRecord(Record{URI: "file:///selected/photo.jpg", Bookmark: []byte("file")}) + if err != nil { + t.Fatal(err) + } + if _, err := Child(file, storage.NewFileURI("/selected/sibling.jpg")); err == nil { + t.Fatal("file grant authorized sibling") + } +} + +func TestRecordRejectsRelativeEscape(t *testing.T) { + for _, relative := range []string{"../other", "sub/../../other", "/absolute", "sub/../other", "sub//other"} { + if _, err := FromRecord(Record{URI: "file:///selected/photo.jpg", Bookmark: []byte("folder"), Directory: true, Relative: relative}); err == nil { + t.Fatalf("accepted %q", relative) + } + } +} + +func TestReaderRetainsScopeUntilCloseAndRollsBackFailure(t *testing.T) { + app := test.NewApp() + t.Cleanup(app.Quit) + path := filepath.Join(t.TempDir(), "source.dat") + if err := os.WriteFile(path, []byte("fixture"), 0600); err != nil { + t.Fatal(err) + } + source, err := FromRecord(Record{URI: "file:///selected/source.dat", Bookmark: []byte("grant")}) + if err != nil { + t.Fatal(err) + } + var active atomic.Int32 + source.(*sourceURI).resolve = func(_ context.Context, _ Record) (string, func(), error) { + active.Add(1) + return path, func() { active.Add(-1) }, nil + } + reader, err := Reader(context.Background(), source) + if err != nil { + t.Fatal(err) + } + content, err := io.ReadAll(reader) + if err != nil || string(content) != "fixture" || active.Load() != 1 { + t.Fatalf("read %q, err %v, active %d", content, err, active.Load()) + } + if err := reader.Close(); err != nil { + t.Fatal(err) + } + if err := reader.Close(); err != nil { + t.Fatal(err) + } + if active.Load() != 0 { + t.Fatal("reader leaked or released twice") + } + if err := os.Remove(path); err != nil { + t.Fatal(err) + } + if _, err := Reader(context.Background(), source); err == nil || active.Load() != 0 { + t.Fatal("failed open leaked its scope") + } +} + +func TestAcquiredDirectoryPropagatesMovedScopeToChild(t *testing.T) { + directory, err := FromRecord(Record{URI: "file:///selected", Bookmark: []byte("folder"), Directory: true}) + if err != nil { + t.Fatal(err) + } + directory.(*sourceURI).resolve = func(_ context.Context, _ Record) (string, func(), error) { + return "/moved", func() {}, nil + } + resolved, release, err := Acquire(context.Background(), directory) + if err != nil { + t.Fatal(err) + } + defer release() + child, err := Child(resolved, storage.NewFileURI("/moved/photo.jpg")) + if err != nil { + t.Fatal(err) + } + record := Snapshot(child) + if record.URI != child.String() || record.Relative != "photo.jpg" || string(record.Bookmark) != "folder" { + t.Fatalf("child lost moved scope: %+v", record) + } +} + +func TestParentRequiresCapturedDirectoryAuthority(t *testing.T) { + directory, err := FromRecord(Record{URI: "file:///selected", Bookmark: []byte("folder"), Directory: true}) + if err != nil { + t.Fatal(err) + } + if _, err := Parent(directory); err == nil { + t.Fatal("selected directory authorized its parent") + } + child, err := Child(directory, storage.NewFileURI("/selected/sub/photo.jpg")) + if err != nil { + t.Fatal(err) + } + parent, err := Parent(child) + if err != nil { + t.Fatal(err) + } + if Snapshot(parent).Relative != "sub" || filepath.ToSlash(filepath.Clean(parent.Path())) != "/selected/sub" { + t.Fatalf("lost parent inside selected scope: path=%q, relative=%q", parent.Path(), Snapshot(parent).Relative) + } + file, err := FromRecord(Record{URI: "file:///selected/photo.jpg", Bookmark: []byte("file")}) + if err != nil { + t.Fatal(err) + } + if _, err := Parent(file); err == nil { + t.Fatal("file grant authorized its parent") + } +} + +func TestAcquireReleasesScopeWhenCancelledDuringResolution(t *testing.T) { + source, err := FromRecord(Record{URI: "file:///photo.jpg", Bookmark: []byte("grant")}) + if err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + released := 0 + source.(*sourceURI).resolve = func(_ context.Context, _ Record) (string, func(), error) { + cancel() + return "/photo.jpg", func() { released++ }, nil + } + uri, release, err := Acquire(ctx, source) + if !errors.Is(err, context.Canceled) || uri != nil || release != nil || released != 1 { + t.Fatalf("cancelled acquisition returned %v, %v; released %d", uri, err, released) + } +} diff --git a/internal/filepicker/darwin.go b/internal/filepicker/darwin.go index ab263dd1..026aa604 100644 --- a/internal/filepicker/darwin.go +++ b/internal/filepicker/darwin.go @@ -11,11 +11,28 @@ package filepicker #include // Both panels and the native transport regression use this serializer. -static char *serializePanelURLs(NSArray *urls) { - NSMutableArray *paths = [NSMutableArray array]; +static char *serializePanelURLs(NSArray *urls, int scopes, char **failure) { + NSMutableArray *paths = [NSMutableArray array]; for (NSURL *url in urls) { if (!url.path) return NULL; - [paths addObject:url.path]; + if (scopes) { + NSError *error = nil; + NSNumber *directory = nil; + if (![url getResourceValue:&directory forKey:NSURLIsDirectoryKey error:&error] || !directory) { + *failure = strdup(error ? error.description.UTF8String : "selected URL directory inspection failed"); + return NULL; + } + NSData *bookmark = [url bookmarkDataWithOptions:NSURLBookmarkCreationWithSecurityScope + includingResourceValuesForKeys:nil relativeToURL:nil error:&error]; + if (!bookmark) { + *failure = strdup(error ? error.description.UTF8String : "selected URL bookmark capture failed"); + return NULL; + } + [paths addObject:@{@"uri":url.absoluteString, + @"bookmark":[bookmark base64EncodedStringWithOptions:0], @"directory":directory}]; + } else { + [paths addObject:url.path]; + } } NSData *data = [NSJSONSerialization dataWithJSONObject:paths options:0 error:NULL]; if (!data) return NULL; @@ -30,16 +47,39 @@ static char *roundTripPanelPaths(const char *input) { NSArray *paths = [NSJSONSerialization JSONObjectWithData:data options:0 error:NULL]; NSMutableArray *urls = [NSMutableArray array]; for (NSString *path in paths) [urls addObject:[NSURL fileURLWithPath:path]]; - return serializePanelURLs(urls); + return serializePanelURLs(urls, 0, NULL); } } // runOpenPanel shows an app-modal NSOpenPanel that allows files, folders, // and multi-select all at once - a combination none of AppleScript's // Standard Additions pickers offer. Must be called on the main thread (an -// AppKit requirement); chooseFilesDarwin below guarantees that. Returns a -// malloc'd JSON path array; JSON null on cancel, NULL on failure. -static char *runOpenPanel(const char *message) { +// AppKit requirement); chooseFilesDarwin below guarantees that. Returns +// retained URLs or an explicit cancellation/failure state. Serialization runs +// on the chooser worker after the modal panel closes. +typedef struct { + int state; // 0 cancellation, 1 selected URLs, -1 native failure. + void *urls; +} panelResult; + +// Keep original native URL authority through delivery to the chooser worker. +static panelResult retainSelection(NSArray *urls) { + if (!urls.count) return (panelResult){-1, NULL}; + return (panelResult){1, (__bridge_retained void *)[urls copy]}; +} + +static char *serializeSelection(panelResult result, int scopes, char **failure) { + @autoreleasepool { + NSArray *urls = (__bridge_transfer NSArray *)result.urls; + char *data = serializePanelURLs(urls, scopes, failure); + if (scopes) { + for (NSURL *url in urls) [url stopAccessingSecurityScopedResource]; + } + return data; + } +} + +static panelResult runOpenPanel(const char *message) { NSOpenPanel *panel = [NSOpenPanel openPanel]; panel.message = [NSString stringWithUTF8String:message]; panel.canChooseFiles = YES; @@ -47,9 +87,9 @@ static char *runOpenPanel(const char *message) { panel.allowsMultipleSelection = YES; NSModalResponse response = [panel runModal]; - if (response == NSModalResponseCancel) return strdup("null"); - if (response != NSModalResponseOK) return NULL; - return serializePanelURLs(panel.URLs); + if (response == NSModalResponseCancel) return (panelResult){0, NULL}; + if (response != NSModalResponseOK) return (panelResult){-1, NULL}; + return retainSelection(panel.URLs); } // runSavePanel shows an app-modal NSSavePanel pre-filled with name, opened @@ -57,8 +97,8 @@ static char *runOpenPanel(const char *message) { // chooseSaveDarwin guarantees it the same way. No allowedContentTypes is // set: the format is already decided by which "Export as…" item the user // picked, and constraining the panel would only stop them naming the file -// whatever they want. Returns the same JSON contract as runOpenPanel. -static char *runSavePanel(const char *message, const char *dir, const char *name) { +// whatever they want. Returns the same retained-URL contract as runOpenPanel. +static panelResult runSavePanel(const char *message, const char *dir, const char *name) { NSSavePanel *panel = [NSSavePanel savePanel]; panel.message = [NSString stringWithUTF8String:message]; panel.nameFieldStringValue = [NSString stringWithUTF8String:name]; @@ -68,20 +108,23 @@ static char *runSavePanel(const char *message, const char *dir, const char *name } NSModalResponse response = [panel runModal]; - if (response == NSModalResponseCancel) return strdup("null"); - if (response != NSModalResponseOK) return NULL; - return serializePanelURLs(panel.URL ? @[panel.URL] : @[]); + if (response == NSModalResponseCancel) return (panelResult){0, NULL}; + if (response != NSModalResponseOK) return (panelResult){-1, NULL}; + return retainSelection(panel.URL ? @[panel.URL] : @[]); } */ import "C" import ( "encoding/json" + "errors" "path/filepath" "unsafe" "fyne.io/fyne/v2" "fyne.io/fyne/v2/lang" + + "github.com/frathe/picfetch/internal/distribution" ) // chooseFilesDarwin runs AppKit's NSOpenPanel in-process rather than @@ -107,13 +150,9 @@ func chooseFilesDarwin() ([]byte, error) { cMsg := C.CString(lang.L("Open images")) defer C.free(unsafe.Pointer(cMsg)) - var out []byte - fyne.DoAndWait(func() { - cOut := C.runOpenPanel(cMsg) - defer C.free(unsafe.Pointer(cOut)) - out = []byte(C.GoString(cOut)) - }) - return out, nil + var result C.panelResult + fyne.DoAndWait(func() { result = C.runOpenPanel(cMsg) }) + return decodeNativePanel(result, distribution.AppleAppStore) } // chooseSaveDarwin is chooseFilesDarwin's save-panel twin, in-process and @@ -130,13 +169,9 @@ func chooseSaveDarwin(suggestedPath string) ([]byte, error) { cName := C.CString(filepath.Base(suggestedPath)) defer C.free(unsafe.Pointer(cName)) - var out []byte - fyne.DoAndWait(func() { - cOut := C.runSavePanel(cMsg, cDir, cName) - defer C.free(unsafe.Pointer(cOut)) - out = []byte(C.GoString(cOut)) - }) - return out, nil + var result C.panelResult + fyne.DoAndWait(func() { result = C.runSavePanel(cMsg, cDir, cName) }) + return decodeNativePanel(result, false) } // darwinPathTransport exercises the panel's actual Objective-C NSURL serializer @@ -152,3 +187,29 @@ func darwinPathTransport(paths []string) ([]byte, error) { defer C.free(unsafe.Pointer(cOut)) return []byte(C.GoString(cOut)), nil } + +// URL/bookmark inspection runs on the tracked chooser worker, after the modal +// panel releases UI. Native URLs remain owned until this serialization ends. +func decodeNativePanel(result C.panelResult, scoped bool) ([]byte, error) { + if result.state == 0 { + return []byte("null"), nil + } + if result.state != 1 { + return nil, errors.New("native file panel failed") + } + scopes := C.int(0) + if scoped { + scopes = 1 + } + var failure *C.char + data := C.serializeSelection(result, scopes, &failure) + defer C.free(unsafe.Pointer(data)) + defer C.free(unsafe.Pointer(failure)) + if failure != nil { + return nil, errors.New(C.GoString(failure)) + } + if data == nil { + return nil, errors.New("native file panel serialization failed") + } + return []byte(C.GoString(data)), nil +} diff --git a/internal/filepicker/filepicker.go b/internal/filepicker/filepicker.go index 8db888c8..bffbbe48 100644 --- a/internal/filepicker/filepicker.go +++ b/internal/filepicker/filepicker.go @@ -20,6 +20,9 @@ import ( "fyne.io/fyne/v2" "fyne.io/fyne/v2/lang" "fyne.io/fyne/v2/storage" + + "github.com/frathe/picfetch/internal/distribution" + "github.com/frathe/picfetch/internal/fileaccess" ) // Choose returns the exact selected file/folder identities in selection order. @@ -67,7 +70,7 @@ func decodePickedDestination(out []byte, err error) (fyne.URI, error) { return picked[0], nil } -// Native adapters emit a JSON array of paths, or null for cancellation. +// Native adapters emit paths, Apple Store scope records, or null for cancellation. // An empty successful selection is distinct from cancellation and cannot be used. func decodePickedPaths(out []byte, err error) ([]fyne.URI, error) { if err != nil { @@ -78,6 +81,9 @@ func decodePickedPaths(out []byte, err error) ([]fyne.URI, error) { } var paths []string if err := json.Unmarshal(out, &paths); err != nil { + if distribution.AppleAppStore { + return decodeScopedSelection(out) + } return nil, fmt.Errorf("invalid file chooser result: %w", err) } if paths == nil { @@ -249,3 +255,27 @@ try { exit 1 }` } + +// decodeScopedSelection accepts only complete native authority records. It +// never upgrades plain paths or a partially captured selection into grants. +func decodeScopedSelection(out []byte) ([]fyne.URI, error) { + var records []fileaccess.Record + if err := json.Unmarshal(out, &records); err != nil { + return nil, fmt.Errorf("invalid scoped file chooser result: %w", err) + } + if len(records) == 0 { + return nil, errors.New("file chooser returned an empty scoped selection") + } + files := make([]fyne.URI, len(records)) + for i, record := range records { + if len(record.Bookmark) == 0 || record.Relative != "" { + return nil, errors.New("file chooser did not capture its selected scope") + } + source, err := fileaccess.FromRecord(record) + if err != nil { + return nil, err + } + files[i] = source + } + return files, nil +} diff --git a/internal/filepicker/filepicker_test.go b/internal/filepicker/filepicker_test.go index 3006db96..578577ec 100644 --- a/internal/filepicker/filepicker_test.go +++ b/internal/filepicker/filepicker_test.go @@ -8,6 +8,11 @@ import ( "slices" "strings" "testing" + + "fyne.io/fyne/v2/storage" + + "github.com/frathe/picfetch/internal/distribution" + "github.com/frathe/picfetch/internal/fileaccess" ) func TestDecodePickedPaths_PreservesBoundaries(t *testing.T) { @@ -297,3 +302,27 @@ func TestZenityResult_DistinguishesCancelAndFailure(t *testing.T) { } } } + +func TestDecodePickedPaths_ScopedRecords(t *testing.T) { + source := storage.NewFileURI("/tmp/ scoped café folder ") + payload, err := json.Marshal([]fileaccess.Record{{URI: source.String(), Bookmark: []byte("selected-scope"), Directory: true}}) + if err != nil { + t.Fatal(err) + } + picked, err := decodePickedPaths(payload, nil) + // The immutable channel changes with the build tags used by this test. + //goland:noinspection GoBoolExpressions + if !distribution.AppleAppStore { + if err == nil { + t.Fatal("ordinary picker accepted a Store authority payload") + } + return + } + if err != nil || len(picked) != 1 { + t.Fatalf("scoped selection: %v, %v", picked, err) + } + record := fileaccess.Snapshot(picked[0]) + if record.URI != picked[0].String() || string(record.Bookmark) != "selected-scope" || !record.Directory { + t.Fatalf("lost selected authority: %+v", record) + } +} diff --git a/internal/filescan/authority_test.go b/internal/filescan/authority_test.go new file mode 100644 index 00000000..4936805d --- /dev/null +++ b/internal/filescan/authority_test.go @@ -0,0 +1,89 @@ +//go:build !darwin || !appleappstore + +package filescan + +import ( + "context" + "image/color" + "os" + "path/filepath" + "testing" + + "fyne.io/fyne/v2" + "fyne.io/fyne/v2/storage" + + "github.com/frathe/picfetch/internal/fileaccess" + "github.com/frathe/picfetch/internal/uitest" +) + +// These structural tests use the non-Store bookmark resolver. Actual native +// security-scope acquisition requires the signed sandbox qualification fixture. +func TestScanPreservesDirectoryAuthority(t *testing.T) { + dir := t.TempDir() + if err := os.Mkdir(filepath.Join(dir, "nested"), 0700); err != nil { + t.Fatal(err) + } + for _, name := range []string{"first.jpg", "nested/second.jpg"} { + if err := os.WriteFile(filepath.Join(dir, name), uitest.EncodeJPEG(t, 4, 4, color.White), 0600); err != nil { + t.Fatal(err) + } + } + root, err := fileaccess.FromRecord(fileaccess.Record{URI: storage.NewFileURI(dir).String(), Bookmark: []byte("folder"), Directory: true}) + if err != nil { + t.Fatal(err) + } + files, truncated := Images(context.Background(), []fyne.URI{root}, 10, nil) + if len(files) != 2 || truncated { + t.Fatalf("scan: %v, truncated=%v", files, truncated) + } + seen := map[string]bool{} + for _, file := range files { + record := fileaccess.Snapshot(file) + if string(record.Bookmark) != "folder" || !record.Directory { + t.Fatal("scan discarded selected scope") + } + seen[record.Relative] = true + } + if !seen["first.jpg"] || !seen["nested/second.jpg"] { + t.Fatalf("relative identities: %v", seen) + } + replayed, _ := ReplayWithAdmission(context.Background(), []fyne.URI{files[0], files[0]}, 10, nil, func(_ fyne.URI) bool { return true }) + if len(replayed) != 2 || replayed[0] != files[0] || replayed[1] != files[0] { + t.Fatal("replay changed captured occurrences") + } +} + +func TestSiblingScanRequiresDirectoryAuthority(t *testing.T) { + dir := t.TempDir() + for _, name := range []string{"first.jpg", "second.jpg"} { + if err := os.WriteFile(filepath.Join(dir, name), uitest.EncodeJPEG(t, 4, 4, color.White), 0600); err != nil { + t.Fatal(err) + } + } + selected := storage.NewFileURI(filepath.Join(dir, "first.jpg")) + file, err := fileaccess.FromRecord(fileaccess.Record{URI: selected.String(), Bookmark: []byte("file")}) + if err != nil { + t.Fatal(err) + } + files, _ := Siblings(context.Background(), file, 10, nil) + if len(files) != 1 || files[0].String() != selected.String() { + t.Fatalf("file grant broadened: %v", files) + } + root, err := fileaccess.FromRecord(fileaccess.Record{URI: storage.NewFileURI(dir).String(), Bookmark: []byte("folder"), Directory: true}) + if err != nil { + t.Fatal(err) + } + child, err := fileaccess.Child(root, selected) + if err != nil { + t.Fatal(err) + } + files, _ = Siblings(context.Background(), child, 10, nil) + if len(files) != 2 { + t.Fatalf("captured directory did not admit siblings: %v", files) + } + for _, source := range files { + if string(fileaccess.Snapshot(source).Bookmark) != "folder" { + t.Fatal("sibling lost authority") + } + } +} diff --git a/internal/filescan/filescan.go b/internal/filescan/filescan.go index 76a2b6ad..1d43d8fe 100644 --- a/internal/filescan/filescan.go +++ b/internal/filescan/filescan.go @@ -25,6 +25,7 @@ import ( "fyne.io/fyne/v2" "fyne.io/fyne/v2/storage" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/imaging" ) @@ -127,7 +128,7 @@ func gather(ctx context.Context, uris []fyne.URI, max int, progress func(int), a // already allows re-adding a file that's already loaded. seenFiles := make(map[string]bool) - process := func(u fyne.URI) { + process := func(u, identity fyne.URI) { if truncated || ctx.Err() != nil { return } @@ -155,7 +156,7 @@ func gather(ctx context.Context, uris []fyne.URI, max int, progress func(int), a seenFiles[pathOf] = true } - images = append(images, u) + images = append(images, identity) count++ if count >= max { truncated = true @@ -166,7 +167,15 @@ func gather(ctx context.Context, uris []fyne.URI, max int, progress func(int), a } for _, u := range uris { - process(u) + resolved, release, err := fileaccess.Acquire(ctx, u) + if err != nil { + // Preserve recorded occurrences; a failed grant does not turn a known + // image into an empty successful collection. Its read reports the failure. + process(u, u) + continue + } + process(resolved, u) + release() } for len(dirs) > 0 && !truncated { @@ -175,13 +184,20 @@ func gather(ctx context.Context, uris []fyne.URI, max int, progress func(int), a } d := dirs[len(dirs)-1] dirs = dirs[:len(dirs)-1] - children, err := storage.List(d) + resolved, release, err := fileaccess.Acquire(ctx, d) if err != nil { continue } - for _, child := range children { - process(child) + children, err := storage.List(resolved) + if err == nil { + for _, child := range children { + granted, err := fileaccess.Child(resolved, child) + if err == nil { + process(granted, granted) + } + } } + release() } return images, truncated @@ -216,7 +232,13 @@ func SiblingsWithAdmission(ctx context.Context, file fyne.URI, max int, progress return nil, false } - origin := realPathOf(file) + resolved, releaseFile, err := fileaccess.Acquire(ctx, file) + if err == nil { + defer releaseFile() + } else { + resolved = file + } + origin := realPathOf(resolved) seen := make(map[string]bool) count := 0 add := func(u fyne.URI) { @@ -247,19 +269,24 @@ func SiblingsWithAdmission(ctx context.Context, file fyne.URI, max int, progress } } - add(file) + add(resolved) if truncated { return images, truncated } seeded := len(images) - parent, err := storage.Parent(file) + parent, err := fileaccess.Parent(resolved) if err != nil { return images, truncated } if parent.Scheme() != "file" { return images, truncated } + parent, releaseParent, err := fileaccess.Acquire(ctx, parent) + if err != nil { + return images, truncated + } + defer releaseParent() directory, err := os.Open(parent.Path()) if err != nil { return images, truncated @@ -274,7 +301,10 @@ func SiblingsWithAdmission(ctx context.Context, file fyne.URI, max int, progress entries, readErr := directory.ReadDir(batch) for _, entry := range entries { entriesRead++ - add(storage.NewFileURI(filepath.Join(parent.Path(), entry.Name()))) + child, err := fileaccess.Child(parent, storage.NewFileURI(filepath.Join(parent.Path(), entry.Name()))) + if err == nil { + add(child) + } } if readErr != nil { break diff --git a/internal/imaging/loader.go b/internal/imaging/loader.go index 1a573e40..d1423825 100644 --- a/internal/imaging/loader.go +++ b/internal/imaging/loader.go @@ -21,7 +21,6 @@ import ( "time" "fyne.io/fyne/v2" - "fyne.io/fyne/v2/storage" _ "github.com/fyne-io/image/xpm" // registers XPM with image.Decode _ "github.com/gen2brain/avif" // registers AVIF with image.Decode (WASM/wazero, no cgo) _ "golang.org/x/image/bmp" // registers BMP with image.Decode @@ -29,6 +28,7 @@ import ( _ "golang.org/x/image/webp" // registers WebP with image.Decode _ "github.com/frathe/picfetch/internal/avifpolicy" // requires the WASM/wazero build + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/heic" ) @@ -258,7 +258,7 @@ func readRawBytes(ctx context.Context, u fyne.URI) ([]byte, error) { return nil, err } - rc, err := storage.Reader(u) + rc, err := fileaccess.Reader(ctx, u) if err != nil { return nil, err } diff --git a/internal/macworker/command_test.go b/internal/macworker/command_test.go index e01a0309..805204e1 100644 --- a/internal/macworker/command_test.go +++ b/internal/macworker/command_test.go @@ -9,12 +9,14 @@ import ( func TestCommandUsesOnlyBundledBrokerAndMode(t *testing.T) { for _, mode := range []Mode{HEIC, Similarity} { - cmd := Command(context.Background(), "/Applications/PicFetch.app/Contents/MacOS/picfetch", mode) - want := []string{"/Applications/PicFetch.app/Contents/MacOS/picfetch-worker-client", string(mode)} + executable := filepath.Join(t.TempDir(), "PicFetch.app", "Contents", "MacOS", "picfetch") + cmd := Command(context.Background(), executable, mode) + broker := filepath.Join(filepath.Dir(executable), "picfetch-worker-client") + want := []string{broker, string(mode)} if !reflect.DeepEqual(cmd.Args, want) { t.Fatalf("arguments = %v, want %v", cmd.Args, want) } - if cmd.Path != filepath.Join("/Applications/PicFetch.app/Contents/MacOS", "picfetch-worker-client") { + if cmd.Path != broker { t.Fatal(cmd.Path) } } diff --git a/internal/session/session.go b/internal/session/session.go index 8e44cc12..fc422418 100644 --- a/internal/session/session.go +++ b/internal/session/session.go @@ -3,10 +3,13 @@ package session import ( + "context" "encoding/json" "fyne.io/fyne/v2" "fyne.io/fyne/v2/storage" + + "github.com/frathe/picfetch/internal/fileaccess" ) // cacheKey names the cache entry Save/Load read and write via app.Cache() - @@ -20,7 +23,8 @@ const cacheKey = "session.json" // format can grow a field later without breaking decode of what's already // on disk. type state struct { - Files []string `json:"files"` + Files []string `json:"files"` + Access *fileaccess.Manifest `json:"access,omitempty"` } // Save records files as the session offered on the next launch. An empty @@ -43,13 +47,22 @@ func Save(app fyne.App, files []fyne.URI) { for i, u := range files { uris[i] = u.String() } + manifest, err := fileaccess.Pack(context.Background(), files) + if err != nil { + fyne.LogError("failed to capture session", err) + return + } + var access *fileaccess.Manifest + if len(manifest.Scopes) != 0 { + access = &manifest + } w, err := cache.Write(cacheKey) if err != nil { fyne.LogError("failed to save session", err) return } - if err := json.NewEncoder(w).Encode(state{Files: uris}); err != nil { + if err := json.NewEncoder(w).Encode(state{Files: uris, Access: access}); err != nil { _ = w.Close() fyne.LogError("failed to save session", err) return @@ -84,6 +97,22 @@ func Load(app fyne.App) []fyne.URI { return nil } + if s.Access != nil { + if len(s.Access.Sources) != len(s.Files) { + return nil + } + for i, source := range s.Access.Sources { + if source.URI != s.Files[i] { + return nil + } + } + uris, err := fileaccess.Unpack(context.Background(), *s.Access) + if err != nil { + return nil + } + return uris + } + uris := make([]fyne.URI, 0, len(s.Files)) for _, u := range s.Files { if parsed, err := storage.ParseURI(u); err == nil { diff --git a/internal/session/session_test.go b/internal/session/session_test.go index 5b468c01..3551693d 100644 --- a/internal/session/session_test.go +++ b/internal/session/session_test.go @@ -1,11 +1,14 @@ package session import ( + "reflect" "testing" "fyne.io/fyne/v2" "fyne.io/fyne/v2/storage" "fyne.io/fyne/v2/test" + + "github.com/frathe/picfetch/internal/fileaccess" ) func TestLoadSession_NothingSavedReturnsNil(t *testing.T) { @@ -64,9 +67,84 @@ func TestLoadSession_CorruptCacheEntryReturnsNil(t *testing.T) { if _, err := w.Write([]byte("not valid json")); err != nil { t.Fatalf("Write: %v", err) } - w.Close() + if err := w.Close(); err != nil { + t.Fatal(err) + } if got := Load(app); got != nil { t.Errorf("Load() = %v, want nil for corrupt cache entry", got) } } + +func TestSaveSession_PreservesAuthorityForEachOccurrence(t *testing.T) { + app := test.NewApp() + t.Cleanup(app.Quit) + var files []fyne.URI + for _, bookmark := range []string{"first-folder", "second-folder"} { + source, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///photos/a.jpg", Bookmark: []byte(bookmark), Directory: true, Relative: "a.jpg"}) + if err != nil { + t.Fatal(err) + } + files = append(files, source) + } + files = append(files, storage.NewFileURI("/photos/plain.jpg"), files[0]) + Save(app, files) + got := Load(app) + if len(got) != 4 { + t.Fatalf("restored %d occurrences, want 4", len(got)) + } + for i := range files { + if !reflect.DeepEqual(fileaccess.Snapshot(got[i]), fileaccess.Snapshot(files[i])) { + t.Fatalf("occurrence %d lost its authority: %+v", i, fileaccess.Snapshot(got[i])) + } + } +} + +func TestLoadSessionRejectsMisalignedAuthority(t *testing.T) { + for _, data := range []string{ + `{"files":["file:///photos/a.jpg"],"access":{"sources":[]}}`, + `{"files":["file:///photos/a.jpg"],"access":{"sources":[{"uri":"file:///photos/b.jpg","scope":-1}]}}`, + `{"files":["file:///photos/a.jpg"],"access":{"scopes":[{"bookmark":"eA==","directory":true}],"sources":[{"uri":"file:///photos/a.jpg","scope":0,"relative":"../a.jpg"}]}}`, + } { + t.Run(data, func(t *testing.T) { + app := test.NewApp() + t.Cleanup(app.Quit) + w, err := app.Cache().Write(cacheKey) + if err != nil { + t.Fatal(err) + } + if _, err := w.Write([]byte(data)); err != nil { + t.Fatal(err) + } + if err := w.Close(); err != nil { + t.Fatal(err) + } + if got := Load(app); got != nil { + t.Fatalf("restored partial or broadened authority: %v", got) + } + }) + } +} + +func TestSaveSession_PreservesScopedOccurrences(t *testing.T) { + app := test.NewApp() + root, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///selected", Bookmark: []byte("scope"), Directory: true}) + if err != nil { + t.Fatal(err) + } + source, err := fileaccess.Child(root, storage.NewFileURI("/selected/photo.jpg")) + if err != nil { + t.Fatal(err) + } + Save(app, []fyne.URI{source, source}) + files := Load(app) + if len(files) != 2 { + t.Fatalf("occurrences = %d", len(files)) + } + for _, uri := range files { + record := fileaccess.Snapshot(uri) + if uri.String() != source.String() || string(record.Bookmark) != "scope" || record.Relative != "photo.jpg" { + t.Fatalf("lost saved authority: %+v", record) + } + } +} diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index fcb1843f..662417d7 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -330,3 +330,135 @@ io.github.frathe.picfetch.grantqualification and only the generated .scratch/apple-store-grant-probe/selected folder. User selection/relaunch/moved folder evidence is pending. No production bookmark readiness is inferred from the successful temporary interprocess grant fixture. + +### Persistent grant probe — actual user-selected folder + +Ronin selected only the generated fixture folder in the temporary native +NSOpenPanel. Its separate ad-hoc signed app has app-sandbox, +files.user-selected.read-write and files.bookmarks.app-scope entitlements. + +- Capture returned a nonempty security-scoped bookmark with no native error. +- Fresh-process restore: original-path read failed with Cocoa error 257, + startAccessingSecurityScopedResource returned true, content read succeeded, + and stale=false. The matching stop call ran before process exit. +- Renamed generated folder: original-path read failed with error 260; the + bookmark resolved the moved directory, returned stale=true, read succeeded, + and regenerating persistent data succeeded. The fixture folder was restored + to its original path afterward. +- Evidence: .scratch/apple-store-grant-probe/restored.json and moved.json; + native source and signed app are alongside them. No user photo was selected. +- Computer-use launched the probe but AX requests timed out. The completed + native output, rather than inaccessible AX state, establishes these results. + Ronin performed the initial selection; subsequent launches need no picker. +- Full Apple-tagged application build to a temporary executable succeeds. + This is compilation evidence only; no production sandbox bundle is ready. + +Source boundary map before T2 edits: + +| Boundary | Existing entry points | Required authority lifetime | +| --- | --- | --- | +| Selection/open | filepicker Darwin panel serializer; openwith native URL bridge; UI open/drop admission | Capture selected URL bookmark before flattening, admit immutable URI metadata; cancelled requests publish no new sources. | +| Decode/EXIF | imaging.readRawBytes | Acquire before storage.Reader; release after ReadCloser closes, including cancelled/failed reads. | +| Scan | filescan.gather and SiblingsWithAdmission | Acquire during each directory listing/stat; attach captured directory grant to discovered children; file grants cannot authorize parent scans. | +| Save/export/metadata | imaging.SaveRotatedContext, ExportContext, StripJPEGMetadataContext | Hold source/destination grant across transaction, temp replacement and committed effects; source metadata reads need their own grant. | +| Version/reconciliation | favthumbs.EntryName; UI filework stats and alias comparisons | Acquire on existing tracked workers; keep captured source/version semantics. | +| Delete | deletion.performDelete -> imaging.WithFileMutation -> trash.Move | Hold captured target grant through submitted native completion even if view closes. | +| Persistence | session state; favstore.Definition.Files and saving list encoding | Preserve complete ordered grant metadata/occurrences; no filesystem widening from path lists. | +| OS handoff | reveal, clipboard file copy and wallpaper native URLs | Hold scope through actual native call; verify recipient access using public URL mechanisms. | +| Analysis | similarity.Client Analyze/Search request and WorkerMain | Parent leases through complete worker wait; temporary bookmark array bounded by existing protocol limits; worker scopes through complete producer/nested decoder retirement. | + +T2 remains open. These probes establish the native bookmark behavior, not +production grant propagation, stale collection reconciliation or complete +sandboxed feature operation. + +### Recovery continuation — source permission persistence + +Resume the uncommitted fileaccess/picker/scan/read patch on `a293137`. +Owner: T0 inline; no delegation. Seams: fileaccess Acquire/Reader/Child/Parent, +filescan Images/Replay/Siblings, session Save/Load, and favstore Save/Open/Files. +Preserve occurrence order and distinct grants for repeated paths; ordinary +path-only records remain readable. Favorite publication must return the same +authority that reopening its complete saved membership returns, without native +scope acquisition during encoding or decoding. Malformed authority fails closed. +Verify: focused ordinary and Apple-tagged tests for fileaccess, filepicker, +filescan, session and favstore; then affected regressions, vet and IDE inspections. +Budget: 0 spawns, 2 inline review rounds, complete verification once at handoff. +This slice does not establish signed full-application qualification. + +### Recovery checkpoint — 2026-09-29 + +Base: `a293137`, plus the recovered and continued working-tree patch; no commit +or push in this continuation. Code scope: 21 Go files, sorted path/NUL/content/NUL +SHA-256 `bcfd4b0b4531e8301d11b5c2b7ad3c0f6a8d7db881cf2091846bc78c3bfac05a`. + +Concurrent detached Codex sessions were still modifying this checkout after the +interrupted run. Ronin explicitly requested stopping all other sessions. They +were terminated, remaining detached processes were force-stopped, and a process +check confirmed only the current session remained. Late manifest/session edits +were reconciled before the final tests. Some shell tool calls fail to report +completion even after command exit; explicit command exit markers and complete +package results are the verification evidence, not those tool session handles. + +Implemented checkpoint: + +- Native open-panel URL ownership survives UI delivery to the tracked chooser + worker, where Apple-channel persistent bookmarks are captured. Canonical image + reads hold one acquired scope through ReadCloser.Close, with idempotent release. +- Folder scans propagate the captured directory grant to descendants; file-only + selections do not acquire parent/sibling authority. Replay preserves occurrences. +- `fileaccess.Manifest` stores each shared scope once and retains every ordered + source occurrence. Session `access` and Favorite `$access` metadata are checked + against the complete saved list before restoration. Ordinary path-only writes + retain their existing format; old path-only records remain readable. Older + PicFetch versions do not understand the new scoped Favorite metadata. +- Committed Favorite save results carry the same source authority as reopening + the published membership. No native scope is acquired while serializing or + restoring metadata. A 1,024-occurrence collection sharing a 128 KiB bookmark + saves and reopens within the existing 64 MiB Favorite definition limit. + +Verification: + +- Session and Favorite authority regressions first failed because bookmarks were + lost; both pass with the implementation. Malformed, mismatched and incomplete + manifests fail closed; duplicate numeric positions and legacy size limits + retain their existing regression coverage. +- Ordinary and Apple-tagged fileaccess, filepicker, filescan, session and favstore + suites pass. Ordinary imaging, favthumbs, UI Favorites, Location Map, similarity + and macworker regressions pass. +- Apple-tagged root UI collection/Favorite and Store-policy integration passes + (`TestCollectionReplay|TestCollection|TestFavorite|TestLaunchPolicyIntegration|TestUpdateCheck_StoreManaged`). +- Temporary Go overlays deliberately discarded bookmarks, omitted cancellation + release, discarded child grants and widened file-only sibling access. Guards + failed on each intended behavior; real sources were unchanged and restored + focused suites pass. The first scan overlay had an unused variable compile + error; it was corrected before the reported behavioral failures. +- `make verify-build` passes (format, TUF/assets/notices/exclusions, vet, build). + Focused Apple-tagged vet and Windows/amd64 no-cgo vet pass for fileaccess, + session, favstore, filescan and filepicker. The two new scan structural tests + use the non-Store resolver explicitly; signed native grant behavior remains + a separate qualification gate. +- GoLand `get_file_problems(errorsOnly=false)` inspected all 21 changed Go files. + Import ordering and a previously unhandled test Close error were corrected. + The build-channel constant warning has a narrow documented suppression. + Reinspection leaves intentional duplicate test fragments in session, + favstore/saving and filepicker; existing exact Qodana exclusions cover them. + The current IDE profile does not select `nodynamic`, so imaging/loader.go has + an AVIF build-constraint diagnostic. CLI vet/build with required tags passes; + that IDE profile's imaging result remains unverified, not suppressed. This is + IDE fallback evidence, not a fresh Qodana SARIF/profile result. +- `make verify` cannot pass its prerequisite: the selected Docker daemon reports + Linux/aarch64. Complete native Linux/amd64 race verification remains unverified; + no isolation test or worker restriction was bypassed. + +Next: native save/Open With/drop capture, missing/stale-grant reselection and +renewal, mutation/version/OS-handoff scope lifetimes, worker grant transport, +signed runtime validation/packaging, and full Intel/Apple Silicon qualification. +Session cache corruption retains its existing nil-result behavior; presenting a +reselection/error surface for unusable authority is still part of T2. The raw +record and manifest tests establish serialization/lifetimes, not a sandboxed +production application's readiness. Signing inputs and Apple submission gates +remain outstanding. No release, upload, credential change or dependency upgrade. + +Recovery budget: 0 spawns; 3 inline review rounds (one additional reconciliation +round for concurrent edits); one attempted complete gate, blocked at the daemon +platform guard. Focused tests and build-only verification completed separately. diff --git a/qodana.yaml b/qodana.yaml index 451275df..3925e558 100644 --- a/qodana.yaml +++ b/qodana.yaml @@ -146,6 +146,9 @@ exclude: - "internal/heic/metadata_test.go" - "internal/heic/native_darwin_test.go" - "internal/heic/primaryfixture_test.go" + - "internal/filescan/authority_test.go" + - "internal/fileaccess/manifest_test.go" + - "internal/fileaccess/source_test.go" - "internal/macworker/command_test.go" - "internal/heic/process_linux_test.go" - "internal/heic/process_unix_test.go" diff --git a/todos.md b/todos.md index 6bb3e63b..a0ef74a9 100644 --- a/todos.md +++ b/todos.md @@ -17,6 +17,7 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/2026-09-29-apple-app-store.md). - [x] Add an immutable Apple Store channel, localized update/repair messages, verified pre-sign runtime staging and a read-only developer-input preflight. +- [x] Preserve captured permissions through image reads, directory scans, and complete session/Favorite serialization; shared folder bookmarks are stored once. - [ ] Qualify persistent security-scoped permissions for opening, saved sessions, Favorites, writes and workers. - [x] Implement the Apple Store XPC worker boundary and qualify native fixture TCP/UDP denial, cancellation and broker crash on Apple Silicon. - [ ] Qualify real HEIC/ONNX execution and granted source/cache access through that worker boundary on both Mac architectures. From 93c3040eacc9cfc2a89e8a31342fe818b2755ff3 Mon Sep 17 00:00:00 2001 From: frathe Date: Tue, 29 Sep 2026 23:03:47 +0200 Subject: [PATCH 05/63] Preserve native opening authority through capture and shutdown --- .github/testshards/internal-ui.tsv | 4 +- ARCHITECTURE.md | 19 ++-- internal/fileaccess/selection.go | 106 ++++++++++++++++++++++ internal/fileaccess/selection_test.go | 83 +++++++++++++++++ internal/filepicker/filepicker.go | 6 ++ internal/filepicker/filepicker_test.go | 24 ++++- internal/openwith/openwith.go | 27 +++++- internal/openwith/openwith_darwin.go | 75 ++++++++++++++- internal/openwith/openwith_darwin.h | 5 + internal/openwith/openwith_darwin.m | 83 +++++++++++++++++ internal/openwith/openwith_darwin_test.go | 33 +++++++ internal/openwith/openwith_test.go | 32 +++++++ internal/ui/build.go | 2 + internal/ui/collection_lifecycle_test.go | 4 + internal/ui/collection_replay_test.go | 2 + internal/ui/drop.go | 90 +++++++++++++++--- internal/ui/drop_test.go | 63 +++++++++++++ internal/ui/harness_test.go | 14 +++ internal/ui/openfiles.go | 1 + internal/ui/openwith.go | 68 +++++++++++++- internal/ui/openwith_test.go | 29 ++++++ internal/ui/run.go | 5 +- internal/ui/viewer.go | 6 +- plans/2026-09-29-apple-app-store.md | 86 ++++++++++++++++++ qodana.yaml | 1 + todos.md | 1 + translations/de.json | 1 + translations/en.json | 1 + 28 files changed, 840 insertions(+), 31 deletions(-) create mode 100644 internal/fileaccess/selection.go create mode 100644 internal/fileaccess/selection_test.go diff --git a/.github/testshards/internal-ui.tsv b/.github/testshards/internal-ui.tsv index 4d8aea06..12de396b 100644 --- a/.github/testshards/internal-ui.tsv +++ b/.github/testshards/internal-ui.tsv @@ -7,7 +7,7 @@ # weights: single CI attempt; projected test sums, not measured shard runtimes # ui-1: 270 entries, 470.520s projected weight sum # ui-2: 255 entries, 470.530s projected weight sum -# ui-3: 221 entries, 504.870s projected weight sum +# ui-3: 223 entries, 504.870s projected weight sum # rebalance: 27 whole-test moves; slowest projected sum within 5% of mean # evidence: finished_refactorings/2026-09-11-ui-shard-rebalance.md # name shard @@ -654,10 +654,12 @@ TestMosaicDrain_ClosesWindowAndClearsTransientState ui-3 TestMosaicShutdown_ClosesWindowBeforeTheEventLoopStops ui-3 TestMosaicSources_UsesExplicitSelectionExclusively ui-3 TestMosaicWallpaper_RejectsOverlapAndDeletesOnlyFailedCopy ui-3 +TestNativeSelectionOpeningOwnership ui-3 TestNormalizePreferenceDefaults ui-3 TestOpenFileDialog_RunsChooserInBackground ui-3 TestOpenInitialFiles_OpensPendingWhenNothingWasDelivered ui-3 TestOpenWithHandler_DeliveryMergesWhenMergeModeIsOn ui-3 +TestOpenWithShutdownReleasesUndeliveredNativeInput ui-3 TestPerformDelete_OSFailureKeepsTheFileAndToastsAnError ui-3 TestPerformUpdate_RecordsRelaunchThenQuits ui-3 TestPictureFrameMenu_DetachedMenuReachesTheWindowEdge ui-3 diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 5d58ee6d..932c40f1 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -702,8 +702,8 @@ The concurrency invariant: see `AGENTS.md` § Concurrency and Fyne. | `menu.go` | Explicit File/Favorites/Actions/Window/Help composition. `menuState` observes one command context and derives named availability decisions; `syncMenus` applies them with presentation facts, updates Favorites and Help, and refreshes the native bar once when rendered state changes. | | `commandpolicy.go`, `commandadmission.go` | Private value-only request/context/decision policy for application commands; root observes feature facts and applies refusal feedback/yield only after admission. Visible surfaces, retained visits, input ownership, capabilities and operation state stay distinct. Payload capture/workers remain with handlers. Delete/export notifications assign their card's single keyboard owner and release it on dismissal; prompt/Fyne-dialog notifications refresh availability. | | `actionmenu.go` | Actions-menu adapters call guarded bare handlers. Duplicate preparation rechecks admission before presenting a group and retires refused presentation instead of replaying it. Progress and accepted-group notifications remain separate. | -| `drop.go` | `openCollection` / `applyScanResult` / `applyScannedFiles` glue over filescan discovery and recorded replay. `handleDrop` discovers, Favorite/session replay; both retain the same scan lifecycle, captured admission and commit path. A non-empty open is refused before any state change while comparison is active. | -| `openwith.go` | macOS "Open With" delivery: `installOpenWithHandler` / `openInitialFiles` / `openFilesFromOS` over `internal/openwith`, both routed through `fyne.Do` so a launch carrying argv files and a delivery makes one `handleDrop`. The combined pending set is cleared before that shared path refuses an active comparison, so deliveries cannot queue behind it. | +| `drop.go` | `openCollection` / `applyScanResult` / `applyScannedFiles` glue over filescan discovery and recorded replay. `handleDrop` discovers, Favorite/session replay; both retain the same scan lifecycle, captured admission and commit path. Native/scoped inputs capture and inspect authority on tracked workers; `scanUI` owns delivery. A non-empty open is refused before any state change while comparison is active. | +| `openwith.go` | macOS "Open With" delivery: `installOpenWithHandler` / `openInitialFiles` / `openFilesFromOS` over `internal/openwith`, both routed through `fyne.Do` so a launch carrying argv files and a delivery makes one `handleDrop`. `osInputQueue` owns native selections until UI admission and discards undelivered requests at shutdown. The combined pending set is cleared before comparison admission. | | `memlimits.go` | `settings` value, `settingsState` / `ApplySettings`, memory-limit get/set that retune caches and `imaging.SetMaxEncodedBytes`. | | `theme.go` | Settings-facing appearance getter/setter; applies `internal/appearance` modes live. | | `favthumbs.go` | Viewer glue for `favthumbs.Sync` and `gridSink`; carries the owner captured by complete opening or committed saving, captures the saved preview limit (default 1000), and retires active work on edits; per-request completion plus all-pass worker tracking and terminal shutdown cancellation. | @@ -784,10 +784,13 @@ Apple Store builds; `native_other.go` preserves ordinary URI behavior. `manifest.go` stores each distinct scope once while retaining every ordered source occurrence. Session and Favorite persistence validate the complete manifest against their saved membership before publishing restored sources. -The Darwin open picker captures native URL bookmarks on its tracked chooser -worker; folder scans pass captured directory authority to children. Native -save/Open With/drop capture, permission reselection, writes, source versions -and analysis transfer remain open in the active Apple Store plan. +`selection.go` owns native selected URLs through admission, worker-side bookmark +capture and exact-once release, including active cancellation and duplicate +occurrences. The Darwin open picker captures native URL bookmarks on its tracked +chooser worker; Open With/Dock selections use the collection worker. Folder scans +pass captured directory authority to children. Native save/window-drop capture, +permission reselection, writes, source versions and analysis transfer remain open +in the active Apple Store plan. ### `internal/macworker` @@ -1141,8 +1144,8 @@ viewer's handler and flushes in the same critical section. | File | Responsibility | |------|----------------| -| `openwith.go` | The queue (`Deliver` / `SetHandler`) and `URIsFromFileURLs`. | -| `openwith_darwin.{go,h,m}` | `Install` / `DelegateRespondsToOpen` + the `application:openURLs:` / `application:openFiles:` graft. | +| `openwith.go` | The queue (`Deliver` / `SetHandler` / terminal `Stop`) and `URIsFromFileURLs`; pending native ownership is discarded at shutdown. | +| `openwith_darwin.{go,h,m}` | `Install` / `DelegateRespondsToOpen` + the `application:openURLs:` / `application:openFiles:` graft; Apple Store deliveries retain original native URLs until worker capture or discard. | | `openwith_notdarwin.go` | Both report false; other OSes use `argv`. | ### `internal/filescan` diff --git a/internal/fileaccess/selection.go b/internal/fileaccess/selection.go new file mode 100644 index 00000000..78b4769b --- /dev/null +++ b/internal/fileaccess/selection.go @@ -0,0 +1,106 @@ +package fileaccess + +import ( + "context" + "errors" + "sync" + + "fyne.io/fyne/v2" +) + +type selectionURI struct { + fyne.URI + capture func(context.Context) (Record, error) + release func() + mu sync.Mutex + active, closed bool +} + +// NewSelection transfers ownership of a native selected URL to an input batch. +// Its capture callback runs only on the consuming worker; release retires the URL. +func NewSelection(uri fyne.URI, capture func(context.Context) (Record, error), release func()) fyne.URI { + return &selectionURI{URI: uri, capture: capture, release: release} +} + +// NeedsCapture identifies selected URLs whose metadata must be captured off UI. +func NeedsCapture(uri fyne.URI) bool { _, ok := uri.(*selectionURI); return ok } + +// CaptureSelected consumes selected input into immutable source URIs. +func CaptureSelected(ctx context.Context, files []fyne.URI) ([]fyne.URI, error) { + defer ReleaseSelected(files) + result := make([]fyne.URI, len(files)) + captured := make(map[*selectionURI]fyne.URI) + for i, file := range files { + if err := ctx.Err(); err != nil { + return nil, err + } + if file == nil { + return nil, errors.New("selected input contains nil URI") + } + selected, ok := file.(*selectionURI) + if !ok { + result[i] = file + continue + } + source, ok := captured[selected] + if !ok { + var err error + source, err = selected.consume(ctx) + if err != nil { + return nil, err + } + captured[selected] = source + } + result[i] = source + } + return result, ctx.Err() +} + +var errSelectionClosed = errors.New("selected input is already consumed or closed") + +func (s *selectionURI) consume(ctx context.Context) (fyne.URI, error) { + s.mu.Lock() + if s.closed || s.active { + s.mu.Unlock() + return nil, errSelectionClosed + } + s.active = true + s.mu.Unlock() + record, err := s.capture(ctx) + s.mu.Lock() + discarded := s.closed + s.active, s.closed = false, true + s.mu.Unlock() + s.release() + if err != nil { + return nil, err + } + if err := ctx.Err(); err != nil { + return nil, err + } + if discarded { + return nil, errSelectionClosed + } + if len(record.Bookmark) == 0 { + return nil, errors.New("selected input has no captured authority") + } + return FromRecord(record) +} + +// ReleaseSelected discards unstarted input without waiting on an active capture. +// An active capture retains native ownership until its callback has returned. +func ReleaseSelected(files []fyne.URI) { + for _, file := range files { + s, ok := file.(*selectionURI) + if !ok { + continue + } + s.mu.Lock() + release := !s.closed && !s.active + s.closed = true + s.mu.Unlock() + if release { + s.release() + } + } +} diff --git a/internal/fileaccess/selection_test.go b/internal/fileaccess/selection_test.go new file mode 100644 index 00000000..d1ba4160 --- /dev/null +++ b/internal/fileaccess/selection_test.go @@ -0,0 +1,83 @@ +package fileaccess + +import ( + "context" + "errors" + "sync/atomic" + "testing" + + "fyne.io/fyne/v2" + "fyne.io/fyne/v2/storage" +) + +func TestSelectedInputCapturesBeforeReleaseAndPreservesOccurrences(t *testing.T) { + selected := storage.NewFileURI("/selected/photo.jpg") + released, captured := 0, 0 + input := NewSelection(selected, func(_ context.Context) (Record, error) { + if released != 0 { + t.Fatal("URL released before capture") + } + captured++ + return Record{URI: selected.String(), Bookmark: []byte("scope")}, nil + }, func() { released++ }) + files := []fyne.URI{input, selected, input} + got, err := CaptureSelected(context.Background(), files) + if err != nil { + t.Fatal(err) + } + if len(got) != 3 || string(Snapshot(got[0]).Bookmark) != "scope" || got[0] != got[2] || got[1] != selected { + t.Fatalf("captured occurrences: %v", got) + } + ReleaseSelected(files) + if captured != 1 || released != 1 { + t.Fatalf("captured=%d released=%d", captured, released) + } +} + +func TestSelectedInputCloseWaitsForActiveNativeCapture(t *testing.T) { + entered, finish := make(chan struct{}), make(chan struct{}) + var released atomic.Int32 + selected := storage.NewFileURI("/selected/photo.jpg") + input := NewSelection(selected, func(_ context.Context) (Record, error) { + close(entered) + <-finish + if released.Load() != 0 { + return Record{}, errors.New("native URL retired during capture") + } + return Record{URI: selected.String(), Bookmark: []byte("scope")}, nil + }, func() { released.Add(1) }) + done := make(chan error, 1) + go func() { _, err := CaptureSelected(context.Background(), []fyne.URI{input}); done <- err }() + <-entered + ReleaseSelected([]fyne.URI{input, input}) + activeRelease := released.Load() + close(finish) + err := <-done + if activeRelease != 0 || released.Load() != 1 || !errors.Is(err, errSelectionClosed) { + t.Fatalf("active releases=%d final releases=%d result=%v", activeRelease, released.Load(), err) + } +} + +func TestSelectedInputFailureDiscardsCompleteBatch(t *testing.T) { + for _, cancelled := range []bool{false, true} { + ctx, cancel := context.WithCancel(context.Background()) + if cancelled { + cancel() + } + released, calls := 0, 0 + var files []fyne.URI + for range 3 { + files = append(files, NewSelection(storage.NewFileURI("/photo.jpg"), func(_ context.Context) (Record, error) { + calls++ + return Record{}, errors.New("capture failed") + }, func() { released++ })) + } + got, err := CaptureSelected(ctx, files) + cancel() + releasedAtReturn := released + ReleaseSelected(files) + if err == nil || got != nil || releasedAtReturn != 3 || released != 3 || (cancelled && calls != 0) || (!cancelled && calls != 1) { + t.Fatalf("cancelled=%v result=%v err=%v calls=%d release at return=%d final=%d", cancelled, got, err, calls, releasedAtReturn, released) + } + } +} diff --git a/internal/filepicker/filepicker.go b/internal/filepicker/filepicker.go index bffbbe48..ba6de19a 100644 --- a/internal/filepicker/filepicker.go +++ b/internal/filepicker/filepicker.go @@ -10,6 +10,7 @@ import ( "encoding/json" "errors" "fmt" + "net/url" "os/exec" "path" "path/filepath" @@ -271,6 +272,11 @@ func decodeScopedSelection(out []byte) ([]fyne.URI, error) { if len(record.Bookmark) == 0 || record.Relative != "" { return nil, errors.New("file chooser did not capture its selected scope") } + nativeURL, err := url.Parse(record.URI) + if err != nil || nativeURL.Scheme != "file" || (nativeURL.Host != "" && nativeURL.Host != "localhost") || nativeURL.Path == "" { + return nil, errors.New("file chooser returned an invalid native file URL") + } + record.URI = storage.NewFileURI(nativeURL.Path).String() source, err := fileaccess.FromRecord(record) if err != nil { return nil, err diff --git a/internal/filepicker/filepicker_test.go b/internal/filepicker/filepicker_test.go index 578577ec..678a23f5 100644 --- a/internal/filepicker/filepicker_test.go +++ b/internal/filepicker/filepicker_test.go @@ -3,6 +3,7 @@ package filepicker import ( "encoding/json" "errors" + "net/url" "os" "os/exec" "slices" @@ -305,7 +306,7 @@ func TestZenityResult_DistinguishesCancelAndFailure(t *testing.T) { func TestDecodePickedPaths_ScopedRecords(t *testing.T) { source := storage.NewFileURI("/tmp/ scoped café folder ") - payload, err := json.Marshal([]fileaccess.Record{{URI: source.String(), Bookmark: []byte("selected-scope"), Directory: true}}) + payload, err := json.Marshal([]fileaccess.Record{{URI: (&url.URL{Scheme: "file", Path: source.Path()}).String(), Bookmark: []byte("selected-scope"), Directory: true}}) if err != nil { t.Fatal(err) } @@ -326,3 +327,24 @@ func TestDecodePickedPaths_ScopedRecords(t *testing.T) { t.Fatalf("lost selected authority: %+v", record) } } + +func TestScopedSelectionDecodesNativeURLExactlyOnce(t *testing.T) { + for _, path := range []string{"/photos/space café.jpg", "/photos/literal%20name.jpg", "/photos/hash#question?.jpg"} { + native := (&url.URL{Scheme: "file", Path: path}).String() + payload, err := json.Marshal([]fileaccess.Record{{URI: native, Bookmark: []byte("scope")}}) + if err != nil { + t.Fatal(err) + } + selected, err := decodeScopedSelection(payload) + if err != nil { + t.Fatal(err) + } + if len(selected) != 1 || selected[0].Path() != path { + t.Fatalf("native URL %q became %v", native, selected) + } + restored, err := fileaccess.FromRecord(fileaccess.Snapshot(selected[0])) + if err != nil || restored.Path() != path { + t.Fatalf("persistent URI decoded twice: %v, %v", restored, err) + } + } +} diff --git a/internal/openwith/openwith.go b/internal/openwith/openwith.go index 14895bea..38f40d40 100644 --- a/internal/openwith/openwith.go +++ b/internal/openwith/openwith.go @@ -34,6 +34,8 @@ import ( "fyne.io/fyne/v2" "fyne.io/fyne/v2/storage" + + "github.com/frathe/picfetch/internal/fileaccess" ) // queue buffers URIs that arrive before a handler is installed, and hands @@ -42,6 +44,7 @@ type queue struct { mu sync.Mutex pending []fyne.URI handler func([]fyne.URI) + stopped bool } // Deliver adds uris to the queue: passed to the installed handler @@ -54,6 +57,11 @@ func (q *queue) Deliver(uris []fyne.URI) { } q.mu.Lock() + if q.stopped { + q.mu.Unlock() + fileaccess.ReleaseSelected(uris) + return + } h := q.handler if h == nil { q.pending = append(q.pending, uris...) @@ -70,13 +78,15 @@ func (q *queue) Deliver(uris []fyne.URI) { // outside the lock. Doing both under one lock acquisition is the point: a // separate "drain, then install" pair would lose anything that arrives via // Deliver between the two steps. SetHandler(nil) clears the handler -// (used at shutdown) without discarding whatever is currently pending - +// without discarding whatever is currently pending - // there's no handler to flush it to, so it's left in the queue rather than // taken and dropped, and picks up right where it left off the next time -// something is delivered or a real handler is installed. +// something is delivered or a real handler is installed. Stop instead discards +// pending ownership and refuses late delivery; SetHandler reopens admission. func (q *queue) SetHandler(h func([]fyne.URI)) { q.mu.Lock() q.handler = h + q.stopped = false var pending []fyne.URI if h != nil { @@ -152,3 +162,16 @@ func URIsFromFileURLs(raw []string) []fyne.URI { return uris } + +// Stop closes native-event admission and discards pending ownership without +// waiting for capture. SetHandler explicitly attaches a fresh viewer afterward. +func Stop() { defaultQueue.stop() } + +func (q *queue) stop() { + q.mu.Lock() + q.stopped, q.handler = true, nil + pending := q.pending + q.pending = nil + q.mu.Unlock() + fileaccess.ReleaseSelected(pending) +} diff --git a/internal/openwith/openwith_darwin.go b/internal/openwith/openwith_darwin.go index 28c9e42c..b0b7f1c7 100644 --- a/internal/openwith/openwith_darwin.go +++ b/internal/openwith/openwith_darwin.go @@ -12,7 +12,17 @@ package openwith */ import "C" -import "unsafe" +import ( + "context" + "encoding/json" + "errors" + "unsafe" + + "fyne.io/fyne/v2" + + "github.com/frathe/picfetch/internal/distribution" + "github.com/frathe/picfetch/internal/fileaccess" +) // picfetchDeliverOpenURLs is the single entry point from Objective-C: both // grafted delegate methods funnel their absolute file:// URL strings here. @@ -127,3 +137,66 @@ var _ = []any{ // here records that the missing call site is the point rather than an // oversight, and keeps an unused-function inspection from flagging it. var _ = picfetchDeliverOpenURLs + +//export picfetchUsesScopedOpen +func picfetchUsesScopedOpen() C.int { + if distribution.AppleAppStore { + return 1 + } + return 0 +} + +//export picfetchDeliverSelectedURLs +func picfetchDeliverSelectedURLs(owners *unsafe.Pointer, addresses **C.char, n C.int) { + if owners == nil || addresses == nil || n <= 0 { + return + } + native := unsafe.Slice(owners, int(n)) + names := unsafe.Slice(addresses, int(n)) + files := make([]fyne.URI, 0, int(n)) + for i, owner := range native { + if owner == nil { + continue + } + if names[i] == nil { + C.picfetchReleaseSelectedURL(owner) + continue + } + uris := URIsFromFileURLs([]string{C.GoString(names[i])}) + if len(uris) != 1 { + C.picfetchReleaseSelectedURL(owner) + continue + } + files = append(files, fileaccess.NewSelection(uris[0], func(ctx context.Context) (fileaccess.Record, error) { + if err := ctx.Err(); err != nil { + return fileaccess.Record{}, err + } + var failure *C.char + data := C.picfetchCaptureSelectedURL(owner, &failure) + defer C.free(unsafe.Pointer(data)) + defer C.free(unsafe.Pointer(failure)) + if failure != nil { + return fileaccess.Record{}, errors.New(C.GoString(failure)) + } + if data == nil { + return fileaccess.Record{}, errors.New("selected URL capture failed") + } + var record fileaccess.Record + if err := json.Unmarshal([]byte(C.GoString(data)), &record); err != nil { + return fileaccess.Record{}, err + } + // Foundation's URL string is escaped; Fyne persists decoded file identities. + record.URI = uris[0].String() + return record, ctx.Err() + }, func() { C.picfetchReleaseSelectedURL(owner) })) + } + Deliver(files) +} + +func testInvokeSelectedURLs(urls []string) { + buf, release := cStrings(urls) + defer release() + C.picfetchTestInvokeSelectedURLs(buf, C.int(len(urls))) +} + +var _ = []any{picfetchUsesScopedOpen, picfetchDeliverSelectedURLs, testInvokeSelectedURLs} diff --git a/internal/openwith/openwith_darwin.h b/internal/openwith/openwith_darwin.h index 63fff60c..4eb2ba0b 100644 --- a/internal/openwith/openwith_darwin.h +++ b/internal/openwith/openwith_darwin.h @@ -28,4 +28,9 @@ int picfetchDelegateRespondsToOpen(void); void picfetchTestInvokeOpenURLs(const char **urls, int n); void picfetchTestInvokeOpenFiles(const char **paths, int n); +// Capture and retirement operate on the original retained NSURL. +char *picfetchCaptureSelectedURL(void *url, char **failure); +void picfetchReleaseSelectedURL(void *url); +void picfetchTestInvokeSelectedURLs(const char **urls, int n); + #endif // PICFETCH_OPENWITH_DARWIN_H diff --git a/internal/openwith/openwith_darwin.m b/internal/openwith/openwith_darwin.m index 8c18733c..1f4c2dcf 100644 --- a/internal/openwith/openwith_darwin.m +++ b/internal/openwith/openwith_darwin.m @@ -80,11 +80,71 @@ static void deliverAbsoluteStrings(NSArray *strings) { free(buf); } +// Transfer original native URLs, not reconstructed paths. Go assumes ownership +// of each retained object even when it refuses the associated identity. +static void deliverSelectedURLs(NSArray *urls) { + NSUInteger count = urls.count; + if (count == 0 || count > INT_MAX) return; + void **owners = calloc(count, sizeof(void *)); + char **addresses = calloc(count, sizeof(char *)); + if (!owners || !addresses) { + free(owners); free(addresses); + for (NSURL *url in urls) [url stopAccessingSecurityScopedResource]; + return; + } + int n = 0; + for (NSURL *url in urls) { + const char *address = url.absoluteString.UTF8String; + char *copy = address ? strdup(address) : NULL; + if (!copy) { [url stopAccessingSecurityScopedResource]; continue; } + owners[n] = (__bridge_retained void *)url; + addresses[n++] = copy; + } + if (n) picfetchDeliverSelectedURLs(owners, addresses, n); + for (int i = 0; i < n; i++) free(addresses[i]); + free(addresses); free(owners); +} + +char *picfetchCaptureSelectedURL(void *owner, char **failure) { + @autoreleasepool { + NSURL *url = (__bridge NSURL *)owner; + NSError *error = nil; + NSNumber *directory = nil; + if (![url getResourceValue:&directory forKey:NSURLIsDirectoryKey error:&error] || !directory) { + *failure = strdup(error ? error.description.UTF8String : "selected URL directory inspection failed"); + return NULL; + } + NSData *bookmark = [url bookmarkDataWithOptions:NSURLBookmarkCreationWithSecurityScope + includingResourceValuesForKeys:nil relativeToURL:nil error:&error]; + if (!bookmark) { + *failure = strdup(error ? error.description.UTF8String : "selected URL bookmark capture failed"); + return NULL; + } + NSDictionary *record = @{@"uri":url.absoluteString, + @"bookmark":[bookmark base64EncodedStringWithOptions:0], @"directory":directory}; + NSData *data = [NSJSONSerialization dataWithJSONObject:record options:0 error:&error]; + if (!data) { + *failure = strdup(error ? error.description.UTF8String : "selected URL serialization failed"); + return NULL; + } + NSString *json = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding]; + return json ? strdup(json.UTF8String) : NULL; + } +} + +void picfetchReleaseSelectedURL(void *owner) { + @autoreleasepool { + NSURL *url = (__bridge_transfer NSURL *)owner; + [url stopAccessingSecurityScopedResource]; + } +} + // picfetchOpenURLs is the IMP grafted on as -application:openURLs:, the // modern (10.13+) open-document callback. Fast enumeration over a nil array // is a no-op, so a nil urls argument simply delivers nothing. static void picfetchOpenURLs(id self, SEL _cmd, id sender, NSArray *urls) { @autoreleasepool { + if (picfetchUsesScopedOpen()) { deliverSelectedURLs(urls); return; } NSMutableArray *absolute = [NSMutableArray arrayWithCapacity:[urls count]]; for (NSURL *url in urls) { NSString *s = [url absoluteString]; @@ -106,6 +166,17 @@ static void picfetchOpenURLs(id self, SEL _cmd, id sender, NSArray *url // when a delegate answers both, so there is no double delivery. static void picfetchOpenFiles(id self, SEL _cmd, id sender, NSArray *paths) { @autoreleasepool { + if (picfetchUsesScopedOpen()) { + NSMutableArray *urls = [NSMutableArray array]; + for (NSString *path in paths) { + if (!path.length) continue; + NSURL *url = [NSURL fileURLWithPath:path]; + if (url) [urls addObject:url]; + } + deliverSelectedURLs(urls); + [(NSApplication *)sender replyToOpenOrPrint:NSApplicationDelegateReplySuccess]; + return; + } NSMutableArray *absolute = [NSMutableArray arrayWithCapacity:[paths count]]; for (NSString *path in paths) { // +fileURLWithPath: answers nil for an empty path on current @@ -207,3 +278,15 @@ void picfetchTestInvokeOpenFiles(const char **paths, int n) { picfetchOpenFiles(nil, @selector(application:openFiles:), nil, arr); } } + +void picfetchTestInvokeSelectedURLs(const char **urls, int n) { + @autoreleasepool { + NSMutableArray *selected = [NSMutableArray array]; + for (int i = 0; urls && i < n; i++) { + if (!urls[i]) continue; + NSURL *url = [NSURL URLWithString:[NSString stringWithUTF8String:urls[i]]]; + if (url) [selected addObject:url]; + } + deliverSelectedURLs(selected); + } +} diff --git a/internal/openwith/openwith_darwin_test.go b/internal/openwith/openwith_darwin_test.go index 2bdc3d14..9eaec172 100644 --- a/internal/openwith/openwith_darwin_test.go +++ b/internal/openwith/openwith_darwin_test.go @@ -3,8 +3,16 @@ package openwith import ( + "context" + "net/url" + "os" + "path/filepath" "testing" + "fyne.io/fyne/v2/storage" + + "github.com/frathe/picfetch/internal/fileaccess" + "fyne.io/fyne/v2" ) @@ -22,6 +30,7 @@ func captureDelivered(t *testing.T) func() []fyne.URI { got = append(got, uris...) }) + t.Cleanup(func() { fileaccess.ReleaseSelected(got) }) return func() []fyne.URI { return got } } @@ -184,5 +193,29 @@ func TestInvokeOpenURLs_BuffersWhenNoHandlerIsInstalledYet(t *testing.T) { got = append(got, uris...) }) + defer fileaccess.ReleaseSelected(got) assertPaths(t, got, "/tmp/a.jpg") } + +func TestNativeSelectedURLCaptureKeepsOriginalAuthority(t *testing.T) { + delivered := captureDelivered(t) + path := filepath.Join(t.TempDir(), "chosen café.jpg") + if err := os.WriteFile(path, []byte("fixture"), 0600); err != nil { + t.Fatal(err) + } + uri := storage.NewFileURI(path) + testInvokeSelectedURLs([]string{(&url.URL{Scheme: "file", Path: path}).String()}) + files := delivered() + if len(files) != 1 || !fileaccess.NeedsCapture(files[0]) { + t.Fatalf("native input was flattened: %v", files) + } + defer fileaccess.ReleaseSelected(files) + captured, err := fileaccess.CaptureSelected(context.Background(), files) + if err != nil { + t.Fatal(err) + } + record := fileaccess.Snapshot(captured[0]) + if len(record.Bookmark) == 0 || record.Directory || captured[0].Path() != uri.Path() { + t.Fatalf("captured path=%q directory=%v bookmark bytes=%d", captured[0].Path(), record.Directory, len(record.Bookmark)) + } +} diff --git a/internal/openwith/openwith_test.go b/internal/openwith/openwith_test.go index df3e2f02..e34a3d0e 100644 --- a/internal/openwith/openwith_test.go +++ b/internal/openwith/openwith_test.go @@ -1,11 +1,16 @@ package openwith import ( + "context" "sync" "sync/atomic" "testing" "time" + "fyne.io/fyne/v2/storage" + + "github.com/frathe/picfetch/internal/fileaccess" + "fyne.io/fyne/v2" "github.com/frathe/picfetch/internal/uitest" @@ -17,9 +22,12 @@ import ( // isn't an exported test seam. func reset() { defaultQueue.mu.Lock() + pending := defaultQueue.pending defaultQueue.pending = nil defaultQueue.handler = nil + defaultQueue.stopped = false defaultQueue.mu.Unlock() + fileaccess.ReleaseSelected(pending) } func TestQueue_DeliverBeforeSetHandler_BuffersThenFlushesInArrivalOrder(t *testing.T) { @@ -319,3 +327,27 @@ func TestURIsFromFileURLs(t *testing.T) { }) } } + +func TestQueueStopDiscardsPendingAndLateNativeSelections(t *testing.T) { + q := &queue{} + released := 0 + makeInput := func() fyne.URI { + return fileaccess.NewSelection(storage.NewFileURI("/selected.jpg"), func(_ context.Context) (fileaccess.Record, error) { + t.Fatal("queue performed native capture") + return fileaccess.Record{}, nil + }, func() { released++ }) + } + first := makeInput() + q.Deliver([]fyne.URI{first, first}) + q.stop() + q.Deliver([]fyne.URI{makeInput()}) + q.stop() + if released != 2 || len(q.pending) != 0 { + t.Fatalf("released=%d pending=%v", released, q.pending) + } + q.SetHandler(func(uris []fyne.URI) { fileaccess.ReleaseSelected(uris) }) + q.Deliver([]fyne.URI{makeInput()}) + if released != 3 { + t.Fatal("fresh handler did not receive ownership") + } +} diff --git a/internal/ui/build.go b/internal/ui/build.go index b018ecd9..aa30c4cb 100644 --- a/internal/ui/build.go +++ b/internal/ui/build.go @@ -112,6 +112,8 @@ func buildViewer(application fyne.App, startup startupState) (*viewer, fyne.Wind } view.chooserUI = fyneChooserQueue{} + view.scanUI = fyneChooserQueue{} + view.osInputs.queue = fyneChooserQueue{} view.maximizeWindow = winpos.Maximize view.unmaximizeWindow = winpos.Unmaximize view.regionCopyDo = fyne.Do diff --git a/internal/ui/collection_lifecycle_test.go b/internal/ui/collection_lifecycle_test.go index 8a43cf68..7c601750 100644 --- a/internal/ui/collection_lifecycle_test.go +++ b/internal/ui/collection_lifecycle_test.go @@ -284,6 +284,10 @@ func collectionPreparationLifecycle(t *testing.T) { } committed, visit := v.state.Observe(), v.browsing.current().binding unblock() + if preparation == "scan" || preparation == "replay" { + v.scanWorkers.Wait() + settleScan(t, v) + } waitHandle(t, "retired preparation", done) v.settleHEIC() v.display.Settle() diff --git a/internal/ui/collection_replay_test.go b/internal/ui/collection_replay_test.go index 4ea83eac..58dc3811 100644 --- a/internal/ui/collection_replay_test.go +++ b/internal/ui/collection_replay_test.go @@ -289,6 +289,8 @@ func TestCollectionReplay(t *testing.T) { v.cancelScan() } unblock() + v.scanWorkers.Wait() + settleScan(t, v) ctx, cancel := context.WithTimeout(context.Background(), testTimeout) defer cancel() if err := completion.Wait(ctx); err != nil { diff --git a/internal/ui/drop.go b/internal/ui/drop.go index e9fff40e..f79f1aeb 100644 --- a/internal/ui/drop.go +++ b/internal/ui/drop.go @@ -12,6 +12,7 @@ import ( "fyne.io/fyne/v2/lang" "fyne.io/fyne/v2/storage" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/filescan" "github.com/frathe/picfetch/internal/heic" "github.com/frathe/picfetch/internal/imaging" @@ -87,6 +88,12 @@ const ( ) func (v *viewer) openCollection(uris []fyne.URI, favoriteDir string, kind collectionInputKind) { + inputOwned := true + defer func() { + if inputOwned { + fileaccess.ReleaseSelected(uris) + } + }() if len(uris) == 0 { return } @@ -94,6 +101,7 @@ func (v *viewer) openCollection(uris []fyne.URI, favoriteDir string, kind collec return } + uris = slices.Clone(uris) // Admission precedes collection replacement and all cancellation effects. v.favorites.CancelOpen() // A replacement request must not inherit --slideshow from the launch @@ -118,6 +126,7 @@ func (v *viewer) openCollection(uris []fyne.URI, favoriteDir string, kind collec v.invalidateSort() v.invalidateLoad() token, scanDone := v.scanOp.begin(v.heicContext(context.Background())) + workerDone := v.scanWorkerDone.Begin() v.syncMenus() v.scanOp.label.SetText(lang.L("Scanning... 0 images")) @@ -135,15 +144,39 @@ func (v *viewer) openCollection(uris []fyne.URI, favoriteDir string, kind collec // scan. See SetMaxScan's doc comment. maxScan := v.settings.maxScan - hasDirs := false - for _, u := range uris { - if canList, err := storage.CanList(u); err == nil && canList { - hasDirs = true - break + protected := false + for _, uri := range uris { + protected = protected || fileaccess.HasScope(uri) || fileaccess.NeedsCapture(uri) + } + inspect := func() (hasDirs, singleImage bool) { + for _, uri := range uris { + resolved, release, err := fileaccess.Acquire(token.Context(), uri) + if err != nil { + continue + } + directory, err := storage.CanList(resolved) + hasDirs = err == nil && directory + if !hasDirs && len(uris) == 1 { + // MIME fallback may read bytes; keep the scope through that read. + singleImage = imaging.IsSupportedImage(resolved) || heic.IsExtension(resolved.Extension()) + } + release() + if hasDirs { + return true, false + } } + return hasDirs, singleImage + } + hasDirs, expandSiblings := false, false + inspectInput := func() { + var singleImage bool + hasDirs, singleImage = inspect() + expandSiblings = kind == discoverCollection && !merging && !hasDirs && singleImage + } + if !protected { + inspectInput() } - expandSiblings := kind == discoverCollection && !merging && !hasDirs && len(uris) == 1 && (imaging.IsSupportedImage(uris[0]) || heic.IsExtension(uris[0].Extension())) gather := filescan.ImagesWithAdmission if kind == replayCollection { gather = filescan.ReplayWithAdmission @@ -249,7 +282,7 @@ func (v *viewer) openCollection(uris []fyne.URI, favoriteDir string, kind collec for _, uri := range uris { explicitHEIC = explicitHEIC || heic.IsExtension(uri.Extension()) } - if !hasDirs && !expandSiblings && (check == nil || !explicitHEIC) { + if !protected && !hasDirs && !expandSiblings && (check == nil || !explicitHEIC) { // nil progress: this path is synchronous and instantaneous, so // there's nothing to show, and it avoids calling fyne.Do from the // UI goroutine. Multi-file loose drops and merge-mode single @@ -257,13 +290,27 @@ func (v *viewer) openCollection(uris []fyne.URI, favoriteDir string, kind collec // directory does not — that listing belongs on the goroutine // below, same as a folder drop. images, truncated := scan(nil) - fyne.Do(func() { + v.scanUI.Do(func() { v.applyScanResult(token, merging, uris, images, truncated, maxScan, scanDone, favoriteDir, skipped, sourceOrder) }) + workerDone() return } - go func() { + inputOwned = false + v.scanWorkers.Go(func() { + defer workerDone() + selected := uris + defer fileaccess.ReleaseSelected(selected) + if protected { + captured, err := fileaccess.CaptureSelected(token.Context(), selected) + if err != nil { + v.scanUI.Do(func() { v.failSelectedInput(token, err, scanDone) }) + return + } + uris = captured + inspectInput() + } // token.Context() is what lets a superseded scan (a newer drop, or // an explicit cancel - see cancelScan) stop walking the tree instead // of racing storage.List calls to completion for a result nobody @@ -271,7 +318,7 @@ func (v *viewer) openCollection(uris []fyne.URI, favoriteDir string, kind collec // discard the result anyway. The same context cancels a sibling // listing (Siblings) as a recursive Images walk. images, truncated := scan(func(n int) { - fyne.Do(func() { + v.scanUI.Do(func() { if !token.Current() { return } @@ -279,10 +326,10 @@ func (v *viewer) openCollection(uris []fyne.URI, favoriteDir string, kind collec }) }) - fyne.Do(func() { + v.scanUI.Do(func() { v.applyScanResult(token, merging, uris, images, truncated, maxScan, scanDone, favoriteDir, skipped, sourceOrder) }) - }() + }) } func admittedSourceOrder(order, images, unavailable []fyne.URI) []fyne.URI { @@ -461,3 +508,22 @@ func (v *viewer) applyScannedCollection(merging bool, images, dropped []fyne.URI }) }) } + +// A failed native capture cannot publish a partial selection or replace the +// existing collection. Its resources have already retired on the worker. +func (v *viewer) failSelectedInput(token requestlife.Token, err error, done func()) { + defer done() + defer token.Release() + if !token.Current() { + return + } + v.scanOp.finish() + v.pendingPictureFrame = false + v.explorerInput.pendingLaunch = false + if v.FileCount() == 0 { + v.presentDropzone() + } + v.syncMenus() + fyne.LogError("failed to capture selected file access", err) + v.ShowToast(fmt.Sprintf(lang.L("could not open selected files: %v"), err)) +} diff --git a/internal/ui/drop_test.go b/internal/ui/drop_test.go index adf2544a..1ce56dd8 100644 --- a/internal/ui/drop_test.go +++ b/internal/ui/drop_test.go @@ -2,14 +2,18 @@ package ui import ( "context" + "errors" "fmt" "image/color" "os" "path/filepath" "strings" "sync" + "sync/atomic" "testing" + "github.com/frathe/picfetch/internal/fileaccess" + "fyne.io/fyne/v2" "fyne.io/fyne/v2/storage" @@ -550,6 +554,8 @@ func TestHandleDrop_SupersededScanGoroutineExits(t *testing.T) { dropAndWait(t, v, jpegB) releaseListing() + v.scanWorkers.Wait() + settleScan(t, v) waitHandle(t, "the superseded scan's goroutine to exit", scanA) if visitedChild { @@ -708,3 +714,60 @@ func TestHandleDrop_SiblingScanTruncationToast(t *testing.T) { }) } } + +func TestNativeSelectionOpeningOwnership(t *testing.T) { + t.Run("cancel_during_capture", func(t *testing.T) { + v := newTestViewer(t) + original := uitest.TempJPEGURI(t, "original.jpg", 4, 4, color.White) + dropAndWait(t, v, original) + var entered completion.Signal + markEntered := entered.Begin() + gate := make(chan struct{}) + releaseCapture := sync.OnceFunc(func() { close(gate) }) + t.Cleanup(releaseCapture) + var released atomic.Int32 + input := fileaccess.NewSelection(original, func(_ context.Context) (fileaccess.Record, error) { + markEntered() + <-gate + return fileaccess.Record{URI: original.String(), Bookmark: []byte("scope")}, nil + }, func() { released.Add(1) }) + v.handleDrop([]fyne.URI{input}) + waitFor(t, "native capture entered without blocking UI", &entered) + v.cancelScan() + if released.Load() != 0 { + t.Fatal("cancel released an actively used native URL") + } + releaseCapture() + waitForScan(t, v) + if released.Load() != 1 || v.FileCount() != 1 || v.state.Observe().FileAt(0).String() != original.String() { + t.Fatalf("cancelled capture changed collection or leaked URL; releases=%d", released.Load()) + } + }) + t.Run("capture_failure_preserves_collection", func(t *testing.T) { + v := newTestViewer(t) + original := uitest.TempJPEGURI(t, "original.jpg", 4, 4, color.White) + dropAndWait(t, v, original) + released := 0 + input := fileaccess.NewSelection(original, func(_ context.Context) (fileaccess.Record, error) { + return fileaccess.Record{}, errors.New("fixture capture denied") + }, func() { released++ }) + v.handleDrop([]fyne.URI{input}) + waitForScan(t, v) + if released != 1 || v.FileCount() != 1 || v.scanOp.active { + t.Fatal("failed capture did not retire cleanly") + } + }) + t.Run("rejected_input_releases_without_capture", func(t *testing.T) { + v := newTestViewer(t) + v.stopping = true + released := 0 + input := fileaccess.NewSelection(storage.NewFileURI("/selected.jpg"), func(_ context.Context) (fileaccess.Record, error) { + t.Fatal("rejected native input was captured") + return fileaccess.Record{}, nil + }, func() { released++ }) + v.handleDrop([]fyne.URI{input}) + if released != 1 || v.scanOp.done.Begun() { + t.Fatal("rejected native selection was retained or admitted") + } + }) +} diff --git a/internal/ui/harness_test.go b/internal/ui/harness_test.go index d0916b2c..1cba81b1 100644 --- a/internal/ui/harness_test.go +++ b/internal/ui/harness_test.go @@ -167,6 +167,7 @@ func newTestUIWithPolicy(t *testing.T, policy launch.Policy) (v *viewer, win fyn v.exif.SetUIQueue(&uitest.UIQueue{}) v.fileWork.ui = &uitest.UIQueue{} v.chooserUI = &uitest.UIQueue{} + v.scanUI = &uitest.UIQueue{} v.clipboardWork.ui = &uitest.UIQueue{} // The auto-hide timer must never fire on its own mid-suite: its inline @@ -322,6 +323,8 @@ func drain(t *testing.T, v *viewer) { // Native chooser delivery can begin scan, which can begin sort and load. // Drain that delivery first, then observe the current root generations. // Display was stopped above and joins all its retired workers afterward. + v.scanWorkers.Wait() + settleScan(t, v) for _, c := range []struct { name string sig *completion.Signal @@ -484,6 +487,7 @@ func waitForScan(t *testing.T, v *viewer) { t.Fatal("the scan never started") } + settleScan(t, v) waitFor(t, "the scan", &v.scanOp.done) } @@ -725,3 +729,13 @@ func namesOfURIs(files []fyne.URI) []string { func beginPendingImageLoad(v *viewer) { v.display.Load(display.Request{Source: storage.NewFileURI("/picfetch-pending-test.png")}) } + +func settleScan(t *testing.T, v *viewer) { + t.Helper() + for { + waitFor(t, "the current scan worker", &v.scanWorkerDone) + if !v.scanUI.Drain() { + return + } + } +} diff --git a/internal/ui/openfiles.go b/internal/ui/openfiles.go index 04d906db..eaa2840e 100644 --- a/internal/ui/openfiles.go +++ b/internal/ui/openfiles.go @@ -61,6 +61,7 @@ func (v *viewer) runFileChooser(token requestlife.Token, choose func() ([]fyne.U // until it returns, so shutdown must not wait for that external interaction. func (v *viewer) closeOpenChooser() { v.openChooserClosed = true + v.osInputs.stop() v.openChooserLifecycle.Invalidate() } diff --git a/internal/ui/openwith.go b/internal/ui/openwith.go index 85965447..03ec15b3 100644 --- a/internal/ui/openwith.go +++ b/internal/ui/openwith.go @@ -5,8 +5,12 @@ package ui import ( + "slices" + "sync" + "fyne.io/fyne/v2" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/openwith" ) @@ -29,7 +33,15 @@ import ( // would fail exactly where this is used. func (v *viewer) installOpenWithHandler() { openwith.SetHandler(func(uris []fyne.URI) { - fyne.Do(func() { v.openFilesFromOS(uris) }) + request := v.osInputs.add(uris) + if request == nil { + return + } + v.osInputs.queue.Do(func() { + if files, ok := v.osInputs.take(request); ok { + v.openFilesFromOS(files) + } + }) }) } @@ -37,7 +49,7 @@ func (v *viewer) installOpenWithHandler() { // installOpenWithHandler's flush already swept it up along with an // "Open With" delivery - the usual macOS cold start. func (v *viewer) openInitialFiles() { - fyne.Do(func() { v.openFilesFromOS(nil) }) + v.osInputs.queue.Do(func() { v.openFilesFromOS(nil) }) } // openFilesFromOS opens uris together with whatever the launch is still @@ -58,3 +70,55 @@ func (v *viewer) openFilesFromOS(uris []fyne.URI) { v.handleDrop(files) } + +// osInputQueue owns URLs handed to a UI callback that may never run at shutdown. +// Taking a request transfers ownership to the admitted collection scan. +type osInputQueue struct { + mu sync.Mutex + pending map[*osInputRequest]struct{} + closed bool + queue chooserUIQueue +} + +type osInputRequest struct{ files []fyne.URI } + +func (q *osInputQueue) add(files []fyne.URI) *osInputRequest { + q.mu.Lock() + if q.closed { + q.mu.Unlock() + fileaccess.ReleaseSelected(files) + return nil + } + if q.pending == nil { + q.pending = make(map[*osInputRequest]struct{}) + } + request := &osInputRequest{files: slices.Clone(files)} + q.pending[request] = struct{}{} + q.mu.Unlock() + return request +} + +func (q *osInputQueue) take(request *osInputRequest) ([]fyne.URI, bool) { + q.mu.Lock() + defer q.mu.Unlock() + if _, ok := q.pending[request]; !ok { + return nil, false + } + delete(q.pending, request) + files := request.files + request.files = nil + return files, true +} + +func (q *osInputQueue) stop() { + q.mu.Lock() + q.closed = true + var files []fyne.URI + for request := range q.pending { + files = append(files, request.files...) + request.files = nil + } + q.pending = nil + q.mu.Unlock() + fileaccess.ReleaseSelected(files) +} diff --git a/internal/ui/openwith_test.go b/internal/ui/openwith_test.go index e82b6f3d..acac5466 100644 --- a/internal/ui/openwith_test.go +++ b/internal/ui/openwith_test.go @@ -1,10 +1,15 @@ package ui import ( + "context" "image/color" "slices" "testing" + "fyne.io/fyne/v2/storage" + + "github.com/frathe/picfetch/internal/fileaccess" + "fyne.io/fyne/v2" "github.com/frathe/picfetch/internal/openwith" @@ -199,3 +204,27 @@ func TestOpenWithHandler_DeliveryMergesWhenMergeModeIsOn(t *testing.T) { t.Errorf("displayed file = %q, want the just-merged b.jpg", got) } } + +func TestOpenWithShutdownReleasesUndeliveredNativeInput(t *testing.T) { + v := newTestViewer(t) + queue := &uitest.UIQueue{} + v.osInputs.queue = queue + v.installOpenWithHandler() + released := 0 + input := fileaccess.NewSelection(storage.NewFileURI("/selected.jpg"), func(_ context.Context) (fileaccess.Record, error) { + t.Fatal("shutdown input reached native capture") + return fileaccess.Record{}, nil + }, func() { released++ }) + openwith.Deliver([]fyne.URI{input}) + if released != 0 { + t.Fatal("queued URL was released before admission") + } + v.closeOpenChooser() + if released != 1 { + t.Fatal("shutdown kept native scope until UI drain") + } + queue.Drain() + if released != 1 || v.scanOp.done.Begun() { + t.Fatal("stale UI delivery revived an input") + } +} diff --git a/internal/ui/run.go b/internal/ui/run.go index a25cb8e9..7fd65af6 100644 --- a/internal/ui/run.go +++ b/internal/ui/run.go @@ -213,9 +213,8 @@ func registerShutdown(application fyne.App, view *viewer) { // state that outlives the viewer: openwith's queue is // process-global, so a delivery landing mid-shutdown would // otherwise reach a viewer whose window is already going away. - // Anything still buffered stays buffered - SetHandler(nil) does - // not discard it - which costs nothing, as the process is exiting. - openwith.SetHandler(nil) + // Native selections still buffered own implicit scope and must retire. + openwith.Stop() session.Save(application, view.state.Observe().Capture(collectionSourceOrder)) preferences.Save(application, view.currentPreferences()) diff --git a/internal/ui/viewer.go b/internal/ui/viewer.go index 8f007537..0f060b9e 100644 --- a/internal/ui/viewer.go +++ b/internal/ui/viewer.go @@ -220,7 +220,11 @@ type viewer struct { // call begins a fresh completion generation before starting its async // work; a stale request's own generation still gets finished, it just // leaves the shared state untouched. - scanOp asyncOpUI + osInputs osInputQueue + scanOp asyncOpUI + scanWorkers sync.WaitGroup + scanWorkerDone completion.Signal + scanUI chooserUIQueue // sortOp is the background-reorder progress UI - see asyncop.go's // asyncOpUI for the shape it shares with the scan. sortOp.active is diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index 662417d7..9f6f48fd 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -462,3 +462,89 @@ remain outstanding. No release, upload, credential change or dependency upgrade. Recovery budget: 0 spawns; 3 inline review rounds (one additional reconciliation round for concurrent edits); one attempted complete gate, blocked at the daemon platform guard. Focused tests and build-only verification completed separately. + +### Next slice — owned native opening requests + +Base: `03a2eaf`. Owner: T0 inline; Deep route, no delegation. +Files: fileaccess selected-input ownership; openwith Darwin bridge/queue; +root collection scan, composition and harness; focused tests, manifests/docs. +Contract: `fileaccess.NewSelection` carries one native selected URL through +admission; `CaptureSelected(ctx, files)` consumes it on the scan worker into +immutable bookmark URIs, and `ReleaseSelected(files)` discards unstarted input. +Release never retires a native URL while capture is active. Capture is one-shot; +repeated occurrences inside one batch share the captured immutable result. +The native bridge retains original URLs before returning from the Apple Event, +performs metadata/bookmark I/O on the tracked collection worker, and balances +implicit access on capture/discard. Queue shutdown discards pending selections. +Collection scans with selected/scoped inputs perform no filesystem work on UI; +scan workers and their per-instance delivery queue are settled by the harness. +Tests: ownership exact-once/active cancellation/duplicate occurrence; queue stop +and cold-start delivery; blocked capture returns to UI, cancellation cannot apply +stale inputs, rejected input releases, capture error preserves current collection. +Verify: ordinary/Apple fileaccess and openwith tests; focused root opening, +collection and shutdown tests; native bridge serialization fixture; shard and +Qodana exclusions, vet/build and GoLand inspections. Native signed production +Open With/drop qualification remains explicit until actually exercised. +Budget: 0 spawns, 2 inline review rounds, complete suite only at final gate. +Apple's file-access guide confirms implicit access on selected/Dock URLs and +requires a matching stop after use; it does not authorize reconstructed path +strings to manufacture permission. +Source: https://developer.apple.com/documentation/security/accessing-files-from-the-macos-app-sandbox?changes=_4 + + +#### Owned opening implementation and verification + +Implemented original-NSURL retention for Apple Store Open With/Dock events, +worker-side capture, and exact-once discard through process and viewer queues. +Scoped scans use a tracked worker and drainable UI delivery; ordinary loose-file +opening retains its synchronous fast path. Cancellation cannot retire an actively +capturing native object or publish a partial collection. Shutdown discards native +inputs even when their scheduled UI callback never runs. Existing collection +replay and superseded-scan tests now explicitly drain retired scan delivery. + +Native serialization exposed an escaped-URL identity defect: Foundation emits +percent escapes while Fyne's persisted URI strings contain decoded paths. Native +picker and Open With boundaries now decode once; persistent records remain +unchanged, including literal percent sequences. MIME fallback inspection stays +inside the acquired operation scope. No new dependency or shipped payload. + +Evidence (base `03a2eaf`, this slice's working-tree changes; local logs retained +under `.scratch/apple-app-store-opening/`): + +- Ordinary focused root opening/collection regressions pass. Apple-tagged root + opening/collection race regressions pass (94.286s); after the final MIME-scope + adjustment, the same Apple-tagged non-race regression set passes (7.812s). +- Final Apple-tagged fileaccess/openwith/filepicker race suites pass. The native + serializer fixture captures a real bookmark for a filename containing a space + and accent; picker cases also cover literal percent, hash and question marks. + This fixture runs outside a signed production sandbox and does not qualify + actual LaunchServices/Dock authorization or restored sandbox access. +- Translation parity and English identity tests pass. Shard inventory validates + 748 root UI tests across three shards. Exact Qodana exclusion added for the new + fileaccess test file; existing exclusions cover modified test files. +- Negative Go overlays fail behaviorally when capture releases early, active + capture loses ownership, failure omits batch discard, process/viewer shutdown retains + queued input, or either native boundary keeps escaped path identities. The + initial batch-discard test masked the defect by explicitly releasing before + asserting; it now asserts release count at capture return and detects the + deliberately broken implementation. Actual sources stayed intact throughout. +- `make verify-build` and the complete Apple-tagged application build pass. + Apple-tagged vet of changed packages and Windows + amd64 no-cgo vet of their shared code pass. GoLand inspections cover every + changed Go file plus the Objective-C/header bridge, including weak warnings; + only existing intentional filepicker test duplication is reported, covered by + its exact Qodana exclusion. Reinspection after fixes is clear. IDE fallback + evidence is not a fresh Qodana SARIF result. +- `make verify` stops at the native Linux/amd64 prerequisite: the local Docker + daemon reports Linux/aarch64. Complete race/golden CI verification remains + unverified, with no relaxed isolation or excluded worker test. + +Remaining T2 work: native save/window-drop ownership, stale/missing grant +reselection and renewal, write/metadata/version/OS-handoff lifetimes, and worker +source transport. Signed production opening, runtime/package validation and +Intel/Apple Silicon qualification remain required before release. T2 remains +open; this is a completed opening-ownership slice, not App Store readiness. + +Cost ledger: 0 spawns (budget 0); 2 inline review rounds (budget 2), with guard +strengthening inside final review; one full gate attempted and blocked before +suite execution. Build-only checks and focused native/race checks recorded above. diff --git a/qodana.yaml b/qodana.yaml index 3925e558..d5665e9f 100644 --- a/qodana.yaml +++ b/qodana.yaml @@ -149,6 +149,7 @@ exclude: - "internal/filescan/authority_test.go" - "internal/fileaccess/manifest_test.go" - "internal/fileaccess/source_test.go" + - "internal/fileaccess/selection_test.go" - "internal/macworker/command_test.go" - "internal/heic/process_linux_test.go" - "internal/heic/process_unix_test.go" diff --git a/todos.md b/todos.md index a0ef74a9..2a4d0332 100644 --- a/todos.md +++ b/todos.md @@ -18,6 +18,7 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Add an immutable Apple Store channel, localized update/repair messages, verified pre-sign runtime staging and a read-only developer-input preflight. - [x] Preserve captured permissions through image reads, directory scans, and complete session/Favorite serialization; shared folder bookmarks are stored once. +- [x] Carry native Open With/Dock selections through worker-side bookmark capture, cancellation and shutdown; decode native URL escapes once. Signed production opening qualification remains pending. - [ ] Qualify persistent security-scoped permissions for opening, saved sessions, Favorites, writes and workers. - [x] Implement the Apple Store XPC worker boundary and qualify native fixture TCP/UDP denial, cancellation and broker crash on Apple Silicon. - [ ] Qualify real HEIC/ONNX execution and granted source/cache access through that worker boundary on both Mac architectures. diff --git a/translations/de.json b/translations/de.json index fc49b20a..f11861ed 100644 --- a/translations/de.json +++ b/translations/de.json @@ -1,4 +1,5 @@ { + "could not open selected files: %v": "Ausgewählte Dateien konnten nicht geöffnet werden: %v", "Could not cache Favorite locations. Locations remain available in memory.": "Standorte des Favoriten konnten nicht zwischengespeichert werden. Sie bleiben im Arbeitsspeicher verfügbar.", "Showing location cluster": "Standortgruppe wird angezeigt", "Location from %s": "Standort von %s", diff --git a/translations/en.json b/translations/en.json index c9ac7ade..24e4867f 100644 --- a/translations/en.json +++ b/translations/en.json @@ -1,4 +1,5 @@ { + "could not open selected files: %v": "could not open selected files: %v", "Could not cache Favorite locations. Locations remain available in memory.": "Could not cache Favorite locations. Locations remain available in memory.", "Showing location cluster": "Showing location cluster", "Location from %s": "Location from %s", From cae44c94d86a355ae9b7db8498712ef01e6d8629 Mon Sep 17 00:00:00 2001 From: frathe Date: Tue, 29 Sep 2026 23:21:04 +0200 Subject: [PATCH 06/63] Retain native save access through scoped atomic image writes --- .github/testshards/internal-ui.tsv | 3 +- ARCHITECTURE.md | 14 ++- internal/fileaccess/destination.go | 67 +++++++++++++ internal/fileaccess/destination_test.go | 57 +++++++++++ internal/fileaccess/source.go | 3 + internal/filepicker/darwin.go | 58 ++++++++++- internal/filepicker/darwin_test.go | 28 ++++++ internal/filepicker/filepicker.go | 4 +- internal/filepicker/other.go | 8 +- internal/imaging/mutations.go | 15 +++ internal/imaging/mutations_test.go | 85 ++++++++++++++++ internal/imaging/save.go | 42 +++++--- internal/imaging/staging_apple.go | 52 ++++++++++ internal/imaging/staging_other.go | 9 ++ internal/ui/export.go | 2 + internal/ui/exportwork_test.go | 53 ++++++++++ internal/ui/mosaicwin/export.go | 2 + internal/ui/mosaicwin/mosaicwin_test.go | 40 ++++++++ plans/2026-09-29-apple-app-store.md | 123 +++++++++++++++++++++++- qodana.yaml | 1 + todos.md | 1 + 21 files changed, 642 insertions(+), 25 deletions(-) create mode 100644 internal/fileaccess/destination.go create mode 100644 internal/fileaccess/destination_test.go create mode 100644 internal/imaging/staging_apple.go create mode 100644 internal/imaging/staging_other.go diff --git a/.github/testshards/internal-ui.tsv b/.github/testshards/internal-ui.tsv index 12de396b..bc874195 100644 --- a/.github/testshards/internal-ui.tsv +++ b/.github/testshards/internal-ui.tsv @@ -7,7 +7,7 @@ # weights: single CI attempt; projected test sums, not measured shard runtimes # ui-1: 270 entries, 470.520s projected weight sum # ui-2: 255 entries, 470.530s projected weight sum -# ui-3: 223 entries, 504.870s projected weight sum +# ui-3: 224 entries, 504.870s projected weight sum # rebalance: 27 whole-test moves; slowest projected sum within 5% of mean # evidence: finished_refactorings/2026-09-11-ui-shard-rebalance.md # name shard @@ -603,6 +603,7 @@ TestExportAs_NoOpWithNothingLoaded ui-3 TestExportAs_SuggestsTheSourceNameWithTheNewExtensionInItsOwnFolder ui-3 TestExportAs_WritesTheDisplayedFrameToThePickedPath ui-3 TestExportItem_DisabledInitiallyAndEnabledOnceAnImageLoads ui-3 +TestExportRetiresNativeDestination ui-3 TestFavoritesMenuItemOpensStoredFilesThroughViewer ui-3 TestFileSnapshot_KeysAndGenerationMoveTogether ui-3 TestFindMoreLikeThisActionsCaptureRankedSources ui-3 diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 932c40f1..5212a416 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -788,9 +788,17 @@ manifest against their saved membership before publishing restored sources. capture and exact-once release, including active cancellation and duplicate occurrences. The Darwin open picker captures native URL bookmarks on its tracked chooser worker; Open With/Dock selections use the collection worker. Folder scans -pass captured directory authority to children. Native save/window-drop capture, -permission reselection, writes, source versions and analysis transfer remain open -in the active Apple Store plan. +pass captured directory authority to children. `destination.go` retains a native +save URL without requiring an existing file; `ReleaseDestination` closes admission +while active `Acquire` borrowers finish. Native save workers own this URI through +export success, failure and cancellation. Window-drop capture, permission +reselection, source versions and analysis transfer remain in the Apple Store plan. + +Image mutations acquire source/destination access before path resolution and hold +it through serialized commit and cleanup; export metadata reads acquire their +source separately. `imaging/staging_apple.go` selects Foundation's private +same-volume replacement directory for Apple Store writes; `staging_other.go` +keeps ordinary sibling staging. ### `internal/macworker` diff --git a/internal/fileaccess/destination.go b/internal/fileaccess/destination.go new file mode 100644 index 00000000..a65268bf --- /dev/null +++ b/internal/fileaccess/destination.go @@ -0,0 +1,67 @@ +package fileaccess + +import ( + "errors" + "sync" + + "fyne.io/fyne/v2" +) + +type destinationURI struct { + fyne.URI + mu sync.Mutex + release func() + borrowers int + closed bool +} + +// NewDestination owns a save-panel URL, even before its file exists. The caller +// must ReleaseDestination on every exit, including cancellation and write errors. +func NewDestination(uri fyne.URI, release func()) fyne.URI { + return &destinationURI{URI: uri, release: release} +} + +func (d *destinationURI) acquire() (fyne.URI, func(), error) { + d.mu.Lock() + if d.closed { + d.mu.Unlock() + return nil, nil, errors.New("save destination is closed") + } + d.borrowers++ + d.mu.Unlock() + return d.URI, sync.OnceFunc(func() { + d.mu.Lock() + d.borrowers-- + release := d.retireLocked() + d.mu.Unlock() + if release != nil { + release() + } + }), nil +} + +func (d *destinationURI) retireLocked() func() { + if !d.closed || d.borrowers != 0 { + return nil + } + release := d.release + d.release = nil + return release +} + +// ReleaseDestination closes admission without waiting for an active operation. +// The original URL retires only after its last Acquire borrower has returned. +// Plain and persistent source URIs have no destination ownership to discard. +func ReleaseDestination(uri fyne.URI) { + d, ok := uri.(*destinationURI) + if !ok { + return + } + d.mu.Lock() + d.closed = true + release := d.retireLocked() + d.mu.Unlock() + if release != nil { + release() + } +} diff --git a/internal/fileaccess/destination_test.go b/internal/fileaccess/destination_test.go new file mode 100644 index 00000000..ea3611e1 --- /dev/null +++ b/internal/fileaccess/destination_test.go @@ -0,0 +1,57 @@ +package fileaccess + +import ( + "context" + "testing" + + "fyne.io/fyne/v2/storage" +) + +func TestDestinationCloseRetainsActiveAccess(t *testing.T) { + released := 0 + uri := storage.NewFileURI("/selected/not-created.png") + destination := NewDestination(uri, func() { released++ }) + first, endFirst, err := Acquire(context.Background(), destination) + if err != nil || first.Path() != uri.Path() { + t.Fatalf("acquire: %v %v", first, err) + } + _, endSecond, err := Acquire(context.Background(), destination) + if err != nil { + t.Fatal(err) + } + ReleaseDestination(destination) + ReleaseDestination(destination) + if released != 0 { + t.Fatal("closed an active destination") + } + if _, release, err := Acquire(context.Background(), destination); err == nil { + release() + t.Error("closed destination admitted access") + } + endFirst() + endFirst() + if released != 0 { + t.Fatal("released before final borrower") + } + endSecond() + endSecond() + if released != 1 { + t.Fatalf("release count = %d", released) + } +} + +func TestDestinationDiscardAndCancelledAdmission(t *testing.T) { + released := 0 + destination := NewDestination(storage.NewFileURI("/selected/missing.png"), func() { released++ }) + ctx, cancel := context.WithCancel(context.Background()) + cancel() + if _, release, err := Acquire(ctx, destination); err == nil { + release() + t.Fatal("cancelled access admitted") + } + ReleaseDestination(destination) + ReleaseDestination(destination) + if released != 1 { + t.Fatalf("release count = %d", released) + } +} diff --git a/internal/fileaccess/source.go b/internal/fileaccess/source.go index 0ae4aaa1..00227775 100644 --- a/internal/fileaccess/source.go +++ b/internal/fileaccess/source.go @@ -109,6 +109,9 @@ func Acquire(ctx context.Context, uri fyne.URI) (fyne.URI, func(), error) { if err := ctx.Err(); err != nil { return nil, nil, err } + if destination, ok := uri.(*destinationURI); ok { + return destination.acquire() + } source, ok := uri.(*sourceURI) if !ok { return uri, func() {}, nil diff --git a/internal/filepicker/darwin.go b/internal/filepicker/darwin.go index 026aa604..d03b1ada 100644 --- a/internal/filepicker/darwin.go +++ b/internal/filepicker/darwin.go @@ -112,6 +112,29 @@ static panelResult runSavePanel(const char *message, const char *dir, const char if (response != NSModalResponseOK) return (panelResult){-1, NULL}; return retainSelection(panel.URL ? @[panel.URL] : @[]); } +// The destination may not exist; retaining the original URL preserves its +// implicit save permission without inspecting or bookmarking a nonexistent file. +static char *saveSelectionPath(panelResult result) { + @autoreleasepool { + NSArray *urls = (__bridge NSArray *)result.urls; + if (urls.count != 1 || !urls.firstObject.isFileURL) return NULL; + const char *path = urls.firstObject.path.UTF8String; + return path ? strdup(path) : NULL; + } +} +static void releaseSaveSelection(panelResult result) { + @autoreleasepool { + NSArray *urls = (__bridge_transfer NSArray *)result.urls; + for (NSURL *url in urls) [url stopAccessingSecurityScopedResource]; + } +} + +// A URL fixture uses the same ownership transfer as the save panel. +static panelResult testSaveSelection(const char *path) { + @autoreleasepool { + return retainSelection(@[[NSURL fileURLWithPath:[NSString stringWithUTF8String:path]]]); + } +} */ import "C" @@ -124,7 +147,10 @@ import ( "fyne.io/fyne/v2" "fyne.io/fyne/v2/lang" + "fyne.io/fyne/v2/storage" + "github.com/frathe/picfetch/internal/distribution" + "github.com/frathe/picfetch/internal/fileaccess" ) // chooseFilesDarwin runs AppKit's NSOpenPanel in-process rather than @@ -161,7 +187,7 @@ func chooseFilesDarwin() ([]byte, error) { // filepath is safe in this file in a way it wouldn't be in the Windows // builder: this only ever compiles for darwin, where filepath's separator // is already the right one. -func chooseSaveDarwin(suggestedPath string) ([]byte, error) { +func chooseSaveDarwin(suggestedPath string) (fyne.URI, error) { cMsg := C.CString(lang.L("Export image")) defer C.free(unsafe.Pointer(cMsg)) cDir := C.CString(filepath.Dir(suggestedPath)) @@ -171,7 +197,11 @@ func chooseSaveDarwin(suggestedPath string) ([]byte, error) { var result C.panelResult fyne.DoAndWait(func() { result = C.runSavePanel(cMsg, cDir, cName) }) - return decodeNativePanel(result, false) + if distribution.AppleAppStore { + return decodeNativeSavePanel(result) + } + out, err := decodeNativePanel(result, false) + return decodePickedDestination(out, err) } // darwinPathTransport exercises the panel's actual Objective-C NSURL serializer @@ -213,3 +243,27 @@ func decodeNativePanel(result C.panelResult, scoped bool) ([]byte, error) { } return []byte(C.GoString(data)), nil } + +func decodeNativeSavePanel(result C.panelResult) (fyne.URI, error) { + if result.state == 0 { + return nil, nil + } + if result.state != 1 { + return nil, errors.New("native save panel failed") + } + path := C.saveSelectionPath(result) + defer C.free(unsafe.Pointer(path)) + if path == nil { + C.releaseSaveSelection(result) + return nil, errors.New("native save panel returned an invalid destination") + } + return fileaccess.NewDestination(storage.NewFileURI(C.GoString(path)), func() { C.releaseSaveSelection(result) }), nil +} + +func darwinSaveTransport(path string) (fyne.URI, error) { + name := C.CString(path) + defer C.free(unsafe.Pointer(name)) + return decodeNativeSavePanel(C.testSaveSelection(name)) +} + +var _ = darwinSaveTransport diff --git a/internal/filepicker/darwin_test.go b/internal/filepicker/darwin_test.go index 50e69b30..3ca198d6 100644 --- a/internal/filepicker/darwin_test.go +++ b/internal/filepicker/darwin_test.go @@ -3,9 +3,12 @@ package filepicker import ( + "context" "os" "path/filepath" "testing" + + "github.com/frathe/picfetch/internal/fileaccess" ) func TestDarwinPathTransport_RoundTripsNativeURLPaths(t *testing.T) { @@ -37,3 +40,28 @@ func TestDarwinPathTransport_RoundTripsNativeURLPaths(t *testing.T) { } } } + +func TestDarwinSaveTransportOwnsMissingDestination(t *testing.T) { + path := filepath.Join(t.TempDir(), "new cafe\u0301 %20 #.png") + destination, err := darwinSaveTransport(path) + if err != nil { + t.Fatal(err) + } + defer fileaccess.ReleaseDestination(destination) + if destination.Path() != path { + t.Fatalf("destination=%q", destination.Path()) + } + if _, err := os.Stat(path); !os.IsNotExist(err) { + t.Fatalf("chooser created destination: %v", err) + } + _, release, err := fileaccess.Acquire(context.Background(), destination) + if err != nil { + t.Fatal(err) + } + fileaccess.ReleaseDestination(destination) + if _, late, err := fileaccess.Acquire(context.Background(), destination); err == nil { + late() + t.Error("native destination lost owned access") + } + release() +} diff --git a/internal/filepicker/filepicker.go b/internal/filepicker/filepicker.go index ba6de19a..f9f649d1 100644 --- a/internal/filepicker/filepicker.go +++ b/internal/filepicker/filepicker.go @@ -46,12 +46,14 @@ var Choose = func() ([]fyne.URI, error) { // ChooseSave returns exactly the destination confirmed by the native panel. // Nil with no error means cancellation; more than one result is a protocol error. // suggestedPath supplies both the initial directory and the proposed name. +// The caller must fileaccess.ReleaseDestination the returned URI after use, +// including cancellation or errors that occur after the panel returns. var ChooseSave = func(suggestedPath string) (fyne.URI, error) { var out []byte var err error switch runtime.GOOS { case "darwin": - out, err = chooseSaveDarwin(suggestedPath) + return chooseSaveDarwin(suggestedPath) case "windows": out, err = chooseSaveWindows(suggestedPath) default: diff --git a/internal/filepicker/other.go b/internal/filepicker/other.go index 03410be5..6eec3cda 100644 --- a/internal/filepicker/other.go +++ b/internal/filepicker/other.go @@ -2,7 +2,11 @@ package filepicker -import "errors" +import ( + "errors" + + "fyne.io/fyne/v2" +) // chooseFilesDarwin's real implementation (darwin.go) is cgo/AppKit and // only compiles on macOS. Choose's GOOS switch makes this stub unreachable; @@ -13,6 +17,6 @@ func chooseFilesDarwin() ([]byte, error) { // chooseSaveDarwin is the same arrangement for the save panel, unreachable // behind ChooseSave's own GOOS switch. -func chooseSaveDarwin(_ string) ([]byte, error) { +func chooseSaveDarwin(_ string) (fyne.URI, error) { return nil, errors.New("the macOS save panel only exists in darwin builds") } diff --git a/internal/imaging/mutations.go b/internal/imaging/mutations.go index 4b34a1f6..ec389c74 100644 --- a/internal/imaging/mutations.go +++ b/internal/imaging/mutations.go @@ -8,6 +8,10 @@ import ( "path/filepath" "strings" "sync" + + "fyne.io/fyne/v2" + + "github.com/frathe/picfetch/internal/fileaccess" ) // WriteResult distinguishes an accomplished replacement from cancelled work. @@ -200,3 +204,14 @@ func (w contextWrite) Write(p []byte) (int, error) { } return n, err } + +// writeWithAccess holds authority across path resolution, serialized admission, +// source reads, atomic replacement and staging cleanup. +func writeWithAccess(ctx context.Context, uri fyne.URI, create bool, write func(string) (bool, error)) (WriteResult, error) { + resolved, release, err := fileaccess.Acquire(ctx, uri) + if err != nil { + return WriteResult{}, err + } + defer release() + return fileTransactions.write(ctx, resolved.Path(), create, write) +} diff --git a/internal/imaging/mutations_test.go b/internal/imaging/mutations_test.go index ccc89588..cb0bea54 100644 --- a/internal/imaging/mutations_test.go +++ b/internal/imaging/mutations_test.go @@ -17,6 +17,8 @@ import ( "fyne.io/fyne/v2/storage" + "github.com/frathe/picfetch/internal/fileaccess" + "github.com/frathe/picfetch/internal/uitest" ) @@ -476,3 +478,86 @@ func TestFileMutationsAllowIndependentDestinations(t *testing.T) { } }) } + +func TestImageMutationsRequireLiveAuthority(t *testing.T) { + for _, operation := range []string{"export", "save", "strip"} { + t.Run(operation, func(t *testing.T) { + uri := uitest.TempGPSJPEGURI(t, "source.jpg", 8, 4, 48.858222, 2.2945) + before, err := os.ReadFile(uri.Path()) + if err != nil { + t.Fatal(err) + } + source := fileaccess.NewDestination(uri, func() {}) + fileaccess.ReleaseDestination(source) + pixels := image.NewRGBA(image.Rect(0, 0, 4, 8)) + var result WriteResult + switch operation { + case "export": + result, err = ExportContext(context.Background(), source, pixels, nil, ExportOptions{}) + case "save": + result, err = SaveRotatedContext(context.Background(), source, pixels) + case "strip": + result, err = StripJPEGMetadataContext(context.Background(), source) + } + after, readErr := os.ReadFile(uri.Path()) + if readErr != nil { + t.Fatal(readErr) + } + if err == nil || result.Committed || !bytes.Equal(before, after) { + t.Fatalf("closed authority wrote: %+v, %v", result, err) + } + }) + } +} + +func TestExportKeepsDestinationAccessThroughEncoding(t *testing.T) { + path := filepath.Join(t.TempDir(), "selected.png") + released := make(chan struct{}) + destination := fileaccess.NewDestination(storage.NewFileURI(path), func() { close(released) }) + defer fileaccess.ReleaseDestination(destination) + pixels := &heldMutationPixels{Image: image.NewRGBA(image.Rect(0, 0, 8, 4)), entered: make(chan struct{}), release: make(chan struct{})} + done := make(chan error, 1) + go func() { + _, err := ExportContext(context.Background(), destination, pixels, nil, ExportOptions{}) + done <- err + }() + <-pixels.entered + fileaccess.ReleaseDestination(destination) + premature := false + select { + case <-released: + premature = true + default: + } + close(pixels.release) + if err := <-done; err != nil { + t.Fatal(err) + } + if premature { + t.Fatal("native destination retired during encode") + } + select { + case <-released: + default: + t.Fatal("destination retained after commit") + } + if _, err := os.Stat(path); err != nil { + t.Fatal(err) + } +} + +func TestExportDoesNotReadMetadataThroughClosedSourceAuthority(t *testing.T) { + source := fileaccess.NewDestination(uitest.TempGPSJPEGURI(t, "source.jpg", 8, 4, 48.858222, 2.2945), func() {}) + fileaccess.ReleaseDestination(source) + destination := storage.NewFileURI(filepath.Join(t.TempDir(), "copy.jpg")) + if _, err := ExportContext(context.Background(), destination, image.NewRGBA(image.Rect(0, 0, 8, 4)), source, ExportOptions{}); err != nil { + t.Fatal(err) + } + data, err := os.ReadFile(destination.Path()) + if err != nil { + t.Fatal(err) + } + if bytes.Contains(data, []byte("Exif")) { + t.Fatal("export read metadata outside source access") + } +} diff --git a/internal/imaging/save.go b/internal/imaging/save.go index 20b3082f..4ef4e8ee 100644 --- a/internal/imaging/save.go +++ b/internal/imaging/save.go @@ -18,6 +18,8 @@ import ( "golang.org/x/image/tiff" "github.com/gen2brain/avif" + + "github.com/frathe/picfetch/internal/fileaccess" ) // jpegSaveQuality is used instead of image/jpeg's own default (75, quite @@ -96,7 +98,7 @@ func SaveRotated(u fyne.URI, img image.Image) error { // SaveRotatedContext holds the selected file's transaction through replacement. func SaveRotatedContext(ctx context.Context, u fyne.URI, img image.Image) (WriteResult, error) { - return fileTransactions.write(ctx, u.Path(), true, func(path string) (bool, error) { + return writeWithAccess(ctx, u, true, func(path string) (bool, error) { err := saveRotated(ctx, path, img) return err == nil, err }) @@ -184,7 +186,7 @@ func Export(dest fyne.URI, img image.Image, src fyne.URI, opts ExportOptions) er // ExportContext participates in the same resolved-destination transaction as // Save Changes and metadata removal, including aliases through parent directories. func ExportContext(ctx context.Context, dest fyne.URI, img image.Image, src fyne.URI, opts ExportOptions) (WriteResult, error) { - return fileTransactions.write(ctx, dest.Path(), true, func(path string) (bool, error) { + return writeWithAccess(ctx, dest, true, func(path string) (bool, error) { err := exportImage(ctx, path, dest.Extension(), img, src, opts) return err == nil, err }) @@ -215,7 +217,7 @@ func exportImage(ctx context.Context, path, ext string, img image.Image, src fyn } if isJPEGExt(ext) && src != nil && src.Path() != "" { - if orig, err := jpegFileBytesContext(ctx, src.Path()); err == nil && orig != nil { + if orig, err := jpegSourceBytesContext(ctx, src); err == nil && orig != nil { // Answered here, once, while both frames are still in scope // under their own names: out is what will be written, img is // what arrived. Inside the closure below only one of them has a @@ -312,10 +314,10 @@ func jpegFileBytesContext(ctx context.Context, path string) ([]byte, error) { return io.ReadAll(contextRead{ctx: ctx, in: f}) } -// writeEncodedContext encodes img into a temp file in path's own directory and +// writeEncodedContext encodes img into a same-volume staging file and // renames it over path only once the encode has fully succeeded, so a // failed or interrupted encode can never leave the destination truncated or -// corrupted - and, since the rename is within one directory, never leaves a +// corrupted - and, since the rename stays on one volume, never leaves a // half-written file where the caller asked for a whole one either. Shared // by SaveRotated (overwriting the file on screen) and Export (writing a // copy elsewhere), which differ only in how they arrive at path, perm, and @@ -324,18 +326,21 @@ func writeEncodedContext(ctx context.Context, path string, perm os.FileMode, enc return writeFileContext(ctx, path, perm, func(w io.Writer) error { return encode(w, img) }) } -// writeFileContext is writeEncodedContext's underlying atomic write, generalized to any -// write func rather than an (encode, img) pair: temp file in path's own -// directory, Chmod(perm), write, Sync, Close, Rename - so a failed or -// interrupted write can never leave the destination truncated or -// corrupted, and never leaves a half-written file behind either, since the -// rename stays within one directory. StripJPEGMetadata uses this directly -// (writing already-encoded bytes rather than encoding an image.Image). +// writeFileContext performs the atomic write shared by image encoding and +// metadata stripping: stage, chmod, write, sync, close, cancellation check, rename. +// Apple Store builds use Foundation's private replacement directory because a +// file-only grant cannot create siblings. Other builds stage beside the target. +// Both routes preserve atomic same-volume replacement and cleanup on failure. func writeFileContext(ctx context.Context, path string, perm os.FileMode, write func(io.Writer) error) error { if err := ctx.Err(); err != nil { return err } - tmp, err := os.CreateTemp(filepath.Dir(path), ".picfetch-save-*"+filepath.Ext(path)) + directory, cleanup, err := writeStagingDirectory(path) + if err != nil { + return err + } + defer cleanup() + tmp, err := os.CreateTemp(directory, ".picfetch-save-*"+filepath.Ext(path)) if err != nil { return err } @@ -380,7 +385,7 @@ func StripJPEGMetadata(u fyne.URI) error { // StripJPEGMetadataContext serializes the source read as well as its rewrite. func StripJPEGMetadataContext(ctx context.Context, u fyne.URI) (WriteResult, error) { - return fileTransactions.write(ctx, u.Path(), false, func(path string) (bool, error) { + return writeWithAccess(ctx, u, false, func(path string) (bool, error) { return stripJPEGMetadata(ctx, path) }) } @@ -457,3 +462,12 @@ type UnsupportedSaveFormatError struct { func (e *UnsupportedSaveFormatError) Error() string { return "saving " + e.ext + " images isn't supported" } + +func jpegSourceBytesContext(ctx context.Context, source fyne.URI) ([]byte, error) { + resolved, release, err := fileaccess.Acquire(ctx, source) + if err != nil { + return nil, err + } + defer release() + return jpegFileBytesContext(ctx, resolved.Path()) +} diff --git a/internal/imaging/staging_apple.go b/internal/imaging/staging_apple.go new file mode 100644 index 00000000..fd01d9b6 --- /dev/null +++ b/internal/imaging/staging_apple.go @@ -0,0 +1,52 @@ +//go:build darwin && appleappstore + +package imaging + +/* +#cgo CFLAGS: -x objective-c -fobjc-arc +#cgo LDFLAGS: -framework Foundation +#import +#include +#include + +static char *replacementDirectory(const char *path, char **failure) { + @autoreleasepool { + NSURL *destination = [NSURL fileURLWithPath:[NSString stringWithUTF8String:path]]; + NSError *error = nil; + NSURL *directory = [NSFileManager.defaultManager URLForDirectory:NSItemReplacementDirectory + inDomain:NSUserDomainMask appropriateForURL:destination create:YES error:&error]; + if (!directory) { + *failure = strdup(error ? error.description.UTF8String : "replacement directory unavailable"); + return NULL; + } + return strdup(directory.fileSystemRepresentation); + } +} +*/ +import "C" + +import ( + "errors" + "os" + "unsafe" +) + +// A file-only sandbox grant does not permit sibling temporary files. Foundation +// supplies a private staging directory on the destination's volume, preserving +// atomic rename without requesting authority over the destination's parent. +func writeStagingDirectory(path string) (string, func(), error) { + name := C.CString(path) + defer C.free(unsafe.Pointer(name)) + var failure *C.char + value := C.replacementDirectory(name, &failure) + defer C.free(unsafe.Pointer(value)) + defer C.free(unsafe.Pointer(failure)) + if failure != nil { + return "", nil, errors.New(C.GoString(failure)) + } + if value == nil { + return "", nil, errors.New("replacement directory unavailable") + } + directory := C.GoString(value) + return directory, func() { _ = os.RemoveAll(directory) }, nil +} diff --git a/internal/imaging/staging_other.go b/internal/imaging/staging_other.go new file mode 100644 index 00000000..ebd38aea --- /dev/null +++ b/internal/imaging/staging_other.go @@ -0,0 +1,9 @@ +//go:build !darwin || !appleappstore + +package imaging + +import "path/filepath" + +func writeStagingDirectory(path string) (string, func(), error) { + return filepath.Dir(path), func() {}, nil +} diff --git a/internal/ui/export.go b/internal/ui/export.go index 9c947157..99baa927 100644 --- a/internal/ui/export.go +++ b/internal/ui/export.go @@ -15,6 +15,7 @@ import ( "fyne.io/fyne/v2" "fyne.io/fyne/v2/lang" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/filepicker" "github.com/frathe/picfetch/internal/imaging" ) @@ -204,6 +205,7 @@ func (req exportRequest) run(ctx context.Context) exportResult { } result.destination, result.err = req.choose(suggestedExportPath(req.source, req.ext, result.edge)) + defer fileaccess.ReleaseDestination(result.destination) if result.err != nil { result.chooserFailure = true return result diff --git a/internal/ui/exportwork_test.go b/internal/ui/exportwork_test.go index 013de5e8..50aba1af 100644 --- a/internal/ui/exportwork_test.go +++ b/internal/ui/exportwork_test.go @@ -17,6 +17,7 @@ import ( "fyne.io/fyne/v2" "fyne.io/fyne/v2/storage" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/imaging" "github.com/frathe/picfetch/internal/uitest" ) @@ -357,3 +358,55 @@ func TestExportBusyAndQueuedFailureAllowRetry(t *testing.T) { t.Error("shutdown allowed a fresh export") } } + +func TestExportRetiresNativeDestination(t *testing.T) { + for _, phase := range []string{"success", "write_error", "chooser_error", "cancel", "shutdown"} { + t.Run(phase, func(t *testing.T) { + v := newTestViewer(t) + sources := uitest.TempDirJPEGURIs(t, "one.jpg", "two.jpg") + dropAndWait(t, v, sources...) + var released atomic.Int32 + destination := fileaccess.NewDestination(storage.NewFileURI(filepath.Join(t.TempDir(), "export.png")), func() { released.Add(1) }) + entered, gate := make(chan struct{}), make(chan struct{}) + unblock := sync.OnceFunc(func() { close(gate) }) + t.Cleanup(unblock) + uitest.StubSaveChooser(t, func(_ string) (fyne.URI, error) { + close(entered) + <-gate + if phase == "chooser_error" { + return destination, errors.New("fixture picker failure") + } + return destination, nil + }) + if phase == "write_error" { + v.fileWork.export = func(_ context.Context, _ fyne.URI, _ image.Image, _ fyne.URI, _ imaging.ExportOptions) (imaging.WriteResult, error) { + return imaging.WriteResult{}, errors.New("fixture write failure") + } + } + v.exportAs(".png") + select { + case <-entered: + case <-time.After(testTimeout): + t.Fatal("save chooser not entered") + } + switch phase { + case "cancel": + v.ShowImage(1) + case "shutdown": + v.closeFileWork() + } + unblock() + settleChooser(t, v) + if released.Load() != 1 { + t.Fatalf("destination released %d times", released.Load()) + } + _, err := os.Stat(destination.Path()) + if phase == "success" && err != nil { + t.Fatal(err) + } + if phase != "success" && !os.IsNotExist(err) { + t.Fatalf("failed/cancelled export wrote file: %v", err) + } + }) + } +} diff --git a/internal/ui/mosaicwin/export.go b/internal/ui/mosaicwin/export.go index d48d4761..c9051b74 100644 --- a/internal/ui/mosaicwin/export.go +++ b/internal/ui/mosaicwin/export.go @@ -10,6 +10,7 @@ import ( "fyne.io/fyne/v2" "fyne.io/fyne/v2/lang" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/filepicker" "github.com/frathe/picfetch/internal/imaging" ) @@ -39,6 +40,7 @@ func (w *Window) SaveImage() { exporter, choose := w.exporter, filepicker.ChooseSave w.workers.Go(func() { destination, err := choose(suggested) + defer fileaccess.ReleaseDestination(destination) if ctx.Err() != nil { return } diff --git a/internal/ui/mosaicwin/mosaicwin_test.go b/internal/ui/mosaicwin/mosaicwin_test.go index bc97d6a1..b28cbd0d 100644 --- a/internal/ui/mosaicwin/mosaicwin_test.go +++ b/internal/ui/mosaicwin/mosaicwin_test.go @@ -11,6 +11,7 @@ import ( "path/filepath" "strings" "sync" + "sync/atomic" "testing" "time" @@ -21,6 +22,7 @@ import ( "fyne.io/fyne/v2/widget" "github.com/frathe/picfetch/internal/displays" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/imaging" "github.com/frathe/picfetch/internal/mosaic" "github.com/frathe/picfetch/internal/ui/widgets" @@ -1883,3 +1885,41 @@ func TestMosaicLivePreview_RestoresAfterCancelOrFailure(t *testing.T) { } } } + +func TestMosaicExportRetiresNativeDestination(t *testing.T) { + for _, phase := range []string{"success", "failure", "close"} { + t.Run(phase, func(t *testing.T) { + w := New(test.NewApp(), successfulHost(t)) + w.SetUIQueue(&uitest.UIQueue{}) + t.Cleanup(w.Close) + w.Show(mustSnapshot(t)) + w.Generate() + settleWindow(t, w) + var released atomic.Int32 + destination := fileaccess.NewDestination(storage.NewFileURI(filepath.Join(t.TempDir(), "mosaic.png")), func() { released.Add(1) }) + entered, gate := make(chan struct{}), make(chan struct{}) + unblock := sync.OnceFunc(func() { close(gate) }) + t.Cleanup(unblock) + uitest.StubSaveChooser(t, func(_ string) (fyne.URI, error) { close(entered); <-gate; return destination, nil }) + if phase == "failure" { + w.SetExporter(func(_ context.Context, _ fyne.URI, _ image.Image, _ fyne.URI, _ imaging.ExportOptions) (imaging.WriteResult, error) { + return imaging.WriteResult{}, errors.New("fixture export failure") + }) + } + w.SaveImage() + select { + case <-entered: + case <-time.After(5 * time.Second): + t.Fatal("save chooser not entered") + } + if phase == "close" { + w.Close() + } + unblock() + settleWindow(t, w) + if released.Load() != 1 { + t.Fatalf("destination released %d times", released.Load()) + } + }) + } +} diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index 9f6f48fd..06547521 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -172,8 +172,10 @@ Record the complete shipped closure and notice delivery before release readiness ## Honest limit A build tag is preparation, not evidence of sandbox compliance. Until T2–T5 -are complete, Apple builds are experimental and must not be submitted. No -signed sandbox test, Apple validation or App Review result exists yet. +are complete, Apple builds are experimental and must not be submitted. Ad-hoc +signed native fixtures qualify selected boundaries on arm64; complete production +sandbox qualification, Apple Distribution signing, validation and App Review +remain outstanding. ## Evidence and cost ledger @@ -548,3 +550,120 @@ open; this is a completed opening-ownership slice, not App Store readiness. Cost ledger: 0 spawns (budget 0); 2 inline review rounds (budget 2), with guard strengthening inside final review; one full gate attempted and blocked before suite execution. Build-only checks and focused native/race checks recorded above. + +### Next slice — native save ownership and scoped image writes + +Base `93c3040`; Deep route, T0 inline, zero delegates. Continue the agreed T2 +fileaccess/picker/imaging/export boundaries with regression tests at those seams. +Task graph: destination lifetime -> native picker/export owners -> scoped writes +and native staging -> integrated verification. No dependency or payload change. + +Contract: `NewDestination(uri, release)` owns the original selected save URL, +including a destination that does not exist. `ReleaseDestination(uri)` closes +admission, retires unborrowed access exactly once, and leaves active `Acquire` +borrows alive until their synchronous operation returns. Native save and mosaic +export workers retire the destination on every exit; imaging holds a bounded +borrow across serialization, encoding, commit and cleanup. Source metadata reads +hold independent access. Save Changes and metadata stripping acquire restored +source authority before path resolution and serialization. + +Apple Store atomic writes use NSItemReplacementDirectory on the destination's +volume; ordinary builds retain sibling staging. Preserve cancellation before +commit, exact destination naming, symlink policy, permissions and Committed facts. +No direct truncate/write fallback. Missing destination bookmarks must not be +manufactured by expanding permission to the parent directory. + +Native recon: ad-hoc signed WriteProbe.app with a genuine implicit grant for a +generated existing file denied a sibling write, allowed item-replacement staging +and succeeded in POSIX atomic rename (sibling=0, staged=1, rename=0). Foundation +bookmark creation for a nonexistent destination failed with Cocoa code 260. +The initial unbundled sandbox helper could not launch (-5); bundle identity fixed +that fixture setup. Evidence: `.scratch/apple-store-write-probe/result.txt`. +This is arm64 fixture qualification, not signed NSSavePanel production validation. +Sources: Apple's Accessing files from the macOS App Sandbox and FileManager +SearchPathDirectory.itemReplacementDirectory documentation. + +Tests/commands: fileaccess destination active close/idempotence/cancellation; +native picker transport for missing and escaped names; imaging write refusal +with closed authority, cancellation/atomicity and source metadata scope; +root/mosaic export success/error/cancel ownership through existing OS stubs. +`go test -tags no_emoji,nodynamic[,appleappstore]` on those focused packages/tests, +focused race checks, negative guard overlays, shard/exclusion checks, GoLand, +`make verify-build`, Apple build/vet and Windows shared-package vet. +Final `make verify` retains the native amd64 prerequisite. Budget: 0 spawns, +2 inline review rounds; full suite once at final gate. Remaining scope excludes +window drops, stale-grant recovery, post-write source-version reconciliation, +OS handoffs, worker transfer and signed Store packaging/Intel qualification. + + +#### Autonomous continuation instruction + +Ronin requested continued implementation and testing across all independently +completable App Store work, with a running protocol of results and human-only +steps collected last. Finish and checkpoint coherent slices; do not stop after a +slice merely because other implementation/testing remains. Preserve the full +feature set and existing commit authorization. No upload/submission/release is +implied. Continue T2/T3/T4 tooling and qualification before asking for credentials, +account setup, external hardware or product decisions that actually block work. + +#### Save/export implementation evidence + +The save-panel destination now retains its original NSURL through export worker +exit, without trying to create a bookmark for a file that does not yet exist. +`ReleaseDestination` is idempotent and nonblocking; active Acquire borrows keep +native access until the complete synchronous write has returned. Root and mosaic +exports discard destinations after chooser errors, cancellation, close, write +failure and success. Source metadata reads acquire their own scope. Save Changes, +Export and metadata stripping acquire authority before resolving/serializing the +write and retain it through staging cleanup and atomic commit. + +Apple builds stage in Foundation's private same-volume replacement directory, +then use the existing sync/close/cancellation/rename transaction. Ordinary builds +retain sibling staging. Original symlink, mode and committed-result behavior is +covered by existing mutation regressions. No direct-write fallback was added. + +Evidence, base `93c3040` plus this slice's working tree: + +- Destination ownership tests failed before implementation (closed admission and + omitted retirement), then passed. Mutation tests demonstrated unauthorized + writes and premature encoding release before the wrapper was added. Both UI + export flows failed all retirement cases before worker defers were added. +- Native save transport tests cover a nonexistent filename with percent/hash + characters and decomposed accents, matching Foundation's filesystem spelling. + The initial fixture used a precomposed accent and failed on normalization; + after correcting that fixture, a deliberate plain-URI overlay fails the actual + ownership assertion. No filesystem object is created by save transport. +- Focused ordinary tests pass in fileaccess, filepicker, imaging, root UI and + mosaicwin. Apple-tagged race checks pass in all five (imaging 12.380s, root UI + 36.052s, mosaic 4.035s), including existing atomicity/cancel/symlink/metadata tests. +- Native ad-hoc signed Go fixture executes the actual imaging.ExportContext and + fileaccess destination code under a file-only implicit sandbox grant. Results: + sibling denied; existing-file replacement committed; newly absent destination + created; cancelled export did not commit and preserved bytes. This is local + arm64 filesystem qualification, not NSSavePanel UI or Intel qualification. + A separately signed ordinary-staging build fails with EPERM on its temporary + sibling and leaves the original fixture bytes unchanged. +- Negative overlays fail for early active-destination release, skipped mutation + acquisition, metadata reads outside source authority, both export retirements, + and flattened native save destinations. Actual sources remain unchanged by + the overlays. Restored ordinary and Apple race tests pass. +- GoLand inspected all 16 changed Go files, including both platform staging files + and tests. Its mutations_test WriteResult/error warning is the existing + intentional Committed-on-error contract already covered by the exact + GoDfaErrorMayBeNotNil exclusion. Existing mosaic test duplicate fragments are + covered by that file's exact DuplicatedCode exclusion. No new suppression. + IDE fallback is not a fresh Qodana SARIF result. +- `make verify-build` and full Apple-tagged application build passed. + Apple-tagged and Windows/amd64 no-cgo vet passed. + A native macOS shard inventory invocation included Darwin-only tests; the + manifest is Linux-specific; the prescribed Docker target passed with 749 + runnable tests across three shards. + Full native Linux/amd64 race verification remains subject to the recorded + ARM-daemon blocker; no isolation policy is bypassed. + +Logs and native fixture evidence are retained in `.scratch/apple-store-write-probe` +and `.scratch/apple-app-store-save/`. No third-party payload or dependency changed. +Remaining: stale grant renewal/reselection, post-write authority/source-version +reconciliation, window drop and OS handoffs, worker source transport, signed +runtime validation and packaging, full artifact/platform qualification. T2 remains +open. Budget: 0 spawns, 2 inline review rounds; focused tests only for this slice. diff --git a/qodana.yaml b/qodana.yaml index d5665e9f..3b3662b2 100644 --- a/qodana.yaml +++ b/qodana.yaml @@ -150,6 +150,7 @@ exclude: - "internal/fileaccess/manifest_test.go" - "internal/fileaccess/source_test.go" - "internal/fileaccess/selection_test.go" + - "internal/fileaccess/destination_test.go" - "internal/macworker/command_test.go" - "internal/heic/process_linux_test.go" - "internal/heic/process_unix_test.go" diff --git a/todos.md b/todos.md index 2a4d0332..14f88c19 100644 --- a/todos.md +++ b/todos.md @@ -19,6 +19,7 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Add an immutable Apple Store channel, localized update/repair messages, verified pre-sign runtime staging and a read-only developer-input preflight. - [x] Preserve captured permissions through image reads, directory scans, and complete session/Favorite serialization; shared folder bookmarks are stored once. - [x] Carry native Open With/Dock selections through worker-side bookmark capture, cancellation and shutdown; decode native URL escapes once. Signed production opening qualification remains pending. +- [x] Own native save destinations through export/mosaic cancellation, scope image mutations and metadata reads, and stage Apple Store atomic writes in a same-volume replacement directory. - [ ] Qualify persistent security-scoped permissions for opening, saved sessions, Favorites, writes and workers. - [x] Implement the Apple Store XPC worker boundary and qualify native fixture TCP/UDP denial, cancellation and broker crash on Apple Silicon. - [ ] Qualify real HEIC/ONNX execution and granted source/cache access through that worker boundary on both Mac architectures. From beb84911399d2cb9f6854a179cc228117fc64a77 Mon Sep 17 00:00:00 2001 From: frathe Date: Tue, 29 Sep 2026 23:38:30 +0200 Subject: [PATCH 07/63] Validate signed Apple runtime code in the application bundle --- ARCHITECTURE.md | 10 ++ internal/macbundle/bundle.go | 99 +++++++++++++ internal/macbundle/bundle_test.go | 25 ++++ internal/macbundle/signature_darwin.go | 69 +++++++++ internal/macbundle/signature_darwin_test.go | 154 ++++++++++++++++++++ internal/macbundle/signature_other.go | 14 ++ internal/similarity/assets.go | 31 +++- internal/similarity/assets_install.go | 2 +- internal/similarity/assets_package.go | 2 +- internal/similarity/assets_test.go | 19 ++- internal/similarity/encoder.go | 11 +- plans/2026-09-29-apple-app-store.md | 77 ++++++++++ qodana.yaml | 2 + todos.md | 1 + 14 files changed, 503 insertions(+), 13 deletions(-) create mode 100644 internal/macbundle/bundle.go create mode 100644 internal/macbundle/bundle_test.go create mode 100644 internal/macbundle/signature_darwin.go create mode 100644 internal/macbundle/signature_darwin_test.go create mode 100644 internal/macbundle/signature_other.go diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 5212a416..b85826e3 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -800,6 +800,16 @@ source separately. `imaging/staging_apple.go` selects Foundation's private same-volume replacement directory for Apple Store writes; `staging_other.go` keeps ordinary sibling staging. +### `internal/macbundle` + +Apple Store native runtime layout and signed admission. `bundle.go` recognizes +only the main app and known XPC image-helper executable, resolves their shared +Contents/Frameworks, refuses escaping links and checks dylib architecture/type. +`signature_darwin.go` validates the library and outer app's complete nested-code +and resource seals through Security.framework; `signature_other.go` refuses +native verification elsewhere. Similarity separates pre-sign payload checksums +from Apple runtime signature checks and revalidates before loading native code. + ### `internal/macworker` Mac App Store worker broker factory and native XPC service. `command.go` selects diff --git a/internal/macbundle/bundle.go b/internal/macbundle/bundle.go new file mode 100644 index 00000000..3877a373 --- /dev/null +++ b/internal/macbundle/bundle.go @@ -0,0 +1,99 @@ +// Package macbundle owns the Apple Store application's installed native layout. +package macbundle + +import ( + "context" + "debug/macho" + "errors" + "fmt" + "os" + "path/filepath" + "strings" +) + +// RuntimeDirectory resolves the shared Frameworks directory from a known app +// executable. Cache paths and arbitrary ancestor searches are never considered. +func RuntimeDirectory(executable string) (string, error) { + if !filepath.IsAbs(executable) || filepath.Clean(executable) != executable { + return "", errors.New("unrecognized app executable") + } + directory := filepath.Dir(executable) + if filepath.Base(directory) != "MacOS" || filepath.Base(filepath.Dir(directory)) != "Contents" { + return "", errors.New("unrecognized app executable") + } + container := filepath.Dir(filepath.Dir(directory)) + if filepath.Base(executable) == "picfetch-image-worker" && filepath.Base(container) == "io.github.frathe.picfetch.worker.xpc" { + services := filepath.Dir(container) + if filepath.Base(services) != "XPCServices" || filepath.Base(filepath.Dir(services)) != "Contents" { + return "", errors.New("unrecognized XPC app layout") + } + container = filepath.Dir(filepath.Dir(services)) + } else if filepath.Base(executable) != "PicFetch" && filepath.Base(executable) != "picfetch" { + return "", errors.New("unrecognized app executable") + } + if !strings.HasSuffix(container, ".app") { + return "", errors.New("runtime requires an application bundle") + } + return filepath.Join(container, "Contents", "Frameworks"), nil +} + +// CurrentRuntimeDirectory uses the native executable identity, independent of +// argv spelling and the sandbox's replacement working directory. +func CurrentRuntimeDirectory() (string, error) { + executable, err := executablePath() + if err != nil { + return "", err + } + return RuntimeDirectory(executable) +} + +// VerifyLibrary checks the selected architecture and both native code/resource +// seals. Ad-hoc signatures are usable for local qualification; distribution trust +// and provisioning are separately enforced by the packaging/submission route. +func VerifyLibrary(ctx context.Context, root, library, arch string) error { + if err := ctx.Err(); err != nil { + return err + } + if !filepath.IsAbs(root) || filepath.Base(root) != "Frameworks" || filepath.Base(filepath.Dir(root)) != "Contents" || !strings.HasSuffix(filepath.Dir(filepath.Dir(root)), ".app") || filepath.Dir(library) != root { + return errors.New("runtime must be directly inside app Frameworks") + } + canonical, err := filepath.EvalSymlinks(library) + if err != nil { + return err + } + expected, err := filepath.EvalSymlinks(root) + if err != nil { + return err + } + app, err := filepath.EvalSymlinks(filepath.Dir(filepath.Dir(root))) + if err != nil { + return err + } + if expected != filepath.Join(app, "Contents", "Frameworks") { + return errors.New("runtime directory escapes its bundle") + } + info, err := os.Lstat(library) + if err != nil { + return err + } + if !info.Mode().IsRegular() || filepath.Dir(canonical) != expected { + return errors.New("runtime library escapes its bundle") + } + cpu, ok := map[string]macho.Cpu{"arm64": macho.CpuArm64, "amd64": macho.CpuAmd64}[arch] + if !ok { + return fmt.Errorf("unsupported runtime architecture %q", arch) + } + file, err := macho.Open(library) + if err != nil { + return err + } + valid := file.Cpu == cpu && file.Type == macho.TypeDylib + _ = file.Close() + if !valid { + return errors.New("runtime library has the wrong architecture or Mach-O type") + } + if err := verifyNative(filepath.Dir(filepath.Dir(root)), library); err != nil { + return err + } + return ctx.Err() +} diff --git a/internal/macbundle/bundle_test.go b/internal/macbundle/bundle_test.go new file mode 100644 index 00000000..8b790eeb --- /dev/null +++ b/internal/macbundle/bundle_test.go @@ -0,0 +1,25 @@ +package macbundle + +import ( + "path/filepath" + "testing" +) + +func TestRuntimeDirectoryUsesKnownAppLayouts(t *testing.T) { + app := filepath.Join(t.TempDir(), "Renamed PicFetch.app") + want := filepath.Join(app, "Contents", "Frameworks") + for _, relative := range []string{"Contents/MacOS/PicFetch", "Contents/XPCServices/io.github.frathe.picfetch.worker.xpc/Contents/MacOS/picfetch-image-worker"} { + got, err := RuntimeDirectory(filepath.Join(app, filepath.FromSlash(relative))) + if err != nil || got != want { + t.Fatalf("layout %s = %q, %v", relative, got, err) + } + } +} + +func TestRuntimeDirectoryRejectsUnrelatedExecutables(t *testing.T) { + for _, path := range []string{"/tmp/PicFetch", "/tmp/PicFetch.app/Contents/Resources/tool", "/tmp/PicFetch.app/Contents/XPCServices/unrelated.xpc/Contents/MacOS/picfetch-image-worker", "/tmp/PicFetch.app/Contents/XPCServices/io.github.frathe.picfetch.worker.xpc/Contents/MacOS/other", "/tmp/not-an-app/Contents/MacOS/PicFetch", "relative.app/Contents/MacOS/PicFetch"} { + if root, err := RuntimeDirectory(filepath.FromSlash(path)); err == nil { + t.Fatalf("unrelated %q selected %q", path, root) + } + } +} diff --git a/internal/macbundle/signature_darwin.go b/internal/macbundle/signature_darwin.go new file mode 100644 index 00000000..8de890b1 --- /dev/null +++ b/internal/macbundle/signature_darwin.go @@ -0,0 +1,69 @@ +//go:build darwin && cgo + +package macbundle + +/* +#cgo LDFLAGS: -framework Security -framework CoreFoundation +#include +#include +#include +#include +#include +#include + +static char *currentExecutablePath(void) { + char path[PROC_PIDPATHINFO_MAXSIZE]; + if (proc_pidpath(getpid(),path,sizeof(path)) <= 0) return NULL; + return strdup(path); +} + +static OSStatus validateCode(const char *path, SecRequirementRef requirement, SecCSFlags flags) { + CFURLRef url = CFURLCreateFromFileSystemRepresentation(NULL, (const UInt8 *)path, strlen(path), false); + if (!url) return errSecParam; + SecStaticCodeRef code = NULL; + OSStatus result = SecStaticCodeCreateWithPath(url, kSecCSDefaultFlags, &code); + CFRelease(url); + if (result == errSecSuccess) { + result = SecStaticCodeCheckValidity(code, flags, requirement); + CFRelease(code); + } + return result; +} +static OSStatus verifyRuntimeCode(const char *app, const char *library) { + SecCSFlags flags = kSecCSStrictValidate | kSecCSCheckAllArchitectures; + OSStatus result = validateCode(library, NULL, flags); + if (result != errSecSuccess) return result; + SecRequirementRef requirement = NULL; + result = SecRequirementCreateWithString(CFSTR("identifier \"io.github.frathe.picfetch\""), kSecCSDefaultFlags, &requirement); + if (result != errSecSuccess) return result; + result = validateCode(app, requirement, flags | kSecCSCheckNestedCode); + CFRelease(requirement); + return result; +} +*/ +import "C" + +import ( + "fmt" + "unsafe" +) + +func verifyNative(app, library string) error { + appPath := C.CString(app) + defer C.free(unsafe.Pointer(appPath)) + libraryPath := C.CString(library) + defer C.free(unsafe.Pointer(libraryPath)) + if status := C.verifyRuntimeCode(appPath, libraryPath); status != 0 { + return fmt.Errorf("bundled runtime code signature is invalid (Security status %d)", int(status)) + } + return nil +} + +func executablePath() (string, error) { + path := C.currentExecutablePath() + defer C.free(unsafe.Pointer(path)) + if path == nil { + return "", fmt.Errorf("cannot locate current native executable") + } + return C.GoString(path), nil +} diff --git a/internal/macbundle/signature_darwin_test.go b/internal/macbundle/signature_darwin_test.go new file mode 100644 index 00000000..da867e9a --- /dev/null +++ b/internal/macbundle/signature_darwin_test.go @@ -0,0 +1,154 @@ +//go:build darwin && cgo + +package macbundle + +import ( + "context" + "debug/macho" + "errors" + "os" + "os/exec" + "path/filepath" + "runtime" + "strings" + "testing" +) + +func TestSignedRuntimeRequiresLibraryAndOuterSeals(t *testing.T) { + app := filepath.Join(t.TempDir(), "Fixture.app") + contents := filepath.Join(app, "Contents") + frameworks := filepath.Join(contents, "Frameworks") + for _, dir := range []string{frameworks, filepath.Join(contents, "MacOS"), filepath.Join(contents, "Resources")} { + if err := os.MkdirAll(dir, 0700); err != nil { + t.Fatal(err) + } + } + put := func(path, content string) { + t.Helper() + if err := os.WriteFile(path, []byte(content), 0600); err != nil { + t.Fatal(err) + } + } + put(filepath.Join(contents, "Info.plist"), `CFBundleIdentifierio.github.frathe.picfetchCFBundleExecutablePicFetchCFBundlePackageTypeAPPL`) + source := filepath.Join(t.TempDir(), "fixture.c") + put(source, "int fixture(void) { return 42; }\nint main(void) {return 0;}\n") + command := func(name string, args ...string) { + t.Helper() + if out, err := exec.Command(name, args...).CombinedOutput(); err != nil { + t.Fatalf("%s: %v\n%s", name, err, out) + } + } + library := filepath.Join(frameworks, "libfixture.dylib") + command("xcrun", "clang", "-dynamiclib", source, "-o", library) + command("xcrun", "clang", source, "-o", filepath.Join(contents, "MacOS", "PicFetch")) + notice := filepath.Join(contents, "Resources", "notice.txt") + put(notice, "sealed notice") + command("codesign", "--force", "--sign", "-", library) + command("codesign", "--force", "--sign", "-", app) + verify := func() error { return VerifyLibrary(context.Background(), frameworks, library, runtime.GOARCH) } + if err := verify(); err != nil { + t.Fatalf("valid signed fixture: %v", err) + } + other := "amd64" + if runtime.GOARCH == other { + other = "arm64" + } + if err := VerifyLibrary(context.Background(), frameworks, library, other); err == nil { + t.Fatal("accepted wrong architecture") + } + put(notice, "modified notice") + if err := verify(); err == nil { + t.Fatal("accepted modified outer bundle") + } + put(notice, "sealed notice") + original, err := os.ReadFile(library) + if err != nil { + t.Fatal(err) + } + parsed, err := macho.Open(library) + if err != nil { + t.Fatal(err) + } + section := parsed.Section("__text") + _ = parsed.Close() + if section == nil { + t.Fatal("native fixture has no text section") + } + offset := section.Offset + modified := append([]byte(nil), original...) + modified[offset] ^= 1 + if err := os.WriteFile(library, modified, 0700); err != nil { + t.Fatal(err) + } + if err := verify(); err == nil { + t.Fatal("accepted tampered library") + } + command("codesign", "--force", "--sign", "-", library) + if err := verify(); err == nil { + t.Fatal("accepted newly signed library outside the outer seal") + } + if err := os.WriteFile(library, original, 0700); err != nil { + t.Fatal(err) + } + if err := verify(); err != nil { + t.Fatalf("restored library: %v", err) + } + command("codesign", "--remove-signature", library) + if err := verify(); err == nil { + t.Fatal("accepted unsigned library") + } +} + +func TestRuntimeVerifierRejectsNonBundleAndCancelledInput(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + cancel() + if err := VerifyLibrary(ctx, "/missing", "/missing/lib.dylib", runtime.GOARCH); !errors.Is(err, context.Canceled) { + t.Fatalf("cancelled: %v", err) + } + if err := VerifyLibrary(context.Background(), t.TempDir(), "/missing/lib.dylib", runtime.GOARCH); err == nil || !strings.Contains(err.Error(), "Frameworks") { + t.Fatalf("non-bundle root: %v", err) + } +} + +func TestRuntimeVerifierRejectsEscapingLinks(t *testing.T) { + app := filepath.Join(t.TempDir(), "Fixture.app") + contents := filepath.Join(app, "Contents") + if err := os.MkdirAll(contents, 0700); err != nil { + t.Fatal(err) + } + outside := t.TempDir() + target := filepath.Join(outside, "libfixture.dylib") + if err := os.WriteFile(target, []byte("fixture"), 0600); err != nil { + t.Fatal(err) + } + frameworks := filepath.Join(contents, "Frameworks") + if err := os.Symlink(outside, frameworks); err != nil { + t.Fatal(err) + } + if err := VerifyLibrary(t.Context(), frameworks, filepath.Join(frameworks, "libfixture.dylib"), runtime.GOARCH); err == nil || !strings.Contains(err.Error(), "directory escapes") { + t.Fatalf("escaping Frameworks: %v", err) + } + if err := os.Remove(frameworks); err != nil { + t.Fatal(err) + } + if err := os.Mkdir(frameworks, 0700); err != nil { + t.Fatal(err) + } + library := filepath.Join(frameworks, "libfixture.dylib") + if err := os.Symlink(target, library); err != nil { + t.Fatal(err) + } + if err := VerifyLibrary(t.Context(), frameworks, library, runtime.GOARCH); err == nil || !strings.Contains(err.Error(), "library escapes") { + t.Fatalf("escaping library: %v", err) + } +} + +func TestNativeExecutablePathIsAbsolute(t *testing.T) { + path, err := executablePath() + if err != nil || !filepath.IsAbs(path) { + t.Fatalf("native executable: %q, %v", path, err) + } + if info, err := os.Stat(path); err != nil || !info.Mode().IsRegular() { + t.Fatalf("native executable not a file: %v", err) + } +} diff --git a/internal/macbundle/signature_other.go b/internal/macbundle/signature_other.go new file mode 100644 index 00000000..ceff95a6 --- /dev/null +++ b/internal/macbundle/signature_other.go @@ -0,0 +1,14 @@ +//go:build !darwin || !cgo + +package macbundle + +import ( + "errors" + "os" +) + +func verifyNative(_, _ string) error { + return errors.New("native code verification requires macOS and cgo") +} + +func executablePath() (string, error) { return os.Executable() } diff --git a/internal/similarity/assets.go b/internal/similarity/assets.go index bbe26ca1..672e2b71 100644 --- a/internal/similarity/assets.go +++ b/internal/similarity/assets.go @@ -14,6 +14,7 @@ import ( "strings" "github.com/frathe/picfetch/internal/distribution" + "github.com/frathe/picfetch/internal/macbundle" ) //go:embed assets.sha256 @@ -111,14 +112,17 @@ func VerifyAssets(ctx context.Context, root string) error { if err != nil { return err } + if err := verifyAssetFiles(ctx, root, "vision_model.onnx", "preprocessor_config.json"); err != nil { + return err + } nativeRoot, err := runtimeDirectory(root) if err != nil { return err } - return verifyAssetDirectories(ctx, root, nativeRoot, asset) + return verifyRuntime(ctx, nativeRoot, asset) } -// Store native code is installed beside the executable. Model/cache overrides +// Store native code has a fixed bundle/package location. Model/cache overrides // must never select another runtime or cause it to be copied into the cache. func runtimeDirectory(modelRoot string) (string, error) { // The Store build tag selects the other branch. @@ -126,6 +130,10 @@ func runtimeDirectory(modelRoot string) (string, error) { if !distribution.StoreManaged { return modelRoot, nil } + //goland:noinspection GoBoolExpressions + if distribution.AppleAppStore { + return macbundle.CurrentRuntimeDirectory() + } executable, err := os.Executable() if err != nil { return "", err @@ -133,14 +141,25 @@ func runtimeDirectory(modelRoot string) (string, error) { return filepath.Dir(executable), nil } -func verifyAssetDirectories(ctx context.Context, modelRoot, nativeRoot string, asset runtimeAsset) error { - if err := verifyAssetFiles(ctx, modelRoot, "vision_model.onnx", "preprocessor_config.json"); err != nil { - return err +func runtimeLibraryPath(root string, asset runtimeAsset) string { + //goland:noinspection GoBoolExpressions + if distribution.AppleAppStore { + return filepath.Join(root, filepath.Base(asset.library)) } - return verifyRuntime(ctx, nativeRoot, asset) + return filepath.Join(root, asset.directory, asset.library) } func verifyRuntime(ctx context.Context, root string, asset runtimeAsset) error { + //goland:noinspection GoBoolExpressions + if distribution.AppleAppStore { + return macbundle.VerifyLibrary(ctx, root, runtimeLibraryPath(root, asset), runtime.GOARCH) + } + return verifyRuntimePayload(ctx, root, asset) +} + +// Packaging verifies original payload bytes before any code signing, regardless +// of the tool's distribution tag. Runtime signature checks are a separate gate. +func verifyRuntimePayload(ctx context.Context, root string, asset runtimeAsset) error { names := []string{asset.directory + "/" + asset.library} for _, name := range asset.supportLibraries { names = append(names, asset.directory+"/"+name) diff --git a/internal/similarity/assets_install.go b/internal/similarity/assets_install.go index b5b6c8e6..6d286cac 100644 --- a/internal/similarity/assets_install.go +++ b/internal/similarity/assets_install.go @@ -94,7 +94,7 @@ func (c Client) InstallAssets(ctx context.Context, progress func(DownloadProgres if distribution.StoreManaged { nativeRoot, err := runtimeDirectory("") if err != nil { - return "", err + return "", fmt.Errorf("%w: %w", ErrBundledRuntimeUnavailable, err) } if err := verifyRuntime(ctx, nativeRoot, assetRuntime); err != nil { return "", fmt.Errorf("%w: %w", ErrBundledRuntimeUnavailable, err) diff --git a/internal/similarity/assets_package.go b/internal/similarity/assets_package.go index 17b4fb00..8a1c26c0 100644 --- a/internal/similarity/assets_package.go +++ b/internal/similarity/assets_package.go @@ -78,7 +78,7 @@ func stageRuntime(ctx context.Context, asset runtimeAsset, archivePath, root str if _, err := unpackRuntimeAsset(ctx, staging, asset); err != nil { return err } - if err := verifyRuntime(ctx, staging, asset); err != nil { + if err := verifyRuntimePayload(ctx, staging, asset); err != nil { return err } for _, name := range asset.files() { diff --git a/internal/similarity/assets_test.go b/internal/similarity/assets_test.go index 7735d8b7..fdcd77e0 100644 --- a/internal/similarity/assets_test.go +++ b/internal/similarity/assets_test.go @@ -50,7 +50,14 @@ func TestStoreDownloadPolicy(t *testing.T) { cache := t.TempDir() t.Setenv("PICFETCH_SIMILARITY_ASSETS", cache) root, err := runtimeDirectory(cache) - if err != nil || root != filepath.Dir(executable) { + // The test executable is outside the installed Apple bundle; it must refuse + // native loading instead of falling back to the model/cache override. + //goland:noinspection GoBoolExpressions + if distribution.AppleAppStore { + if err == nil || root != "" { + t.Fatalf("Apple runtime accepted an unbundled executable: %s, %v", root, err) + } + } else if err != nil || root != filepath.Dir(executable) { t.Fatalf("Store runtime followed a model/cache override: %s, %v", root, err) } requests := 0 @@ -117,13 +124,17 @@ func TestStageMacRuntime(t *testing.T) { func TestStageMacRuntimePinnedArchive(t *testing.T) { archive := os.Getenv("PICFETCH_TEST_MAC_RUNTIME_ARCHIVE") if archive == "" { - t.Skip("explicit pinned macOS arm64 archive required") + t.Skip("explicit pinned macOS archive required") + } + arch := os.Getenv("PICFETCH_TEST_MAC_RUNTIME_ARCH") + if arch == "" { + arch = "arm64" } root := t.TempDir() - if err := StageMacRuntime(t.Context(), "arm64", archive, root); err != nil { + if err := StageMacRuntime(t.Context(), arch, archive, root); err != nil { t.Fatal(err) } - asset, _ := platformRuntime("darwin", "arm64") + asset, _ := platformRuntime("darwin", arch) asset.privacyNotice = true for _, name := range asset.files() { if info, err := os.Stat(filepath.Join(root, name)); err != nil || !info.Mode().IsRegular() || info.Size() == 0 { diff --git a/internal/similarity/encoder.go b/internal/similarity/encoder.go index c2466cb8..982dec97 100644 --- a/internal/similarity/encoder.go +++ b/internal/similarity/encoder.go @@ -13,6 +13,7 @@ import ( "golang.org/x/image/draw" + "github.com/frathe/picfetch/internal/distribution" "github.com/frathe/picfetch/internal/ort" ) @@ -36,7 +37,15 @@ func NewEncoder(assets, provider string) (_ *Encoder, err error) { if err != nil { return nil, err } - ort.SetSharedLibraryPath(filepath.Join(nativeRoot, asset.directory, asset.library)) + // Revalidate signed code immediately before dlopen, including retained query + // workers whose last asset check predates this encoder initialization. + //goland:noinspection GoBoolExpressions + if distribution.AppleAppStore { + if err := verifyRuntime(context.Background(), nativeRoot, asset); err != nil { + return nil, err + } + } + ort.SetSharedLibraryPath(runtimeLibraryPath(nativeRoot, asset)) if err := ort.InitializeEnvironment(); err != nil { return nil, err } diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index 06547521..5aef411b 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -667,3 +667,80 @@ Remaining: stale grant renewal/reselection, post-write authority/source-version reconciliation, window drop and OS handoffs, worker source transport, signed runtime validation and packaging, full artifact/platform qualification. T2 remains open. Budget: 0 spawns, 2 inline review rounds; focused tests only for this slice. + +### Next slice — signed bundled runtime validation + +Base `cae44c9`; T4, T0 inline, Deep route. Native dylib bytes change when signed, +but runtime admission still compares them with upstream archive hashes. The +worker executable also lives in an XPC bundle, so its executable directory does +not identify a shared app runtime. Fix these before building the full package. + +Contract: `internal/macbundle` locates only the main app or the known nested XPC +image-helper layout and validates a regular, architecture-matching dylib in that +app's Contents/Frameworks. Native Security checks validate the library signature +and the outer app's complete resource/nested-code seal under PicFetch's signing +identifier. Apple runtime loading revalidates before dlopen. Models/cache cannot +redirect native code. Ordinary/MS Store checksums and directory policy remain. +Pre-sign staging always checks upstream payload hashes independent of build tags. +Native libraries go in Frameworks; license/privacy notices go in Resources, +following Apple's standard nested-code locations rather than mixing data/code. + +Files: new macbundle locator/native verifier plus tests; similarity asset +admission/encoder/staging and focused tests; architecture/exclusions/evidence. +Tests: accepted main/helper layout, refused unrelated executable, wrong library +location/architecture, unsigned/modified library and modified outer resources; +valid ad-hoc signed temporary native bundle; pre-sign checks still reject bad +archives under Apple tags. Tests use generated C fixtures, no user application. +Verification: focused ordinary/Apple tests, native signing fixture, negative +guards, package build/vet, GoLand. Full Store packaging is the next dependent +slice. No new dependency; preserve pinned ONNX versions and notice obligations. +Budget: zero spawns, two inline review rounds, no extra complete race suite. +References: Apple Code Signing Tasks; TN2206 standard code locations and nested +resource seals; current code-signature format guidance. Ad-hoc qualification is +separate from Apple Distribution signing, provisioning and App Store processing. + + +#### Signed runtime admission evidence + +Added `internal/macbundle`: known main/XPC-helper layout only, shared Frameworks +location, no escaping symlinks, exact dylib architecture/type, native library +signature and complete outer PicFetch resource/nested-code seal. Kernel +`proc_pidpath` identifies the running executable independently of argv and the +sandbox's changed working directory. Apple loading checks these seals again +immediately before dlopen. Pre-sign staging retains upstream checksum validation; +ordinary and Microsoft Store policies remain unchanged. Native system Security +and CoreFoundation APIs add no redistributed dependency. + +- Generated native fixtures pass valid signing and reject modified/re-signed/ + unsigned libraries, altered outer resources, wrong architecture and escaping + Frameworks/library links. Deliberately bypassing native validity or architecture + causes the regression tests to fail behaviorally. +- Apple-tagged full affected-package race tests pass (macbundle 1.417s, similarity + 38.848s). Ordinary and Microsoft Store focused policy/staging tests pass. + One existing checksum-cancellation test initially hit unbundled-executable + refusal first; model checks now retain their cancellation behavior before the + native runtime lookup, while Store installation still refuses missing runtime + before any download. +- The real pinned arm64 ONNX 1.29.0 archive was staged with original hashes and + all three notices. Its dylib was then signed inside a sandboxed app without + network entitlement. Real CPU inference through CheckAssets/NewEncoder/Encode + returned 768 finite values. The first relative-launch fixture used os.Executable + after the sandbox changed cwd and failed to find models; the corrected native + executable lookup passes both absolute and relative launches. One intermediate + scratch fixture rebuild failed on an unused import; only the subsequent + fail-fast rebuilt execution counts as qualification. +- The existing pinned Intel ONNX 1.23.2 archive was fetched from the exact upstream + release and passed repository size/SHA-256 admission and complete notice staging + under Apple tags. Rosetta execution is available for further Intel testing. + ARM runtime load commands declare macOS 14.0; Intel's existing policy is 13.4. +- GoLand inspected every changed Go file, including warnings. Wrapped-error + comparison and a fixture section nil check were corrected; reinspection is + clean. `make verify-build`, Apple build/vet and Windows no-cgo vet passed. An + initial format gate caught an unformatted scratch qualification helper; the + corrected complete build gate passed. No full Linux race gate is implied. + +Evidence: `.scratch/apple-app-store-runtime/` and +`.scratch/apple-store-runtime-probe/`; base `cae44c9` plus this slice. Ad-hoc code +validity is not Apple Distribution trust, provisioning or App Store approval. +The full packaged app/XPC workers and Intel execution remain next qualification +steps. Cost: 0 spawns, 2 inline review rounds; no repeated complete suite. diff --git a/qodana.yaml b/qodana.yaml index 3b3662b2..15d2ac75 100644 --- a/qodana.yaml +++ b/qodana.yaml @@ -151,6 +151,8 @@ exclude: - "internal/fileaccess/source_test.go" - "internal/fileaccess/selection_test.go" - "internal/fileaccess/destination_test.go" + - "internal/macbundle/bundle_test.go" + - "internal/macbundle/signature_darwin_test.go" - "internal/macworker/command_test.go" - "internal/heic/process_linux_test.go" - "internal/heic/process_unix_test.go" diff --git a/todos.md b/todos.md index 14f88c19..dda7f88d 100644 --- a/todos.md +++ b/todos.md @@ -23,6 +23,7 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [ ] Qualify persistent security-scoped permissions for opening, saved sessions, Favorites, writes and workers. - [x] Implement the Apple Store XPC worker boundary and qualify native fixture TCP/UDP denial, cancellation and broker crash on Apple Silicon. - [ ] Qualify real HEIC/ONNX execution and granted source/cache access through that worker boundary on both Mac architectures. +- [x] Validate post-sign Apple runtime code and outer bundle seals, use a shared Frameworks path for the app and worker, and qualify real arm64 CPU inference in a network-disabled sandbox fixture. - [ ] Bundle pinned native runtimes and notices, and validate signed Mach-O code without comparing its post-sign bytes with upstream archive hashes. - [ ] Add the signed Store packaging route and validate the final artifact on Intel and Apple Silicon. - [ ] Complete privacy/dependency/export assessment, listing metadata and Apple submission validation. From f0565d1dd69b5a95a73ded8765c974d52367ebc0 Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 07:49:28 +0200 Subject: [PATCH 08/63] Qualify universal Apple Store builds and sandbox file access Package both native runtimes, retain source grants across workers and OS handoffs, capture native window drops, and refresh saved bookmarks. Add signed artifact and production worker qualification with submission evidence and remaining release gates. --- .gitignore | 1 + ARCHITECTURE.md | 29 +- Makefile | 5 +- PRIVACY.md | 20 +- ...apple-app-store-requirements-2026-09-29.md | 2 +- internal/favthumbs/name.go | 6 +- internal/favthumbs/name_test.go | 8 +- internal/fileaccess/native_darwin.go | 36 ++- internal/fileaccess/native_other.go | 6 +- internal/fileaccess/selection.go | 23 +- internal/fileaccess/source.go | 14 +- internal/fileaccess/source_test.go | 81 ++++- internal/fileaccess/stat.go | 22 ++ internal/fileaccess/transfer.go | 89 +++++ internal/fileaccess/transfer_darwin.go | 81 +++++ internal/fileaccess/transfer_other.go | 12 + internal/fileaccess/transfer_test.go | 73 +++++ internal/openwith/drop_darwin_store.go | 38 +++ internal/openwith/drop_darwin_store_test.go | 35 ++ internal/openwith/drop_other.go | 8 + internal/openwith/openwith_darwin.h | 3 + internal/openwith/openwith_darwin.m | 74 +++++ internal/similarity/client.go | 15 +- internal/similarity/search_client.go | 6 +- internal/similarity/worker_access.go | 124 +++++++ internal/similarity/worker_access_darwin.go | 34 ++ internal/similarity/worker_access_other.go | 7 + internal/similarity/worker_access_test.go | 66 ++++ internal/ui/batch.go | 36 ++- internal/ui/batch_test.go | 16 + internal/ui/deletion/deletion.go | 16 +- internal/ui/deletion/deletion_test.go | 23 ++ internal/ui/explorer.go | 4 +- internal/ui/explorer/feature.go | 1 + internal/ui/explorer/feature_test.go | 16 +- internal/ui/explorer/workflow.go | 3 +- internal/ui/explorer_test.go | 4 + internal/ui/filework.go | 25 +- internal/ui/reveal.go | 11 +- internal/ui/reveal_test.go | 12 +- internal/ui/run.go | 3 + internal/ui/visualsearch.go | 4 +- internal/ui/visualsearch/feature.go | 6 + internal/ui/visualsearch/feature_test.go | 33 ++ internal/ui/visualsearch/session.go | 3 +- packaging/apple-app-store/README.md | 87 ++++- .../privacy-dependency-audit.md | 92 ++++++ packaging/apple-app-store/submission-draft.md | 129 ++++++++ plans/2026-09-29-apple-app-store.md | 303 ++++++++++++++++++ qodana.yaml | 19 +- scripts/apple-store-preflight.sh | 4 +- scripts/macstorepackage/package.py | 247 ++++++++++++++ scripts/macstorepackage/qualify.py | 95 ++++++ scripts/macstorepackage/test_package.py | 92 ++++++ scripts/macstorequalify/main.go | 110 +++++++ scripts/macstorestage/main.go | 22 ++ todos.md | 15 +- 57 files changed, 2266 insertions(+), 83 deletions(-) create mode 100644 internal/fileaccess/stat.go create mode 100644 internal/fileaccess/transfer.go create mode 100644 internal/fileaccess/transfer_darwin.go create mode 100644 internal/fileaccess/transfer_other.go create mode 100644 internal/fileaccess/transfer_test.go create mode 100644 internal/openwith/drop_darwin_store.go create mode 100644 internal/openwith/drop_darwin_store_test.go create mode 100644 internal/openwith/drop_other.go create mode 100644 internal/similarity/worker_access.go create mode 100644 internal/similarity/worker_access_darwin.go create mode 100644 internal/similarity/worker_access_other.go create mode 100644 internal/similarity/worker_access_test.go create mode 100644 packaging/apple-app-store/privacy-dependency-audit.md create mode 100644 packaging/apple-app-store/submission-draft.md create mode 100644 scripts/macstorepackage/package.py create mode 100644 scripts/macstorepackage/qualify.py create mode 100644 scripts/macstorepackage/test_package.py create mode 100644 scripts/macstorequalify/main.go create mode 100644 scripts/macstorestage/main.go diff --git a/.gitignore b/.gitignore index b780eae0..b60df5d5 100644 --- a/.gitignore +++ b/.gitignore @@ -11,3 +11,4 @@ node_modules/ .env.local .scratch/ .tools/ +__pycache__/ diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index b85826e3..8a1c81e0 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -301,6 +301,11 @@ Its README records the libyuv/WASI source-provenance limits. checks full Xcode and caller-supplied Store signing prerequisites without changing credentials or uploading an artifact. `packaging/apple-app-store/README.md` records the experimental channel and outstanding signed qualification. +`scripts/macstorestage` adapts verified runtime staging for native packaging. +`scripts/macstorepackage` builds and validates universal ad-hoc bundles through +`make apple-store-package-local`; its disposable thin derivatives run production +workers through the Go driver in `scripts/macstorequalify`, without changing the +original app or accessing an Apple account. `packaging/tools.mk` owns reviewed CLI versions and multiarchitecture image digests consumed by Makefile and the release/Store workflows. @@ -780,19 +785,29 @@ restores/snapshots bookmark records, propagates selected directory grants to validated children and resolves an independent scope for each operation. `reader.go` holds the scope through the actual ReadCloser.Close; imaging's canonical read path uses it. `native_darwin.go` resolves explicit scopes in -Apple Store builds; `native_other.go` preserves ordinary URI behavior. +Apple Store builds and renews stale bookmarks while access is active; +`native_other.go` preserves ordinary URI behavior. Acquisition returns renewed +metadata without mutating previously captured records. `manifest.go` stores each distinct scope once while retaining every ordered source occurrence. Session and Favorite persistence validate the complete manifest against their saved membership before publishing restored sources. `selection.go` owns native selected URLs through admission, worker-side bookmark capture and exact-once release, including active cancellation and duplicate -occurrences. The Darwin open picker captures native URL bookmarks on its tracked +occurrences. On restored opening it also refreshes scoped paths/records on the +worker, retaining unavailable entries for existing offline handling. The Darwin open picker captures native URL bookmarks on its tracked chooser worker; Open With/Dock selections use the collection worker. Folder scans pass captured directory authority to children. `destination.go` retains a native save URL without requiring an existing file; `ReleaseDestination` closes admission while active `Acquire` borrowers finish. Native save workers own this URI through -export success, failure and cancellation. Window-drop capture, permission -reselection, source versions and analysis transfer remain in the Apple Store plan. +export success, failure and cancellation. Native window drops retain original URLs before GLFW flattens paths. Permission +reselection and full source-version qualification remain in the Apple Store plan. +`transfer.go` captures immutable URI authority in request contexts and owns +ephemeral interprocess grants; `transfer_darwin.go` creates/resolves implicit +Foundation bookmarks while original scopes remain active, with an unsupported +platform pair. These grants are never persisted in sessions/Favorites. `stat.go` bounds source +metadata reads with the same acquisition contract; preview versioning and file +reconciliation retain authority before stat/alias checks. Root clipboard/reveal +and deletion workers hold scopes through their native OS handoff. Image mutations acquire source/destination access before path resolution and hold it through serialized commit and cleanup; export metadata reads acquire their @@ -821,7 +836,10 @@ descendants. The image worker lives inside the XPC service's `Contents/MacOS`. `scripts/macworkerqualify` builds an ad-hoc signed native fixture to exercise actual TCP/UDP denial, pipe transfer, cancellation, broker crash, group exit and implicit bookmark transfer from a separate app container. -Source grant transport and final Store signing remain qualification work. +Similarity `worker_access*.go` captures app/model/cache and exact source grants +in the existing bounded request stream, retains parent scopes through process +Wait and releases receiving scopes at worker exit. Explorer/search capture +authority before leaving UI. Final Store signing remains qualification work. ### `internal/heic` @@ -1164,6 +1182,7 @@ viewer's handler and flushes in the same critical section. |------|----------------| | `openwith.go` | The queue (`Deliver` / `SetHandler` / terminal `Stop`) and `URIsFromFileURLs`; pending native ownership is discarded at shutdown. | | `openwith_darwin.{go,h,m}` | `Install` / `DelegateRespondsToOpen` + the `application:openURLs:` / `application:openFiles:` graft; Apple Store deliveries retain original native URLs until worker capture or discard. | +| `drop_darwin_store.go` / `drop_other.go` | Installs retained-NSURL drops on only the shown main Cocoa content view through a native per-instance subclass; other views and distribution channels keep Fyne handling. | | `openwith_notdarwin.go` | Both report false; other OSes use `argv`. | ### `internal/filescan` diff --git a/Makefile b/Makefile index b795f319..58e61c25 100644 --- a/Makefile +++ b/Makefile @@ -66,7 +66,7 @@ COVERAGE_HTML := $(COVERAGE_DIR)/coverage.html .PHONY: verify-build --skip-local-tests .PHONY: generate-updater-notices check-updater-notices generate-avif-notices check-avif-notices .PHONY: fossa-findings -.PHONY: apple-store-preflight apple-worker-test +.PHONY: apple-store-preflight apple-worker-test apple-store-package-local .PHONY: loc all: build @@ -106,6 +106,9 @@ fossa-findings: ## Retrieve current FOSSA licensing findings (PR=61 or FOSSA_REV apple-worker-test: ## Qualify signed Mac XPC worker isolation and retirement with native fixtures bash scripts/macworkerqualify/run.sh +apple-store-package-local: install-fyne generate-tag-vectors ## Build an ad-hoc universal sandbox app for local qualification (see packaging/apple-app-store/README.md) + python3 scripts/macstorepackage/package.py + apple-store-preflight: ## Check native Xcode and Mac App Store signing inputs (see packaging/apple-app-store/README.md) bash scripts/apple-store-preflight.sh diff --git a/PRIVACY.md b/PRIVACY.md index f530e472..6516a5cb 100644 --- a/PRIVACY.md +++ b/PRIVACY.md @@ -1,6 +1,6 @@ # PicFetch privacy policy -Effective date: September 29, 2026 +Effective date: September 30, 2026 PicFetch is a free and open-source desktop image viewer. It has no accounts, advertising, analytics, or telemetry. The PicFetch developer does not collect, @@ -84,6 +84,14 @@ downloads only the model and processor configuration from Hugging Face (about 372 MB); it does not download executable code from GitHub or place runtime DLLs in the model cache. +The Mac App Store build includes the Apple Silicon and Intel ONNX Runtime +libraries and their license notices inside its signed application bundle. +Explorer setup downloads only model data and the processor configuration from +Hugging Face (about 372 MB). The App Store delivers runtime code updates. Its +analysis worker runs in a separate sandbox without network permission; the +viewer transfers temporary access to the selected inputs and local model/cache +files for the operation. Persistent file permissions remain on the device. + On supported macOS and Linux systems, the operating system blocks the analysis worker's network access. On Windows 11, analysis runs in a normal local worker process with ONNX Runtime telemetry disabled. PicFetch does not configure an @@ -92,6 +100,16 @@ port does not prevent it from making outgoing connections; local processing and the telemetry opt-out are not a guarantee of network isolation. The Windows setup page explains this difference before analysis starts. +## Release-note artwork + +Release notes are bundled with the app. If a displayed release note contains a +remote image, PicFetch downloads that image while the note is open. The image +host receives the requested URL, the device's IP address, and ordinary HTTP +connection information. PicFetch does not attach local photos, metadata, or +usage reports. Closing the notes cancels pending image work; requests already +sent cannot be recalled. Release notes without remote images make no image +requests. + ## Voluntary community discussions The GitHub Discussions link opens diff --git a/docs/apple-app-store-requirements-2026-09-29.md b/docs/apple-app-store-requirements-2026-09-29.md index 367b974d..49d17e71 100644 --- a/docs/apple-app-store-requirements-2026-09-29.md +++ b/docs/apple-app-store-requirements-2026-09-29.md @@ -52,7 +52,7 @@ Verification proposals: ## Signing, package and submission -[Certificates overview](https://developer.apple.com/help/account/certificates/certificates-overview) distinguishes store app/installer certificates from Developer ID certificates. Apple's helper guide shows Apple Distribution for the app and tool; certificate selection must match the chosen distribution workflow. [Provisioning profiles](https://developer.apple.com/help/account/provisioning-profiles/create-an-app-store-provisioning-profile) require an app record with an explicit App ID and a Mac App Store Connect distribution profile; Xcode can manage profiles automatically. +[Certificates overview](https://developer.apple.com/help/account/certificates/certificates-overview) distinguishes store app/installer certificates from Developer ID certificates. Apple's helper guide shows Apple Distribution for the app and tool; certificate selection must match the chosen distribution workflow. [Apple TN3125](https://developer.apple.com/documentation/technotes/tn3125-inside-code-signing-provisioning-profiles) distinguishes unrestricted macOS entitlements (which can omit a profile even for the Mac App Store) from restricted capabilities. TestFlight always requires a profile. When required, use a Mac App Store Connect profile matching the explicit App ID and signing identity; Xcode can manage it. Inherited helpers have only sandbox/inherit entitlements and must not be assigned restricted capabilities casually. [Packaging Mac software](https://developer.apple.com/documentation/xcode/packaging-mac-software-for-distribution) specifies a signed `.pkg` for store submission and Mac Installer Distribution signing, whose identity is named `3rd Party Mac Developer Installer`. Upload using Transporter or Apple's documented tooling. [Upload builds](https://developer.apple.com/help/app-store-connect/manage-builds/upload-builds) explains that bundle ID/version associate the uploaded artifact with its record and build string identifies the build. [Notarization](https://developer.apple.com/documentation/security/notarizing-macos-software-before-distribution) is a separate direct-distribution process; store submission includes equivalent checks. diff --git a/internal/favthumbs/name.go b/internal/favthumbs/name.go index 2813ac15..5718c144 100644 --- a/internal/favthumbs/name.go +++ b/internal/favthumbs/name.go @@ -3,14 +3,16 @@ package favthumbs import ( + "context" "crypto/sha256" "encoding/hex" "fmt" - "os" "path/filepath" "time" "fyne.io/fyne/v2" + + "github.com/frathe/picfetch/internal/fileaccess" ) // SubDir is the directory name, under a favorite, that holds previews. @@ -45,7 +47,7 @@ func EntryName(src fyne.URI) (string, bool) { return "", false } - info, err := os.Stat(src.Path()) + info, err := fileaccess.Stat(context.Background(), src) if err != nil { return "", false } diff --git a/internal/favthumbs/name_test.go b/internal/favthumbs/name_test.go index d4edd67d..2dcdffad 100644 --- a/internal/favthumbs/name_test.go +++ b/internal/favthumbs/name_test.go @@ -8,6 +8,8 @@ import ( "time" "fyne.io/fyne/v2/storage" + + "github.com/frathe/picfetch/internal/fileaccess" ) func TestEntryNameStableAcrossCalls(t *testing.T) { @@ -17,7 +19,7 @@ func TestEntryNameStableAcrossCalls(t *testing.T) { if err := os.WriteFile(path, []byte("hello"), 0o644); err != nil { t.Fatal(err) } - src := storage.NewFileURI(path) + src := fileaccess.NewDestination(storage.NewFileURI(path), func() {}) first, ok := EntryName(src) if !ok { @@ -30,6 +32,10 @@ func TestEntryNameStableAcrossCalls(t *testing.T) { if first != second { t.Errorf("EntryName not stable: first %q, second %q", first, second) } + fileaccess.ReleaseDestination(src) + if _, ok := EntryName(src); ok { + t.Fatal("source version read without active authority") + } } func TestEntryNameDiffersForDifferentPaths(t *testing.T) { diff --git a/internal/fileaccess/native_darwin.go b/internal/fileaccess/native_darwin.go index 8fea6c35..3ea1fc94 100644 --- a/internal/fileaccess/native_darwin.go +++ b/internal/fileaccess/native_darwin.go @@ -8,14 +8,16 @@ package fileaccess #import #include #include +#include -static char *acquireBookmark(const void *bytes, size_t length, int directory, void **owner, char **failure) { +static char *acquireBookmark(const void *bytes, size_t length, int directory, void **owner, void **renewed, int *renewedLength, char **failure) { @autoreleasepool { NSData *bookmark = [NSData dataWithBytes:bytes length:length]; NSError *error = nil; + BOOL stale = NO; NSURL *url = [NSURL URLByResolvingBookmarkData:bookmark options:NSURLBookmarkResolutionWithSecurityScope | NSURLBookmarkResolutionWithoutUI - relativeToURL:nil bookmarkDataIsStale:NULL error:&error]; + relativeToURL:nil bookmarkDataIsStale:&stale error:&error]; if (!url || ![url startAccessingSecurityScopedResource]) { *failure = strdup(error ? error.description.UTF8String : "security-scoped source access denied"); return NULL; @@ -27,8 +29,24 @@ static char *acquireBookmark(const void *bytes, size_t length, int directory, vo *failure = strdup("security-scoped source type changed"); return NULL; } + NSData *replacement = nil; + if (stale) { + replacement = [url bookmarkDataWithOptions:NSURLBookmarkCreationWithSecurityScope + includingResourceValuesForKeys:nil relativeToURL:nil error:&error]; + if (!replacement || replacement.length == 0 || replacement.length > INT_MAX) { + [url stopAccessingSecurityScopedResource]; + *failure = strdup(error ? error.description.UTF8String : "unable to renew security-scoped bookmark"); + return NULL; + } + } char *path = strdup(url.fileSystemRepresentation); if (!path) { [url stopAccessingSecurityScopedResource]; return NULL; } + if (replacement) { + *renewed = malloc(replacement.length); + if (!*renewed) { free(path); [url stopAccessingSecurityScopedResource]; return NULL; } + memcpy(*renewed, replacement.bytes, replacement.length); + *renewedLength = (int)replacement.length; + } *owner = (__bridge_retained void *)url; return path; } @@ -48,21 +66,23 @@ import ( "unsafe" ) -func resolveBookmark(_ context.Context, record Record) (string, func(), error) { - var owner unsafe.Pointer +func resolveBookmark(_ context.Context, record Record) (resolution, func(), error) { + var owner, renewed unsafe.Pointer + var renewedLength C.int var failure *C.char directory := C.int(0) if record.Directory { directory = 1 } - path := C.acquireBookmark(unsafe.Pointer(&record.Bookmark[0]), C.size_t(len(record.Bookmark)), directory, &owner, &failure) + path := C.acquireBookmark(unsafe.Pointer(&record.Bookmark[0]), C.size_t(len(record.Bookmark)), directory, &owner, &renewed, &renewedLength, &failure) defer C.free(unsafe.Pointer(path)) + defer C.free(renewed) defer C.free(unsafe.Pointer(failure)) if path == nil { if failure != nil { - return "", nil, errors.New(C.GoString(failure)) + return resolution{}, nil, errors.New(C.GoString(failure)) } - return "", nil, errors.New("unable to acquire security-scoped source") + return resolution{}, nil, errors.New("unable to acquire security-scoped source") } - return C.GoString(path), func() { C.releaseBookmark(owner) }, nil + return resolution{path: C.GoString(path), bookmark: C.GoBytes(renewed, renewedLength)}, func() { C.releaseBookmark(owner) }, nil } diff --git a/internal/fileaccess/native_other.go b/internal/fileaccess/native_other.go index 088d9077..20b0d27c 100644 --- a/internal/fileaccess/native_other.go +++ b/internal/fileaccess/native_other.go @@ -10,10 +10,10 @@ import ( "fyne.io/fyne/v2/storage" ) -func resolveBookmark(_ context.Context, record Record) (string, func(), error) { +func resolveBookmark(_ context.Context, record Record) (resolution, func(), error) { uri, err := storage.ParseURI(record.URI) if err != nil { - return "", nil, err + return resolution{}, nil, err } root := uri.Path() if record.Relative != "" { @@ -21,5 +21,5 @@ func resolveBookmark(_ context.Context, record Record) (string, func(), error) { root = filepath.Dir(root) } } - return root, func() {}, nil + return resolution{path: root}, func() {}, nil } diff --git a/internal/fileaccess/selection.go b/internal/fileaccess/selection.go index 78b4769b..d1cb3eca 100644 --- a/internal/fileaccess/selection.go +++ b/internal/fileaccess/selection.go @@ -25,11 +25,14 @@ func NewSelection(uri fyne.URI, capture func(context.Context) (Record, error), r // NeedsCapture identifies selected URLs whose metadata must be captured off UI. func NeedsCapture(uri fyne.URI) bool { _, ok := uri.(*selectionURI); return ok } -// CaptureSelected consumes selected input into immutable source URIs. +// CaptureSelected consumes selected input and refreshes restored source metadata +// on the opening worker. Unavailable sources retain their previous record for +// the scanner's per-item/offline handling; cancellation aborts the opening. func CaptureSelected(ctx context.Context, files []fyne.URI) ([]fyne.URI, error) { defer ReleaseSelected(files) result := make([]fyne.URI, len(files)) captured := make(map[*selectionURI]fyne.URI) + refreshed := make(map[*sourceURI]fyne.URI) for i, file := range files { if err := ctx.Err(); err != nil { return nil, err @@ -40,6 +43,19 @@ func CaptureSelected(ctx context.Context, files []fyne.URI) ([]fyne.URI, error) selected, ok := file.(*selectionURI) if !ok { result[i] = file + if saved, scoped := file.(*sourceURI); scoped { + current, seen := refreshed[saved] + if !seen { + current = file + resolved, release, err := Acquire(ctx, saved) + if err == nil { + current = resolved + release() + } + refreshed[saved] = current + } + result[i] = current + } continue } source, ok := captured[selected] @@ -53,7 +69,10 @@ func CaptureSelected(ctx context.Context, files []fyne.URI) ([]fyne.URI, error) } result[i] = source } - return result, ctx.Err() + if err := ctx.Err(); err != nil { + return nil, err + } + return result, nil } var errSelectionClosed = errors.New("selected input is already consumed or closed") diff --git a/internal/fileaccess/source.go b/internal/fileaccess/source.go index 00227775..c55aa117 100644 --- a/internal/fileaccess/source.go +++ b/internal/fileaccess/source.go @@ -24,10 +24,15 @@ type Record struct { Relative string `json:"relative,omitempty"` } +type resolution struct { + path string + bookmark []byte +} + type sourceURI struct { fyne.URI record Record - resolve func(context.Context, Record) (string, func(), error) + resolve func(context.Context, Record) (resolution, func(), error) } // FromRecord restores metadata without starting native access or inspecting disk. @@ -126,13 +131,16 @@ func Acquire(ctx context.Context, uri fyne.URI) (fyne.URI, func(), error) { release() return nil, nil, err } - resolved := root + resolved := root.path if source.record.Relative != "" { - resolved = filepath.Join(root, filepath.FromSlash(source.record.Relative)) + resolved = filepath.Join(root.path, filepath.FromSlash(source.record.Relative)) } resolvedURI := storage.NewFileURI(resolved) record := source.record record.URI = resolvedURI.String() + if len(root.bookmark) != 0 { + record.Bookmark = append([]byte(nil), root.bookmark...) + } return &sourceURI{URI: resolvedURI, record: record, resolve: source.resolve}, release, nil } diff --git a/internal/fileaccess/source_test.go b/internal/fileaccess/source_test.go index ab9a81d1..cb26abc7 100644 --- a/internal/fileaccess/source_test.go +++ b/internal/fileaccess/source_test.go @@ -9,6 +9,7 @@ import ( "sync/atomic" "testing" + "fyne.io/fyne/v2" "fyne.io/fyne/v2/storage" "fyne.io/fyne/v2/test" ) @@ -19,9 +20,9 @@ func TestSourceAcquireKeepsIndependentOperationLifetimes(t *testing.T) { if err != nil { t.Fatal(err) } - source.(*sourceURI).resolve = func(_ context.Context, _ Record) (string, func(), error) { + source.(*sourceURI).resolve = func(_ context.Context, _ Record) (resolution, func(), error) { active.Add(1) - return "/moved/photo.jpg", func() { active.Add(-1) }, nil + return resolution{path: "/moved/photo.jpg"}, func() { active.Add(-1) }, nil } first, releaseFirst, err := Acquire(context.Background(), source) if err != nil { @@ -101,9 +102,9 @@ func TestReaderRetainsScopeUntilCloseAndRollsBackFailure(t *testing.T) { t.Fatal(err) } var active atomic.Int32 - source.(*sourceURI).resolve = func(_ context.Context, _ Record) (string, func(), error) { + source.(*sourceURI).resolve = func(_ context.Context, _ Record) (resolution, func(), error) { active.Add(1) - return path, func() { active.Add(-1) }, nil + return resolution{path: path}, func() { active.Add(-1) }, nil } reader, err := Reader(context.Background(), source) if err != nil { @@ -135,8 +136,8 @@ func TestAcquiredDirectoryPropagatesMovedScopeToChild(t *testing.T) { if err != nil { t.Fatal(err) } - directory.(*sourceURI).resolve = func(_ context.Context, _ Record) (string, func(), error) { - return "/moved", func() {}, nil + directory.(*sourceURI).resolve = func(_ context.Context, _ Record) (resolution, func(), error) { + return resolution{path: "/moved"}, func() {}, nil } resolved, release, err := Acquire(context.Background(), directory) if err != nil { @@ -189,12 +190,76 @@ func TestAcquireReleasesScopeWhenCancelledDuringResolution(t *testing.T) { ctx, cancel := context.WithCancel(context.Background()) defer cancel() released := 0 - source.(*sourceURI).resolve = func(_ context.Context, _ Record) (string, func(), error) { + source.(*sourceURI).resolve = func(_ context.Context, _ Record) (resolution, func(), error) { cancel() - return "/photo.jpg", func() { released++ }, nil + return resolution{path: "/photo.jpg"}, func() { released++ }, nil } uri, release, err := Acquire(ctx, source) if !errors.Is(err, context.Canceled) || uri != nil || release != nil || released != 1 { t.Fatalf("cancelled acquisition returned %v, %v; released %d", uri, err, released) } } + +func TestOpeningRefreshesImmutableBookmarkOccurrences(t *testing.T) { + original, err := FromRecord(Record{URI: "file:///old/sub/photo.jpg", Bookmark: []byte("old"), Directory: true, Relative: "sub/photo.jpg"}) + if err != nil { + t.Fatal(err) + } + calls, released := 0, 0 + renewed := []byte("renewed") + original.(*sourceURI).resolve = func(_ context.Context, record Record) (resolution, func(), error) { + calls++ + if string(record.Bookmark) != "old" { + t.Fatal("captured input changed") + } + return resolution{path: "/moved", bookmark: renewed}, func() { released++ }, nil + } + opened, err := CaptureSelected(context.Background(), []fyne.URI{original, original}) + if err != nil { + t.Fatal(err) + } + record := Snapshot(opened[0]) + if record.URI != "file:///moved/sub/photo.jpg" || string(record.Bookmark) != "renewed" || record.Relative != "sub/photo.jpg" || !record.Directory { + t.Fatalf("opening retained stale authority: %+v", record) + } + if calls != 1 || released != 1 || opened[0] != opened[1] { + t.Fatalf("duplicate opening: calls %d, releases %d", calls, released) + } + manifest, err := Pack(context.Background(), opened) + if err != nil { + t.Fatal(err) + } + restored, err := Unpack(context.Background(), manifest) + if err != nil || len(restored) != 2 || restored[1].String() != record.URI || string(Snapshot(restored[1]).Bookmark) != "renewed" { + t.Fatalf("renewed authority did not survive persistence: %v %v", restored, err) + } + renewed[0] = 'X' + if string(Snapshot(opened[0]).Bookmark) != "renewed" || string(Snapshot(original).Bookmark) != "old" || original.Path() != "/old/sub/photo.jpg" { + t.Fatal("renewal changed immutable metadata") + } +} + +func TestOpeningPreservesUnavailableSourceButAbortsCancellation(t *testing.T) { + for _, cancelled := range []bool{false, true} { + source, err := FromRecord(Record{URI: "file:///offline/photo.jpg", Bookmark: []byte("saved")}) + if err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithCancel(context.Background()) + source.(*sourceURI).resolve = func(_ context.Context, _ Record) (resolution, func(), error) { + if cancelled { + cancel() + } + return resolution{}, nil, errors.New("offline") + } + opened, err := CaptureSelected(ctx, []fyne.URI{source}) + cancel() + if cancelled { + if !errors.Is(err, context.Canceled) || opened != nil { + t.Fatalf("cancelled opening: %v %v", opened, err) + } + } else if err != nil || len(opened) != 1 || opened[0] != source { + t.Fatalf("lost offline source: %v %v", opened, err) + } + } +} diff --git a/internal/fileaccess/stat.go b/internal/fileaccess/stat.go new file mode 100644 index 00000000..1cf064f6 --- /dev/null +++ b/internal/fileaccess/stat.go @@ -0,0 +1,22 @@ +package fileaccess + +import ( + "context" + "errors" + "os" + + "fyne.io/fyne/v2" +) + +// Stat observes source metadata within its operation-bound native authority. +func Stat(ctx context.Context, uri fyne.URI) (os.FileInfo, error) { + if uri == nil { + return nil, errors.New("missing source") + } + resolved, release, err := Acquire(ctx, uri) + if err != nil { + return nil, err + } + defer release() + return os.Stat(resolved.Path()) +} diff --git a/internal/fileaccess/transfer.go b/internal/fileaccess/transfer.go new file mode 100644 index 00000000..1bf28600 --- /dev/null +++ b/internal/fileaccess/transfer.go @@ -0,0 +1,89 @@ +package fileaccess + +import ( + "context" + "errors" + "sync" + + "fyne.io/fyne/v2" + "fyne.io/fyne/v2/storage" +) + +// Transfer is ephemeral interprocess authority, never a persisted source record. +// Path identifies the captured source; Bookmark has implicit scope for another +// process. Its rights are bounded by the exporting process's existing access. +type Transfer struct { + Path string + Bookmark []byte +} + +type sourcesKey struct{} + +// WithSources captures an immutable path lookup before work leaves the UI. +// It performs no native I/O and does not start security-scoped access. +func WithSources(ctx context.Context, sources []fyne.URI) context.Context { + captured := make(map[string]fyne.URI, len(sources)) + for _, uri := range sources { + if uri != nil { + captured[uri.Path()] = uri + } + } + return context.WithValue(ctx, sourcesKey{}, captured) +} + +// SourceForPath returns captured authority, or a plain URI for an app-owned path. +// A plain URI does not manufacture permission in a sandbox. +func SourceForPath(ctx context.Context, path string) fyne.URI { + sources, _ := ctx.Value(sourcesKey{}).(map[string]fyne.URI) + if uri := sources[path]; uri != nil { + return uri + } + return storage.NewFileURI(path) +} + +// Export captures a fresh implicit bookmark while the original authority is +// active. The caller retains release until the receiving worker has finished. +func Export(ctx context.Context, uri fyne.URI) (Transfer, func(), error) { + return exportTransfer(ctx, uri, createTransfer) +} + +func exportTransfer(ctx context.Context, uri fyne.URI, create func(string) ([]byte, error)) (Transfer, func(), error) { + if uri == nil || uri.Scheme() != "file" { + return Transfer{}, nil, errors.New("worker source must be a local file") + } + resolved, release, err := Acquire(ctx, uri) + if err != nil { + return Transfer{}, nil, err + } + bookmark, err := create(resolved.Path()) + if err == nil { + err = ctx.Err() + } + if err != nil { + release() + return Transfer{}, nil, err + } + return Transfer{Path: resolved.Path(), Bookmark: bookmark}, release, nil +} + +// Import activates only the transferred authority for one bounded worker. +// Native implicit resolution starts access; release balances that acquisition. +func Import(ctx context.Context, transfer Transfer) (func(), error) { + if err := ctx.Err(); err != nil { + return nil, err + } + if len(transfer.Bookmark) == 0 || len(transfer.Bookmark) > 1024*1024 { + return nil, errors.New("invalid worker bookmark size") + } + release, err := resolveTransfer(transfer) + if err != nil { + return nil, err + } + var once sync.Once + stop := func() { once.Do(release) } + if err := ctx.Err(); err != nil { + stop() + return nil, err + } + return stop, nil +} diff --git a/internal/fileaccess/transfer_darwin.go b/internal/fileaccess/transfer_darwin.go new file mode 100644 index 00000000..13cbe748 --- /dev/null +++ b/internal/fileaccess/transfer_darwin.go @@ -0,0 +1,81 @@ +//go:build darwin && appleappstore + +package fileaccess + +/* +#cgo CFLAGS: -x objective-c -fobjc-arc +#cgo LDFLAGS: -framework Foundation +#import +#include +#include + +static void *exportTransfer(const char *path, size_t *length, char **failure) { + @autoreleasepool { + NSURL *url = [NSURL fileURLWithFileSystemRepresentation:path isDirectory:NO relativeToURL:nil]; + NSError *error = nil; + NSData *data = [url bookmarkDataWithOptions:0 includingResourceValuesForKeys:nil relativeToURL:nil error:&error]; + if (!data) { *failure = strdup(error.description.UTF8String); return NULL; } + void *bytes = malloc(data.length); + if (bytes) { memcpy(bytes, data.bytes, data.length); *length = data.length; } + return bytes; + } +} +static void *importTransfer(const void *bytes, size_t length, char **failure) { + @autoreleasepool { + NSError *error = nil; + NSURL *url = [NSURL URLByResolvingBookmarkData:[NSData dataWithBytes:bytes length:length] + options:NSURLBookmarkResolutionWithoutUI relativeToURL:nil bookmarkDataIsStale:NULL error:&error]; + if (!url) { *failure = strdup(error.description.UTF8String); return NULL; } + return (__bridge_retained void *)url; + } +} +static void stopTransfer(void *owner) { + @autoreleasepool { + NSURL *url = (__bridge_transfer NSURL *)owner; + [url stopAccessingSecurityScopedResource]; + } +} +*/ +import "C" + +import ( + "errors" + "strings" + "unsafe" +) + +func createTransfer(path string) ([]byte, error) { + if strings.ContainsRune(path, 0) { + return nil, errors.New("invalid worker path") + } + native := C.CString(path) + defer C.free(unsafe.Pointer(native)) + var length C.size_t + var failure *C.char + bytes := C.exportTransfer(native, &length, &failure) + defer C.free(bytes) + defer C.free(unsafe.Pointer(failure)) + if bytes == nil { + if failure != nil { + return nil, errors.New(C.GoString(failure)) + } + return nil, errors.New("unable to create worker grant") + } + if length == 0 || length > 1024*1024 { + return nil, errors.New("invalid worker bookmark size") + } + return C.GoBytes(bytes, C.int(length)), nil +} + +func resolveTransfer(transfer Transfer) (func(), error) { + var failure *C.char + owner := C.importTransfer(unsafe.Pointer(&transfer.Bookmark[0]), C.size_t(len(transfer.Bookmark)), &failure) + defer C.free(unsafe.Pointer(failure)) + if owner == nil { + if failure != nil { + return nil, errors.New(C.GoString(failure)) + } + return nil, errors.New("unable to resolve worker grant") + } + return func() { C.stopTransfer(owner) }, nil +} diff --git a/internal/fileaccess/transfer_other.go b/internal/fileaccess/transfer_other.go new file mode 100644 index 00000000..49541e38 --- /dev/null +++ b/internal/fileaccess/transfer_other.go @@ -0,0 +1,12 @@ +//go:build !darwin || !appleappstore + +package fileaccess + +import "errors" + +func createTransfer(_ string) ([]byte, error) { + return nil, errors.New("native worker grants unavailable") +} +func resolveTransfer(_ Transfer) (func(), error) { + return nil, errors.New("native worker grants unavailable") +} diff --git a/internal/fileaccess/transfer_test.go b/internal/fileaccess/transfer_test.go new file mode 100644 index 00000000..17e5130e --- /dev/null +++ b/internal/fileaccess/transfer_test.go @@ -0,0 +1,73 @@ +package fileaccess + +import ( + "context" + "errors" + "testing" + + "fyne.io/fyne/v2" + "fyne.io/fyne/v2/storage" +) + +func TestTransferRetainsOriginalScope(t *testing.T) { + active, stopped := false, 0 + uri := &sourceURI{URI: storage.NewFileURI("/selected.png"), record: Record{URI: "file:///selected.png", Bookmark: []byte("persistent")}, + resolve: func(_ context.Context, _ Record) (resolution, func(), error) { + active = true + return resolution{path: "/resolved.png"}, func() { active = false; stopped++ }, nil + }} + sources := []fyne.URI{uri} + ctx := WithSources(context.Background(), sources) + sources[0] = storage.NewFileURI("/replacement.png") + if SourceForPath(ctx, "/selected.png") != uri { + t.Fatal("source authority was not captured") + } + transfer, release, err := exportTransfer(ctx, SourceForPath(ctx, "/selected.png"), func(path string) ([]byte, error) { + if !active || path != "/resolved.png" { + t.Fatal("bookmark capture outside resolved authority") + } + return []byte("implicit"), nil + }) + if err != nil { + t.Fatal(err) + } + if !active || string(transfer.Bookmark) != "implicit" || transfer.Path != "/resolved.png" { + t.Fatal("transfer lost access") + } + release() + release() + if active || stopped != 1 { + t.Fatalf("released %d times", stopped) + } +} + +func TestTransferCaptureFailureReleasesScope(t *testing.T) { + for _, cancel := range []bool{false, true} { + ctx, stop := context.WithCancel(context.Background()) + released := false + uri := &sourceURI{URI: storage.NewFileURI("/selected"), record: Record{URI: "file:///selected", Bookmark: []byte("scope")}, + resolve: func(_ context.Context, _ Record) (resolution, func(), error) { + return resolution{path: "/selected"}, func() { released = true }, nil + }} + _, release, err := exportTransfer(ctx, uri, func(_ string) ([]byte, error) { + if cancel { + stop() + return []byte("implicit"), nil + } + return nil, errors.New("refused") + }) + stop() + if err == nil || release != nil || !released { + t.Fatal("failed capture retained access") + } + } +} + +func TestTransferCancelledBeforeCapture(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + cancel() + _, _, err := exportTransfer(ctx, storage.NewFileURI("/selected"), func(_ string) ([]byte, error) { t.Fatal("cancelled transfer created a grant"); return nil, nil }) + if !errors.Is(err, context.Canceled) { + t.Fatalf("got %v", err) + } +} diff --git a/internal/openwith/drop_darwin_store.go b/internal/openwith/drop_darwin_store.go new file mode 100644 index 00000000..1b49265b --- /dev/null +++ b/internal/openwith/drop_darwin_store.go @@ -0,0 +1,38 @@ +//go:build darwin && appleappstore + +package openwith + +/* +#include +#include "openwith_darwin.h" +*/ +import "C" + +import ( + "fyne.io/fyne/v2" + "fyne.io/fyne/v2/driver" +) + +// InstallWindowDrop preserves the native NSURLs before GLFW reduces them to +// paths. Call after Show, when the main window's native content view exists. +func InstallWindowDrop(window fyne.Window) bool { + native, ok := window.(driver.NativeWindow) + if !ok { + return false + } + installed := false + native.RunNative(func(value any) { + if mac, ok := value.(driver.MacWindowContext); ok && mac.NSWindow != 0 { + installed = C.picfetchInstallWindowDrop(C.uintptr_t(mac.NSWindow)) != 0 + } + }) + return installed +} + +func testWindowDrop(path string) int { + paths, release := cStrings([]string{path}) + defer release() + return int(C.picfetchTestWindowDrop(paths)) +} + +var _ = testWindowDrop diff --git a/internal/openwith/drop_darwin_store_test.go b/internal/openwith/drop_darwin_store_test.go new file mode 100644 index 00000000..24b3610e --- /dev/null +++ b/internal/openwith/drop_darwin_store_test.go @@ -0,0 +1,35 @@ +//go:build darwin && appleappstore + +package openwith + +import ( + "context" + "os" + "path/filepath" + "testing" + + "github.com/frathe/picfetch/internal/fileaccess" +) + +func TestNativeWindowDropRetainsSelectionWithoutChangingOtherViews(t *testing.T) { + got := captureDelivered(t) + path := filepath.Join(t.TempDir(), "cafe\u0301 # %20.png") + if err := os.WriteFile(path, []byte("fixture"), 0600); err != nil { + t.Fatal(err) + } + if result := testWindowDrop(path); result != 0 { + t.Fatalf("native hook fixture failed: %d", result) + } + files := got() + if len(files) != 1 || files[0].Path() != path || !fileaccess.NeedsCapture(files[0]) { + t.Fatalf("drop lost native ownership: %v", files) + } + captured, err := fileaccess.CaptureSelected(context.Background(), files) + if err != nil { + t.Fatal(err) + } + if len(captured) != 1 || !fileaccess.HasScope(captured[0]) { + t.Fatal("drop did not capture persistent authority") + } + fileaccess.ReleaseSelected(files) +} diff --git a/internal/openwith/drop_other.go b/internal/openwith/drop_other.go new file mode 100644 index 00000000..1570f9c6 --- /dev/null +++ b/internal/openwith/drop_other.go @@ -0,0 +1,8 @@ +//go:build !darwin || !appleappstore + +package openwith + +import "fyne.io/fyne/v2" + +// InstallWindowDrop leaves the ordinary Fyne/platform drop callback unchanged. +func InstallWindowDrop(_ fyne.Window) bool { return true } diff --git a/internal/openwith/openwith_darwin.h b/internal/openwith/openwith_darwin.h index 4eb2ba0b..12986ddd 100644 --- a/internal/openwith/openwith_darwin.h +++ b/internal/openwith/openwith_darwin.h @@ -10,6 +10,9 @@ #ifndef PICFETCH_OPENWITH_DARWIN_H #define PICFETCH_OPENWITH_DARWIN_H +#include +int picfetchInstallWindowDrop(uintptr_t window); +int picfetchTestWindowDrop(char **path); // picfetchInstallOpenHandler grafts -application:openURLs: and // -application:openFiles: onto GLFW's application delegate class. Returns 1 diff --git a/internal/openwith/openwith_darwin.m b/internal/openwith/openwith_darwin.m index 1f4c2dcf..06b6f0ca 100644 --- a/internal/openwith/openwith_darwin.m +++ b/internal/openwith/openwith_darwin.m @@ -290,3 +290,77 @@ void picfetchTestInvokeSelectedURLs(const char **urls, int n) { deliverSelectedURLs(selected); } } + +// The main view gets an instance-specific subclass. Other GLFW windows retain +// their existing drag/drop handler, and no GLFW implementation is patched. +static BOOL picfetchPerformDrop(id self, SEL selector, id sender) { + @autoreleasepool { + NSDictionary *options = @{NSPasteboardURLReadingFileURLsOnlyKey:@YES}; + NSArray *urls = [sender.draggingPasteboard readObjectsForClasses:@[NSURL.class] options:options]; + if (!urls.count) return NO; + deliverSelectedURLs(urls); + return YES; + } +} + +static BOOL installDropView(id view) { + if (!view) return NO; + Class original = object_getClass(view); + SEL selector = @selector(performDragOperation:); + Method method = class_getInstanceMethod(original, selector); + if (!method) return NO; + if (method_getImplementation(method) == (IMP)picfetchPerformDrop) return YES; + NSString *name = [@"PicFetchScopedDrop_" stringByAppendingString:NSStringFromClass(original)]; + Class subclass = NSClassFromString(name); + if (!subclass) { + subclass = objc_allocateClassPair(original, name.UTF8String, 0); + if (!subclass) return NO; + if (!class_addMethod(subclass, selector, (IMP)picfetchPerformDrop, method_getTypeEncoding(method))) { + objc_disposeClassPair(subclass); return NO; + } + objc_registerClassPair(subclass); + } else if (class_getSuperclass(subclass) != original || + class_getMethodImplementation(subclass, selector) != (IMP)picfetchPerformDrop) { + return NO; + } + object_setClass(view, subclass); + return YES; +} + +int picfetchInstallWindowDrop(uintptr_t pointer) { + @autoreleasepool { + NSWindow *window = (__bridge NSWindow *)(void *)pointer; + return installDropView(window.contentView); + } +} + +// Private pasteboard and synthetic receiver: no visible window or desktop input. +@interface PicFetchDropFixture : NSObject +@property(nonatomic) int calls; +- (BOOL)performDragOperation:(id)sender; +@end +@implementation PicFetchDropFixture +- (BOOL)performDragOperation:(id)sender { self.calls++; return YES; } +@end +@interface PicFetchDragFixture : NSObject +@property(nonatomic, strong) NSPasteboard *draggingPasteboard; +@end +@implementation PicFetchDragFixture +@end + +int picfetchTestWindowDrop(char **path) { + @autoreleasepool { + PicFetchDropFixture *target = [PicFetchDropFixture new]; + PicFetchDropFixture *other = [PicFetchDropFixture new]; + if (!installDropView(target) || !installDropView(target)) return 1; + PicFetchDragFixture *drag = [PicFetchDragFixture new]; + drag.draggingPasteboard = [NSPasteboard pasteboardWithUniqueName]; + NSURL *url = [NSURL fileURLWithPath:[NSString stringWithUTF8String:path[0]]]; + if (![drag.draggingPasteboard writeObjects:@[url]]) return 2; + BOOL accepted = [target performDragOperation:drag]; + [other performDragOperation:drag]; + [drag.draggingPasteboard releaseGlobally]; + if (!accepted || target.calls || other.calls != 1) return 3; + return 0; + } +} diff --git a/internal/similarity/client.go b/internal/similarity/client.go index d36cba59..66e699d9 100644 --- a/internal/similarity/client.go +++ b/internal/similarity/client.go @@ -15,6 +15,7 @@ import ( "syscall" "time" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/heic" "github.com/frathe/picfetch/internal/imaging" ) @@ -101,7 +102,8 @@ type Client struct { } type request struct { - Search *SearchRequest `json:",omitempty"` + Access []fileaccess.Transfer `json:",omitempty"` + Search *SearchRequest `json:",omitempty"` Assets string FavoritesDir string GeneralAnalysisDir string @@ -136,7 +138,11 @@ func (c Client) Analyze(ctx context.Context, paths []string, controls <-chan Con assets = defaultAssets(executable) } req := request{Assets: assets, FavoritesDir: c.FavoritesDir, GeneralAnalysisDir: c.GeneralAnalysisDir, GeneralAnalysisLimitBytes: c.GeneralAnalysisLimitBytes, DisableFavoriteCache: c.DisableFavoriteCache, Paths: paths, MaxEncodedBytes: imaging.MaxEncodedBytes(), AnalysisLimits: limits} - req.captureHEIC(heic.FromContext(ctx)) + release, err := c.captureAccess(ctx, &req) + if err != nil { + return err + } + defer release() cmd := c.workerCommand(ctx, executable) cmd.Env = append(os.Environ(), workerEnvironment+"=1") return analyzeCommand(ctx, cmd, req, controls, emit) @@ -297,6 +303,11 @@ func runWorker() error { err = req.AnalysisLimits.validateSourceCount(len(req.Paths)) } if err == nil { + releaseAccess, accessErr := importWorkerAccess(ctx, req.Access) + if accessErr != nil { + return accessErr + } + defer releaseAccess() imaging.SetMaxEncodedBytes(req.MaxEncodedBytes) var release func() ctx, release = workerHEICContext(ctx, req) diff --git a/internal/similarity/search_client.go b/internal/similarity/search_client.go index 549eb3ad..f0390695 100644 --- a/internal/similarity/search_client.go +++ b/internal/similarity/search_client.go @@ -35,7 +35,11 @@ func (c Client) Search(ctx context.Context, search SearchRequest, queries <-chan } search.Paths = slices.Clone(search.Paths) req := request{Assets: assets, MaxEncodedBytes: imaging.MaxEncodedBytes(), Search: &search} - req.captureHEIC(heic.FromContext(ctx)) + release, err := c.captureAccess(ctx, &req) + if err != nil { + return err + } + defer release() cmd := c.workerCommand(ctx, executable) cmd.Env = append(os.Environ(), workerEnvironment+"=1") return searchCommand(ctx, cmd, req, queries, emit) diff --git a/internal/similarity/worker_access.go b/internal/similarity/worker_access.go new file mode 100644 index 00000000..6dee8669 --- /dev/null +++ b/internal/similarity/worker_access.go @@ -0,0 +1,124 @@ +package similarity + +import ( + "context" + "encoding/json" + "fmt" + "path/filepath" + "sync" + + "fyne.io/fyne/v2" + "fyne.io/fyne/v2/storage" + + "github.com/frathe/picfetch/internal/fileaccess" + "github.com/frathe/picfetch/internal/heic" +) + +type transferExport func(context.Context, fyne.URI) (fileaccess.Transfer, func(), error) + +// captureWorkerAccess retains exact source authority through process exit. +// Optional cache/source failures keep the existing per-item/cache error paths; +// cancellation or unavailable model/code authority refuses launch entirely. +func captureWorkerAccess(ctx context.Context, req *request, app string, export transferExport) (func(), error) { + var releases []func() + var once sync.Once + release := func() { + once.Do(func() { + for i := len(releases) - 1; i >= 0; i-- { + releases[i]() + } + }) + } + seen := map[string]bool{} + add := func(uri fyne.URI, required bool) error { + path := uri.Path() + if seen[path] { + return nil + } + seen[path] = true + grant, stop, err := export(ctx, uri) + if err != nil { + if ctx.Err() != nil { + return ctx.Err() + } + if required { + return err + } + return nil + } + releases = append(releases, stop) + req.Access = append(req.Access, grant) + return nil + } + for _, path := range []string{app, filepath.Join(req.Assets, "vision_model.onnx"), filepath.Join(req.Assets, "preprocessor_config.json")} { + if err := add(storage.NewFileURI(path), true); err != nil { + release() + return nil, err + } + } + paths := req.Paths + caches := []string{req.FavoritesDir, req.GeneralAnalysisDir} + if req.Search != nil { + paths = req.Search.Paths + caches = []string{req.Search.Cache.Roots.FavoritesDir} + if req.Search.Cache.LooseEnabled { + caches = append(caches, req.Search.Cache.Roots.GeneralDir) + } + } + for _, path := range paths { + if err := add(fileaccess.SourceForPath(ctx, path), false); err != nil { + release() + return nil, err + } + } + for _, path := range caches { + if path == "" { + continue + } + if err := add(storage.NewFileURI(path), false); err != nil { + release() + return nil, err + } + } + // Bound the complete encoded request, including base64 bookmark expansion. + payload, err := json.Marshal(req) + if err == nil && len(payload) > workerRequestLimit { + err = fmt.Errorf("worker request exceeds %d bytes", workerRequestLimit) + } + if err == nil { + err = ctx.Err() + } + if err != nil { + release() + return nil, err + } + return release, nil +} + +func importWorkerAccess(ctx context.Context, grants []fileaccess.Transfer) (func(), error) { + var releases []func() + release := func() { + for i := len(releases) - 1; i >= 0; i-- { + releases[i]() + } + } + for _, grant := range grants { + stop, err := fileaccess.Import(ctx, grant) + if err != nil { + release() + return nil, err + } + releases = append(releases, stop) + } + return release, nil +} + +// Test command adapters own their own transport. Production captures authority +// for both worker protocols before admission and retains it through Wait. +func (c Client) captureAccess(ctx context.Context, req *request) (func(), error) { + req.captureHEIC(heic.FromContext(ctx)) + if c.command != nil { + return func() {}, nil + } + return prepareWorkerAccess(ctx, req) +} diff --git a/internal/similarity/worker_access_darwin.go b/internal/similarity/worker_access_darwin.go new file mode 100644 index 00000000..594bddd6 --- /dev/null +++ b/internal/similarity/worker_access_darwin.go @@ -0,0 +1,34 @@ +//go:build darwin && appleappstore + +package similarity + +import ( + "context" + "os" + "path/filepath" + + "github.com/frathe/picfetch/internal/fileaccess" + "github.com/frathe/picfetch/internal/macbundle" +) + +func prepareWorkerAccess(ctx context.Context, req *request) (func(), error) { + root, err := macbundle.CurrentRuntimeDirectory() + if err != nil { + return nil, err + } + // These app-owned roots must exist before Foundation can grant their child + // creation. Cache failures remain best-effort and are reported by the worker. + caches := []string{req.FavoritesDir, req.GeneralAnalysisDir} + if req.Search != nil { + caches = []string{req.Search.Cache.Roots.FavoritesDir} + if req.Search.Cache.LooseEnabled { + caches = append(caches, req.Search.Cache.Roots.GeneralDir) + } + } + for _, path := range caches { + if path != "" { + _ = os.MkdirAll(path, 0700) + } + } + return captureWorkerAccess(ctx, req, filepath.Dir(filepath.Dir(root)), fileaccess.Export) +} diff --git a/internal/similarity/worker_access_other.go b/internal/similarity/worker_access_other.go new file mode 100644 index 00000000..0277862b --- /dev/null +++ b/internal/similarity/worker_access_other.go @@ -0,0 +1,7 @@ +//go:build !darwin || !appleappstore + +package similarity + +import "context" + +func prepareWorkerAccess(_ context.Context, _ *request) (func(), error) { return func() {}, nil } diff --git a/internal/similarity/worker_access_test.go b/internal/similarity/worker_access_test.go new file mode 100644 index 00000000..38ae6e24 --- /dev/null +++ b/internal/similarity/worker_access_test.go @@ -0,0 +1,66 @@ +package similarity + +import ( + "context" + "errors" + "path/filepath" + "testing" + + "fyne.io/fyne/v2" + + "github.com/frathe/picfetch/internal/fileaccess" +) + +func TestWorkerAccessCapturesExactPathsUntilRetirement(t *testing.T) { + source, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///photos/one.png", Bookmark: []byte("persistent")}) + if err != nil { + t.Fatal(err) + } + ctx := fileaccess.WithSources(context.Background(), []fyne.URI{source}) + req := request{Assets: "/models", Paths: []string{source.Path(), source.Path()}, GeneralAnalysisDir: "/cache"} + active := map[string]bool{} + release, err := captureWorkerAccess(ctx, &req, "/Applications/PicFetch.app", func(_ context.Context, uri fyne.URI) (fileaccess.Transfer, func(), error) { + path := uri.Path() + if path == source.Path() && uri != source { + t.Fatal("source authority flattened") + } + active[path] = true + return fileaccess.Transfer{Path: path, Bookmark: []byte("implicit")}, func() { active[path] = false }, nil + }) + if err != nil { + t.Fatal(err) + } + want := []string{"/Applications/PicFetch.app", filepath.Join("/models", "vision_model.onnx"), filepath.Join("/models", "preprocessor_config.json"), "/photos/one.png", "/cache"} + if len(req.Access) != len(want) { + t.Fatalf("unexpected grants: %+v", req.Access) + } + for _, path := range want { + if !active[path] { + t.Fatalf("access lost: %s", path) + } + } + if active["/photos"] { + t.Fatal("source expanded to parent") + } + release() + release() + for path, on := range active { + if on { + t.Fatalf("access leaked: %s", path) + } + } +} + +func TestWorkerAccessFailureRetiresPriorGrants(t *testing.T) { + req := request{Assets: "/models"} + retired := 0 + _, err := captureWorkerAccess(context.Background(), &req, "/App.app", func(_ context.Context, uri fyne.URI) (fileaccess.Transfer, func(), error) { + if uri.Path() != "/App.app" { + return fileaccess.Transfer{}, nil, errors.New("missing models") + } + return fileaccess.Transfer{Path: uri.Path()}, func() { retired++ }, nil + }) + if err == nil || retired != 1 { + t.Fatalf("err=%v retired=%d", err, retired) + } +} diff --git a/internal/ui/batch.go b/internal/ui/batch.go index b74b5b6e..ea4729d4 100644 --- a/internal/ui/batch.go +++ b/internal/ui/batch.go @@ -12,6 +12,7 @@ package ui import ( + "context" "errors" "fmt" @@ -19,6 +20,7 @@ import ( "fyne.io/fyne/v2/lang" "github.com/frathe/picfetch/internal/clipboard" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/ui/deletion" ) @@ -108,14 +110,14 @@ func (v *viewer) copyGridSelection() { } targets := v.grid.Targets() - paths := make([]string, 0, len(targets)) + sources := make([]fyne.URI, 0, len(targets)) collection := v.state.Observe() for _, i := range targets { if i >= 0 && i < collection.Count() { - paths = append(paths, collection.FileAt(i).Path()) + sources = append(sources, collection.FileAt(i)) } } - if len(paths) == 0 { + if len(sources) == 0 { return } @@ -128,18 +130,18 @@ func (v *viewer) copyGridSelection() { v.completeClipboardCopy(token, done, nil) return } - err := clipboard.CopyFiles(paths) + err := copyFilesWithAccess(token.Context(), sources) v.completeClipboardCopy(token, done, func() { if err != nil { v.reportFileCopyError(err) return } - if len(paths) == 1 { + if len(sources) == 1 { v.ShowToast(lang.L("copied 1 file")) return } - v.ShowToast(fmt.Sprintf(lang.L("copied %d files"), len(paths))) + v.ShowToast(fmt.Sprintf(lang.L("copied %d files"), len(sources))) }) }) } @@ -163,3 +165,25 @@ func (v *viewer) selectAllInGrid() { v.grid.SelectAll() } + +func copyFilesWithAccess(ctx context.Context, sources []fyne.URI) error { + paths := make([]string, 0, len(sources)) + var releases []func() + defer func() { + for i := len(releases) - 1; i >= 0; i-- { + releases[i]() + } + }() + for _, uri := range sources { + resolved, release, err := fileaccess.Acquire(ctx, uri) + if err != nil { + return err + } + releases = append(releases, release) + paths = append(paths, resolved.Path()) + } + if err := ctx.Err(); err != nil { + return err + } + return clipboard.CopyFiles(paths) +} diff --git a/internal/ui/batch_test.go b/internal/ui/batch_test.go index 346f200b..b4a9a5d8 100644 --- a/internal/ui/batch_test.go +++ b/internal/ui/batch_test.go @@ -8,6 +8,7 @@ import ( "fyne.io/fyne/v2" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/uitest" ) @@ -222,9 +223,21 @@ func TestBatchDelete_LeavesTheWindowMaximizedOnAColdReload(t *testing.T) { func TestCopy_WhileGridVisibleCopiesTheSelectionAsFileReferences(t *testing.T) { v := openGridWith(t, "a.jpg", "b.jpg", "c.jpg") + released := 0 + sources := slices.Clone(v.state.Observe().DisplayFiles()) + for i, uri := range sources { + sources[i] = fileaccess.NewDestination(uri, func() { released++ }) + } + v.state.Replace(collectionInput{source: sources, display: sources}) var got []string uitest.StubClipboardCopyFiles(t, func(paths []string) error { got = paths + for _, uri := range sources { + fileaccess.ReleaseDestination(uri) + } + if released != 0 { + t.Error("source access ended during pasteboard publication") + } return nil }) uitest.StubClipboardCopy(t, func([]byte) error { @@ -239,6 +252,9 @@ func TestCopy_WhileGridVisibleCopiesTheSelectionAsFileReferences(t *testing.T) { handler.TypedShortcut(&fyne.ShortcutCopy{}) waitForClipboard(t, v) + if released != len(sources) { + t.Errorf("released %d sources", released) + } want := []string{v.state.Observe().DisplayFiles()[0].Path(), v.state.Observe().DisplayFiles()[1].Path(), v.state.Observe().DisplayFiles()[2].Path()} if !slices.Equal(got, want) { t.Errorf("CopyFiles paths = %v, want %v", got, want) diff --git a/internal/ui/deletion/deletion.go b/internal/ui/deletion/deletion.go index 5226626b..8cadb702 100644 --- a/internal/ui/deletion/deletion.go +++ b/internal/ui/deletion/deletion.go @@ -22,6 +22,7 @@ import ( "fyne.io/fyne/v2/lang" "fyne.io/fyne/v2/widget" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/imaging" "github.com/frathe/picfetch/internal/trash" "github.com/frathe/picfetch/internal/ui/widgets" @@ -256,9 +257,7 @@ func (c *Confirmer) performDelete() { // The claim includes Save/Strip/Export so their atomic replacement // cannot recreate a source after its successful move to Trash. // Pass the original path to Trash: a symlink is itself the target. - err := imaging.WithFileMutation(c.ctx, t.URI.Path(), func() error { - return trash.Move(t.URI.Path()) - }) + err := trashWithAccess(c.ctx, t.URI) if err != nil { if firstErr == nil { firstErr, firstFailed = err, t.URI.Name() @@ -333,3 +332,14 @@ func ShortcutHandler(request func()) func(fyne.Shortcut) { } } } + +// Keep the confirmed path: bookmark refresh must not redirect deletion to a +// different name, and a selected symlink itself is the Trash target. +func trashWithAccess(ctx context.Context, uri fyne.URI) error { + _, release, err := fileaccess.Acquire(ctx, uri) + if err != nil { + return err + } + defer release() + return imaging.WithFileMutation(ctx, uri.Path(), func() error { return trash.Move(uri.Path()) }) +} diff --git a/internal/ui/deletion/deletion_test.go b/internal/ui/deletion/deletion_test.go index 72609f7f..a05d61c2 100644 --- a/internal/ui/deletion/deletion_test.go +++ b/internal/ui/deletion/deletion_test.go @@ -14,6 +14,7 @@ import ( "fyne.io/fyne/v2/storage" "fyne.io/fyne/v2/test" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/imaging" "github.com/frathe/picfetch/internal/trash" "github.com/frathe/picfetch/internal/uitest" @@ -457,3 +458,25 @@ func TestShortcutHandler_RunsOnlyOnSecondaryCut(t *testing.T) { t.Errorf("requests = %d after Shift+Delete, want 1", requests) } } + +func TestTrashRetainsAccessThroughNativeCompletion(t *testing.T) { + files := tempFiles(t, "scoped.jpg") + released := false + source := fileaccess.NewDestination(files[0], func() { released = true }) + host := &fakeHost{files: []fyne.URI{source}} + c := newConfirmer(t, host) + uitest.StubTrashMove(t, func(_ string) error { + fileaccess.ReleaseDestination(source) + if released { + t.Error("source released during native Trash call") + } + return nil + }) + c.Request() + c.setSelection(true) + c.confirmSelection() + c.Settle() + if !released { + t.Error("source access leaked after Trash completion") + } +} diff --git a/internal/ui/explorer.go b/internal/ui/explorer.go index 9df66e75..10825804 100644 --- a/internal/ui/explorer.go +++ b/internal/ui/explorer.go @@ -61,13 +61,15 @@ func (v *viewer) beginExplorerAnalysis() { } collection := v.state.Observe() paths := make([]string, 0, collection.Count()) + access := make([]fyne.URI, 0, collection.Count()) visibility := v.dupes.Visibility() for i := range collection.Count() { if visibility.Visible(i) { paths = append(paths, collection.FileAt(i).Path()) + access = append(access, collection.FileAt(i)) } } - request := explorerui.OpenRequest{Sources: paths, FavoriteDir: collection.Favorite(), FavoritesDir: v.favorites.Dir()} + request := explorerui.OpenRequest{Sources: paths, Access: access, FavoriteDir: collection.Favorite(), FavoritesDir: v.favorites.Dir()} cache := v.searchCachePolicy() if cache.LooseEnabled { request.GeneralAnalysisDir, request.GeneralAnalysisLimitBytes = cache.Roots.GeneralDir, cache.GeneralLimitBytes diff --git a/internal/ui/explorer/feature.go b/internal/ui/explorer/feature.go index 45c98b64..c70b47a4 100644 --- a/internal/ui/explorer/feature.go +++ b/internal/ui/explorer/feature.go @@ -70,6 +70,7 @@ type Options struct { // OpenRequest captures a prepared collection and its cache policy at admission. type OpenRequest struct { + Access []fyne.URI Sources []string FavoriteDir, FavoritesDir string GeneralAnalysisDir string diff --git a/internal/ui/explorer/feature_test.go b/internal/ui/explorer/feature_test.go index 762663e7..d76c3e11 100644 --- a/internal/ui/explorer/feature_test.go +++ b/internal/ui/explorer/feature_test.go @@ -18,6 +18,7 @@ import ( "github.com/frathe/picfetch/internal/explorerpresets" "github.com/frathe/picfetch/internal/explorertrial" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/similarity" "github.com/frathe/picfetch/internal/ui/explorer" "github.com/frathe/picfetch/internal/uitest" @@ -173,25 +174,36 @@ func TestFeatureOpenCopiesSourcesAndRejectsClosedDelivery(t *testing.T) { queue := &uitest.UIQueue{} started, release := make(chan struct{}), make(chan struct{}) var captured []string - provider := func(_ context.Context, paths []string, _ <-chan similarity.Control, emit func(similarity.Event)) error { + uri, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///original.jpg", Bookmark: []byte("authority")}) + if err != nil { + t.Fatal(err) + } + access := []fyne.URI{uri} + var capturedURI fyne.URI + provider := func(ctx context.Context, paths []string, _ <-chan similarity.Control, emit func(similarity.Event)) error { close(started) <-release captured = slices.Clone(paths) + capturedURI = fileaccess.SourceForPath(ctx, paths[0]) emit(similarity.Event{Complete: true, Successful: 1, Total: 1, Items: []similarity.Item{{Path: paths[0], Cohort: "a", Position: []float32{0, 0}}}}) return nil } f := explorer.NewFeature(host, explorer.Options{Analyze: provider, Queue: queue}) t.Cleanup(func() { f.Stop(); f.Settle() }) paths := []string{"/original.jpg"} - if !f.Open(explorer.OpenRequest{Sources: paths}) { + if !f.Open(explorer.OpenRequest{Sources: paths, Access: access}) { t.Fatal("opening a prepared collection did not admit analysis") } <-started paths[0] = "/replacement.jpg" + access[0] = nil close(release) f.Wait() f.Close() f.Settle() + if capturedURI != uri { + t.Fatal("captured source authority changed") + } if !slices.Equal(captured, []string{"/original.jpg"}) { t.Fatalf("analysis source snapshot changed with its caller: %v", captured) } diff --git a/internal/ui/explorer/workflow.go b/internal/ui/explorer/workflow.go index 9a18e3ab..18e14ca4 100644 --- a/internal/ui/explorer/workflow.go +++ b/internal/ui/explorer/workflow.go @@ -11,6 +11,7 @@ import ( "fyne.io/fyne/v2/lang" "github.com/frathe/picfetch/internal/favstore" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/similarity" ) @@ -33,7 +34,7 @@ func (f *Feature) Open(request OpenRequest) bool { f.complete = false f.hasMap = false token := f.lifecycle.begin() - token.ctx = f.heic.CaptureContext(token.context()) + token.ctx = fileaccess.WithSources(f.heic.CaptureContext(token.context()), request.Access) f.token = token f.surface.SetResult(nil, nil) f.surface.Status(lang.L("Analyzing images...")) diff --git a/internal/ui/explorer_test.go b/internal/ui/explorer_test.go index cbe66b18..8d017558 100644 --- a/internal/ui/explorer_test.go +++ b/internal/ui/explorer_test.go @@ -549,6 +549,9 @@ func TestVisualSimilarityExplorer(t *testing.T) { } }) v.handleDrop(files) + if stage == "sort" { + settleScan(t, v) + } select { case <-entered: case <-time.After(testTimeout): @@ -892,6 +895,7 @@ func TestVisualSimilarityExplorer(t *testing.T) { v.OpenFiles(files) } else { v.OpenFavorite("replacement-favorite", files) + settleScan(t, v) } select { case <-entered: diff --git a/internal/ui/filework.go b/internal/ui/filework.go index e97801bd..4437737f 100644 --- a/internal/ui/filework.go +++ b/internal/ui/filework.go @@ -12,6 +12,7 @@ import ( "fyne.io/fyne/v2/storage" "github.com/frathe/picfetch/internal/completion" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/imaging" "github.com/frathe/picfetch/internal/requestlife" ) @@ -93,7 +94,7 @@ func (v *viewer) reconcileSearchOrigin() { if source == nil || source.Scheme() != "file" { continue } - if _, err := os.Stat(source.Path()); errors.Is(err, os.ErrNotExist) { + if _, err := fileaccess.Stat(ctx, source); errors.Is(err, os.ErrNotExist) { missing = append(missing, i) } } @@ -147,7 +148,6 @@ func writtenFileSources(ctx context.Context, path string, files collectionSnapsh if ctx.Err() != nil { return nil } - written, _ := os.Stat(path) var affected []fyne.URI for i := range files.Count() { if ctx.Err() != nil { @@ -159,8 +159,17 @@ func writtenFileSources(ctx context.Context, path string, files collectionSnapsh } if filepath.Clean(u.Path()) == filepath.Clean(path) { affected = append(affected, u) - } else if source, err := os.Stat(u.Path()); err == nil && written != nil && os.SameFile(source, written) { - affected = append(affected, u) + } else { + resolved, release, err := fileaccess.Acquire(ctx, u) + if err != nil { + continue + } + source, sourceErr := os.Stat(resolved.Path()) + written, writtenErr := os.Stat(path) + if sourceErr == nil && writtenErr == nil && os.SameFile(source, written) { + affected = append(affected, u) + } + release() } } return affected @@ -184,7 +193,13 @@ func (v *viewer) refreshWrittenFile(result imaging.WriteResult, reload, refreshE done() return } - path, err := filepath.Abs(u.Path()) + resolved, release, err := fileaccess.Acquire(ctx, u) + if err != nil { + done() + return + } + defer release() + path, err := filepath.Abs(resolved.Path()) if err == nil { path, err = filepath.EvalSymlinks(path) } diff --git a/internal/ui/reveal.go b/internal/ui/reveal.go index 002ee326..ec32e042 100644 --- a/internal/ui/reveal.go +++ b/internal/ui/reveal.go @@ -11,6 +11,7 @@ import ( "fyne.io/fyne/v2" "fyne.io/fyne/v2/lang" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/filemanager" ) @@ -29,7 +30,7 @@ func (v *viewer) revealCurrentFile() { // Capture the admitted choice now; later navigation must not retarget it. source, _, _ := v.CurrentFile() - path := source.Path() + ctx := v.fileWork.ctx // reveal is finished once this goroutine has fully run, error // reporting included, so a test can wait for the whole operation @@ -39,7 +40,13 @@ func (v *viewer) revealCurrentFile() { go func() { defer done() - if err := filemanager.Reveal(path); err != nil { + resolved, release, err := fileaccess.Acquire(ctx, source) + if err != nil { + v.reportRevealError(err) + return + } + defer release() + if err := filemanager.Reveal(resolved.Path()); err != nil { v.reportRevealError(err) } }() diff --git a/internal/ui/reveal_test.go b/internal/ui/reveal_test.go index 79324e31..3bd35649 100644 --- a/internal/ui/reveal_test.go +++ b/internal/ui/reveal_test.go @@ -9,6 +9,7 @@ import ( "fyne.io/fyne/v2" "fyne.io/fyne/v2/driver/desktop" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/uitest" ) @@ -22,18 +23,27 @@ import ( func TestRevealCurrentFile_HandsThePathToTheFileManager(t *testing.T) { v, _, _ := newTestUI(t) - jpegURI := uitest.TempJPEGURI(t, "picked.jpg", 4, 4, color.RGBA{R: 100, A: 255}) + uri := uitest.TempJPEGURI(t, "picked.jpg", 4, 4, color.RGBA{R: 100, A: 255}) + released := false + jpegURI := fileaccess.NewDestination(uri, func() { released = true }) v.state.Replace(collectionInput{source: []fyne.URI{jpegURI}, display: []fyne.URI{jpegURI}}) var got string uitest.StubReveal(t, func(path string) error { got = path + fileaccess.ReleaseDestination(jpegURI) + if released { + t.Error("source released before Finder handoff returned") + } return nil }) v.revealCurrentFile() waitForReveal(t, v) + if !released { + t.Error("Finder handoff leaked access") + } if got != jpegURI.Path() { t.Errorf("revealed path = %q, want %q", got, jpegURI.Path()) } diff --git a/internal/ui/run.go b/internal/ui/run.go index 7fd65af6..3bc7fd1f 100644 --- a/internal/ui/run.go +++ b/internal/ui/run.go @@ -71,6 +71,9 @@ func Run(application fyne.App, initial []fyne.URI, opts launch.Options, prepared // defers CLI drops until the event loop is running, as handleDrop // touches widgets directly. window.Show() + if !openwith.InstallWindowDrop(window) { + fyne.LogError("native window drop bridge unavailable", nil) + } view.syncNativeMenuBar() registerStartup(application, view, initial) stopSignals := func() {} diff --git a/internal/ui/visualsearch.go b/internal/ui/visualsearch.go index 18c58359..69d72183 100644 --- a/internal/ui/visualsearch.go +++ b/internal/ui/visualsearch.go @@ -80,6 +80,7 @@ func (v *viewer) startVisualSearch(reference string) { return } var paths []string + var access []fyne.URI seen := map[string]bool{} collection := v.state.Observe() for i := range collection.Count() { @@ -87,6 +88,7 @@ func (v *viewer) startVisualSearch(reference string) { if uri != nil && !seen[uri.Path()] { seen[uri.Path()] = true paths = append(paths, uri.Path()) + access = append(access, uri) } } origin := searchHost{v}.CaptureVisit() @@ -96,7 +98,7 @@ func (v *viewer) startVisualSearch(reference string) { after := v.explorer.Suspend() v.explorer.Surface().Hide() policy := v.searchCachePolicy() - if v.visualsearch.Start(searchui.StartRequest{Paths: paths, ReferencePath: reference, Cache: policy, After: after}) { + if v.visualsearch.Start(searchui.StartRequest{Paths: paths, Access: access, ReferencePath: reference, Cache: policy, After: after}) { v.presentSearch(searchui.Visit{ReferencePath: reference}) } else { v.browsing.detachSearch() diff --git a/internal/ui/visualsearch/feature.go b/internal/ui/visualsearch/feature.go index 6bf3cb11..bfaf4c8b 100644 --- a/internal/ui/visualsearch/feature.go +++ b/internal/ui/visualsearch/feature.go @@ -4,6 +4,8 @@ package visualsearch import ( "slices" + "fyne.io/fyne/v2" + "github.com/frathe/picfetch/internal/fileidentity" "github.com/frathe/picfetch/internal/heic" "github.com/frathe/picfetch/internal/similarity" @@ -41,6 +43,7 @@ type Options struct { // StartRequest fixes the original scope and waits for prior native retirement. type StartRequest struct { + Access []fyne.URI Paths []string ReferencePath string Cache similarity.CachePolicy @@ -57,6 +60,7 @@ type State struct { // Feature is UI-owned. Workers capture immutable requests and deliver through // Queue; Wait and Settle are the only blocking lifecycle observations. type Feature struct { + access []fyne.URI heic *heic.Capability host Host provider similarity.SearchProvider @@ -115,6 +119,7 @@ func (f *Feature) Start(request StartRequest) bool { f.clear() f.active = true f.scope = slices.Clone(request.Paths) + f.access = slices.Clone(request.Access) f.cache = request.Cache f.progress = grid.Progress{Total: len(f.scope)} f.preparing = true @@ -263,6 +268,7 @@ func (f *Feature) clear() { f.invalidateQuery() f.active, f.preparing = false, false f.scope, f.history = nil, nil + f.access = nil f.initialGrid = nil f.progress = grid.Progress{} } diff --git a/internal/ui/visualsearch/feature_test.go b/internal/ui/visualsearch/feature_test.go index eb27e0b6..6fb81e60 100644 --- a/internal/ui/visualsearch/feature_test.go +++ b/internal/ui/visualsearch/feature_test.go @@ -8,6 +8,9 @@ import ( "testing" "testing/synctest" + "fyne.io/fyne/v2" + + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/fileidentity" "github.com/frathe/picfetch/internal/similarity" "github.com/frathe/picfetch/internal/ui/grid" @@ -730,3 +733,33 @@ func TestVisualSearchCanceledSuccessorRetainsExternalRetirementBarrier(t *testin } }) } + +func TestVisualSearchRetainsSourceAuthorityAcrossProducerRestart(t *testing.T) { + uri, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///source.png", Bookmark: []byte("authority")}) + if err != nil { + t.Fatal(err) + } + captured := make(chan fyne.URI, 2) + host := &visitHost{} + f := visualsearch.New(host, visualsearch.Options{Queue: &uitest.UIQueue{}, Provider: func(ctx context.Context, req similarity.SearchRequest, _ <-chan similarity.SearchQuery, _ func(similarity.SearchEvent)) error { + captured <- fileaccess.SourceForPath(ctx, req.Paths[0]) + <-ctx.Done() + return ctx.Err() + }}) + t.Cleanup(func() { f.Stop(); f.Settle() }) + access := []fyne.URI{uri} + if !f.Start(visualsearch.StartRequest{Paths: []string{uri.Path()}, ReferencePath: uri.Path(), Access: access}) { + t.Fatal("not started") + } + access[0] = nil + if got := <-captured; got != uri { + t.Fatal("source authority lost") + } + <-f.Suspend() + if !f.Explore(uri.Path()) { + t.Fatal("not restarted") + } + if got := <-captured; got != uri { + t.Fatal("restart lost captured authority") + } +} diff --git a/internal/ui/visualsearch/session.go b/internal/ui/visualsearch/session.go index 8167060c..3391c848 100644 --- a/internal/ui/visualsearch/session.go +++ b/internal/ui/visualsearch/session.go @@ -8,6 +8,7 @@ import ( "fyne.io/fyne/v2" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/similarity" "github.com/frathe/picfetch/internal/ui/grid" ) @@ -37,7 +38,7 @@ func (f *Feature) beginProducer(after <-chan struct{}) { f.sessionID++ f.cacheWarned = false f.revision = 0 - ctx, cancel := context.WithCancel(f.heic.CaptureContext(context.Background())) + ctx, cancel := context.WithCancel(fileaccess.WithSources(f.heic.CaptureContext(context.Background()), f.access)) s := &producer{id: f.sessionID, cancel: cancel, queries: make(chan similarity.SearchQuery, 1), done: make(chan struct{}), notice: make(chan struct{}, 1)} f.producer = s request := similarity.SearchRequest{SessionID: s.id, Paths: slices.Clone(f.scope), Limit: 30, Cache: f.cache} diff --git a/packaging/apple-app-store/README.md b/packaging/apple-app-store/README.md index 31e2ec81..d28f5498 100644 --- a/packaging/apple-app-store/README.md +++ b/packaging/apple-app-store/README.md @@ -1,8 +1,8 @@ # Mac App Store preparation -The Apple channel is experimental. No submission package or signed sandbox -qualification of the complete app has been completed. The native worker -fixture is qualified on this Apple Silicon host. Retain all PicFetch features; do not use +The Apple channel is experimental. A universal ad-hoc sandbox bundle and real +HEIC/ONNX workers are locally qualified on ARM and Intel under Rosetta. No Apple +Distribution-signed submission or complete GUI sandbox qualification is claimed. Retain all PicFetch features; do not use temporary filesystem exceptions or weaken the analysis worker's network denial to pass a packaging check. @@ -24,7 +24,7 @@ produce the direct-download distribution and must not be submitted to the Store. archive and stages only its verified native library, LICENSE, ThirdPartyNotices.txt and Privacy.md. The complete archive is verified in a temporary directory before any payload enters the package. It is a pre-sign -step; signed Mach-O validation is separate work. The same library's SHA-256 +step; the local packager then validates signed Mach-O code and outer resource seals. The same library's SHA-256 changes when signed, so checking it against the original upstream digest after signing is incorrect. @@ -66,6 +66,7 @@ Run the read-only prerequisite check: ```sh APPLE_STORE_TEAM_ID=YOURTEAMID \ +APPLE_STORE_TESTFLIGHT=1 \ APPLE_STORE_PROFILE=/absolute/path/PicFetch.provisionprofile \ make apple-store-preflight ``` @@ -73,7 +74,9 @@ make apple-store-preflight It uses `/Applications/Xcode.app/Contents/Developer` without changing the system developer-directory selection. Override `APPLE_STORE_DEVELOPER_DIR` for another full Xcode installation. It checks for Store application and installer signing -identities with private keys and confirms that a profile file was supplied. +identities with private keys. `APPLE_STORE_TESTFLIGHT=1` requires a profile file; +without TestFlight, an app using only unrestricted macOS entitlements may omit +a profile, per [Apple TN3125](https://developer.apple.com/documentation/technotes/tn3125-inside-code-signing-provisioning-profiles). It does not install credentials, access an Apple account, upload a package or certify the profile or app. Supplying a profile does not establish its signature, expiry, registered identifier or entitlement compatibility. @@ -84,8 +87,9 @@ Required inputs: `io.github.frathe.picfetch` available to that team. - Apple Distribution / Mac App Store application signing certificate with its private key, plus a Mac Installer Distribution identity. -- Mac App Store Connect provisioning profile for that identifier and signing - certificate; exact helpers and their profiles depend on the sandbox design. +- Mac App Store Connect provisioning profile for TestFlight or restricted + entitlements, matching the identifier and signing certificate. Inherited + helpers have exactly sandbox/inherit entitlements, without their own profile. - App Store Connect app record, public support/contact and privacy-policy URLs, localized listing, screenshots, category, rating and intended countries. - Encryption questionnaire determination covering Go's bundled TLS/cryptography; @@ -117,6 +121,71 @@ Required inputs: TestFlight/Store installation and upgrade. Submission/upload needs Ronin's authorization separately; this preparation does not publish anything. -No `package-mac-store` target is advertised until these runtime and signing -contracts are implemented and exercised. A successful prerequisite check cannot +The local target below is explicitly an ad-hoc qualification route; distribution +signing, installer packaging and submission validation are still separate work. A successful prerequisite check cannot substitute for signed sandbox or submission evidence. + + +## Local universal sandbox bundle + +Python 3.11+, full Xcode, Go and the pinned Fyne CLI are used; no developer +certificate or account is needed. Supply both reviewed ONNX archives and a fresh +output directory (an existing output is refused): + +```sh +APPLE_STORE_ARM64_ARCHIVE=/absolute/path/onnxruntime-osx-arm64-1.29.0.tgz \ +APPLE_STORE_AMD64_ARCHIVE=/absolute/path/onnxruntime-osx-x86_64-1.23.2.tgz \ +APPLE_STORE_OUTPUT_DIR=/absolute/path/local-qualification \ +make apple-store-package-local +``` + +The builder compiles arm64 and x86_64 app/broker/service binaries, stages both +pinned native libraries and complete upstream notices, signs nested code before +the app, and validates exact entitlements, native dependency paths, architecture, +minimum OS and code/resource seals. ARM requires macOS 14.0 and Intel 13.4. +Deployment flags are explicit cgo cache inputs. Fyne metadata writes happen in a +temporary directory; the repository build number is unchanged. The output includes +`PicFetch.app` and a JSON manifest containing source/diff evidence and file hashes. +These ad-hoc signatures are local test signatures and cannot be submitted. + +```sh +python3 scripts/macstorepackage/package.py --verify /path/local-qualification/PicFetch.app +PICFETCH_TEST_MAC_APP_BUNDLE=/path/local-qualification/PicFetch.app \ +python3 -m unittest discover -s scripts/macstorepackage -v +python3 scripts/macstorepackage/qualify.py \ + --app /path/local-qualification/PicFetch.app \ + --models /path/to/verified-model-data --result /path/worker-results.json +``` + +The worker qualifier creates disposable thin copies and substitutes only the +main GUI executable with a Go test driver. Production broker/service/image worker +code remains. It checks HEIC 8/10-bit pixels, ONNX inference, model/app access, +private-container source/cache transfer, cache reuse, retained search and actual +TCP/UDP denial. Models are verified by production checks; they are only added to +the temporary fixture. Intel results on this host are Rosetta evidence, not +physical Intel or minimum-OS testing. Generated private fixture directories are +removed on normal exit. The original local app is never changed. + +Explorer and visual search capture URI authority before leaving UI. Production +clients transfer fresh implicit bookmarks while original source scopes are active, +keep acquisitions through worker return, and release receiving scopes when the +worker exits. Exact source files are transferred; no source parent is granted. +The outer app grant permits full signature validation, and internal model/cache +locations carry separate grants. Missing sources/cache grants keep existing +per-item/cache failure behavior. This does not yet qualify stale-grant recovery, +window drops or every OS file handoff. + +## Submission preparation records + +[Submission draft](submission-draft.md) contains listing copy, review steps, +screenshot plans and the human-input checklist. [Dependency/privacy audit](privacy-dependency-audit.md) +records a remaining release blocker: both pinned ONNX binaries include Apple-listed +SDKs, but their archives contain no Apple privacy manifests. Local sandbox and +signature checks do not clear that submission requirement. + +The worker qualifier also supports `--deny-source`. It compiles only its test +parent with a Go overlay omitting source grants, keeps the packaged production +broker/service/helper unchanged, and requires the private source to fail with an +OS permission error while an allowed bundle source still completes inference. +Run it with the same `--app`, `--models` and a separate `--result` output as the +positive qualification. The overlay and diagnostic driver are never shipped. diff --git a/packaging/apple-app-store/privacy-dependency-audit.md b/packaging/apple-app-store/privacy-dependency-audit.md new file mode 100644 index 00000000..5fbe1e2c --- /dev/null +++ b/packaging/apple-app-store/privacy-dependency-audit.md @@ -0,0 +1,92 @@ +# Apple Store privacy and dependency audit + +Checked 2026-09-30 against the local qualification bundle and primary sources. +This record does not certify App Store acceptance. + +## Native closure and listed SDKs + +The package uses ONNX Runtime **1.29.0 arm64** and **1.23.2 x86_64**, staged +through the existing pinned size/hash verifier. Both contain only system-linked +native dependencies according to the package Mach-O checks. That check does not +exclude statically linked third-party code. + +| Runtime | Abseil evidence | Protobuf evidence | +|---|---|---| +| arm64 1.29.0 | Binary strings include `absl::` and `lts_20250814`, plus an Abseil 20250814.0 build path | Binary strings include `google::protobuf` and a `v3.21.12` build path | +| x86_64 1.23.2 | Binary strings include `lts_20250512` | Binary RTTI includes `google8protobuf`; upstream pins v21.12 | + +The exact upstream dependency declarations are +[ONNX 1.29.0 deps.txt](https://raw.githubusercontent.com/microsoft/onnxruntime/v1.29.0/cmake/deps.txt) +and [ONNX 1.23.2 deps.txt](https://raw.githubusercontent.com/microsoft/onnxruntime/v1.23.2/cmake/deps.txt). +Their Abseil versions agree with the binary symbols; both pin Protobuf v21.12. +The runtime's bundled ThirdPartyNotices.txt also names both libraries. + +Apple's [listed-SDK requirements](https://developer.apple.com/support/third-party-SDK-requirements/) +include Abseil and Protobuf and cover SDKs repackaging them. Binary dependencies +also have SDK-signature requirements. **The current local bundle is not cleared +for this gate.** Neither pinned runtime archive contains an `.xcprivacy` file; +its `Privacy.md` is explanatory text, not an Apple privacy manifest. An ad-hoc +app signature is not evidence of upstream SDK provenance or distribution trust. + +The [Abseil 20250814.0 tree](https://github.com/abseil/abseil-cpp/tree/20250814.0) +lists a PrivacyInfo.xcprivacy file. Retrieving exact manifest files and recursive +source trees failed in this environment (GitHub API timeouts; research-browser +cache/fetch failures). A newer default-branch manifest is not silently substituted +for an exact pinned dependency. The Protobuf CocoaPods manifest packaging found +in current upstream is specifically for its Objective-C runtime; it does not by +itself qualify the older C++ runtime in these ONNX binaries. + +Next technical work: obtain and hash manifests for the exact shipped components, +verify their applicable declarations and intended bundle placement, and resolve +upstream binary-signature provenance or use a reproducible, reviewed source build. +Then validate the final package through Apple's tools. Do not invent empty +manifests or infer compliance from an archive's license notice. This is an +external-artifact/research blocker, not a request to remove Similarity Explorer. + +The native libraries live in Contents/Frameworks. Complete runtime LICENSE, +ThirdPartyNotices.txt and Privacy.md files live under versioned directories in +Contents/Resources. PicFetch's LICENSE, THIRD-PARTY-NOTICES.md and PRIVACY.md are +also included there. Existing Go/Fyne/font/AVIF notices remain required; the +Make verification gate checks the retained AVIF notice inputs. + +## Network and local-data inventory + +| Behavior | Trigger and destination | Information exposed | +|---|---|---| +| Image viewing, EXIF, similarity, local caches | Local processing; isolated Apple analysis worker | No image or metadata upload | +| Location Map and EXIF map | User opens map; `tile.openstreetmap.org` | Tile coordinates, IP, user agent and connection data; map area can reveal photo location | +| Model setup | User accepts download; pinned Hugging Face model/configuration URLs with permitted provider redirects | Asset URL, IP, user agent and connection data | +| Release-note artwork | Displayed notes contain remote images | Image URL, IP and ordinary HTTP details; current bundled 1.1.11 notes contain no images | +| Help links | User opens public GitHub pages in their browser | Browser/provider policies apply | +| App updates | Mac App Store delivery | Apple-managed; GitHub self-update is disabled for this channel | + +Implementation evidence: `internal/similarity/assets_install.go`, +`internal/ui/locationmap/tiles.go`, `internal/ui/exifwin/tiles.go`, +`internal/ui/help/releaseart.go`, `internal/ui/help/releaseimage.go` and the +captured launch/distribution policy. Permitted asset redirect host families are +huggingface.co, hf.co, github.com and githubusercontent.com; executable runtime +downloads are excluded from Store setup. + +OSMF explicitly describes network access records, including IP/request data, +and third-party tile CDN processing in its +[privacy policy](https://osmfoundation.org/wiki/Privacy_Policy). Therefore a +blanket claim that all app-related requests are unretained is unsupported. +Apple's [app privacy guidance](https://developer.apple.com/app-store/app-privacy-details/) +distinguishes on-device processing from data transmitted and retained by third +parties. The exact App Store Connect categories/purposes/linkage answers need +assessment of map-derived location and provider retention. This record does not +select answers or claim that every optional request is exempt from disclosure. + +The app privacy policy now explains Apple bundled runtime delivery, temporary +worker access and conditional release-note image requests. The public policy +URL must be rechecked after these uncommitted changes are published. + +## Verification boundaries + +Local artifact checks, retained ARM/Intel-Rosetta worker results and binary +samples are under `.scratch/apple-store-package/`; the implementation plan +records their commands and limits. No SDK privacy upload validation, physical +Intel run, minimum-OS run, final Store signing, account questionnaire or App +Review has completed. Required-reason API rules for other Apple platforms should +not be copied wholesale onto native macOS; independently apply the current +[privacy manifest documentation](https://developer.apple.com/documentation/bundleresources/privacy-manifest-files). diff --git a/packaging/apple-app-store/submission-draft.md b/packaging/apple-app-store/submission-draft.md new file mode 100644 index 00000000..0ac7cb7c --- /dev/null +++ b/packaging/apple-app-store/submission-draft.md @@ -0,0 +1,129 @@ +# Mac App Store submission draft + +Prepared 2026-09-30. Local preparation only: nothing uploaded or published. +The current ad-hoc bundle is a qualification artifact, not a submission build. +Release readiness still depends on the technical and human gates below. + +## Listing copy + +Name: **PicFetch** + +Subtitle: **Explore your photo collection** + +Suggested category: Photography. Secondary category, price, territories, age +rating, copyright owner and seller/trader details require Ronin's decision. + +Description draft: + +> PicFetch helps you browse and explore image collections on your Mac. Open +> individual pictures or folders, switch between an image and a thumbnail grid, +> compare pictures, run a slideshow, and organize reusable collections with +> Favorites. +> +> Explore image metadata and discover where geotagged photos were taken with +> Location Map. Maps use OpenStreetMap when you choose to open them. +> +> Similarity Explorer and visual search analyze images locally on your Mac. +> Their optional first-use model download is about 372 MB. After setup, +> analysis works offline; your pictures are not uploaded for analysis. +> +> PicFetch has no account requirement, advertising or analytics. You choose +> which files and folders it can access. Updates arrive through the Mac App +> Store. + +Keyword draft: `photo,image,viewer,slideshow,compare,metadata,EXIF,favorites,similarity` + +Do not advertise minimum-OS or physical Intel qualification until those tests +pass. The generated bundle declares macOS 14.0 on Apple Silicon and 13.4 on +Intel, based on the pinned runtime requirements. Rosetta tests are separate +from a physical Intel run. + +Candidate support URL: https://github.com/frathe/picfetch/issues + +Candidate privacy URL: https://github.com/frathe/picfetch/blob/main/PRIVACY.md + +Both URLs still require successful unauthenticated content checks after the +updated policy reaches the public branch. The research browser could not fetch +these pages on 2026-09-30; that is not evidence that the links are public or +broken. No contact email or legal identity has been invented. + +## Review notes draft + +No login, subscription or reviewer account is required. + +1. Use Open to select a folder of local images. The app uses the macOS file + chooser and stores security-scoped bookmarks to reopen granted files. A file + selection authorizes that file; select a folder to browse its siblings. +2. Browse images, switch to the grid, compare pictures and save a Favorite. + Relaunch and reopen the Favorite to exercise saved access. Permission failure + can be addressed by explicitly selecting the source again with Open. +3. Similarity Explorer offers an optional download of pinned model data and a + processor configuration from Hugging Face. Declining leaves ordinary viewing + available. Executable ONNX code is bundled and updated with the app. Analysis + uses an embedded XPC service and inherited image helper without network + permission; operation-scoped grants let it read selected inputs and local + model/cache files. +4. HEIC decoding uses Apple's system decoder in that isolated worker boundary. +5. Maps contact OpenStreetMap only after opening Location Map or expanding the + EXIF Location section. Tile requests disclose the viewed map area and normal + network request information, but contain no pictures or filenames. +6. Store builds do not check GitHub for app updates or install replacement app + code. Help includes the privacy policy and third-party notices offline. + +Replace these notes with observed instructions from the final signed artifact. +Do not submit the disposable worker-qualification app; its main executable is a +command-line test driver, not the viewer. + +## Screenshot plan + +Capture the real, final sandboxed viewer with a small synthetic or clearly +licensed image collection: main image and filmstrip, grid, comparison, +Similarity Explorer, and Location Map. Show actual features and readable UI; +exclude private filenames, user images and personal coordinates. Capture both +light and dark appearance if useful. App Store screenshots remain outstanding +because the computer-use connection did not respond. Generated mockups are not +substitutes for these screenshots. Confirm current upload dimensions in App +Store Connect when the app record exists. + +## Privacy and encryption facts + +See [the dependency/privacy audit](privacy-dependency-audit.md). Do not select +“No Data Collected” solely because image analysis is local: optional map and +model requests reach third-party servers, whose retention policies matter. + +The app uses Go HTTPS/TLS for map and model downloads. The distribution closure +also contains hashing and signature-verification code. No custom cryptographic +algorithm, end-user encrypted messaging feature, VPN or user-file encryption +feature was added for the Store port. This is a technical inventory, not an +export classification. Do not set `ITSAppUsesNonExemptEncryption` until the +export questionnaire has been assessed for the actual shipped closure. Apple +requires the developer to determine the applicable documentation; see +[export compliance](https://developer.apple.com/help/app-store-connect/manage-app-information/overview-of-export-compliance/). + +## Technical gates before account work + +- Resolve the listed-SDK privacy manifest/signature gap in the bundled ONNX + closure; see the audit. Keep source/version and notice delivery evidence. +- Qualify the final signed GUI: picker, Finder/Dock/window drops, restart, + Favorites, moved and unavailable sources, Save/export, overwrite, Trash, + clipboard, Finder reveal and wallpaper. Scope renewal has unit/race coverage; + moved-folder relaunch and final UI behavior remain unverified. +- Finish negative grant/cancellation tests against the production worker and + qualify real Intel hardware and both declared minimum OS versions. +- Run the full native Linux/amd64 CI suite, CodeQL and fresh post-suppression + Qodana analysis. ARM Docker emulation cannot replace the isolation gate. +- Complete the Store certificate signing and productbuild route; validate the + profile when supplied and inspect the final installer payload. Ad-hoc tests + cannot establish distribution trust or App Store acceptance. + +## Human inputs last + +1. Signed Git commits and pushing the preparation branch were reauthorized by + Ronin after the overnight work; submission/release still require a separate decision. +2. Supply the developer Team ID and Store app/installer signing identities with + their private keys. TestFlight also needs a matching provisioning profile. +3. Create or select the App Store Connect record for `io.github.frathe.picfetch`. +4. Confirm legal contact, seller/trader status, age rating, pricing, availability, + privacy questionnaire and encryption/export answers from the prepared facts. +5. Review the final screenshots/listing, authorize upload and TestFlight testing, + then decide whether to submit for review and release. diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index 5aef411b..d336a88a 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -744,3 +744,306 @@ Evidence: `.scratch/apple-app-store-runtime/` and validity is not Apple Distribution trust, provisioning or App Store approval. The full packaged app/XPC workers and Intel execution remain next qualification steps. Cost: 0 spawns, 2 inline review rounds; no repeated complete suite. + +### Next slice — reproducible local Store bundle + +Base `beb8491`; T4, T0 inline, Deep route; zero spawns/two reviews. Build an +ad-hoc sandboxed universal app using pinned Fyne v1.7.2, both pinned runtime +archives, current app metadata and native XPC components. Keep local qualification +explicitly separate from Apple Distribution signing and submission. No new +runtime dependencies or model-distribution decision; models remain optional data. + +Files: scripts/macstorestage Go archive adapter; scripts/macstorepackage Python +orchestrator/validator and tests; Make entry point; packaging guide, architecture, +requirements, todos and evidence. Contracts: fresh output only, explicit metadata +(no source Build mutation), fixed app/XPC identifiers and sandbox entitlement +sets, architecture-specific minimum OS, standard nested-code paths, upstream +hashes before signing, complete upstream notices in Resources, deepest-first +signing and strict final validation. Missing prerequisites fail before builds. + +Acceptance commands: Python packaging guard tests (bad entitlements, unexpected +code/dependencies/architecture, wrong minima, edited resources); actual universal +local build and strict code-signature readback; real HEIC worker probes through +the packaged broker on arm64 and Rosetta where available; all existing archive +staging guards, GoLand changed code inspections, make verify-build. Final package +hash/manifest and build log retained in .scratch. No credential requirement for +this local route; no distribution-ready or App Store acceptance claim. + +Apple TN3125 distinguishes unrestricted macOS entitlements from restricted +profile-backed capabilities: TestFlight always requires a profile, but an +ordinary Store Mac app without restricted entitlements may not. Correct the +preflight/docs accordingly; do not assume every inherited helper needs a profile. + +Ronin's overnight instruction: continue autonomous implementation/testing, but +make **no further commits** while Ronin is away because commits require Ronin's +signature. Leave reviewed changes and evidence in the working tree for return. + +#### Packaging progress and dependent worker finding + +The universal arm64/x86_64 ad-hoc bundle builds and passes nested/outer signing, +exact sandbox entitlements, architecture, dependency-path and deployment-minimum +validation. Pinned Fyne requires a Go source in its working directory even with +--executable; a staging-only stub satisfies it without changing FyneApp.toml. +The first minimum-OS parser incorrectly included LC_BUILD_VERSION's linker tool +version; the corrected parser distinguishes minos from the linker version. + +A disposable thin arm64 derivative retaining production worker code passes real +HEIC 8/10-bit pixel probes through XPC. ONNX then fails EPERM reading model data +in the outer app Resources. The service has its own bundle/container; shared +bundle location alone does not confer authority. This is useful failing native +acceptance evidence, not an ONNX pass. Continue the dependent T3 grant transport +before declaring worker/package qualification complete. + +### Dependent slice — transient worker authority + +T3/T2, T0 inline, zero spawns/two review rounds. Preserve captured URI authority +in Explorer/search request contexts. Main exports fresh implicit bookmarks only +while source access is active; transfer exact source files, internal model/cache +roots, and the immutable outer app needed for signature validation. Do not widen +an external source to its parent or pass persistent app-scoped bookmarks to a +different process. Main retains acquisitions until worker return; worker resolves +and releases transferred scopes within its existing tracked process lifetime. +Invalid/cancelled transfer refuses launch. No new goroutines or global test seams. + +Files: fileaccess transfer/context API with native build pair and tests; similarity +request capture/import build pair and clients; Explorer/search captured request +wiring; existing feature tests, architecture, qodana exclusions and evidence. +Verification: scoped capture/release/cancellation guards, request serialization +limits, existing client/search regressions, native packaged workers with real +models/source/cache permissions, no-network checks, focused race/build/vet/IDE. +Apple's Accessing files from the macOS App Sandbox documents options=0 implicit +bookmark transfer across processes. Such grants convey the rights the OS permits; +do not claim a read-only attenuation that the public implicit API does not offer. + +#### Local bundle / worker evidence (overnight, no commits) + +The local universal bundle passes full nested/outer strict signatures, exact +sandbox entitlements, architecture/dependency/minimum-OS checks. Seven Python +policy/artifact tests pass, including resource tampering, a validly re-signed +network-enabled service, a hidden non-executable-bit Mach-O resource and a +re-signed wrong deployment minimum. Both ONNX archives retain original checksums +and notices before signing. Make target `apple-store-package-local` produces the +same qualification layout and an external source/input/payload hash manifest. + +Production HEIC/analysis/search clients now transfer fresh implicit authority +while retaining original source access until worker return. Disposable thin +arm64 and x86_64/Rosetta derivatives of the bundle pass real 8/10-bit HEIC pixels, +ONNX inference, private-container source/cache grants, cache reuse, retained +search, actual TCP/UDP denial and worker retirement. The GUI is replaced only by +a qualification driver; production worker/broker/service code is retained. +This is not physical Intel/minimum-OS or complete GUI qualification. + +Affected Apple-tagged race suites pass: fileaccess 1.280s, similarity 38.031s, +Explorer 4.309s and visualsearch 1.292s. Four deliberate Go overlays fail for +premature release, flattened worker source, lost Explorer authority and lost +search authority. GoLand inspected all new transfer code and changed feature +code including warnings; shared client capture removed a confirmed duplication, +and goimports corrected one import grouping. Seven signed artifact guards pass. + +Broader root UI tests exposed five legacy sort-admission waits that did not drain +scanUI after the earlier scan-queue change. The diagnosing-bugs workflow narrowed +the failure to scan completion queued before sort admission (other candidates: +source wrapper loss, changed admission). Adding the existing settleScan barrier +only to sort-stage fixtures makes the whole affected replacement/cancellation +subgroup pass in 1.030s, without a production behavior change or sleeps. + +One qualification-driver build tried to replace an existing universal binary, +which Go refused; disposable copies now unlink that main executable first. +Explicit minimum-OS cgo flags also avoid reusing cache objects compiled for a +newer deployment target. A format gate caught a deliberately faulty scratch +overlay; formatting that fixture fixed the gate prerequisite. Final build/vet +and complete root-group output remain tracked in /tmp/picfetch-store-* and +/tmp/picfetch-worker-access-* until archived. No commit has been attempted since +Ronin withdrew overnight signing authorization. + +### Next slice — OS handoff and source metadata scopes + +T2, T0 inline, zero spawns/two reviews. Acquire captured authority on existing +workers before Trash serialization/native completion, clipboard file-reference +publication and Finder handoff. Keep original confirmed Trash paths to preserve +symlink semantics; never retarget a destructive action from a refreshed bookmark. +Scope favorite-preview source stat calls and write reconciliation before alias +resolution/version reads. Do not add desktop effects to unit tests: existing +uitest OS stubs prove active scope and exact release; closed authority must +refuse the native operation. Existing mutation/symlink and completion tests remain. +Files: fileaccess metadata helper/test; root batch/reveal/filework and tests; +delete feature and tests; favthumbs source-version helper/tests. Focused Apple +and ordinary tests/race, negative guards, GoLand and final Make build checks. +Wallpaper already writes an app-owned persistent PNG; native wallpaper and +Finder/pasteboard integration still require separate signed GUI qualification. + +#### Handoff test evidence and GUI limitation + +Four new assertions failed before fixes: clipboard/Finder/Trash released native +ownership during the OS call, and Favorite preview versioning stat-ed a closed +source. After scoped acquisition, ordinary focused tests pass (root UI 5.300s, +delete 0.451s, favthumbs 0.348s). Apple race checks pass in fileaccess, favthumbs +and deletion. The broad root regex also selected the direct-distribution updater +`TestApplyStagedUpdate_SavesNotesAndCallsApply`, which correctly cannot apply an +update in an Apple Store build; rerun with the actual file-action test prefixes. +Do not relax Store update policy. GoLand found a loop-defer warning in clipboard +scope collection; one explicit reverse-release list replaces it, preserving the +required simultaneous grants through pasteboard publication. + +Computer-use skill was read for full GUI/picker qualification. Its node_repl +get_app_state call did not return after more than a minute and was terminated; +no generated PicFetch app process was observed afterward. Native picker/GUI +interaction is therefore unverified, not passed. Continue independent native +fixtures and implementation; leave the inaccessible GUI check in the final list. + +### Next slice — native window-drop authority + +T2, T0 inline, zero spawns/two reviews. Fyne/GLFW converts dropped NSURLs to plain +paths before the viewer sees them. Install a per-instance Cocoa content-view +subclass only on the shown main window in Apple Store builds. Override its drop +method to read native file URLs and deliver the existing retained-selection queue; +other views and ordinary builds retain the original method. Preserve ordering, +command admission, worker capture/discard and shutdown through the existing +Open With path; no new queues/goroutines or permissions. The root callback ignores +drop coordinates already. Native tests use synthetic objects/private pasteboards, +not desktop input, to pin one delivery, ownership, repeated install and unaffected +other views. Full real GLFW/picker/drop UI qualification stays unverified until +the GUI tool is available. Files: openwith native bridge/build pair/test, root Run, +architecture/qodana/evidence. Verify focused Darwin/Apple tests, builds, inspections. + +#### Window-drop and handoff completion + +Native drop ownership/repeated-install/isolation guard failed before the hook, +then Apple openwith race passed (1.291s); ordinary build passed. Corrected Apple +file-action root race passed (44.450s). GoLand inspected the new Go bridge/tests, +root Run, clipboard release fix and qualifier code. One ordinary-build constant +condition in Run is replaced by an unconditional platform dispatcher whose +ordinary implementation is a successful no-op. Native Objective-C bridge remains +covered by cgo compilation and native tests, not a GoLand Objective-C scan. + +### Next slice — refresh restored bookmark authority + +T2, T0 inline, zero spawns/two reviews. Resolution returns an operation-owned +path plus optional renewed bookmark bytes. Native resolution checks Apple's +stale flag and regenerates explicit app-scope data while access is active. +Acquire returns new immutable metadata; it never mutates a captured source. +The existing opening worker refreshes restored sources once per shared URI, +releases access before delivery, preserves duplicate occurrences, and leaves +unavailable sources intact for current per-item/offline handling. Cancellation +still aborts the opening. Refreshed records follow existing session/Favorite +serialization. No UI prompt or global mutable bookmark cache. Tests pin changed +paths/bytes, directory-relative ownership, exact release, cancellation and +unavailable fallback. Verify fileaccess race plus existing collection tests, +Apple native compilation, vet and GoLand. Real moved-folder relaunch still needs +signed native/GUI qualification. +Apple API contract: https://developer.apple.com/documentation/foundation/nsurl/urlbyresolvingbookmarkdata:options:relativetourl:bookmarkdataisstale:error: + +#### Bookmark refresh verification + +The new opening guards first failed on stale authority and missing cancellation. +After implementation, ordinary fileaccess race passed (1.365s); Apple +fileaccess/openwith race passed (1.272s/1.275s). GoLand inspected source, selection, +both native build pairs, source/transfer tests, Run and the ordinary drop bridge, +including warnings; no findings remain. Persistence roundtrip coverage added +before the final run. Scope renewal is operation-owned and copied before release; +original records remain immutable. Offline failure leaves the entry unchanged, +so it cannot silently replace a saved collection with only currently online files. +The collection/opening regression group is running separately. + +### Next slice — submission facts and dependency privacy audit + +T5, T0 inline, zero spawns/two reviews. Record exact dependency evidence, service +endpoints, draft listing/review notes and export-encryption facts. Do not publish +or select legal questionnaire answers. Apple lists Abseil and Protobuf, including +SDKs that repackage them; both ONNX notice files contain both components. The +pinned runtime tarballs include Privacy.md but no xcprivacy manifest. Investigate +exact upstream revisions and binary evidence before claiming SDK compliance. +Any unresolved upstream manifests/signatures stay explicit release blockers. + +#### Submission records and remaining SDK artifact gap + +Prepared submission-draft.md and privacy-dependency-audit.md with explicit +technical/account gates, not legal answers or a submission claim. Both runtime +binaries confirm Abseil/Protobuf via symbols; exact upstream dependency versions +match. Pinned tarballs lack xcprivacy files. Abseil's exact ARM-tag tree lists one, +but content retrieval and recursive GitHub API requests failed/time out. Do not +substitute default-branch or Objective-C Protobuf declarations without evidence. +PRIVACY.md now covers Apple bundled-runtime delivery and optional note artwork. +Collection/opening/Favorite Apple race regressions passed in 99.087s. + +### Next slice — production worker negative source grant + +T4, T0 inline, zero spawns/two reviews. Extend only the disposable qualification +main/script. A Go build overlay removes source export from the parent, retaining +app/model/cache grants and the unchanged production broker/service/image worker. +The same private PNG must fail per-item with no successful inference. Without +that overlay the refusal assertion must fail because the positive path succeeds. +Run the final overlay on both native ARM and Intel under Rosetta, alongside the +normal worker qualification. Do not ship the diagnostic flag or overlay. + +#### Final local checks and inspection disposition + +`make verify-build` passes after all production changes: format, pinned/generated +assets, notice checks, vet and build. Full Apple-tagged vet and Windows/amd64 +no-cgo internal vet are clean. Apple race checks: fileaccess 1.269s, openwith +cached from its passing current-code run, Help 28.138s. Collection/opening root +race passed 99.087s. The latest universal artifact is +`.scratch/apple-store-package/overnight-final/PicFetch.app`; its external manifest +records the build source and every bundled file hash. It includes all production +changes, unchanged version/build 1.1.11/477, both architectures and strict nested/ +outer code seals. Subsequent qualifier-only changes do not modify that app. + +GoLand inspection requested every one of 46 changed Go/Python/shell/native bridge +code files with warnings enabled. Four production files showed duplicate-code +weak warnings caused by deliberately faulty `.scratch` Go overlay copies. +Renaming retained fixture payloads to `.fixture` and updating their overlay JSON +removed those false comparisons; all four affected inspections and the final +qualifier Go file are clean on reinspection. No production duplication suppression +was added. Native code is additionally compiled into both architectures; an +empty GoLand report is not a complete Objective-C/Qodana profile guarantee. +The JSON reports are archived in `.scratch/apple-store-package/`. + +The first negative source fixture correctly hit the existing all-images-failed +terminal error before a Complete event. It now uses an allowed bundle image as +a control: the worker must initialize/infer successfully for that image and +report an OS permission error for the private ungranted image. Removing the +overlay makes the new assertion fail with two successful images, zero failures +and two inferences, confirming the guard. No production policy was weakened. + +#### Final production worker and artifact results + +Latest universal app positive qualification passes native ARM and Intel under +Rosetta: real HEIC pixels, ONNX inference, private source/cache access, cache reuse, +retained search, TCP/UDP denial and orderly worker exit. Negative source-grant +qualification passes both architectures: OS-denied private source plus successful +allowed control image, with exactly one inference. JSON results: +`overnight-workers.json` and `overnight-source-denial.json` in the scratch package +directory. Signed-artifact/policy suite: seven tests pass (1.381s), including +modified resource, re-signed network-enabled service, hidden Mach-O and minimum +OS tampering. The required full-suite platform check still refuses this host's +linux/aarch64 Docker daemon; no isolation test was skipped or policy relaxed. + +SDK manifests/signature provenance, native GUI interaction (unresponsive tool), +physical Intel/minimum-OS runs and full native-amd64 CI remain unverified. The +Store distribution signing/installer route cannot be qualified without matching +identities/profile; the ad-hoc route remains deliberately non-submittable. +GoLand fallback evidence is not fresh Qodana SARIF, CodeQL or a GitHub review. +These gates precede the account/legal/submission checklist in submission-draft.md. + +Cost ledger for overnight continuation: all slices T0 inline, 0 spawns, two +review rounds per implemented slice. Focused tests and native fixtures ran as +recorded; final broad local build/vet passed. Full race suite unavailable on the +selected Docker platform. No commits or pushes; HEAD remains beb8491 and Ronin's +signing authorization is still withheld. Remaining work stays in todos.md. + +Final formatting and qualifier vet pass; Docker shard inventory passes with +749 runnable root UI tests assigned across three shards. Shell syntax and Python +compile checks pass. Evidence is archived under `.scratch/apple-store-package/logs/`; +`overnight-source.json` fingerprints the uncommitted source and +`overnight-worktree.patch` preserves its tracked diff. Human-only tasks remain +last; no credentials, questionnaire answers, uploads, commits or release actions +were attempted. + +### Commit and push authorization + +Ronin subsequently authorized committing and pushing the prepared work. Before +staging, all 57 changed files matched the archived overnight source hashes, with +no additional paths. Existing runtime test/inspection evidence therefore carries +forward unchanged. Only this authorization record, the TODO entry and submission +checklist were updated. Use the configured SSH commit signing; do not disable +signing. Push the existing feature branch, without merging or releasing. diff --git a/qodana.yaml b/qodana.yaml index 15d2ac75..19f7c7f1 100644 --- a/qodana.yaml +++ b/qodana.yaml @@ -128,11 +128,17 @@ exclude: - "internal/favthumbs/store_test.go" - "internal/favthumbs/sweep_test.go" - "internal/favthumbs/sync_test.go" + - "internal/fileaccess/destination_test.go" + - "internal/fileaccess/manifest_test.go" + - "internal/fileaccess/selection_test.go" + - "internal/fileaccess/source_test.go" + - "internal/fileaccess/transfer_test.go" - "internal/fileidentity/occurrence_test.go" - "internal/filemanager/filemanager_test.go" - "internal/filepicker/darwin_test.go" - "internal/filepicker/filepicker_test.go" - "internal/filepicker/windows_test.go" + - "internal/filescan/authority_test.go" - "internal/filescan/filescan_test.go" - "internal/filesort/filesort_test.go" - "internal/hdbscan/hdbscan_test.go" @@ -146,14 +152,6 @@ exclude: - "internal/heic/metadata_test.go" - "internal/heic/native_darwin_test.go" - "internal/heic/primaryfixture_test.go" - - "internal/filescan/authority_test.go" - - "internal/fileaccess/manifest_test.go" - - "internal/fileaccess/source_test.go" - - "internal/fileaccess/selection_test.go" - - "internal/fileaccess/destination_test.go" - - "internal/macbundle/bundle_test.go" - - "internal/macbundle/signature_darwin_test.go" - - "internal/macworker/command_test.go" - "internal/heic/process_linux_test.go" - "internal/heic/process_unix_test.go" - "internal/heic/restrict_linux_test.go" @@ -186,11 +184,15 @@ exclude: - "internal/launch/policy_test.go" - "internal/launch/preparation_test.go" - "internal/locationtrial/recorder_test.go" + - "internal/macbundle/bundle_test.go" + - "internal/macbundle/signature_darwin_test.go" + - "internal/macworker/command_test.go" - "internal/mosaic/generator_test.go" - "internal/mosaic/heic_test.go" - "internal/mosaic/layout_test.go" - "internal/mosaic/mosaic_test.go" - "internal/mosaic/quality_test.go" + - "internal/openwith/drop_darwin_store_test.go" - "internal/openwith/openwith_darwin_test.go" - "internal/openwith/openwith_test.go" - "internal/ort/binding_test.go" @@ -217,6 +219,7 @@ exclude: - "internal/similarity/search_rank_test.go" - "internal/similarity/search_session_test.go" - "internal/similarity/tags_test.go" + - "internal/similarity/worker_access_test.go" - "internal/similarity/worker_cancel_unix_test.go" - "internal/similarity/worker_linux_test.go" - "internal/trash/trash_test.go" diff --git a/scripts/apple-store-preflight.sh b/scripts/apple-store-preflight.sh index 84408f0f..daa6dd80 100644 --- a/scripts/apple-store-preflight.sh +++ b/scripts/apple-store-preflight.sh @@ -64,7 +64,9 @@ else fi profile="${APPLE_STORE_PROFILE:-}" -if [ -z "$profile" ] || [ ! -f "$profile" ]; then +if [ -z "$profile" ] && [ "${APPLE_STORE_TESTFLIGHT:-0}" != 1 ]; then + printf 'No profile supplied; unrestricted Mac App Store entitlements may omit it. TestFlight always requires one.\n' +elif [ -z "$profile" ] || [ ! -f "$profile" ]; then fail 'Set APPLE_STORE_PROFILE to the Mac App Store Connect provisioning profile for io.github.frathe.picfetch.' else printf 'Provisioning profile supplied; its signature, identity, expiry and entitlements still require artifact validation.\n' diff --git a/scripts/macstorepackage/package.py b/scripts/macstorepackage/package.py new file mode 100644 index 00000000..ade1c005 --- /dev/null +++ b/scripts/macstorepackage/package.py @@ -0,0 +1,247 @@ +"""Build and validate an ad-hoc, universal Apple Store qualification app. + +This route has no distribution signing or upload mode. Apple account credentials +are unnecessary; the result cannot be submitted as an App Store release. +""" +import argparse +import hashlib +import json +import os +from pathlib import Path +import plistlib +import re +import shutil +import subprocess +import tempfile +import tomllib + + +ENTITLEMENTS = { + "app": {"com.apple.security.app-sandbox": True, + "com.apple.security.network.client": True, + "com.apple.security.files.user-selected.read-write": True, + "com.apple.security.files.bookmarks.app-scope": True}, + "service": {"com.apple.security.app-sandbox": True}, + "helper": {"com.apple.security.app-sandbox": True, + "com.apple.security.inherit": True}, +} + + +APP_ID = "io.github.frathe.picfetch" +TARGETS = {"arm64": ("arm64", "14.0"), "amd64": ("x86_64", "13.4")} +TAGS = "no_emoji,nodynamic,appleappstore" + + +def check_dependencies(paths): + for path in paths.splitlines(): + if not path.startswith(("/System/Library/", "/usr/lib/")): + raise ValueError(f"non-system native dependency: {path}") + + +def check_entitlements(role, actual): + if actual != ENTITLEMENTS[role]: + raise ValueError(f"unexpected {role} entitlements: {actual}") + + +def check_macho(actual_arches, expected_arches, load_commands, minimum): + if set(actual_arches.split()) != expected_arches: + raise ValueError(f"unexpected architectures: {actual_arches}") + versions = re.findall(r"minos (\d+\.\d+(?:\.\d+)?)", load_commands) + if not versions: + versions = re.findall(r"version (\d+\.\d+(?:\.\d+)?)", load_commands) + normalize = lambda v: tuple((list(map(int, v.split("."))) + [0, 0])[:3]) + if not versions or any(normalize(v) > normalize(minimum) for v in versions): + raise ValueError(f"native minimum OS exceeds {minimum}: {versions}") + + +def run(*args, cwd=None, env=None, capture=False): + print("+", " ".join(map(str, args)), flush=True) + result = subprocess.run(list(map(str, args)), cwd=cwd, env=env, check=True, + stdout=subprocess.PIPE if capture else None) + return result.stdout if capture else None + + +def digest(path): + with path.open("rb") as source: + return hashlib.file_digest(source, "sha256").hexdigest() + + +def write_plist(path, data): + path.write_bytes(plistlib.dumps(data, sort_keys=True)) + + +def native_check(path, architectures, minima, env): + actual = run("xcrun", "lipo", "-archs", path, env=env, capture=True).decode().strip() + for arch in architectures: + commands = run("xcrun", "otool", "-arch", arch, "-l", path, env=env, capture=True).decode() + # Keep only the deployment command; dylib compatibility versions are unrelated. + deployment = re.findall(r"cmd LC_(?:BUILD_VERSION|VERSION_MIN_MACOSX)\n(.*?)(?=Load command|\Z)", commands, re.S) + check_macho(actual, architectures, "\n".join(deployment), minima[arch]) + linked = run("xcrun", "otool", "-arch", arch, "-L", path, env=env, capture=True).decode().splitlines()[1:] + dependencies = [line.strip().split(" (", 1)[0] for line in linked] + # otool lists a dylib's install ID first; it is not a dependency. + if path.suffix == ".dylib": + dependencies = dependencies[1:] + check_dependencies("\n".join(dependencies)) + + +def verify(app, env): + contents = app / "Contents" + service = contents / "XPCServices" / (APP_ID + ".worker.xpc") + roles = {app: "app", service: "service", + contents / "MacOS/picfetch-worker-client": "helper", + service / "Contents/MacOS/picfetch-image-worker": "helper"} + for path, role in roles.items(): + data = run("codesign", "-d", "--entitlements", ":-", path, capture=True, env=env) + check_entitlements(role, plistlib.loads(data)) + run("codesign", "--verify", "--strict", path, env=env) + plist = plistlib.loads((contents / "Info.plist").read_bytes()) + if plist.get("CFBundleIdentifier") != APP_ID or plist.get("CFBundleExecutable") != "PicFetch": + raise ValueError("unexpected app identity") + if plist.get("LSMinimumSystemVersionByArchitecture") != {a: m for a, m in TARGETS.values()}: + raise ValueError("unexpected per-architecture deployment minimum") + expected_code = {contents / "MacOS/PicFetch", contents / "MacOS/picfetch-worker-client", + service / "Contents/MacOS/worker", service / "Contents/MacOS/picfetch-image-worker"} + for path in expected_code: + native_check(path, {"arm64", "x86_64"}, {a: m for a, m in TARGETS.values()}, env) + libraries = {"libonnxruntime.1.29.0.dylib": ("arm64", "14.0"), + "libonnxruntime.1.23.2.dylib": ("x86_64", "13.4")} + if {p.name for p in (contents / "Frameworks").iterdir()} != set(libraries): + raise ValueError("unexpected bundled runtime files") + for name, (arch, minimum) in libraries.items(): + path = contents / "Frameworks" / name + native_check(path, {arch}, {arch: minimum}, env) + run("codesign", "--verify", "--strict", path, env=env) + data = run("codesign", "-d", "--entitlements", ":-", path, capture=True, env=env) + if data.strip() and plistlib.loads(data): + raise ValueError("native libraries must not carry entitlements") + expected_code.add(path) + # Reject extra executable payloads or symlinks rather than silently signing them. + for path in contents.rglob("*"): + if path.is_symlink(): + raise ValueError(f"unexpected bundle symlink: {path}") + if path.is_file() and path not in expected_code: + with path.open("rb") as source: + magic = source.read(4) + if os.access(path, os.X_OK) or magic in (b"\xcf\xfa\xed\xfe", b"\xce\xfa\xed\xfe", b"\xfe\xed\xfa\xcf", b"\xfe\xed\xfa\xce", b"\xca\xfe\xba\xbe", b"\xca\xfe\xba\xbf"): + raise ValueError(f"unexpected executable payload: {path}") + run("codesign", "--verify", "--deep", "--strict", app, env=env) + + +def build_environment(env, arch, minimum): + # cgo hashes these flags into its cache key. MACOSX_DEPLOYMENT_TARGET alone + # can reuse objects compiled for a newer SDK deployment target. + flags = "-O2 -g -mmacosx-version-min=" + minimum + return env | {"GOOS": "darwin", "GOARCH": arch, "CGO_ENABLED": "1", + "MACOSX_DEPLOYMENT_TARGET": minimum, "CC": "clang", "CXX": "clang++", + "CGO_CFLAGS": flags, "CGO_CXXFLAGS": flags, "CGO_LDFLAGS": flags} + + +def build(args): + repo = Path(__file__).resolve().parents[2] + env = os.environ | {"DEVELOPER_DIR": os.environ.get("APPLE_STORE_DEVELOPER_DIR", "/Applications/Xcode.app/Contents/Developer")} + if args.verify: + verify(Path(args.verify).resolve(), env) + return + out = Path(args.out).absolute() + if out.exists() or out.is_symlink(): + raise ValueError("output already exists; choose a fresh directory") + archives = {"arm64": Path(args.arm64_archive).resolve(), "amd64": Path(args.amd64_archive).resolve()} + if any(not p.is_file() for p in archives.values()): + raise ValueError("both pinned ONNX archives are required") + fyne = repo / ".tools/fyne-v1.7.2/fyne" + if not fyne.is_file(): + raise ValueError("run make install-fyne first") + details = tomllib.loads((repo / "FyneApp.toml").read_text())["Details"] + if details["ID"] != APP_ID or details["Name"] != "PicFetch": + raise ValueError("unexpected source app identity") + run("xcodebuild", "-version", env=env) + out.parent.mkdir(parents=True, exist_ok=True) + with tempfile.TemporaryDirectory(prefix=".picfetch-store-", dir=out.parent) as temporary: + stage = Path(temporary) + runtime = stage / "runtime" + # Verify all downloaded code before spending time compiling the application. + for arch, archive in archives.items(): + run("go", "run", "-tags", TAGS, "./scripts/macstorestage", "-arch", arch, + "-archive", archive, "-out", runtime / arch, cwd=repo, env=env) + for arch, (native, minimum) in TARGETS.items(): + archdir = stage / arch + archdir.mkdir() + buildenv = build_environment(env, arch, minimum) + run("go", "build", "-tags", TAGS, "-trimpath", "-ldflags=-s -w", "-o", archdir / "PicFetch", ".", cwd=repo, env=buildenv) + for source, name in (("client.m", "picfetch-worker-client"), ("service.m", "worker")): + run("xcrun", "clang", "-arch", native, "-mmacosx-version-min=" + minimum, + "-fobjc-arc", "-Wall", "-Wextra", "-Werror", "-framework", "Foundation", + repo / "internal/macworker/native" / source, "-o", archdir / name, env=env) + for name in ("PicFetch", "picfetch-worker-client", "worker"): + run("xcrun", "lipo", "-create", stage / "arm64" / name, stage / "amd64" / name, + "-output", stage / name, env=env) + # Fyne validates a source directory even with an explicit executable. + # A staging-only stub keeps its metadata/build-number writes off the repo. + (stage / "main.go").write_text("package main\n\nfunc main() {}\n") + run(fyne, "package", "--os", "darwin", "--executable", stage / "PicFetch", + "--name", "PicFetch", "--appID", APP_ID, "--appVersion", details["Version"], + "--appBuild", details["Build"], "--icon", repo / details["Icon"], "--release", cwd=stage, env=env) + app = stage / "PicFetch.app" + contents = app / "Contents" + service = contents / "XPCServices" / (APP_ID + ".worker.xpc") + (service / "Contents/MacOS").mkdir(parents=True) + shutil.copy2(stage / "PicFetch", service / "Contents/MacOS/picfetch-image-worker") + shutil.copy2(stage / "worker", service / "Contents/MacOS/worker") + shutil.copy2(stage / "picfetch-worker-client", contents / "MacOS/picfetch-worker-client") + write_plist(service / "Contents/Info.plist", { + "CFBundleIdentifier": APP_ID + ".worker", "CFBundleExecutable": "worker", + "CFBundlePackageType": "XPC!", "CFBundleVersion": str(details["Build"]), + "XPCService": {"ServiceType": "Application"}}) + run("go", "run", "-tags", TAGS, "./scripts/plistdoctypes", contents / "Info.plist", cwd=repo, env=env) + info = plistlib.loads((contents / "Info.plist").read_bytes()) + info.update({"LSMinimumSystemVersion": "13.4", + "LSMinimumSystemVersionByArchitecture": {a: m for a, m in TARGETS.values()}, + "LSApplicationCategoryType": "public.app-category.photography"}) + write_plist(contents / "Info.plist", info) + (contents / "Frameworks").mkdir() + for arch in TARGETS: + extracted = next((runtime / arch).iterdir()) + library = next((extracted / "lib").glob("*.dylib")) + shutil.copy2(library, contents / "Frameworks" / library.name) + notices = contents / "Resources" / extracted.name + notices.mkdir() + for name in ("LICENSE", "ThirdPartyNotices.txt", "Privacy.md"): + shutil.copyfile(extracted / name, notices / name) + for name in ("LICENSE", "THIRD-PARTY-NOTICES.md", "PRIVACY.md"): + shutil.copyfile(repo / name, contents / "Resources" / name) + for role, entitlements in ENTITLEMENTS.items(): + write_plist(stage / (role + ".plist"), entitlements) + for library in (contents / "Frameworks").iterdir(): + run("codesign", "--force", "--sign", "-", library, env=env) + for path, role in ((contents / "MacOS/picfetch-worker-client", "helper"), + (service / "Contents/MacOS/picfetch-image-worker", "helper"), + (service, "service"), (app, "app")): + run("codesign", "--force", "--sign", "-", "--entitlements", stage / (role + ".plist"), path, env=env) + verify(app, env) + manifest = {"purpose": "ad-hoc local qualification; not for submission", + "version": details["Version"], "build": details["Build"], + "source": run("git", "rev-parse", "HEAD", cwd=repo, capture=True).decode().strip(), + "archives": {arch: digest(path) for arch, path in archives.items()}, + "source_diff_sha256": hashlib.sha256(run("git", "diff", "--binary", "HEAD", cwd=repo, capture=True)).hexdigest(), + "dirty": bool(run("git", "status", "--porcelain", cwd=repo, capture=True)), + "untracked_sources": {name: digest(repo / name) for name in run("git", "ls-files", "--others", "--exclude-standard", cwd=repo, capture=True).decode().splitlines() if (repo / name).is_file()}, + "files": {str(p.relative_to(app)): digest(p) + for p in sorted(app.rglob("*")) if p.is_file()}} + # Reserve only after validation; never replace an existing output directory. + out.mkdir() + shutil.move(app, out / app.name) + (out / "manifest.json").write_text(json.dumps(manifest, indent=2) + "\n") + print(f"Qualified local bundle: {out / 'PicFetch.app'}", flush=True) + + +if __name__ == "__main__": + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--arm64-archive", default=os.environ.get("APPLE_STORE_ARM64_ARCHIVE", "")) + parser.add_argument("--amd64-archive", default=os.environ.get("APPLE_STORE_AMD64_ARCHIVE", "")) + parser.add_argument("--out", default=os.environ.get("APPLE_STORE_OUTPUT_DIR", "")) + parser.add_argument("--verify", help="validate an existing local app without modifying it") + options = parser.parse_args() + if not options.verify and not all((options.arm64_archive, options.amd64_archive, options.out)): + parser.error("both runtime archives and a fresh --out are required") + build(options) diff --git a/scripts/macstorepackage/qualify.py b/scripts/macstorepackage/qualify.py new file mode 100644 index 00000000..b5f6c8c9 --- /dev/null +++ b/scripts/macstorepackage/qualify.py @@ -0,0 +1,95 @@ +"""Qualify packaged production workers in disposable per-architecture apps. + +The original app is never modified. Only the main GUI executable is replaced by +a Go driver that calls the production clients and unchanged packaged broker. Models and a generated +image live in the derived app resources. The driver copies the image into its +private container and tests transfer to the worker along with a private cache. Intel execution on Apple Silicon uses Rosetta. +""" +import argparse +import json +import os +from pathlib import Path +import shutil +import struct +import subprocess +import tempfile +import zlib + +from package import APP_ID, ENTITLEMENTS, TARGETS, build_environment, run, write_plist + + +def png(): + def chunk(kind, data): + return struct.pack(">I", len(data)) + kind + data + struct.pack(">I", zlib.crc32(kind + data)) + return (b"\x89PNG\r\n\x1a\n" + chunk(b"IHDR", struct.pack(">IIBBBBB", 32, 32, 8, 2, 0, 0, 0)) + + chunk(b"IDAT", zlib.compress((b"\x00" + b"\x30\x80\xc0" * 32) * 32)) + chunk(b"IEND", b"")) + + +def qualify(app, models, deny_source=False): + repo = Path(__file__).resolve().parents[2] + env = os.environ | {"DEVELOPER_DIR": os.environ.get("APPLE_STORE_DEVELOPER_DIR", "/Applications/Xcode.app/Contents/Developer")} + results = [] + for native, minimum in TARGETS.values(): + with tempfile.TemporaryDirectory(prefix="picfetch-packaged-worker-", dir="/private/tmp") as temporary: + stage = Path(temporary) + derived = stage / "PicFetch.app" + shutil.copytree(app, derived) + contents = derived / "Contents" + service = contents / "XPCServices" / (APP_ID + ".worker.xpc") + executable = contents / "MacOS/PicFetch" + for path in (contents / "MacOS/picfetch-worker-client", service / "Contents/MacOS/worker", + service / "Contents/MacOS/picfetch-image-worker"): + thin = stage / path.name + run("xcrun", "lipo", path, "-thin", native, "-output", thin, env=env) + shutil.copy2(thin, path) + executable.unlink() + goarch = "arm64" if native == "arm64" else "amd64" + build_flags = [] + if deny_source: + source = repo / "internal/similarity/worker_access.go" + original = source.read_text() + marker = "for _, path := range paths {" + if original.count(marker) != 1: + raise ValueError("source-grant qualification overlay no longer matches") + patched = stage / "worker_access.go" + patched.write_text(original.replace(marker, "for _, path := range paths[:0] {")) + overlay = stage / "source-denial.json" + overlay.write_text(json.dumps({"Replace": {str(source): str(patched)}})) + build_flags = ["-overlay", str(overlay)] + run_flags = ["--expect-source-denial"] if deny_source else [] + run("go", "build", *build_flags, "-tags", "no_emoji,nodynamic,appleappstore", "-o", executable, + "./scripts/macstorequalify", cwd=repo, + env=build_environment(env, goarch, minimum)) + resources = contents / "Resources" + (resources / "models").mkdir() + for name in ("vision_model.onnx", "preprocessor_config.json"): + shutil.copyfile(models / name, resources / "models" / name) + (resources / "fixture.png").write_bytes(png()) + for role, value in ENTITLEMENTS.items(): + write_plist(stage / (role + ".plist"), value) + for path, role in ((contents / "MacOS/picfetch-worker-client", "helper"), + (service / "Contents/MacOS/picfetch-image-worker", "helper"), + (service, "service"), (derived, "app")): + run("codesign", "--force", "--sign", "-", "--entitlements", stage / (role + ".plist"), path, env=env) + run("codesign", "--verify", "--deep", "--strict", derived, env=env) + result = subprocess.run([str(executable), *run_flags], capture_output=True, timeout=180) + if result.returncode: + raise ValueError(f"worker qualification failed ({result.returncode}): {result.stdout!r} {result.stderr!r}") + evidence = json.loads(result.stdout) + if deny_source: + print(f"PASS {native}: production worker refused the private source without its grant", flush=True) + else: + print(f"PASS {native}: real HEIC pixels, ONNX inference, private source/cache grants, cache reuse, retained search, TCP/UDP denial and worker exit", flush=True) + results.append({"architecture": native, **evidence}) + return results + + +if __name__ == "__main__": + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--app", required=True) + parser.add_argument("--models", required=True) + parser.add_argument("--result", required=True) + parser.add_argument("--deny-source", action="store_true", help="qualify refusal with source export disabled in the test parent") + args = parser.parse_args() + evidence = qualify(Path(args.app).resolve(), Path(args.models).resolve(), args.deny_source) + Path(args.result).write_text(json.dumps(evidence, indent=2) + "\n") diff --git a/scripts/macstorepackage/test_package.py b/scripts/macstorepackage/test_package.py new file mode 100644 index 00000000..3860bd02 --- /dev/null +++ b/scripts/macstorepackage/test_package.py @@ -0,0 +1,92 @@ +"""Policy guards complement the actual signed bundle qualification.""" +import importlib.util +import os +import shutil +import subprocess +import tempfile +from pathlib import Path +import unittest + +spec = importlib.util.spec_from_file_location("package", Path(__file__).with_name("package.py")) +package = importlib.util.module_from_spec(spec) +spec.loader.exec_module(package) + + +class PackagingPolicy(unittest.TestCase): + def test_dependency_paths(self): + package.check_dependencies("/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation\n/usr/lib/libSystem.B.dylib") + for path in ("/opt/homebrew/lib/libfoo.dylib", "@rpath/libfoo.dylib", "libfoo.dylib"): + with self.subTest(path=path), self.assertRaises(ValueError): + package.check_dependencies(path) + + def test_entitlements_are_exact(self): + for role, expected in package.ENTITLEMENTS.items(): + package.check_entitlements(role, expected.copy()) + for key in expected: + altered = expected.copy() + del altered[key] + with self.assertRaises(ValueError): + package.check_entitlements(role, altered) + altered = expected | {"com.apple.security.network.server": True} + with self.assertRaises(ValueError): + package.check_entitlements(role, altered) + with self.assertRaises(ValueError): + package.check_entitlements("helper", package.ENTITLEMENTS["app"]) + + def test_architecture_and_minimum(self): + package.check_macho("arm64 x86_64", {"arm64", "x86_64"}, "minos 14.0\n tool LD\n version 27037.1\n", "14.0") + for architectures, minimum in (("arm64", "14.0"), ("arm64 x86_64", "15.0")): + with self.assertRaises(ValueError): + package.check_macho(architectures, {"arm64", "x86_64"}, "minos " + minimum, "14.0") + with self.assertRaises(ValueError): + package.check_macho("arm64 x86_64", {"arm64", "x86_64"}, "", "14.0") + + +@unittest.skipUnless(os.environ.get("PICFETCH_TEST_MAC_APP_BUNDLE"), "requires a built signed app") +class SignedArtifactGuards(unittest.TestCase): + def setUp(self): + self.directory = tempfile.TemporaryDirectory(prefix="picfetch-package-guard-") + self.addCleanup(self.directory.cleanup) + self.app = Path(self.directory.name) / "PicFetch.app" + shutil.copytree(os.environ["PICFETCH_TEST_MAC_APP_BUNDLE"], self.app) + self.env = os.environ.copy() + + def sign(self, path, role): + entitlement = Path(self.directory.name) / "entitlements.plist" + package.write_plist(entitlement, package.ENTITLEMENTS[role]) + package.run("codesign", "--force", "--sign", "-", "--entitlements", entitlement, path) + + def test_edited_resource_breaks_seal(self): + with (self.app / "Contents/Resources/LICENSE").open("a") as target: + target.write("tampered") + with self.assertRaises(subprocess.CalledProcessError): + package.verify(self.app, self.env) + + def test_resigned_network_service_is_refused(self): + service = self.app / "Contents/XPCServices" / (package.APP_ID + ".worker.xpc") + self.sign(service, "app") + self.sign(self.app, "app") + package.run("codesign", "--verify", "--deep", "--strict", self.app) + with self.assertRaisesRegex(ValueError, "service entitlements"): + package.verify(self.app, self.env) + + def test_hidden_native_payload_is_refused(self): + hidden = self.app / "Contents/Resources/hidden-data" + shutil.copyfile(self.app / "Contents/MacOS/PicFetch", hidden) + hidden.chmod(0o644) + self.sign(self.app, "app") + with self.assertRaisesRegex(ValueError, "unexpected executable payload"): + package.verify(self.app, self.env) + + def test_resigned_wrong_minimum_is_refused(self): + path = self.app / "Contents/Info.plist" + data = package.plistlib.loads(path.read_bytes()) + data["LSMinimumSystemVersionByArchitecture"]["arm64"] = "10.0" + package.write_plist(path, data) + self.sign(self.app, "app") + with self.assertRaisesRegex(ValueError, "deployment minimum"): + package.verify(self.app, self.env) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/macstorequalify/main.go b/scripts/macstorequalify/main.go new file mode 100644 index 00000000..df570dd4 --- /dev/null +++ b/scripts/macstorequalify/main.go @@ -0,0 +1,110 @@ +// Command macstorequalify is a qualification-only main executable. Packaging +// tests put it in a disposable signed app while preserving production workers. +package main + +import ( + "context" + "encoding/json" + "flag" + "fmt" + "os" + "path/filepath" + "strings" + "time" + + "github.com/frathe/picfetch/internal/heic" + "github.com/frathe/picfetch/internal/macbundle" + "github.com/frathe/picfetch/internal/similarity" +) + +func main() { + denySource := flag.Bool("expect-source-denial", false, "require an ungranted private source to be refused") + flag.Parse() + if err := qualify(*denySource); err != nil { + _, _ = fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } +} + +func qualify(denySource bool) error { + ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute) + defer cancel() + root, err := macbundle.CurrentRuntimeDirectory() + if err != nil { + return err + } + resources := filepath.Join(filepath.Dir(root), "Resources") + executable := filepath.Join(filepath.Dir(root), "MacOS", "PicFetch") + decoder := heic.NewClient(executable) + defer func() { decoder.Stop(); decoder.Wait() }() + if err := decoder.Check(ctx); err != nil { + return err + } + cache, err := os.UserCacheDir() + if err != nil { + return err + } + if err := os.MkdirAll(cache, 0700); err != nil { + return err + } + private, err := os.MkdirTemp(cache, "picfetch-worker-qualification-") + if err != nil { + return err + } + defer func() { _ = os.RemoveAll(private) }() + data, err := os.ReadFile(filepath.Join(resources, "fixture.png")) + if err != nil { + return err + } + path := filepath.Join(private, "cafe\u0301 # %20.png") + if err := os.WriteFile(path, data, 0600); err != nil { + return err + } + client := similarity.Client{Assets: filepath.Join(resources, "models"), GeneralAnalysisDir: filepath.Join(private, "analysis"), GeneralAnalysisLimitBytes: 16 * 1024 * 1024, DisableFavoriteCache: true} + var fresh, cached similarity.Event + paths := []string{path} + if denySource { + // A granted bundle source proves worker/model setup still works. + paths = append(paths, filepath.Join(resources, "fixture.png")) + } + if err := client.Analyze(ctx, paths, nil, func(e similarity.Event) { fresh = e }); err != nil { + return err + } + if denySource { + denied := false + for _, item := range fresh.Items { + if item.Path == path && (strings.Contains(item.Error, "operation not permitted") || strings.Contains(item.Error, "permission denied")) { + denied = true + } + } + if !denied || !fresh.Complete || fresh.Successful != 1 || fresh.Failed != 1 || !fresh.OfflineVerified || fresh.Measurements.InferenceAttempts != 1 { + return fmt.Errorf("ungranted private source: complete=%t successful=%d failed=%d offline=%t inference=%d", fresh.Complete, fresh.Successful, fresh.Failed, fresh.OfflineVerified, fresh.Measurements.InferenceAttempts) + } + return json.NewEncoder(os.Stdout).Encode(map[string]any{"source_denied": true, "event": fresh}) + } + if err := client.Analyze(ctx, []string{path}, nil, func(e similarity.Event) { cached = e }); err != nil { + return err + } + if !fresh.Complete || fresh.Successful != 1 || fresh.Failed != 0 || !fresh.OfflineVerified || fresh.Measurements.InferenceAttempts != 1 || fresh.CacheWarning != "" { + return fmt.Errorf("fresh analysis failed: %+v", fresh) + } + if !cached.Complete || cached.Successful != 1 || cached.Reused != 1 || !cached.OfflineVerified || cached.CacheWarning != "" { + return fmt.Errorf("cache replay failed: %+v", cached) + } + queries := make(chan similarity.SearchQuery, 1) + queries <- similarity.SearchQuery{ID: 1, ReferencePath: path} + var search similarity.SearchEvent + request := similarity.SearchRequest{SessionID: 1, Paths: []string{path}, Limit: 30, Cache: similarity.CachePolicy{Roots: similarity.CacheRoots{GeneralDir: client.GeneralAnalysisDir}, LooseEnabled: true, GeneralLimitBytes: client.GeneralAnalysisLimitBytes}} + if err := client.Search(ctx, request, queries, func(event similarity.SearchEvent) { + if event.Kind == similarity.SearchFinal { + search = event + close(queries) + } + }); err != nil { + return err + } + if search.Kind != similarity.SearchFinal || search.Failed != 0 || !search.OfflineVerified || search.CacheWarning != "" { + return fmt.Errorf("search failed: %+v", search) + } + return json.NewEncoder(os.Stdout).Encode(map[string]any{"heic": true, "fresh": fresh, "cached": cached, "search": search}) +} diff --git a/scripts/macstorestage/main.go b/scripts/macstorestage/main.go new file mode 100644 index 00000000..4d858aa7 --- /dev/null +++ b/scripts/macstorestage/main.go @@ -0,0 +1,22 @@ +// Command macstorestage verifies and extracts a pinned ONNX archive before signing. +package main + +import ( + "context" + "flag" + "fmt" + "os" + + "github.com/frathe/picfetch/internal/similarity" +) + +func main() { + arch := flag.String("arch", "", "Go architecture: arm64 or amd64") + archive := flag.String("archive", "", "pinned upstream ONNX archive") + out := flag.String("out", "", "new staging directory") + flag.Parse() + if err := similarity.StageMacRuntime(context.Background(), *arch, *archive, *out); err != nil { + _, _ = fmt.Fprintf(os.Stderr, "macstorestage: %v\n", err) + os.Exit(1) + } +} diff --git a/todos.md b/todos.md index dda7f88d..878a4525 100644 --- a/todos.md +++ b/todos.md @@ -22,13 +22,20 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Own native save destinations through export/mosaic cancellation, scope image mutations and metadata reads, and stage Apple Store atomic writes in a same-volume replacement directory. - [ ] Qualify persistent security-scoped permissions for opening, saved sessions, Favorites, writes and workers. - [x] Implement the Apple Store XPC worker boundary and qualify native fixture TCP/UDP denial, cancellation and broker crash on Apple Silicon. -- [ ] Qualify real HEIC/ONNX execution and granted source/cache access through that worker boundary on both Mac architectures. +- [x] Qualify real HEIC/ONNX execution and granted private source/cache access through that worker boundary on ARM and Intel under Rosetta. - [x] Validate post-sign Apple runtime code and outer bundle seals, use a shared Frameworks path for the app and worker, and qualify real arm64 CPU inference in a network-disabled sandbox fixture. -- [ ] Bundle pinned native runtimes and notices, and validate signed Mach-O code without comparing its post-sign bytes with upstream archive hashes. +- [x] Bundle both pinned native runtimes and notices in a universal ad-hoc app; validate signed Mach-O code, architecture/minimum OS, dependencies and nested/outer seals. +- [x] Transfer exact source/model/cache and app-signature access to analysis workers; qualify real HEIC/ONNX, private source/cache access and reuse on ARM and Intel under Rosetta. +- [x] Qualify retained visual search through production ARM/Intel-Rosetta workers; keep scopes through clipboard/Finder/Trash calls and source metadata reads. +- [x] Capture native window-drop URLs before GLFW flattens paths; refresh restored moved/stale bookmarks with immutable persistence coverage. +- [x] Verify production workers deny an ungranted private source while an allowed control image still completes inference, on ARM and Intel under Rosetta. +- [ ] Finish GUI sandbox workflows, moved-folder relaunch, and physical Intel/minimum-OS validation. - [ ] Add the signed Store packaging route and validate the final artifact on Intel and Apple Silicon. -- [ ] Complete privacy/dependency/export assessment, listing metadata and Apple submission validation. +- [x] Prepare [listing/review notes and human-input checklist](packaging/apple-app-store/submission-draft.md); inventory privacy/network/export facts. +- [ ] Resolve [ONNX listed-SDK privacy manifests/signature provenance](packaging/apple-app-store/privacy-dependency-audit.md), confirm final privacy/export answers and validate Apple submission. - [x] Enable the full Xcode compiler after local license acceptance. -- [ ] Obtain developer Team ID, Store application/installer certificates and provisioning profile for final signed qualification. +- [ ] Human inputs last: developer Team ID, Store application/installer certificates, TestFlight provisioning profile, App Store Connect record and final submission decisions. +- [x] Ronin reauthorized signed commits and pushing the preparation branch after the overnight work. ## Deferred From cd063e9cc8645c904ee822e2f9d275660f906e10 Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 09:26:44 +0200 Subject: [PATCH 09/63] Verify Apple runtime provenance and retain Abseil privacy manifests Check original Microsoft library signatures before re-signing, preserve exact upstream privacy/license inputs, and reject modified resources even after signing. Run portable packaging guards in CI and document the remaining Protobuf and submission requirements. --- .github/workflows/ci.yml | 5 + ARCHITECTURE.md | 5 +- THIRD-PARTY-NOTICES.md | 10 + packaging/apple-app-store/README.md | 12 +- .../privacy-dependency-audit.md | 64 ++++-- .../apple-app-store/privacy/abseil/LICENSE | 203 ++++++++++++++++++ .../privacy/abseil/PrivacyInfo.xcprivacy | 14 ++ .../privacy/abseil/provenance.json | 20 ++ packaging/apple-app-store/submission-draft.md | 10 +- plans/2026-09-29-apple-app-store.md | 75 +++++++ scripts/macstorepackage/package.py | 53 +++++ scripts/macstorepackage/test_package.py | 41 ++++ todos.md | 3 +- 13 files changed, 486 insertions(+), 29 deletions(-) create mode 100644 packaging/apple-app-store/privacy/abseil/LICENSE create mode 100644 packaging/apple-app-store/privacy/abseil/PrivacyInfo.xcprivacy create mode 100644 packaging/apple-app-store/privacy/abseil/provenance.json diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 59c46287..63fcfbae 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -40,6 +40,11 @@ jobs: - name: Check formatting run: make fmt-check + - name: Check Apple Store packaging policy + # Portable policy/resource tests run here; signed native artifact guards + # require a locally built macOS bundle and are qualified separately. + run: python3 -m unittest discover -s scripts/macstorepackage -p test_package.py + - name: Check search review controls run: | node --version diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 8a1c81e0..4750a65f 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -305,7 +305,10 @@ records the experimental channel and outstanding signed qualification. `scripts/macstorepackage` builds and validates universal ad-hoc bundles through `make apple-store-package-local`; its disposable thin derivatives run production workers through the Go driver in `scripts/macstorequalify`, without changing the -original app or accessing an Apple account. +original app or accessing an Apple account. The package builder also verifies and +stages exact upstream Abseil privacy/license/provenance inputs from +`packaging/apple-app-store/privacy/abseil` into a macOS resource bundle; final +artifact validation refuses missing or changed declarations even after re-signing. `packaging/tools.mk` owns reviewed CLI versions and multiarchitecture image digests consumed by Makefile and the release/Store workflows. diff --git a/THIRD-PARTY-NOTICES.md b/THIRD-PARTY-NOTICES.md index a3021a74..a2a9fa94 100644 --- a/THIRD-PARTY-NOTICES.md +++ b/THIRD-PARTY-NOTICES.md @@ -5742,6 +5742,16 @@ retains the complete upstream `LICENSE`, `ThirdPartyNotices.txt`, and library in the eventual Store bundle; PicFetch's source license does not replace the runtime's component licenses. Signed Store packaging remains under qualification. +The local Mac App Store packaging route also retains the unmodified Abseil +privacy manifest supplied by both Abseil 20250512.0 (Intel runtime) and 20250814.0 +(Apple Silicon runtime). These identical declarations are distributed once in +`Contents/Resources/AbseilPrivacy.bundle`, alongside their Apache-2.0 license +and exact source/version/hash provenance. Sources: +https://github.com/abseil/abseil-cpp/blob/20250512.0/PrivacyInfo.xcprivacy +https://github.com/abseil/abseil-cpp/blob/20250814.0/PrivacyInfo.xcprivacy +The retained manifest does not cover PicFetch, Protobuf or other ONNX components; +remaining SDK manifest and signature requirements are still under qualification. + Intel macOS setup retains the complete upstream `LICENSE`, `ThirdPartyNotices.txt` and `Privacy.md` beside `onnxruntime-osx-x86_64-1.23.2/lib/libonnxruntime.1.23.2.dylib`. diff --git a/packaging/apple-app-store/README.md b/packaging/apple-app-store/README.md index d28f5498..dffda647 100644 --- a/packaging/apple-app-store/README.md +++ b/packaging/apple-app-store/README.md @@ -140,8 +140,9 @@ make apple-store-package-local ``` The builder compiles arm64 and x86_64 app/broker/service binaries, stages both -pinned native libraries and complete upstream notices, signs nested code before -the app, and validates exact entitlements, native dependency paths, architecture, +pinned native libraries and complete upstream notices, verifies the original libraries against Microsoft's Apple-anchored signing +identity, retains Abseil's exact privacy declaration/license/provenance resource, +signs nested code before the app, and validates exact entitlements, native dependency paths, architecture, minimum OS and code/resource seals. ARM requires macOS 14.0 and Intel 13.4. Deployment flags are explicit cgo cache inputs. Fyne metadata writes happen in a temporary directory; the repository build number is unchanged. The output includes @@ -179,9 +180,10 @@ window drops or every OS file handoff. [Submission draft](submission-draft.md) contains listing copy, review steps, screenshot plans and the human-input checklist. [Dependency/privacy audit](privacy-dependency-audit.md) -records a remaining release blocker: both pinned ONNX binaries include Apple-listed -SDKs, but their archives contain no Apple privacy manifests. Local sandbox and -signature checks do not clear that submission requirement. +records the remaining Protobuf/ONNX privacy coverage and submission validation +gap. Exact Abseil manifests are retained separately and verified during packaging, +and original Microsoft runtime identities are checked before local re-signing. +These input checks do not clear all Apple submission requirements. The worker qualifier also supports `--deny-source`. It compiles only its test parent with a Go overlay omitting source grants, keeps the packaged production diff --git a/packaging/apple-app-store/privacy-dependency-audit.md b/packaging/apple-app-store/privacy-dependency-audit.md index 5fbe1e2c..5d9c3c03 100644 --- a/packaging/apple-app-store/privacy-dependency-audit.md +++ b/packaging/apple-app-store/privacy-dependency-audit.md @@ -25,23 +25,45 @@ Apple's [listed-SDK requirements](https://developer.apple.com/support/third-part include Abseil and Protobuf and cover SDKs repackaging them. Binary dependencies also have SDK-signature requirements. **The current local bundle is not cleared for this gate.** Neither pinned runtime archive contains an `.xcprivacy` file; -its `Privacy.md` is explanatory text, not an Apple privacy manifest. An ad-hoc -app signature is not evidence of upstream SDK provenance or distribution trust. - -The [Abseil 20250814.0 tree](https://github.com/abseil/abseil-cpp/tree/20250814.0) -lists a PrivacyInfo.xcprivacy file. Retrieving exact manifest files and recursive -source trees failed in this environment (GitHub API timeouts; research-browser -cache/fetch failures). A newer default-branch manifest is not silently substituted -for an exact pinned dependency. The Protobuf CocoaPods manifest packaging found -in current upstream is specifically for its Objective-C runtime; it does not by -itself qualify the older C++ runtime in these ONNX binaries. - -Next technical work: obtain and hash manifests for the exact shipped components, -verify their applicable declarations and intended bundle placement, and resolve -upstream binary-signature provenance or use a reproducible, reviewed source build. -Then validate the final package through Apple's tools. Do not invent empty -manifests or infer compliance from an archive's license notice. This is an -external-artifact/research blocker, not a request to remove Similarity Explorer. +its `Privacy.md` is explanatory text, not an Apple privacy manifest. + +Both exact Abseil manifests are now retained in +[privacy/abseil](privacy/abseil/provenance.json). The authenticated GitHub read API +succeeded where the earlier research-browser/raw requests failed. Versions +20250512.0 and 20250814.0 contain identical bytes, git blob +`3ff4a9d98b13eafdc813fb5394796afd6cb8486b`, SHA-256 +`f232217ae9edf2ab6a541a28eff50cfe05303c2b4756fff95cca72eccbc3b898`. +Their upstream Apache-2.0 LICENSE files are also identical. The builder preserves +the declaration, license and provenance once in +`Contents/Resources/AbseilPrivacy.bundle/Contents/Resources`; the surrounding +resource bundle has macOS BNDL metadata. It checks pinned hashes before staging +and during artifact validation, including validly re-signed app tampering. This +retains Abseil's declaration; it does not declare PicFetch or ONNX as collecting +no data and does not establish Apple's acceptance of the completed SDK assembly. + +The complete, non-truncated Protobuf v21.12 source tree contains no privacy +manifest. Current upstream CocoaPods manifest packaging is specifically for its +Objective-C runtime; it does not by itself qualify the older C++ runtime in +these ONNX binaries. No replacement declaration has been invented or copied +from another version. Protobuf/ONNX manifest coverage and final Apple validation +remain open. + +Original dylibs from both pinned archives have Developer ID Application +signatures from Microsoft Corporation, Team `UBF8T346G9`, identifier +`libonnxruntime.1`. Before replacing them with local test signatures, packaging +now verifies strict/all-architecture code integrity with an Apple-anchored +requirement for that exact Team and identifier. The output manifest records the +verified identity, requirement and original library SHA-256 per architecture. +An otherwise valid ad-hoc signature fails this check. This closes the previously +unverified input-identity step; it does not establish final Store distribution +trust or substitute for Apple's SDK/submission validation. + +Next technical work: obtain an applicable declaration for the pinned Protobuf +C++/ONNX assembly, or qualify a reviewed source build/new runtime with complete +privacy artifacts. Then validate final manifest placement and SDK requirements +with Apple's tooling. Source builds/upgrades require renewed license review, +architecture/minimum-OS, model, performance and sandbox qualification. Keep +Similarity Explorer available while resolving this release gate. The native libraries live in Contents/Frameworks. Complete runtime LICENSE, ThirdPartyNotices.txt and Privacy.md files live under versioned directories in @@ -79,7 +101,13 @@ select answers or claim that every optional request is exempt from disclosure. The app privacy policy now explains Apple bundled runtime delivery, temporary worker access and conditional release-note image requests. The public policy -URL must be rechecked after these uncommitted changes are published. +URL must be rechecked after the preparation branch reaches the public main branch. + +The pinned ONNX 1.29.0 privacy documentation warns that an API-only opt-out may +follow initialization events. `internal/similarity/encoder.go` already sets +`ORT_DISABLE_TELEMETRY=1` before loading the runtime and also disables telemetry +through the API before session creation. The worker network sandbox is a separate +verified protection; no opt-out code change was necessary. ## Verification boundaries diff --git a/packaging/apple-app-store/privacy/abseil/LICENSE b/packaging/apple-app-store/privacy/abseil/LICENSE new file mode 100644 index 00000000..ccd61dcf --- /dev/null +++ b/packaging/apple-app-store/privacy/abseil/LICENSE @@ -0,0 +1,203 @@ + + Apache License + Version 2.0, January 2004 + https://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + https://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + diff --git a/packaging/apple-app-store/privacy/abseil/PrivacyInfo.xcprivacy b/packaging/apple-app-store/privacy/abseil/PrivacyInfo.xcprivacy new file mode 100644 index 00000000..3ff4a9d9 --- /dev/null +++ b/packaging/apple-app-store/privacy/abseil/PrivacyInfo.xcprivacy @@ -0,0 +1,14 @@ + + + + + NSPrivacyTracking + + NSPrivacyCollectedDataTypes + + NSPrivacyTrackingDomains + + NSPrivacyAccessedAPITypes + + + diff --git a/packaging/apple-app-store/privacy/abseil/provenance.json b/packaging/apple-app-store/privacy/abseil/provenance.json new file mode 100644 index 00000000..f64d0644 --- /dev/null +++ b/packaging/apple-app-store/privacy/abseil/provenance.json @@ -0,0 +1,20 @@ +{ + "component": "Abseil", + "license": "Apache-2.0", + "sources": [ + { + "version": "20250512.0", + "url": "https://raw.githubusercontent.com/abseil/abseil-cpp/20250512.0/PrivacyInfo.xcprivacy", + "git_blob": "3ff4a9d98b13eafdc813fb5394796afd6cb8486b", + "sha256": "f232217ae9edf2ab6a541a28eff50cfe05303c2b4756fff95cca72eccbc3b898" + }, + { + "version": "20250814.0", + "url": "https://raw.githubusercontent.com/abseil/abseil-cpp/20250814.0/PrivacyInfo.xcprivacy", + "git_blob": "3ff4a9d98b13eafdc813fb5394796afd6cb8486b", + "sha256": "f232217ae9edf2ab6a541a28eff50cfe05303c2b4756fff95cca72eccbc3b898" + } + ], + "license_source": "https://raw.githubusercontent.com/abseil/abseil-cpp/20250814.0/LICENSE", + "license_sha256": "c79a7fea0e3cac04cd43f20e7b648e5a0ff8fa5344e644b0ee09ca1162b62747" +} diff --git a/packaging/apple-app-store/submission-draft.md b/packaging/apple-app-store/submission-draft.md index 0ac7cb7c..a3855564 100644 --- a/packaging/apple-app-store/submission-draft.md +++ b/packaging/apple-app-store/submission-draft.md @@ -102,14 +102,16 @@ requires the developer to determine the applicable documentation; see ## Technical gates before account work -- Resolve the listed-SDK privacy manifest/signature gap in the bundled ONNX - closure; see the audit. Keep source/version and notice delivery evidence. +- Resolve the remaining Protobuf/ONNX privacy manifest coverage and final SDK + validation. Abseil declarations and pre-sign Microsoft identity checks are now + implemented; see the audit for source/version and notice delivery evidence. - Qualify the final signed GUI: picker, Finder/Dock/window drops, restart, Favorites, moved and unavailable sources, Save/export, overwrite, Trash, clipboard, Finder reveal and wallpaper. Scope renewal has unit/race coverage; moved-folder relaunch and final UI behavior remain unverified. -- Finish negative grant/cancellation tests against the production worker and - qualify real Intel hardware and both declared minimum OS versions. +- Negative source-grant tests pass on ARM and Intel under Rosetta. Finish full + application cancellation/reopen testing and qualify real Intel hardware and + both declared minimum OS versions. - Run the full native Linux/amd64 CI suite, CodeQL and fresh post-suppression Qodana analysis. ARM Docker emulation cannot replace the isolation gate. - Complete the Store certificate signing and productbuild route; validate the diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index d336a88a..09eb1695 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -1047,3 +1047,78 @@ no additional paths. Existing runtime test/inspection evidence therefore carries forward unchanged. Only this authorization record, the TODO entry and submission checklist were updated. Use the configured SSH commit signing; do not disable signing. Push the existing feature branch, without merging or releasing. + +### Continuation — exact SDK privacy inputs and retained Abseil resource + +T5/T4, T0 inline, zero spawns/two reviews. GitHub's authenticated read API now +retrieves both exact Abseil manifests (20250512.0 and 20250814.0): same git blob +3ff4a9d98b13eafdc813fb5394796afd6cb8486b. Protobuf v21.12's complete non-truncated +source tree has no privacy manifest. Preserve exact upstream bytes, license and +source/version/hash provenance; do not substitute a current Protobuf manifest. +Package the identical Abseil declaration once in a macOS resource bundle, retaining +flat ONNX libraries and existing runtime admission. Validate its exact content +before signing and when inspecting a final app; signed-but-altered/missing resource +must fail. This supplies the known declaration, not an App Store acceptance claim: +Protobuf/ONNX declarations and upstream binary provenance still need resolution. +Files: packaging/apple-app-store/privacy/abseil inputs; package.py/test_package.py; +notices/README/audit/todos/evidence. Test existing packaging boundary first with +missing/changed manifest, then real signed artifact tampering. Verify Python suite, +new universal package, strict seals, worker regression and GoLand. The user's +f0565d1 E2E bundle remains untouched. + +Pinned 1.29.0 Privacy.md was checked for its initialization-event warning. +Encoder already sets ORT_DISABLE_TELEMETRY=1 before dlopen and also invokes the +API opt-out before creating a session; no telemetry code change is needed. + +#### Upstream runtime signature finding + +Original versioned runtime dylibs extracted from the pinned archives both carry +Developer ID Application signatures from Microsoft Corporation, Team UBF8T346G9, +identifier libonnxruntime.1 (ARM timestamp 2026-08-12; Intel 2025-10-22). Debug and +test dylibs in the archive are not the shipped payload and are excluded. Add a +pre-sign guard after existing archive verification: native codesign strict/all- +architecture validation with Apple anchor, exact Team and identifier. Only then +may the local packager replace that signature. A valid ad-hoc signature must fail +this upstream provenance check. Keep post-sign app/nested seal validation separate. +Existing packaging tests are the agreed seam; add one native negative guard and +verify original libraries as positive fixtures. Preserve both signature records +in the build manifest. This establishes input provenance, not final Store trust. + +#### Privacy/provenance implementation evidence + +Missing-resource and staged-byte tests failed before implementation. The native +signed-artifact guard also failed when an altered Abseil declaration was re-signed +but final validation omitted the privacy check; it now rejects that artifact. +The upstream identity guard failed before implementation on a valid ad-hoc dylib; +strict Apple-anchor/Team/identifier checks now reject it and accept both original +Microsoft-signed libraries. codesign inline requirements require the leading '='; +positive fixtures exposed and corrected an initial command syntax error. + +Eleven packaging tests pass (1.884s). A fresh universal package with the retained +resources and original signature records built successfully at +`.scratch/apple-store-package/privacy-provenance/PicFetch.app`. The preceding +resource-identical app passed production HEIC/ONNX/cache/retained-search/network +qualification on native ARM and Intel under Rosetta. The later change only adds +pre-sign identity verification/manifest records; it changes no worker code or +resource content. GoLand inspected both Python code files including weak warnings; +no findings. Reinspection follows the small provenance addition before commit. +The older f0565d1 E2E app/ZIP given to Ronin is untouched. + +The final Make notice check caught normalized CRLF inside unrelated retained +updater license blocks. Restored those original bytes and limited the notice +change to the new ten-line Abseil paragraph. `make verify-build` then passed, +including the reviewed updater-source notice test. No license text was removed. +Both changed Python files were reinspected after the provenance guard; no findings. + +Add the existing portable packaging policy/resource suite to CI's Ubuntu +validation job (five runnable tests; six macOS signed-artifact tests explicitly +skip without an artifact). Native eleven-test evidence remains a separate local +gate. This small workflow addition is within T4/T5, zero spawns/two reviews. +A draft PR will activate full native Linux/amd64 tests, CodeQL and Qodana; it does +not mark the unfinished Store release ready or authorize a merge/submission. + +Final artifact at `.scratch/apple-store-package/privacy-final/PicFetch.app` +includes preserved upstream notice bytes and both provenance records. All eleven +native packaging tests pass (1.753s). Portable invocation passes five tests with +six explicit native skips. The CI workflow inspection is clean. All current +local gates passed before staging; full native-amd64 CI is pending the draft PR. diff --git a/scripts/macstorepackage/package.py b/scripts/macstorepackage/package.py index ade1c005..cdfdbc79 100644 --- a/scripts/macstorepackage/package.py +++ b/scripts/macstorepackage/package.py @@ -30,6 +30,14 @@ APP_ID = "io.github.frathe.picfetch" TARGETS = {"arm64": ("arm64", "14.0"), "amd64": ("x86_64", "13.4")} TAGS = "no_emoji,nodynamic,appleappstore" +# Reviewed exact upstream declarations; changes require a source/license audit. +PRIVACY_FILES = { + 'PrivacyInfo.xcprivacy': 'f232217ae9edf2ab6a541a28eff50cfe05303c2b4756fff95cca72eccbc3b898', + 'LICENSE': 'c79a7fea0e3cac04cd43f20e7b648e5a0ff8fa5344e644b0ee09ca1162b62747', + 'provenance.json': 'dc61cbd210161cb9cea8b0dda99d446147ab9def03e739ffe224d63588dbb576'} +PRIVACY_INFO = {"CFBundleIdentifier": APP_ID + ".abseil-privacy", + "CFBundleName": "AbseilPrivacy", "CFBundleVersion": "1", + "CFBundlePackageType": "BNDL"} def check_dependencies(paths): @@ -70,6 +78,44 @@ def write_plist(path, data): path.write_bytes(plistlib.dumps(data, sort_keys=True)) +def check_privacy_files(resources): + for name, expected in PRIVACY_FILES.items(): + path = resources / name + if not path.is_file() or path.is_symlink() or digest(path) != expected: + raise ValueError(f"missing or changed Abseil privacy input: {name}") + + +def verify_privacy(contents): + bundle = contents / "Resources/AbseilPrivacy.bundle/Contents" + check_privacy_files(bundle / "Resources") + try: + actual = plistlib.loads((bundle / "Info.plist").read_bytes()) + except (OSError, ValueError) as error: + raise ValueError("invalid Abseil privacy bundle metadata") from error + if actual != PRIVACY_INFO: + raise ValueError("unexpected Abseil privacy bundle metadata") + + +def stage_privacy(repo, contents): + source = repo / "packaging/apple-app-store/privacy/abseil" + check_privacy_files(source) + bundle = contents / "Resources/AbseilPrivacy.bundle/Contents" + resources = bundle / "Resources" + resources.mkdir(parents=True, exist_ok=True) + for name in PRIVACY_FILES: + shutil.copyfile(source / name, resources / name) + write_plist(bundle / "Info.plist", PRIVACY_INFO) + verify_privacy(contents) + + +def verify_upstream_library(path, env): + team, identifier = "UBF8T346G9", "libonnxruntime.1" + requirement = f'=anchor apple generic and certificate leaf[subject.OU] = "{team}" and identifier "{identifier}"' + run("codesign", "--verify", "--strict", "--all-architectures", "-R", requirement, path, env=env) + return {"team": team, "identifier": identifier, "requirement": requirement, + "sha256_before_resigning": digest(path)} + + def native_check(path, architectures, minima, env): actual = run("xcrun", "lipo", "-archs", path, env=env, capture=True).decode().strip() for arch in architectures: @@ -100,6 +146,7 @@ def verify(app, env): raise ValueError("unexpected app identity") if plist.get("LSMinimumSystemVersionByArchitecture") != {a: m for a, m in TARGETS.values()}: raise ValueError("unexpected per-architecture deployment minimum") + verify_privacy(contents) expected_code = {contents / "MacOS/PicFetch", contents / "MacOS/picfetch-worker-client", service / "Contents/MacOS/worker", service / "Contents/MacOS/picfetch-image-worker"} for path in expected_code: @@ -161,9 +208,13 @@ def build(args): stage = Path(temporary) runtime = stage / "runtime" # Verify all downloaded code before spending time compiling the application. + upstream_signatures = {} for arch, archive in archives.items(): run("go", "run", "-tags", TAGS, "./scripts/macstorestage", "-arch", arch, "-archive", archive, "-out", runtime / arch, cwd=repo, env=env) + extracted = next((runtime / arch).iterdir()) + library = next((extracted / "lib").glob("*.dylib")) + upstream_signatures[arch] = verify_upstream_library(library, env) for arch, (native, minimum) in TARGETS.items(): archdir = stage / arch archdir.mkdir() @@ -210,6 +261,7 @@ def build(args): shutil.copyfile(extracted / name, notices / name) for name in ("LICENSE", "THIRD-PARTY-NOTICES.md", "PRIVACY.md"): shutil.copyfile(repo / name, contents / "Resources" / name) + stage_privacy(repo, contents) for role, entitlements in ENTITLEMENTS.items(): write_plist(stage / (role + ".plist"), entitlements) for library in (contents / "Frameworks").iterdir(): @@ -223,6 +275,7 @@ def build(args): "version": details["Version"], "build": details["Build"], "source": run("git", "rev-parse", "HEAD", cwd=repo, capture=True).decode().strip(), "archives": {arch: digest(path) for arch, path in archives.items()}, + "upstream_signatures": upstream_signatures, "source_diff_sha256": hashlib.sha256(run("git", "diff", "--binary", "HEAD", cwd=repo, capture=True)).hexdigest(), "dirty": bool(run("git", "status", "--porcelain", cwd=repo, capture=True)), "untracked_sources": {name: digest(repo / name) for name in run("git", "ls-files", "--others", "--exclude-standard", cwd=repo, capture=True).decode().splitlines() if (repo / name).is_file()}, diff --git a/scripts/macstorepackage/test_package.py b/scripts/macstorepackage/test_package.py index 3860bd02..049e8d2d 100644 --- a/scripts/macstorepackage/test_package.py +++ b/scripts/macstorepackage/test_package.py @@ -13,6 +13,30 @@ class PackagingPolicy(unittest.TestCase): + def test_missing_privacy_resource_is_refused(self): + with tempfile.TemporaryDirectory() as folder: + with self.assertRaisesRegex(ValueError, "Abseil privacy"): + package.verify_privacy(Path(folder)) + + def test_privacy_resource_preserves_upstream_bytes_and_refuses_changes(self): + repo = Path(__file__).resolve().parents[2] + upstream = repo / "packaging/apple-app-store/privacy/abseil" + with tempfile.TemporaryDirectory() as folder: + contents = Path(folder) + package.stage_privacy(repo, contents) + package.verify_privacy(contents) + resources = contents / "Resources/AbseilPrivacy.bundle/Contents/Resources" + for name in ("PrivacyInfo.xcprivacy", "LICENSE", "provenance.json"): + self.assertEqual((resources / name).read_bytes(), (upstream / name).read_bytes()) + original = (resources / name).read_bytes() + (resources / name).write_bytes(original + b"changed") + with self.assertRaisesRegex(ValueError, "Abseil privacy"): + package.verify_privacy(contents) + (resources / name).write_bytes(original) + declaration = package.plistlib.loads((resources / "PrivacyInfo.xcprivacy").read_bytes()) + self.assertIs(declaration["NSPrivacyTracking"], False) + self.assertEqual(declaration["NSPrivacyCollectedDataTypes"], []) + def test_dependency_paths(self): package.check_dependencies("/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation\n/usr/lib/libSystem.B.dylib") for path in ("/opt/homebrew/lib/libfoo.dylib", "@rpath/libfoo.dylib", "libfoo.dylib"): @@ -56,6 +80,23 @@ def sign(self, path, role): package.write_plist(entitlement, package.ENTITLEMENTS[role]) package.run("codesign", "--force", "--sign", "-", "--entitlements", entitlement, path) + def test_ad_hoc_runtime_cannot_claim_upstream_identity(self): + runtime = self.app / "Contents/Frameworks/libonnxruntime.1.29.0.dylib" + package.run("codesign", "--verify", "--strict", runtime) + with self.assertRaises(subprocess.CalledProcessError): + package.verify_upstream_library(runtime, self.env) + + def test_resigned_privacy_change_is_refused(self): + package.stage_privacy(Path(__file__).resolve().parents[2], self.app / "Contents") + resource = self.app / "Contents/Resources/AbseilPrivacy.bundle/Contents/Resources/PrivacyInfo.xcprivacy" + declaration = package.plistlib.loads(resource.read_bytes()) + declaration["NSPrivacyTracking"] = True + package.write_plist(resource, declaration) + self.sign(self.app, "app") + package.run("codesign", "--verify", "--deep", "--strict", self.app) + with self.assertRaisesRegex(ValueError, "Abseil privacy"): + package.verify(self.app, self.env) + def test_edited_resource_breaks_seal(self): with (self.app / "Contents/Resources/LICENSE").open("a") as target: target.write("tampered") diff --git a/todos.md b/todos.md index 878a4525..6d57b8a8 100644 --- a/todos.md +++ b/todos.md @@ -32,7 +32,8 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [ ] Finish GUI sandbox workflows, moved-folder relaunch, and physical Intel/minimum-OS validation. - [ ] Add the signed Store packaging route and validate the final artifact on Intel and Apple Silicon. - [x] Prepare [listing/review notes and human-input checklist](packaging/apple-app-store/submission-draft.md); inventory privacy/network/export facts. -- [ ] Resolve [ONNX listed-SDK privacy manifests/signature provenance](packaging/apple-app-store/privacy-dependency-audit.md), confirm final privacy/export answers and validate Apple submission. +- [x] Preserve exact Abseil privacy manifests/license/provenance; verify original Microsoft runtime signatures before local re-signing. +- [ ] Resolve remaining [Protobuf/ONNX privacy coverage and SDK validation](packaging/apple-app-store/privacy-dependency-audit.md), confirm final privacy/export answers and validate Apple submission. - [x] Enable the full Xcode compiler after local license acceptance. - [ ] Human inputs last: developer Team ID, Store application/installer certificates, TestFlight provisioning profile, App Store Connect record and final submission decisions. - [x] Ronin reauthorized signed commits and pushing the preparation branch after the overnight work. From 38ec88ecbe408601f20ff72b83194350fb6d80fa Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 09:50:39 +0200 Subject: [PATCH 10/63] Restore Store picker folder navigation with explicit consent --- ARCHITECTURE.md | 4 +- internal/filepicker/darwin.go | 60 ++++++++++++++++ internal/filepicker/filepicker.go | 32 ++++++++- internal/filepicker/filepicker_test.go | 57 ++++++++++++++++ internal/filepicker/other.go | 4 ++ internal/openwith/openwith_darwin_test.go | 32 +++++++++ internal/similarity/worker_access_test.go | 4 +- internal/ui/heic_test.go | 2 + internal/ui/step_test.go | 50 ++++++++++---- packaging/apple-app-store/README.md | 13 +++- plans/2026-09-29-apple-app-store.md | 83 +++++++++++++++++++++++ scripts/macstorepackage/package.py | 12 +++- scripts/macstorepackage/test_package.py | 37 ++++++++++ todos.md | 3 + translations/de.json | 2 + translations/en.json | 2 + 16 files changed, 378 insertions(+), 19 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 4750a65f..b393d084 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1125,8 +1125,8 @@ can pick folders; Windows is files-only. | File | Responsibility | |------|----------------| -| `filepicker.go` | Typed `Choose` (URI list) / `ChooseSave` (one URI), strict native result decoding, canonical-path Zenity framing and UTF-8 JSON PowerShell transport. | -| `darwin.go` / `other.go` | `NSOpenPanel` / `NSSavePanel` with a shared NSURL-to-JSON transport / stubs; `darwin_test.go` exercises the actual native serializer. | +| `filepicker.go` | Typed `Choose` (URI list) / `ChooseSave` (one URI), strict native result decoding, Store single-file parent-consent validation, canonical-path Zenity framing and UTF-8 JSON PowerShell transport. | +| `darwin.go` / `other.go` | `NSOpenPanel` / `NSSavePanel`, including explicit Store sibling-folder permission, with a shared NSURL-to-JSON transport / stubs; `darwin_test.go` exercises the actual native serializer. | | `windows.go` / `notwindows.go` | `hideConsoleWindow` pair. | ### `internal/trash` diff --git a/internal/filepicker/darwin.go b/internal/filepicker/darwin.go index d03b1ada..393b2462 100644 --- a/internal/filepicker/darwin.go +++ b/internal/filepicker/darwin.go @@ -92,6 +92,38 @@ static panelResult runOpenPanel(const char *message) { return retainSelection(panel.URLs); } +// Only the matching folder can be confirmed. Navigating elsewhere never grants +// broader authority to the selected image through our result transport. +@interface PicFetchSiblingFolderDelegate : NSObject +@property(nonatomic, strong) NSURL *folder; +@end +@implementation PicFetchSiblingFolderDelegate +- (BOOL)panel:(id)sender validateURL:(NSURL *)url error:(NSError **)error { + (void)sender; (void)error; + return [url.URLByStandardizingPath isEqual:self.folder.URLByStandardizingPath]; +} +@end + +static panelResult runSiblingFolderPanel(const char *message, const char *prompt, const char *directory) { + NSOpenPanel *panel = [NSOpenPanel openPanel]; + NSURL *folder = [NSURL fileURLWithPath:[NSString stringWithUTF8String:directory] isDirectory:YES]; + __attribute__((objc_precise_lifetime)) PicFetchSiblingFolderDelegate *delegate = [PicFetchSiblingFolderDelegate new]; + delegate.folder = folder; + panel.delegate = delegate; + panel.message = [NSString stringWithUTF8String:message]; + panel.prompt = [NSString stringWithUTF8String:prompt]; + panel.canChooseFiles = NO; + panel.canChooseDirectories = YES; + panel.allowsMultipleSelection = NO; + panel.canCreateDirectories = NO; + panel.directoryURL = folder; + NSModalResponse response = [panel runModal]; + panel.delegate = nil; + if (response == NSModalResponseCancel) return (panelResult){0, NULL}; + if (response != NSModalResponseOK) return (panelResult){-1, NULL}; + return retainSelection(panel.URLs); +} + // runSavePanel shows an app-modal NSSavePanel pre-filled with name, opened // on dir. Same main-thread requirement as runOpenPanel above; // chooseSaveDarwin guarantees it the same way. No allowedContentTypes is @@ -267,3 +299,31 @@ func darwinSaveTransport(path string) (fyne.URI, error) { } var _ = darwinSaveTransport + +// This optional second panel runs on the existing chooser worker. Its accepted +// native URL is captured before returning to the ordinary collection pipeline. +func chooseSiblingFolderDarwin(directory string) (fyne.URI, error) { + message := C.CString(lang.L("Allow access to this image's folder to browse its other images. Cancel to open only the selected image.")) + defer C.free(unsafe.Pointer(message)) + prompt := C.CString(lang.L("Allow Folder Access")) + defer C.free(unsafe.Pointer(prompt)) + path := C.CString(directory) + defer C.free(unsafe.Pointer(path)) + var result C.panelResult + fyne.DoAndWait(func() { result = C.runSiblingFolderPanel(message, prompt, path) }) + out, err := decodeNativePanel(result, true) + if err != nil { + return nil, err + } + if string(out) == "null" { + return nil, nil + } + folders, err := decodeScopedSelection(out) + if err != nil { + return nil, err + } + if len(folders) != 1 { + return nil, errors.New("folder permission panel returned multiple selections") + } + return folders[0], nil +} diff --git a/internal/filepicker/filepicker.go b/internal/filepicker/filepicker.go index f9f649d1..c047f687 100644 --- a/internal/filepicker/filepicker.go +++ b/internal/filepicker/filepicker.go @@ -40,7 +40,13 @@ var Choose = func() ([]fyne.URI, error) { default: out, err = chooseFilesLinux() } - return decodePickedPaths(out, err) + files, err := decodePickedPaths(out, err) + // The immutable channel differs between ordinary and Store builds. + //goland:noinspection GoBoolExpressions + if err != nil || !distribution.AppleAppStore { + return files, err + } + return authorizeSiblingFolder(files, chooseSiblingFolderDarwin) } // ChooseSave returns exactly the destination confirmed by the native panel. @@ -287,3 +293,27 @@ func decodeScopedSelection(out []byte) ([]fyne.URI, error) { } return files, nil } + +// authorizeSiblingFolder keeps the opened file while attaching only a separately +// selected parent directory's authority. Cancelling leaves file-only access. +func authorizeSiblingFolder(files []fyne.URI, choose func(string) (fyne.URI, error)) ([]fyne.URI, error) { + if len(files) != 1 || !fileaccess.HasScope(files[0]) || fileaccess.Snapshot(files[0]).Directory { + return files, nil + } + parent := storage.NewFileURI(filepath.Dir(files[0].Path())) + folder, err := choose(parent.Path()) + if err != nil { + return nil, err + } + if folder == nil { + return files, nil + } + if !fileaccess.HasScope(folder) || !fileaccess.Snapshot(folder).Directory || filepath.Clean(folder.Path()) != filepath.Clean(parent.Path()) { + return nil, errors.New("folder access does not match the selected image's parent") + } + child, err := fileaccess.Child(folder, files[0]) + if err != nil { + return nil, err + } + return []fyne.URI{child}, nil +} diff --git a/internal/filepicker/filepicker_test.go b/internal/filepicker/filepicker_test.go index 678a23f5..9d4e0b56 100644 --- a/internal/filepicker/filepicker_test.go +++ b/internal/filepicker/filepicker_test.go @@ -10,6 +10,7 @@ import ( "strings" "testing" + "fyne.io/fyne/v2" "fyne.io/fyne/v2/storage" "github.com/frathe/picfetch/internal/distribution" @@ -348,3 +349,59 @@ func TestScopedSelectionDecodesNativeURLExactlyOnce(t *testing.T) { } } } + +func TestAuthorizeSiblingFolderPreservesOpenedFileAndFolderAuthority(t *testing.T) { + source, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///photos/a.jpg", Bookmark: []byte("file")}) + if err != nil { + t.Fatal(err) + } + folder, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///photos", Bookmark: []byte("folder"), Directory: true}) + if err != nil { + t.Fatal(err) + } + calls := 0 + files, err := authorizeSiblingFolder([]fyne.URI{source}, func(path string) (fyne.URI, error) { + calls++ + if path != "/photos" { + t.Fatalf("requested folder %q", path) + } + return folder, nil + }) + if err != nil || calls != 1 || len(files) != 1 { + t.Fatalf("files=%v calls=%d err=%v", files, calls, err) + } + record := fileaccess.Snapshot(files[0]) + if files[0].String() != source.String() || !record.Directory || record.Relative != "a.jpg" || string(record.Bookmark) != "folder" { + t.Fatalf("selected file lost folder authority: %+v", record) + } +} + +func TestAuthorizeSiblingFolderCancellationAndInvalidGrants(t *testing.T) { + source, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///photos/a.jpg", Bookmark: []byte("file")}) + if err != nil { + t.Fatal(err) + } + folder, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///other", Bookmark: []byte("other"), Directory: true}) + if err != nil { + t.Fatal(err) + } + cancelled, err := authorizeSiblingFolder([]fyne.URI{source}, func(_ string) (fyne.URI, error) { return nil, nil }) + if err != nil || len(cancelled) != 1 || cancelled[0] != source { + t.Fatalf("cancel lost original: %v %v", cancelled, err) + } + for _, invalid := range []fyne.URI{folder, source, storage.NewFileURI("/photos")} { + if _, err := authorizeSiblingFolder([]fyne.URI{source}, func(_ string) (fyne.URI, error) { return invalid, nil }); err == nil { + t.Fatalf("accepted unrelated or unscoped folder: %v", invalid) + } + } + failure := errors.New("native permission failure") + if _, err := authorizeSiblingFolder([]fyne.URI{source}, func(_ string) (fyne.URI, error) { return nil, failure }); !errors.Is(err, failure) { + t.Fatalf("lost permission error: %v", err) + } + for _, files := range [][]fyne.URI{nil, {source, source}, {folder}, {storage.NewFileURI("/photos/a.jpg")}} { + result, err := authorizeSiblingFolder(files, func(_ string) (fyne.URI, error) { t.Fatal("unexpected permission panel"); return nil, nil }) + if err != nil || !slices.Equal(result, files) { + t.Fatalf("changed selection: %v %v", result, err) + } + } +} diff --git a/internal/filepicker/other.go b/internal/filepicker/other.go index 6eec3cda..603f2fe9 100644 --- a/internal/filepicker/other.go +++ b/internal/filepicker/other.go @@ -20,3 +20,7 @@ func chooseFilesDarwin() ([]byte, error) { func chooseSaveDarwin(_ string) (fyne.URI, error) { return nil, errors.New("the macOS save panel only exists in darwin builds") } + +func chooseSiblingFolderDarwin(_ string) (fyne.URI, error) { + return nil, errors.New("the macOS folder permission panel only exists in darwin builds") +} diff --git a/internal/openwith/openwith_darwin_test.go b/internal/openwith/openwith_darwin_test.go index 9eaec172..411dba53 100644 --- a/internal/openwith/openwith_darwin_test.go +++ b/internal/openwith/openwith_darwin_test.go @@ -6,8 +6,10 @@ import ( "context" "net/url" "os" + "os/exec" "path/filepath" "testing" + "time" "fyne.io/fyne/v2/storage" @@ -198,6 +200,36 @@ func TestInvokeOpenURLs_BuffersWhenNoHandlerIsInstalledYet(t *testing.T) { } func TestNativeSelectedURLCaptureKeepsOriginalAuthority(t *testing.T) { + + // Go's Intel linker leaves test binaries unsigned. Foundation needs a signing + // identity for an app-scoped bookmark, even outside App Sandbox. Exercise the + // same test in a signed disposable copy instead of skipping native capture. + if os.Getenv("PICFETCH_TEST_SIGNED_BOOKMARK") != "1" { + executable, err := os.Executable() + if err != nil { + t.Fatal(err) + } + data, err := os.ReadFile(executable) + if err != nil { + t.Fatal(err) + } + fixture := filepath.Join(t.TempDir(), "bookmark.test") + if err := os.WriteFile(fixture, data, 0700); err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithTimeout(t.Context(), time.Minute) + defer cancel() + sign := exec.CommandContext(ctx, "/usr/bin/codesign", "--force", "--sign", "-", "--identifier", "io.github.frathe.picfetch.bookmark-tests", fixture) + if out, err := sign.CombinedOutput(); err != nil { + t.Fatalf("sign fixture: %v\n%s", err, out) + } + child := exec.CommandContext(ctx, fixture, "-test.run=^TestNativeSelectedURLCaptureKeepsOriginalAuthority$", "-test.v") + child.Env = append(os.Environ(), "PICFETCH_TEST_SIGNED_BOOKMARK=1") + if out, err := child.CombinedOutput(); err != nil { + t.Fatalf("signed bookmark fixture: %v\n%s", err, out) + } + return + } delivered := captureDelivered(t) path := filepath.Join(t.TempDir(), "chosen café.jpg") if err := os.WriteFile(path, []byte("fixture"), 0600); err != nil { diff --git a/internal/similarity/worker_access_test.go b/internal/similarity/worker_access_test.go index 38ae6e24..d951053f 100644 --- a/internal/similarity/worker_access_test.go +++ b/internal/similarity/worker_access_test.go @@ -3,7 +3,6 @@ package similarity import ( "context" "errors" - "path/filepath" "testing" "fyne.io/fyne/v2" @@ -30,7 +29,8 @@ func TestWorkerAccessCapturesExactPathsUntilRetirement(t *testing.T) { if err != nil { t.Fatal(err) } - want := []string{"/Applications/PicFetch.app", filepath.Join("/models", "vision_model.onnx"), filepath.Join("/models", "preprocessor_config.json"), "/photos/one.png", "/cache"} + // Fyne URI paths use slashes on Windows too. + want := []string{"/Applications/PicFetch.app", "/models/vision_model.onnx", "/models/preprocessor_config.json", "/photos/one.png", "/cache"} if len(req.Access) != len(want) { t.Fatalf("unexpected grants: %+v", req.Access) } diff --git a/internal/ui/heic_test.go b/internal/ui/heic_test.go index da07d5ea..913e01d4 100644 --- a/internal/ui/heic_test.go +++ b/internal/ui/heic_test.go @@ -269,6 +269,8 @@ func TestHEICUnavailableFiles(t *testing.T) { case <-time.After(testTimeout): t.Fatal("pending HEIC check blocked traversal before later supported files") } + // Traversal observed the JPEG; its earlier UI progress is queued. + v.scanUI.Drain() if got := v.scanOp.label.Text; got != "Scanning... 1 images" { t.Fatalf("supported-file progress while checking = %q", got) } diff --git a/internal/ui/step_test.go b/internal/ui/step_test.go index c0426c53..d380fd2c 100644 --- a/internal/ui/step_test.go +++ b/internal/ui/step_test.go @@ -8,6 +8,8 @@ import ( "fyne.io/fyne/v2" "fyne.io/fyne/v2/lang" + "github.com/frathe/picfetch/internal/distribution" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/filesort" "github.com/frathe/picfetch/internal/imaging" "github.com/frathe/picfetch/internal/uitest" @@ -82,18 +84,42 @@ func TestStepImage_SingleFileDropWalksFolderSiblings(t *testing.T) { } func TestHandleKeyEvent_LeftRightWalkFolderSiblings(t *testing.T) { - v := newTestViewer(t) - files := uitest.TempDirJPEGURIs(t, "a.jpg", "b.jpg") - dropAndWait(t, v, files[0]) - v.handleKeyEvent(&fyne.KeyEvent{Name: fyne.KeyRight}) - waitUntilLoaded(t, v) - if v.state.Observe().DisplayFiles()[v.state.Observe().index].Name() != "b.jpg" { - t.Fatalf("Right showing %q, want b.jpg", v.state.Observe().DisplayFiles()[v.state.Observe().index].Name()) - } - v.handleKeyEvent(&fyne.KeyEvent{Name: fyne.KeyLeft}) - waitUntilLoaded(t, v) - if v.state.Observe().DisplayFiles()[v.state.Observe().index].Name() != "a.jpg" { - t.Fatalf("Left showing %q, want a.jpg", v.state.Observe().DisplayFiles()[v.state.Observe().index].Name()) + for _, scoped := range []bool{false, true} { + name := "ordinary" + if scoped { + name = "authorized-folder" + } + t.Run(name, func(t *testing.T) { + v := newTestViewer(t) + files := uitest.TempDirJPEGURIs(t, "a.jpg", "b.jpg") + selected := files[0] + if scoped { + if distribution.AppleAppStore { + t.Skip("synthetic bookmark requires non-Store resolver; native grants are qualified separately") + } + var err error + selected, err = fileaccess.FromRecord(fileaccess.Record{URI: selected.String(), Bookmark: []byte("folder"), Directory: true, Relative: selected.Name()}) + if err != nil { + t.Fatal(err) + } + } + uitest.StubChooser(t, []fyne.URI{selected}, nil) + v.openFileDialog() + settleChooser(t, v) + waitForScan(t, v) + waitForSort(t, v) + waitUntilLoaded(t, v) + v.handleKeyEvent(&fyne.KeyEvent{Name: fyne.KeyRight}) + waitUntilLoaded(t, v) + if v.state.Observe().DisplayFiles()[v.state.Observe().index].Name() != "b.jpg" { + t.Fatalf("Right showing %q, want b.jpg", v.state.Observe().DisplayFiles()[v.state.Observe().index].Name()) + } + v.handleKeyEvent(&fyne.KeyEvent{Name: fyne.KeyLeft}) + waitUntilLoaded(t, v) + if v.state.Observe().DisplayFiles()[v.state.Observe().index].Name() != "a.jpg" { + t.Fatalf("Left showing %q, want a.jpg", v.state.Observe().DisplayFiles()[v.state.Observe().index].Name()) + } + }) } } diff --git a/packaging/apple-app-store/README.md b/packaging/apple-app-store/README.md index dffda647..d3fc50a1 100644 --- a/packaging/apple-app-store/README.md +++ b/packaging/apple-app-store/README.md @@ -143,7 +143,8 @@ The builder compiles arm64 and x86_64 app/broker/service binaries, stages both pinned native libraries and complete upstream notices, verifies the original libraries against Microsoft's Apple-anchored signing identity, retains Abseil's exact privacy declaration/license/provenance resource, signs nested code before the app, and validates exact entitlements, native dependency paths, architecture, -minimum OS and code/resource seals. ARM requires macOS 14.0 and Intel 13.4. +minimum OS, copyright/category metadata and code/resource seals. Final validation +rejects quarantine attributes on the app or any nested item without changing them. ARM requires macOS 14.0 and Intel 13.4. Deployment flags are explicit cgo cache inputs. Fyne metadata writes happen in a temporary directory; the repository build number is unchanged. The output includes `PicFetch.app` and a JSON manifest containing source/diff evidence and file hashes. @@ -191,3 +192,13 @@ broker/service/helper unchanged, and requires the private source to fail with an OS permission error while an allowed bundle source still completes inference. Run it with the same `--app`, `--models` and a separate `--result` output as the positive qualification. The overlay and diagnostic driver are never shipped. + +## Single-image folder navigation + +In the Store Open dialog, choosing one image now offers a second native folder +permission panel. Confirm **Allow Folder Access** for the image's containing +folder to enable Left/Right browsing. The selected image remains the initial +image; siblings retain the confirmed directory bookmark for later access. +Cancel opens only the selected image. Folder and multiple-file selections do not +show this extra panel. Finder/Open With and drag/drop do not yet offer equivalent +folder consent; opening a folder explicitly remains the available route there. diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index 09eb1695..dc256b4c 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -1122,3 +1122,86 @@ includes preserved upstream notice bytes and both provenance records. All eleven native packaging tests pass (1.753s). Portable invocation passes five tests with six explicit native skips. The CI workflow inspection is clean. All current local gates passed before staging; full native-amd64 CI is pending the draft PR. + +### Continuation — submission metadata and quarantine guards + +Draft PR #75 now runs CI, CodeQL and Qodana against cd063e9. T4 Standard slice, +T0 inline, zero spawns/two reviews. Inspecting the built app found the required +NSHumanReadableCopyright key absent. Final validation also omitted quarantine +attributes. Apple preparation/upload requirements were rechecked 2026-09-30. +Use the existing LICENSE copyright line and photography category; validate both +in the final plist. Reject quarantine on the app root, nested directories or files +without mutating the inspected app or clearing downloaded user files. +Files: existing package.py/test_package.py, packaging README, todos and this record. +Seam: existing final signed-artifact validator. AC: re-signed missing/changed +metadata and quarantined copies fail; a fresh universal package passes all native +packaging tests. Run the Python suite, Make package target and GoLand inspections. +The user-facing f0565d1 E2E artifact remains unchanged. Distribution signing and +submission remain separate gates; this slice does not need account credentials. + +### User regression — single-image Open dialog cannot navigate siblings + +Ronin reproduced in the ad-hoc Store app using PicFetch's Open dialog. A temporary +Go overlay on the existing Left/Right UI test changes only the input to a +file-scoped record: `go test -tags no_emoji,nodynamic -overlay +.scratch/apple-store-package/sibling-repro-overlay.json ./internal/ui -run +'^TestHandleKeyEvent_LeftRightWalkFolderSiblings$' -count=1` fails: Right still +shows a.jpg, expected b.jpg. Changing only that record to captured folder authority +passes. The picker captures a single file grant; Parent correctly refuses to +broaden it. Keyboard handling and retained directory propagation work. + +T3 Deep continuation, T0 inline, zero spawns/two reviews. Add an explicit native +folder grant after single-file selection in the Store Open dialog, preserving the +selected image as the initial image. Cancellation keeps the original file; no +parent access is manufactured from a path. Multiple selections/directories and +non-Store channels retain current behavior. Native UI work stays in the existing +tracked chooser and capture remains on its worker. Filepicker behavior tests cover +accept/cancel/wrong-folder and multi-file cases; existing Left/Right test adds a +scoped-folder chooser case. Test native transport, focused Go race suites, locale +parity, Make package and signed guards, then GoLand. Build a fresh E2E app without +overwriting the original. Finder/Drop entry-point folder consent remains a +separate open workflow; do not silently claim this Open-dialog fix covers it. + +### CI follow-up at cd063e9 + +PR #75's first run passed CodeQL, validation, Linux native/non-UI/ui-2, ARM native +and FOSSA. Three failures have concrete causes: Windows worker-access expected +backslashes even though Fyne URI paths use slashes; the HEIC progress test observed +worker traversal before draining scanUI; Intel's unsigned Go test binary could +not retrieve Foundation's app-scope key. The latter reproduced locally under +Rosetta, then passed with the identical binary ad-hoc signed. T0 inline fixes +only these test assumptions: URI-form expected paths, drain already-submitted +progress, and run the bookmark test in a signed disposable self-copy (no skip). +Verify focused race tests plus unsigned Intel parent invoking its signed child; +CI must still rerun on actual Windows/Linux/Intel. No production sandbox changes. + +Qodana post-suppression `/qodana.sarif.json` reports eight unused-function warnings. +Each has live callers: Stat (favthumbs/name.go and ui/filework.go), NeedsCapture +(ui/drop.go), HasScope (favstore and ui/drop.go), Export (worker_access_darwin.go), +Import (worker_access.go), CurrentRuntimeDirectory (similarity/assets.go and native +worker launch), InstallWindowDrop and Stop (ui/run.go). These are cross-package/ +platform analysis false positives, not removable code. The `/start/` SARIF is +baseline output and must not overwrite the actual result. Keep the warnings and +these exact dispositions visible; no broad unused-code exclusion is introduced. + +Folder-consent policy, cancellation, wrong/unscoped directory, native picker, +Apple-tagged picker and ordinary/authorized-folder Left/Right tests pass. English/ +German locale parity passes. Universal test app is `bin/apple-store-e2e-folder-navigation/PicFetch.app` +and its matching ZIP/hash are beside the output directory. All 13 signed artifact +guards pass (3.231s). The app's manifest records the exact working tree used; +subsequent changes are inspection comments/test fixes/docs, not runtime behavior. +The native live permission panel still needs Ronin's E2E confirmation; original +f0565d1 app is untouched. GoLand inspected all seven changed packaging/picker/UI +files. Existing test duplication remains covered by exact qodana.yaml test +exclusions; immutable distribution-channel condition has a narrow documented +GoBoolExpressions suppression. No other findings in that scope. + +Focused race verification passed: similarity 1.427s, openwith 2.665s, root UI +7.655s. Unsigned Intel parent now runs the real signed child guard successfully +(0.64s), preserving native capture assertions. GoLand reinspection of all three +CI test fixes and the picker condition suppression is clean. `make verify-build` +passes; native package suite is 13/13. No top-level root UI test was added, so +existing shard assignments remain exact. Source-local test duplication exclusions +are unchanged. Final review: permission scope remains explicit, original image +identity survives consent, cancellation does not widen access, and no production +sandbox policy is relaxed. CI results for the next pushed commit remain pending. diff --git a/scripts/macstorepackage/package.py b/scripts/macstorepackage/package.py index cdfdbc79..ae1922f5 100644 --- a/scripts/macstorepackage/package.py +++ b/scripts/macstorepackage/package.py @@ -30,6 +30,10 @@ APP_ID = "io.github.frathe.picfetch" TARGETS = {"arm64": ("arm64", "14.0"), "amd64": ("x86_64", "13.4")} TAGS = "no_emoji,nodynamic,appleappstore" +SUBMISSION_METADATA = { + "NSHumanReadableCopyright": "Copyright (c) 2026 Florian Rathe", + "LSApplicationCategoryType": "public.app-category.photography", +} # Reviewed exact upstream declarations; changes require a source/license audit. PRIVACY_FILES = { 'PrivacyInfo.xcprivacy': 'f232217ae9edf2ab6a541a28eff50cfe05303c2b4756fff95cca72eccbc3b898', @@ -132,6 +136,9 @@ def native_check(path, architectures, minima, env): def verify(app, env): + attributes = run("xattr", "-r", app, capture=True, env=env) + if any(line.rsplit(b": ", 1)[-1] == b"com.apple.quarantine" for line in attributes.splitlines()): + raise ValueError("bundle contains com.apple.quarantine; rebuild from unquarantined inputs") contents = app / "Contents" service = contents / "XPCServices" / (APP_ID + ".worker.xpc") roles = {app: "app", service: "service", @@ -146,6 +153,9 @@ def verify(app, env): raise ValueError("unexpected app identity") if plist.get("LSMinimumSystemVersionByArchitecture") != {a: m for a, m in TARGETS.values()}: raise ValueError("unexpected per-architecture deployment minimum") + for key, expected in SUBMISSION_METADATA.items(): + if plist.get(key) != expected: + raise ValueError(f"unexpected submission metadata: {key}") verify_privacy(contents) expected_code = {contents / "MacOS/PicFetch", contents / "MacOS/picfetch-worker-client", service / "Contents/MacOS/worker", service / "Contents/MacOS/picfetch-image-worker"} @@ -248,7 +258,7 @@ def build(args): info = plistlib.loads((contents / "Info.plist").read_bytes()) info.update({"LSMinimumSystemVersion": "13.4", "LSMinimumSystemVersionByArchitecture": {a: m for a, m in TARGETS.values()}, - "LSApplicationCategoryType": "public.app-category.photography"}) + **SUBMISSION_METADATA}) write_plist(contents / "Info.plist", info) (contents / "Frameworks").mkdir() for arch in TARGETS: diff --git a/scripts/macstorepackage/test_package.py b/scripts/macstorepackage/test_package.py index 049e8d2d..07949d81 100644 --- a/scripts/macstorepackage/test_package.py +++ b/scripts/macstorepackage/test_package.py @@ -86,6 +86,43 @@ def test_ad_hoc_runtime_cannot_claim_upstream_identity(self): with self.assertRaises(subprocess.CalledProcessError): package.verify_upstream_library(runtime, self.env) + def test_resigned_missing_submission_metadata_is_refused(self): + path = self.app / "Contents/Info.plist" + original = package.plistlib.loads(path.read_bytes()) + for key in ("NSHumanReadableCopyright", "LSApplicationCategoryType"): + for value in (None, "unexpected"): + with self.subTest(key=key, value=value): + data = original.copy() + data["NSHumanReadableCopyright"] = "Copyright (c) 2026 Florian Rathe" + data["LSApplicationCategoryType"] = "public.app-category.photography" + if value is None: + del data[key] + else: + data[key] = value + package.write_plist(path, data) + self.sign(self.app, "app") + package.run("codesign", "--verify", "--deep", "--strict", self.app) + with self.assertRaisesRegex(ValueError, "submission metadata"): + package.verify(self.app, self.env) + + def test_quarantined_bundle_is_refused_without_mutation(self): + metadata = self.app / "Contents/Info.plist" + data = package.plistlib.loads(metadata.read_bytes()) + data["NSHumanReadableCopyright"] = "Copyright (c) 2026 Florian Rathe" + package.write_plist(metadata, data) + self.sign(self.app, "app") + for path in (self.app, self.app / "Contents/Resources", + self.app / "Contents/Resources/LICENSE"): + with self.subTest(path=path): + attribute = "0081;00000000;PicFetchTest;" + package.run("xattr", "-w", "com.apple.quarantine", attribute, path) + try: + with self.assertRaisesRegex(ValueError, "quarantine"): + package.verify(self.app, self.env) + self.assertEqual(package.run("xattr", "-p", "com.apple.quarantine", path, capture=True).decode().strip(), attribute) + finally: + package.run("xattr", "-d", "com.apple.quarantine", path) + def test_resigned_privacy_change_is_refused(self): package.stage_privacy(Path(__file__).resolve().parents[2], self.app / "Contents") resource = self.app / "Contents/Resources/AbseilPrivacy.bundle/Contents/Resources/PrivacyInfo.xcprivacy" diff --git a/todos.md b/todos.md index 6d57b8a8..3c6c781e 100644 --- a/todos.md +++ b/todos.md @@ -29,6 +29,9 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Qualify retained visual search through production ARM/Intel-Rosetta workers; keep scopes through clipboard/Finder/Trash calls and source metadata reads. - [x] Capture native window-drop URLs before GLFW flattens paths; refresh restored moved/stale bookmarks with immutable persistence coverage. - [x] Verify production workers deny an ungranted private source while an allowed control image still completes inference, on ARM and Intel under Rosetta. +- [x] Add explicit Store Open-dialog folder consent for single-image sibling navigation; preserve cancellation and the initially selected image. Rebuilt E2E app is in `bin/apple-store-e2e-folder-navigation/`. +- [ ] Confirm the native folder-consent interaction E2E; extend explicit folder consent to Finder/Open With and drag/drop. +- [x] Validate copyright/category metadata and reject quarantine attributes in local Store packages. - [ ] Finish GUI sandbox workflows, moved-folder relaunch, and physical Intel/minimum-OS validation. - [ ] Add the signed Store packaging route and validate the final artifact on Intel and Apple Silicon. - [x] Prepare [listing/review notes and human-input checklist](packaging/apple-app-store/submission-draft.md); inventory privacy/network/export facts. diff --git a/translations/de.json b/translations/de.json index f11861ed..156ec931 100644 --- a/translations/de.json +++ b/translations/de.json @@ -1,4 +1,6 @@ { + "Allow Folder Access": "Ordnerzugriff erlauben", + "Allow access to this image's folder to browse its other images. Cancel to open only the selected image.": "Erlaube den Zugriff auf den Ordner dieses Bildes, um seine anderen Bilder durchzusehen. Mit Abbrechen öffnest du nur das ausgewählte Bild.", "could not open selected files: %v": "Ausgewählte Dateien konnten nicht geöffnet werden: %v", "Could not cache Favorite locations. Locations remain available in memory.": "Standorte des Favoriten konnten nicht zwischengespeichert werden. Sie bleiben im Arbeitsspeicher verfügbar.", "Showing location cluster": "Standortgruppe wird angezeigt", diff --git a/translations/en.json b/translations/en.json index 24e4867f..79e22fa7 100644 --- a/translations/en.json +++ b/translations/en.json @@ -1,4 +1,6 @@ { + "Allow Folder Access": "Allow Folder Access", + "Allow access to this image's folder to browse its other images. Cancel to open only the selected image.": "Allow access to this image's folder to browse its other images. Cancel to open only the selected image.", "could not open selected files: %v": "could not open selected files: %v", "Could not cache Favorite locations. Locations remain available in memory.": "Could not cache Favorite locations. Locations remain available in memory.", "Showing location cluster": "Showing location cluster", From 75356d99786d809cf73b059a5f28ad6fc858b6fb Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 10:05:40 +0200 Subject: [PATCH 11/63] Share Store folder consent across dropped and opened images --- .github/testshards/internal-ui.tsv | 3 +- ARCHITECTURE.md | 7 +- internal/filepicker/darwin.go | 11 +- internal/filepicker/filepicker.go | 30 +++- internal/filepicker/filepicker_test.go | 16 +- internal/filepicker/other.go | 3 +- internal/similarity/worker_access_test.go | 14 +- internal/ui/build.go | 2 + internal/ui/drop.go | 28 ++++ internal/ui/harness_test.go | 1 + internal/ui/step_test.go | 185 ++++++++++++++++++---- internal/ui/viewer.go | 4 + packaging/apple-app-store/README.md | 16 +- plans/2026-09-29-apple-app-store.md | 50 ++++++ todos.md | 5 +- 15 files changed, 305 insertions(+), 70 deletions(-) diff --git a/.github/testshards/internal-ui.tsv b/.github/testshards/internal-ui.tsv index bc874195..087545de 100644 --- a/.github/testshards/internal-ui.tsv +++ b/.github/testshards/internal-ui.tsv @@ -5,7 +5,7 @@ # baseline-run: 34627413846 # baseline-attempts: 1 # weights: single CI attempt; projected test sums, not measured shard runtimes -# ui-1: 270 entries, 470.520s projected weight sum +# ui-1: 271 entries, 470.520s projected weight sum # ui-2: 255 entries, 470.530s projected weight sum # ui-3: 224 entries, 504.870s projected weight sum # rebalance: 27 whole-test moves; slowest projected sum within 5% of mean @@ -234,6 +234,7 @@ TestShowExifWindow_OpensAndRaisesSameWindow ui-1 TestShowImage_StartsLoadLifecycle ui-1 TestShow_ResetsZoomOnNavigation ui-1 TestShutdownStopsPictureFrameWithQueuedAdvance ui-1 +TestSiblingFolderConsentLifecycle ui-1 TestSlideInterval_DefaultsBeforePictureFrameModeEverRuns ui-1 TestStartWindowPosPolling_TestDriverGetsNoopStop ui-1 TestStartup_LoadsSavedSecondaryWindowGeometry ui-1 diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index b393d084..6eaacb3a 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1120,13 +1120,14 @@ PNG image data (`CopyImage`) and file-reference lists (`CopyFiles`). ### `internal/filepicker` -Native open chooser (`Choose`) and save panel (`ChooseSave`). Linux/macOS +Native open chooser (`Choose`), sibling-folder authorization (`AuthorizeSiblingFolder`) +and save panel (`ChooseSave`). Linux/macOS can pick folders; Windows is files-only. | File | Responsibility | |------|----------------| -| `filepicker.go` | Typed `Choose` (URI list) / `ChooseSave` (one URI), strict native result decoding, Store single-file parent-consent validation, canonical-path Zenity framing and UTF-8 JSON PowerShell transport. | -| `darwin.go` / `other.go` | `NSOpenPanel` / `NSSavePanel`, including explicit Store sibling-folder permission, with a shared NSURL-to-JSON transport / stubs; `darwin_test.go` exercises the actual native serializer. | +| `filepicker.go` | Typed `Choose` (URI list) / `ChooseSave` (one URI), strict native result decoding, explicit `AuthorizeSiblingFolder` consent policy for shared collection discovery, canonical-path Zenity framing and UTF-8 JSON PowerShell transport. | +| `darwin.go` / `other.go` | `NSOpenPanel` / `NSSavePanel`, including cancellable Store sibling-folder permission, with a shared NSURL-to-JSON transport / stubs; `darwin_test.go` exercises the actual native serializer. | | `windows.go` / `notwindows.go` | `hideConsoleWindow` pair. | ### `internal/trash` diff --git a/internal/filepicker/darwin.go b/internal/filepicker/darwin.go index 393b2462..090fbba6 100644 --- a/internal/filepicker/darwin.go +++ b/internal/filepicker/darwin.go @@ -171,6 +171,7 @@ static panelResult testSaveSelection(const char *path) { import "C" import ( + "context" "encoding/json" "errors" "path/filepath" @@ -300,9 +301,9 @@ func darwinSaveTransport(path string) (fyne.URI, error) { var _ = darwinSaveTransport -// This optional second panel runs on the existing chooser worker. Its accepted +// This optional panel runs on a tracked opening worker. Its accepted // native URL is captured before returning to the ordinary collection pipeline. -func chooseSiblingFolderDarwin(directory string) (fyne.URI, error) { +func chooseSiblingFolderDarwin(ctx context.Context, directory string) (fyne.URI, error) { message := C.CString(lang.L("Allow access to this image's folder to browse its other images. Cancel to open only the selected image.")) defer C.free(unsafe.Pointer(message)) prompt := C.CString(lang.L("Allow Folder Access")) @@ -310,7 +311,11 @@ func chooseSiblingFolderDarwin(directory string) (fyne.URI, error) { path := C.CString(directory) defer C.free(unsafe.Pointer(path)) var result C.panelResult - fyne.DoAndWait(func() { result = C.runSiblingFolderPanel(message, prompt, path) }) + fyne.DoAndWait(func() { + if ctx.Err() == nil { + result = C.runSiblingFolderPanel(message, prompt, path) + } + }) out, err := decodeNativePanel(result, true) if err != nil { return nil, err diff --git a/internal/filepicker/filepicker.go b/internal/filepicker/filepicker.go index c047f687..bb7a6e66 100644 --- a/internal/filepicker/filepicker.go +++ b/internal/filepicker/filepicker.go @@ -7,6 +7,7 @@ package filepicker import ( + "context" "encoding/json" "errors" "fmt" @@ -40,13 +41,7 @@ var Choose = func() ([]fyne.URI, error) { default: out, err = chooseFilesLinux() } - files, err := decodePickedPaths(out, err) - // The immutable channel differs between ordinary and Store builds. - //goland:noinspection GoBoolExpressions - if err != nil || !distribution.AppleAppStore { - return files, err - } - return authorizeSiblingFolder(files, chooseSiblingFolderDarwin) + return decodePickedPaths(out, err) } // ChooseSave returns exactly the destination confirmed by the native panel. @@ -317,3 +312,24 @@ func authorizeSiblingFolder(files []fyne.URI, choose func(string) (fyne.URI, err } return []fyne.URI{child}, nil } + +// AuthorizeSiblingFolder optionally obtains explicit directory access for a +// captured single-image input. Call on a tracked worker, after admission for +// sibling discovery; saved collections and merge inputs do not need this grant. +func AuthorizeSiblingFolder(ctx context.Context, files []fyne.URI) ([]fyne.URI, error) { + if err := ctx.Err(); err != nil { + return nil, err + } + // The channel is fixed at build time, independently of the host inspector. + //goland:noinspection GoBoolExpressions + if !distribution.AppleAppStore { + return files, nil + } + files, err := authorizeSiblingFolder(files, func(directory string) (fyne.URI, error) { + return chooseSiblingFolderDarwin(ctx, directory) + }) + if cancelled := ctx.Err(); cancelled != nil { + return nil, cancelled + } + return files, err +} diff --git a/internal/filepicker/filepicker_test.go b/internal/filepicker/filepicker_test.go index 9d4e0b56..d5fda6f1 100644 --- a/internal/filepicker/filepicker_test.go +++ b/internal/filepicker/filepicker_test.go @@ -351,18 +351,19 @@ func TestScopedSelectionDecodesNativeURLExactlyOnce(t *testing.T) { } func TestAuthorizeSiblingFolderPreservesOpenedFileAndFolderAuthority(t *testing.T) { - source, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///photos/a.jpg", Bookmark: []byte("file")}) + root := storage.NewFileURI(t.TempDir()).Path() + source, err := fileaccess.FromRecord(fileaccess.Record{URI: storage.NewFileURI(root + "/photos/a.jpg").String(), Bookmark: []byte("file")}) if err != nil { t.Fatal(err) } - folder, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///photos", Bookmark: []byte("folder"), Directory: true}) + folder, err := fileaccess.FromRecord(fileaccess.Record{URI: storage.NewFileURI(root + "/photos").String(), Bookmark: []byte("folder"), Directory: true}) if err != nil { t.Fatal(err) } calls := 0 files, err := authorizeSiblingFolder([]fyne.URI{source}, func(path string) (fyne.URI, error) { calls++ - if path != "/photos" { + if path != root+"/photos" { t.Fatalf("requested folder %q", path) } return folder, nil @@ -377,11 +378,12 @@ func TestAuthorizeSiblingFolderPreservesOpenedFileAndFolderAuthority(t *testing. } func TestAuthorizeSiblingFolderCancellationAndInvalidGrants(t *testing.T) { - source, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///photos/a.jpg", Bookmark: []byte("file")}) + root := storage.NewFileURI(t.TempDir()).Path() + source, err := fileaccess.FromRecord(fileaccess.Record{URI: storage.NewFileURI(root + "/photos/a.jpg").String(), Bookmark: []byte("file")}) if err != nil { t.Fatal(err) } - folder, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///other", Bookmark: []byte("other"), Directory: true}) + folder, err := fileaccess.FromRecord(fileaccess.Record{URI: storage.NewFileURI(root + "/other").String(), Bookmark: []byte("other"), Directory: true}) if err != nil { t.Fatal(err) } @@ -389,7 +391,7 @@ func TestAuthorizeSiblingFolderCancellationAndInvalidGrants(t *testing.T) { if err != nil || len(cancelled) != 1 || cancelled[0] != source { t.Fatalf("cancel lost original: %v %v", cancelled, err) } - for _, invalid := range []fyne.URI{folder, source, storage.NewFileURI("/photos")} { + for _, invalid := range []fyne.URI{folder, source, storage.NewFileURI(root + "/photos")} { if _, err := authorizeSiblingFolder([]fyne.URI{source}, func(_ string) (fyne.URI, error) { return invalid, nil }); err == nil { t.Fatalf("accepted unrelated or unscoped folder: %v", invalid) } @@ -398,7 +400,7 @@ func TestAuthorizeSiblingFolderCancellationAndInvalidGrants(t *testing.T) { if _, err := authorizeSiblingFolder([]fyne.URI{source}, func(_ string) (fyne.URI, error) { return nil, failure }); !errors.Is(err, failure) { t.Fatalf("lost permission error: %v", err) } - for _, files := range [][]fyne.URI{nil, {source, source}, {folder}, {storage.NewFileURI("/photos/a.jpg")}} { + for _, files := range [][]fyne.URI{nil, {source, source}, {folder}, {storage.NewFileURI(root + "/photos/a.jpg")}} { result, err := authorizeSiblingFolder(files, func(_ string) (fyne.URI, error) { t.Fatal("unexpected permission panel"); return nil, nil }) if err != nil || !slices.Equal(result, files) { t.Fatalf("changed selection: %v %v", result, err) diff --git a/internal/filepicker/other.go b/internal/filepicker/other.go index 603f2fe9..d0ea67aa 100644 --- a/internal/filepicker/other.go +++ b/internal/filepicker/other.go @@ -3,6 +3,7 @@ package filepicker import ( + "context" "errors" "fyne.io/fyne/v2" @@ -21,6 +22,6 @@ func chooseSaveDarwin(_ string) (fyne.URI, error) { return nil, errors.New("the macOS save panel only exists in darwin builds") } -func chooseSiblingFolderDarwin(_ string) (fyne.URI, error) { +func chooseSiblingFolderDarwin(_ context.Context, _ string) (fyne.URI, error) { return nil, errors.New("the macOS folder permission panel only exists in darwin builds") } diff --git a/internal/similarity/worker_access_test.go b/internal/similarity/worker_access_test.go index d951053f..ee6db392 100644 --- a/internal/similarity/worker_access_test.go +++ b/internal/similarity/worker_access_test.go @@ -6,19 +6,21 @@ import ( "testing" "fyne.io/fyne/v2" + "fyne.io/fyne/v2/storage" "github.com/frathe/picfetch/internal/fileaccess" ) func TestWorkerAccessCapturesExactPathsUntilRetirement(t *testing.T) { - source, err := fileaccess.FromRecord(fileaccess.Record{URI: "file:///photos/one.png", Bookmark: []byte("persistent")}) + root := storage.NewFileURI(t.TempDir()).Path() + source, err := fileaccess.FromRecord(fileaccess.Record{URI: storage.NewFileURI(root + "/photos/one.png").String(), Bookmark: []byte("persistent")}) if err != nil { t.Fatal(err) } ctx := fileaccess.WithSources(context.Background(), []fyne.URI{source}) - req := request{Assets: "/models", Paths: []string{source.Path(), source.Path()}, GeneralAnalysisDir: "/cache"} + req := request{Assets: root + "/models", Paths: []string{source.Path(), source.Path()}, GeneralAnalysisDir: root + "/cache"} active := map[string]bool{} - release, err := captureWorkerAccess(ctx, &req, "/Applications/PicFetch.app", func(_ context.Context, uri fyne.URI) (fileaccess.Transfer, func(), error) { + release, err := captureWorkerAccess(ctx, &req, root+"/PicFetch.app", func(_ context.Context, uri fyne.URI) (fileaccess.Transfer, func(), error) { path := uri.Path() if path == source.Path() && uri != source { t.Fatal("source authority flattened") @@ -29,8 +31,8 @@ func TestWorkerAccessCapturesExactPathsUntilRetirement(t *testing.T) { if err != nil { t.Fatal(err) } - // Fyne URI paths use slashes on Windows too. - want := []string{"/Applications/PicFetch.app", "/models/vision_model.onnx", "/models/preprocessor_config.json", "/photos/one.png", "/cache"} + // Use absolute host paths: Windows roots need a volume as well as slashes. + want := []string{root + "/PicFetch.app", root + "/models/vision_model.onnx", root + "/models/preprocessor_config.json", root + "/photos/one.png", root + "/cache"} if len(req.Access) != len(want) { t.Fatalf("unexpected grants: %+v", req.Access) } @@ -39,7 +41,7 @@ func TestWorkerAccessCapturesExactPathsUntilRetirement(t *testing.T) { t.Fatalf("access lost: %s", path) } } - if active["/photos"] { + if active[root+"/photos"] { t.Fatal("source expanded to parent") } release() diff --git a/internal/ui/build.go b/internal/ui/build.go index aa30c4cb..2d83fe96 100644 --- a/internal/ui/build.go +++ b/internal/ui/build.go @@ -16,6 +16,7 @@ import ( "fyne.io/fyne/v2/widget" "github.com/frathe/picfetch/internal/appearance" + "github.com/frathe/picfetch/internal/filepicker" "github.com/frathe/picfetch/internal/filesort" "github.com/frathe/picfetch/internal/heic" "github.com/frathe/picfetch/internal/imaging" @@ -111,6 +112,7 @@ func buildViewer(application fyne.App, startup startupState) (*viewer, fyne.Wind updateExecutable: os.Executable, } + view.authorizeSiblingFolder = filepicker.AuthorizeSiblingFolder view.chooserUI = fyneChooserQueue{} view.scanUI = fyneChooserQueue{} view.osInputs.queue = fyneChooserQueue{} diff --git a/internal/ui/drop.go b/internal/ui/drop.go index f79f1aeb..96473b0b 100644 --- a/internal/ui/drop.go +++ b/internal/ui/drop.go @@ -297,9 +297,24 @@ func (v *viewer) openCollection(uris []fyne.URI, favoriteDir string, kind collec return } + // Reserve the native prefix before starting the worker so chooser settlement + // never races a new WaitGroup admission. It ends before the directory scan. + needsConsent := protected && kind == discoverCollection && !merging && len(uris) == 1 + authorize := v.authorizeSiblingFolder + if needsConsent { + v.openChooserWorkers.Add(1) + } inputOwned = false v.scanWorkers.Go(func() { defer workerDone() + nativePending := needsConsent + finishNative := func() { + if nativePending { + nativePending = false + v.openChooserWorkers.Done() + } + } + defer finishNative() selected := uris defer fileaccess.ReleaseSelected(selected) if protected { @@ -311,6 +326,19 @@ func (v *viewer) openCollection(uris []fyne.URI, favoriteDir string, kind collec uris = captured inspectInput() } + if needsConsent && expandSiblings && !fileaccess.Snapshot(uris[0]).Directory { + if !token.Current() { + v.scanUI.Do(func() { v.failSelectedInput(token, token.Context().Err(), scanDone) }) + return + } + permitted, err := authorize(token.Context(), uris) + if err != nil { + v.scanUI.Do(func() { v.failSelectedInput(token, err, scanDone) }) + return + } + uris = permitted + } + finishNative() // token.Context() is what lets a superseded scan (a newer drop, or // an explicit cancel - see cancelScan) stop walking the tree instead // of racing storage.List calls to completion for a result nobody diff --git a/internal/ui/harness_test.go b/internal/ui/harness_test.go index 1cba81b1..69d1c8b8 100644 --- a/internal/ui/harness_test.go +++ b/internal/ui/harness_test.go @@ -166,6 +166,7 @@ func newTestUIWithPolicy(t *testing.T, policy launch.Policy) (v *viewer, win fyn v.slides.SetUIQueue(&uitest.UIQueue{}) v.exif.SetUIQueue(&uitest.UIQueue{}) v.fileWork.ui = &uitest.UIQueue{} + v.authorizeSiblingFolder = func(_ context.Context, files []fyne.URI) ([]fyne.URI, error) { return files, nil } v.chooserUI = &uitest.UIQueue{} v.scanUI = &uitest.UIQueue{} v.clipboardWork.ui = &uitest.UIQueue{} diff --git a/internal/ui/step_test.go b/internal/ui/step_test.go index d380fd2c..a37eecf2 100644 --- a/internal/ui/step_test.go +++ b/internal/ui/step_test.go @@ -1,9 +1,12 @@ package ui import ( + "context" "image" "image/color" + "sync" "testing" + "time" "fyne.io/fyne/v2" "fyne.io/fyne/v2/lang" @@ -84,42 +87,59 @@ func TestStepImage_SingleFileDropWalksFolderSiblings(t *testing.T) { } func TestHandleKeyEvent_LeftRightWalkFolderSiblings(t *testing.T) { - for _, scoped := range []bool{false, true} { - name := "ordinary" - if scoped { - name = "authorized-folder" - } - t.Run(name, func(t *testing.T) { - v := newTestViewer(t) - files := uitest.TempDirJPEGURIs(t, "a.jpg", "b.jpg") - selected := files[0] + for _, route := range []string{"picker", "drop", "os"} { + for _, scoped := range []bool{false, true} { + name := route + "/ordinary" if scoped { - if distribution.AppleAppStore { - t.Skip("synthetic bookmark requires non-Store resolver; native grants are qualified separately") + name = route + "/selected-file" + } + t.Run(name, func(t *testing.T) { + if scoped { + if distribution.AppleAppStore { + t.Skip("synthetic bookmark requires non-Store resolver; native grants are qualified separately") + } } - var err error - selected, err = fileaccess.FromRecord(fileaccess.Record{URI: selected.String(), Bookmark: []byte("folder"), Directory: true, Relative: selected.Name()}) - if err != nil { - t.Fatal(err) + v := newTestViewer(t) + files := uitest.TempDirJPEGURIs(t, "a.jpg", "b.jpg") + selected := files[0] + if scoped { + original := selected + selected = fileaccess.NewSelection(original, func(_ context.Context) (fileaccess.Record, error) { + return fileaccess.Record{URI: original.String(), Bookmark: []byte("file")}, nil + }, func() {}) + v.authorizeSiblingFolder = func(_ context.Context, inputs []fyne.URI) ([]fyne.URI, error) { + if len(inputs) != 1 || inputs[0].String() != original.String() || fileaccess.NeedsCapture(inputs[0]) { + t.Error("consent did not receive captured original selection") + } + child, err := fileaccess.FromRecord(fileaccess.Record{URI: original.String(), Bookmark: []byte("folder"), Directory: true, Relative: original.Name()}) + return []fyne.URI{child}, err + } } - } - uitest.StubChooser(t, []fyne.URI{selected}, nil) - v.openFileDialog() - settleChooser(t, v) - waitForScan(t, v) - waitForSort(t, v) - waitUntilLoaded(t, v) - v.handleKeyEvent(&fyne.KeyEvent{Name: fyne.KeyRight}) - waitUntilLoaded(t, v) - if v.state.Observe().DisplayFiles()[v.state.Observe().index].Name() != "b.jpg" { - t.Fatalf("Right showing %q, want b.jpg", v.state.Observe().DisplayFiles()[v.state.Observe().index].Name()) - } - v.handleKeyEvent(&fyne.KeyEvent{Name: fyne.KeyLeft}) - waitUntilLoaded(t, v) - if v.state.Observe().DisplayFiles()[v.state.Observe().index].Name() != "a.jpg" { - t.Fatalf("Left showing %q, want a.jpg", v.state.Observe().DisplayFiles()[v.state.Observe().index].Name()) - } - }) + switch route { + case "picker": + uitest.StubChooser(t, []fyne.URI{selected}, nil) + v.openFileDialog() + settleChooser(t, v) + case "drop": + v.handleDrop([]fyne.URI{selected}) + case "os": + v.openFilesFromOS([]fyne.URI{selected}) + } + waitForScan(t, v) + waitForSort(t, v) + waitUntilLoaded(t, v) + v.handleKeyEvent(&fyne.KeyEvent{Name: fyne.KeyRight}) + waitUntilLoaded(t, v) + if got := v.state.Observe().DisplayFiles()[v.state.Observe().index].Name(); got != "b.jpg" { + t.Fatalf("Right showing %q, want b.jpg", got) + } + v.handleKeyEvent(&fyne.KeyEvent{Name: fyne.KeyLeft}) + waitUntilLoaded(t, v) + if got := v.state.Observe().DisplayFiles()[v.state.Observe().index].Name(); got != "a.jpg" { + t.Fatalf("Left showing %q, want a.jpg", got) + } + }) + } } } @@ -721,3 +741,102 @@ func TestClearToDropzone_ClearsInspect(t *testing.T) { t.Fatal("clearToDropzone must ClearInspect") } } + +func TestSiblingFolderConsentLifecycle(t *testing.T) { + if distribution.AppleAppStore { + t.Skip("synthetic scopes use non-Store resolver") + } + for _, mode := range []string{"cancel-panel", "cancel-scan", "replace", "merge", "replay", "multiple", "directory-grant"} { + t.Run(mode, func(t *testing.T) { + v := newTestViewer(t) + files := uitest.TempDirJPEGURIs(t, "a.jpg", "b.jpg") + record := fileaccess.Record{URI: files[0].String(), Bookmark: []byte("file")} + if mode == "directory-grant" { + record.Directory = true + record.Relative = files[0].Name() + } + source, err := fileaccess.FromRecord(record) + if err != nil { + t.Fatal(err) + } + entered, returned := make(chan struct{}), make(chan struct{}) + release := sync.OnceFunc(func() { close(returned) }) + t.Cleanup(release) + v.authorizeSiblingFolder = func(ctx context.Context, inputs []fyne.URI) ([]fyne.URI, error) { + close(entered) + if mode == "cancel-panel" { + return inputs, nil + } + if mode != "cancel-scan" && mode != "replace" { + t.Error("unexpected folder consent") + return inputs, nil + } + <-returned + if ctx.Err() == nil { + t.Error("retired opening did not cancel folder consent") + } + granted, err := fileaccess.FromRecord(fileaccess.Record{URI: inputs[0].String(), Bookmark: []byte("folder"), Directory: true, Relative: inputs[0].Name()}) + return []fyne.URI{granted}, err + } + if mode == "merge" { + dropAndWait(t, v, uitest.TempJPEGURI(t, "existing.jpg", 4, 4, color.White)) + v.SetMergeMode(true) + } + switch mode { + case "replay": + v.openCollection([]fyne.URI{source}, "", replayCollection) + case "multiple": + v.handleDrop([]fyne.URI{source, files[1]}) + default: + v.handleDrop([]fyne.URI{source}) + } + if mode == "cancel-scan" || mode == "replace" { + select { + case <-entered: + case <-time.After(testTimeout): + t.Fatal("folder consent did not begin") + } + pending := v.scanWorkerDone.Current() + if mode == "cancel-scan" { + v.cancelScan() + } else { + dropAndWait(t, v, uitest.TempJPEGURI(t, "replacement.jpg", 4, 4, color.White)) + } + release() + ctx, cancel := context.WithTimeout(t.Context(), testTimeout) + defer cancel() + if err := pending.Wait(ctx); err != nil { + t.Fatal(err) + } + v.scanUI.Drain() + if mode == "cancel-scan" && v.FileCount() != 0 { + t.Fatal("cancelled consent populated collection") + } + if mode == "replace" && v.state.Observe().DisplayFiles()[v.state.Observe().index].Name() != "replacement.jpg" { + t.Fatal("late consent replaced current image") + } + return + } + waitForScan(t, v) + waitForSort(t, v) + waitUntilLoaded(t, v) + want := 2 + if mode == "cancel-panel" || mode == "replay" { + want = 1 + } + if v.FileCount() != want { + t.Fatalf("files=%d, want %d", v.FileCount(), want) + } + if mode == "cancel-panel" { + select { + case <-entered: + default: + t.Fatal("folder consent did not run") + } + if string(fileaccess.Snapshot(v.state.Observe().DisplayFiles()[0]).Bookmark) != "file" { + t.Fatal("cancelled panel changed source authority") + } + } + }) + } +} diff --git a/internal/ui/viewer.go b/internal/ui/viewer.go index 0f060b9e..4122e84e 100644 --- a/internal/ui/viewer.go +++ b/internal/ui/viewer.go @@ -1,6 +1,7 @@ package ui import ( + "context" "fmt" "sync" "sync/atomic" @@ -406,6 +407,9 @@ type viewer struct { openChooserWorkers sync.WaitGroup openChooserClosed bool + // Captured by an admitted scan before worker-side native folder consent. + authorizeSiblingFolder func(context.Context, []fyne.URI) ([]fyne.URI, error) + // reveal is begun by revealCurrentFile (reveal.go) and finished once // that goroutine has fully run, error toast included - the same // contract clipboard keeps, for the same reason: every file manager diff --git a/packaging/apple-app-store/README.md b/packaging/apple-app-store/README.md index d3fc50a1..a472ba6a 100644 --- a/packaging/apple-app-store/README.md +++ b/packaging/apple-app-store/README.md @@ -195,10 +195,12 @@ positive qualification. The overlay and diagnostic driver are never shipped. ## Single-image folder navigation -In the Store Open dialog, choosing one image now offers a second native folder -permission panel. Confirm **Allow Folder Access** for the image's containing -folder to enable Left/Right browsing. The selected image remains the initial -image; siblings retain the confirmed directory bookmark for later access. -Cancel opens only the selected image. Folder and multiple-file selections do not -show this extra panel. Finder/Open With and drag/drop do not yet offer equivalent -folder consent; opening a folder explicitly remains the available route there. +Opening a single image in the Store build offers a native folder permission panel +when sibling discovery needs a wider grant. This shared path covers the Open +dialog, window drops and Open With/Dock delivery. Confirm **Allow Folder Access** +for the containing folder to enable Left/Right browsing. The selected image stays +on screen initially; siblings retain the confirmed directory bookmark. +Cancel opens only the selected image. Existing directory grants, folders, +multiple-file selections, saved-collection replay and merge additions do not +request this extra permission. A replaced or cancelled opening discards late +permission results. Native presentation is tracked without blocking shutdown. diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index dc256b4c..eb7103bb 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -1205,3 +1205,53 @@ existing shard assignments remain exact. Source-local test duplication exclusion are unchanged. Final review: permission scope remains explicit, original image identity survives consent, cancellation does not widen access, and no production sandbox policy is relaxed. CI results for the next pushed commit remain pending. + +### User regression — dropped single files need the same folder consent + +Ronin confirmed the Open-dialog fix, then reproduced the same symptom with one +file dropped on the dropzone. Extend consent at the shared collection-discovery +boundary, after native URL capture and before sibling scan. This also covers OS +Open With. Move consent out of Choose so there is no duplicate prompt. Ask only +for a single protected image being expanded, excluding merge/replay/multiple/ +directory inputs. Preserve native permission code and the folder policy guards. + +T3 Deep continuation, T0 inline, zero spawns/two reviews. A per-viewer consent +function is captured on UI; the existing scan worker owns capture/permission/scan. +Reserve its native-prefix membership in openChooserWorkers on UI before launch; +retire that prefix before scanning, including every early error. Context is checked +before native presentation and after its return; the scan token still gates result +delivery. No extra goroutine or blocking shutdown wait. Tests stub per-viewer +native consent, never the real desktop. The root harness installs that stub and +already cancels scans before draining native and scan workers. Verify the original +Left/Right test through picker/drop/OS delivery with native selected-file records, +plus cancellation/replacement and no prompts for merge/replay. Rebuild a new E2E +artifact once focused race tests, Store native compile and GoLand pass. + +Shared-consent verification: picker/drop/OS selected-file tests initially failed +with Right still showing a.jpg. All now pass with consent in the shared path. +Focused race suite covering opening, discovery and lifetimes passed (UI 16.466s, +filepicker 1.290s); Store-tagged picker passed (0.279s). Native-prefix worker counts +are reserved on UI and retired before scanning; harness cancellation precedes +both native and scan waits. Existing folder authority bypasses the prompt. +GoLand inspected all eight changed production/harness/test files; only existing +exact-excluded test duplication remains. `make verify-build` passes. The direct +host shard check correctly exposed macOS-only tests absent from the Linux manifest; +required `make check-test-shards` via Docker then passed all 750 tests/three shards. + +New universal E2E app: `bin/apple-store-e2e-drop-navigation/PicFetch.app`, with +TESTING.md and manifest; ZIP and SHA-256 are beside its output directory. All 13 +artifact guards pass (3.228s), and ZIP extraction preserves all 24 file hashes and +strict nested signatures. Earlier E2E artifacts remain untouched. User-facing +Open-dialog navigation was confirmed by Ronin; live drop/Open With confirmation +is pending for this newer bundle. + +The 38ec88e CI round passed both macOS architectures, the repaired Linux ui-3 +shard, CodeQL and validation. Windows showed the path fixtures still lacked a +Windows volume; changing separators alone was insufficient. Both affected tests +now derive absolute fixture roots from t.TempDir and compare fully qualified URI +paths. Focused local race tests pass (filepicker 1.469s, similarity 1.520s); actual +Windows confirmation remains pending the next CI round. Fresh Qodana has exactly +the same eight documented unused-function false positives and no new findings. +Final review covered consent ownership/currentness, no duplicate picker prompt, +merge/replay exclusions, native cancellation and artifact provenance. Two reviews, +zero agents. No Store submission or distribution-signing readiness is claimed. diff --git a/todos.md b/todos.md index 3c6c781e..9a8614cd 100644 --- a/todos.md +++ b/todos.md @@ -29,8 +29,9 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Qualify retained visual search through production ARM/Intel-Rosetta workers; keep scopes through clipboard/Finder/Trash calls and source metadata reads. - [x] Capture native window-drop URLs before GLFW flattens paths; refresh restored moved/stale bookmarks with immutable persistence coverage. - [x] Verify production workers deny an ungranted private source while an allowed control image still completes inference, on ARM and Intel under Rosetta. -- [x] Add explicit Store Open-dialog folder consent for single-image sibling navigation; preserve cancellation and the initially selected image. Rebuilt E2E app is in `bin/apple-store-e2e-folder-navigation/`. -- [ ] Confirm the native folder-consent interaction E2E; extend explicit folder consent to Finder/Open With and drag/drop. +- [x] Add explicit Store Open-dialog folder consent for single-image sibling navigation; preserve cancellation and the initially selected image. Latest rebuilt E2E app is in `bin/apple-store-e2e-drop-navigation/`. +- [x] Ronin confirmed native Open-dialog folder consent fixes navigation; extend the same consent to shared drop/Open With discovery with cancellation and replacement coverage. +- [ ] Confirm the rebuilt shared-consent app with a single-file window drop and Open With. - [x] Validate copyright/category metadata and reject quarantine attributes in local Store packages. - [ ] Finish GUI sandbox workflows, moved-folder relaunch, and physical Intel/minimum-OS validation. - [ ] Add the signed Store packaging route and validate the final artifact on Intel and Apple Silicon. From b98d848a0df465bde7acddd28f6e88f7b4f4551f Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 10:20:59 +0200 Subject: [PATCH 12/63] Restore Store guard inventory and probe multiple-file navigation --- .../distribution/distribution_store_test.go | 2 +- internal/ui/step_test.go | 45 +++++++++++++++++++ plans/2026-09-29-apple-app-store.md | 27 +++++++++++ todos.md | 1 + 4 files changed, 74 insertions(+), 1 deletion(-) diff --git a/internal/distribution/distribution_store_test.go b/internal/distribution/distribution_store_test.go index d2afc00b..5ea54a00 100644 --- a/internal/distribution/distribution_store_test.go +++ b/internal/distribution/distribution_store_test.go @@ -4,7 +4,7 @@ package distribution import "testing" -func TestStoreManaged_StoreBuildIsTrue(t *testing.T) { +func TestStoreManaged_MicrosoftStoreBuildIsTrue(t *testing.T) { if !StoreManaged { t.Error("Store build does not report Store-managed updates") } diff --git a/internal/ui/step_test.go b/internal/ui/step_test.go index a37eecf2..9c036c09 100644 --- a/internal/ui/step_test.go +++ b/internal/ui/step_test.go @@ -87,6 +87,51 @@ func TestStepImage_SingleFileDropWalksFolderSiblings(t *testing.T) { } func TestHandleKeyEvent_LeftRightWalkFolderSiblings(t *testing.T) { + for _, route := range []string{"picker", "drop", "os"} { + t.Run(route+"/multiple-selected-files", func(t *testing.T) { + if distribution.AppleAppStore { + t.Skip("synthetic bookmark requires non-Store resolver; native grants are qualified separately") + } + v := newTestViewer(t) + files := uitest.TempDirJPEGURIs(t, "a.jpg", "b.jpg", "c.jpg") + selected := make([]fyne.URI, 2) + for i, original := range files[:2] { + selected[i] = fileaccess.NewSelection(original, func(_ context.Context) (fileaccess.Record, error) { + return fileaccess.Record{URI: original.String(), Bookmark: []byte(original.Name())}, nil + }, func() {}) + } + v.authorizeSiblingFolder = func(_ context.Context, inputs []fyne.URI) ([]fyne.URI, error) { + t.Error("selected files should not require additional folder consent") + return inputs, nil + } + switch route { + case "picker": + uitest.StubChooser(t, selected, nil) + v.openFileDialog() + settleChooser(t, v) + case "drop": + v.handleDrop(selected) + case "os": + v.openFilesFromOS(selected) + } + waitForScan(t, v) + waitForSort(t, v) + waitUntilLoaded(t, v) + if got := v.FileCount(); got != 2 { + t.Fatalf("selected image count = %d, want 2", got) + } + for _, step := range []struct { + key fyne.KeyName + want string + }{{fyne.KeyRight, "b.jpg"}, {fyne.KeyLeft, "a.jpg"}} { + v.handleKeyEvent(&fyne.KeyEvent{Name: step.key}) + waitUntilLoaded(t, v) + if got := v.state.Observe().DisplayFiles()[v.state.Observe().index].Name(); got != step.want { + t.Fatalf("%s showing %q, want %s", step.key, got, step.want) + } + } + }) + } for _, route := range []string{"picker", "drop", "os"} { for _, scoped := range []bool{false, true} { name := route + "/ordinary" diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index eb7103bb..f02df5c5 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -1255,3 +1255,30 @@ the same eight documented unused-function false positives and no new findings. Final review covered consent ownership/currentness, no duplicate picker prompt, merge/replay exclusions, native cancellation and artifact provenance. Two reviews, zero agents. No Store submission or distribution-signing readiness is claimed. + +### Follow-up — native guard inventory and multiple-selection report + +75356d9 CI passed every Linux race partition, Linux native guards, both macOS +architectures, validation, CodeQL and FOSSA. Windows passed the repaired path +fixtures, then stopped because the Store inventory requires +TestStoreManaged_MicrosoftStoreBuildIsTrue but the test was renamed to +TestStoreManaged_StoreBuildIsTrue. Thin T0 fix: restore the explicit Microsoft +name in the existing test; no production change. Store-tagged test discovery now +selects that exact required name and scripts/nativeguards tests pass (0.455s). +GoLand inspected the changed distribution test with no findings. Actual Windows +confirmation awaits the next CI run. Zero agents, one review. + +Fresh 75356d9 Qodana post-suppression /qodana.sarif.json has nine unused-exported- +function warnings: the eight previously recorded false positives plus +filepicker.AuthorizeSiblingFolder, assigned in internal/ui/build.go:115. Rechecked +all nine callers; none is unused. No new actionable finding or broad suppression. + +Ronin subsequently reported multiple-file Open-dialog/drop navigation still +failing. A new focused diagnostic subcase in the existing Left/Right test opens +two distinct selected-file bookmarks through picker/drop/OS; all three retain +two images and navigate Right to b.jpg then Left to a.jpg (UI 0.793s). This does +not reproduce the packaged-app report. Clarification is pending whether selected +images themselves cannot be navigated or unselected folder siblings are missing. +Do not widen multi-selection to whole-folder discovery without resolving that +ambiguity. Latest E2E app remains unchanged; no fix is claimed. GoLand inspected +the diagnostic test: only the existing exact-excluded duplicate fixture remains. diff --git a/todos.md b/todos.md index 9a8614cd..f49e20b1 100644 --- a/todos.md +++ b/todos.md @@ -32,6 +32,7 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Add explicit Store Open-dialog folder consent for single-image sibling navigation; preserve cancellation and the initially selected image. Latest rebuilt E2E app is in `bin/apple-store-e2e-drop-navigation/`. - [x] Ronin confirmed native Open-dialog folder consent fixes navigation; extend the same consent to shared drop/Open With discovery with cancellation and replacement coverage. - [ ] Confirm the rebuilt shared-consent app with a single-file window drop and Open With. +- [ ] Diagnose Ronin's multiple-file picker/drop navigation report; selected-file navigation passes the automated harness, packaged symptom needs clarification. - [x] Validate copyright/category metadata and reject quarantine attributes in local Store packages. - [ ] Finish GUI sandbox workflows, moved-folder relaunch, and physical Intel/minimum-OS validation. - [ ] Add the signed Store packaging route and validate the final artifact on Intel and Apple Silicon. From 62661a19291e5f936dc02751b88914158c50beaa Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 11:40:01 +0200 Subject: [PATCH 13/63] Remember folder approvals and prepare signed Store packaging --- .github/workflows/ci.yml | 2 +- ARCHITECTURE.md | 13 +- Makefile | 5 +- internal/filepicker/filepicker.go | 22 -- internal/filepicker/folders.go | 141 +++++++++++ internal/filepicker/folders_test.go | 165 +++++++++++++ internal/ui/build.go | 2 +- packaging/apple-app-store/README.md | 63 ++++- plans/2026-09-29-apple-app-store.md | 140 ++++++++++- qodana.yaml | 1 + scripts/apple-store-preflight.sh | 26 +- scripts/macstorepackage/distribution.py | 237 ++++++++++++++++++ scripts/macstorepackage/package.py | 13 +- scripts/macstorepackage/test_distribution.py | 245 +++++++++++++++++++ todos.md | 8 +- 15 files changed, 1032 insertions(+), 51 deletions(-) create mode 100644 internal/filepicker/folders.go create mode 100644 internal/filepicker/folders_test.go create mode 100644 scripts/macstorepackage/distribution.py create mode 100644 scripts/macstorepackage/test_distribution.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 63fcfbae..7778d77f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -43,7 +43,7 @@ jobs: - name: Check Apple Store packaging policy # Portable policy/resource tests run here; signed native artifact guards # require a locally built macOS bundle and are qualified separately. - run: python3 -m unittest discover -s scripts/macstorepackage -p test_package.py + run: python3 -m unittest discover -s scripts/macstorepackage - name: Check search review controls run: | diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 6eaacb3a..92e52549 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -309,6 +309,14 @@ original app or accessing an Apple account. The package builder also verifies an stages exact upstream Abseil privacy/license/provenance inputs from `packaging/apple-app-store/privacy/abseil` into a macOS resource bundle; final artifact validation refuses missing or changed declarations even after re-signing. +`distribution.py`, through `make apple-store-package-signed`, consumes a qualified +local app without modifying it. It checks explicit Store identities and profile +fields, signs inside-out with exact per-role claims, verifies Apple-anchored code +identities, builds a component installer and checks its signature and expanded +payload. Profile field checks do not establish Apple's CMS/DER or submission +acceptance. `test_distribution.py` covers policy and assembly, with opt-in native +unsigned installer roundtrip and ad-hoc signature rejection guards. Neither route +uploads or installs the result. `packaging/tools.mk` owns reviewed CLI versions and multiarchitecture image digests consumed by Makefile and the release/Store workflows. @@ -1120,13 +1128,14 @@ PNG image data (`CopyImage`) and file-reference lists (`CopyFiles`). ### `internal/filepicker` -Native open chooser (`Choose`), sibling-folder authorization (`AuthorizeSiblingFolder`) +Native open chooser (`Choose`), persistent sibling-folder authorization (`FolderAuthorizer.AuthorizeSiblingFolder`) and save panel (`ChooseSave`). Linux/macOS can pick folders; Windows is files-only. | File | Responsibility | |------|----------------| -| `filepicker.go` | Typed `Choose` (URI list) / `ChooseSave` (one URI), strict native result decoding, explicit `AuthorizeSiblingFolder` consent policy for shared collection discovery, canonical-path Zenity framing and UTF-8 JSON PowerShell transport. | +| `filepicker.go` | Typed `Choose` (URI list) / `ChooseSave` (one URI), strict native result decoding, explicit folder consent policy for shared collection discovery, canonical-path Zenity framing and UTF-8 JSON PowerShell transport. | +| `folders.go` | Per-viewer `FolderAuthorizer` stores explicit sibling-folder bookmarks in app preferences; resolves and validates saved grants on tracked opening workers, refreshes moved/stale authority, and falls back to native consent. Native scopes are bounded, and preference locks never cover native calls. | | `darwin.go` / `other.go` | `NSOpenPanel` / `NSSavePanel`, including cancellable Store sibling-folder permission, with a shared NSURL-to-JSON transport / stubs; `darwin_test.go` exercises the actual native serializer. | | `windows.go` / `notwindows.go` | `hideConsoleWindow` pair. | diff --git a/Makefile b/Makefile index 58e61c25..8a92821c 100644 --- a/Makefile +++ b/Makefile @@ -66,7 +66,7 @@ COVERAGE_HTML := $(COVERAGE_DIR)/coverage.html .PHONY: verify-build --skip-local-tests .PHONY: generate-updater-notices check-updater-notices generate-avif-notices check-avif-notices .PHONY: fossa-findings -.PHONY: apple-store-preflight apple-worker-test apple-store-package-local +.PHONY: apple-store-preflight apple-worker-test apple-store-package-local apple-store-package-signed .PHONY: loc all: build @@ -109,6 +109,9 @@ apple-worker-test: ## Qualify signed Mac XPC worker isolation and retirement wit apple-store-package-local: install-fyne generate-tag-vectors ## Build an ad-hoc universal sandbox app for local qualification (see packaging/apple-app-store/README.md) python3 scripts/macstorepackage/package.py +apple-store-package-signed: ## Sign a qualified Store app and package a candidate installer (requires Apple credentials) + python3 scripts/macstorepackage/distribution.py + apple-store-preflight: ## Check native Xcode and Mac App Store signing inputs (see packaging/apple-app-store/README.md) bash scripts/apple-store-preflight.sh diff --git a/internal/filepicker/filepicker.go b/internal/filepicker/filepicker.go index bb7a6e66..b1f84df1 100644 --- a/internal/filepicker/filepicker.go +++ b/internal/filepicker/filepicker.go @@ -7,7 +7,6 @@ package filepicker import ( - "context" "encoding/json" "errors" "fmt" @@ -312,24 +311,3 @@ func authorizeSiblingFolder(files []fyne.URI, choose func(string) (fyne.URI, err } return []fyne.URI{child}, nil } - -// AuthorizeSiblingFolder optionally obtains explicit directory access for a -// captured single-image input. Call on a tracked worker, after admission for -// sibling discovery; saved collections and merge inputs do not need this grant. -func AuthorizeSiblingFolder(ctx context.Context, files []fyne.URI) ([]fyne.URI, error) { - if err := ctx.Err(); err != nil { - return nil, err - } - // The channel is fixed at build time, independently of the host inspector. - //goland:noinspection GoBoolExpressions - if !distribution.AppleAppStore { - return files, nil - } - files, err := authorizeSiblingFolder(files, func(directory string) (fyne.URI, error) { - return chooseSiblingFolderDarwin(ctx, directory) - }) - if cancelled := ctx.Err(); cancelled != nil { - return nil, cancelled - } - return files, err -} diff --git a/internal/filepicker/folders.go b/internal/filepicker/folders.go new file mode 100644 index 00000000..17979419 --- /dev/null +++ b/internal/filepicker/folders.go @@ -0,0 +1,141 @@ +package filepicker + +import ( + "context" + "encoding/json" + "path/filepath" + "sync" + + "fyne.io/fyne/v2" + "fyne.io/fyne/v2/storage" + + "github.com/frathe/picfetch/internal/distribution" + "github.com/frathe/picfetch/internal/fileaccess" +) + +const keySiblingFolderGrants = "appleSiblingFolderGrants" + +// FolderAuthorizer remembers explicit sibling-folder approvals across launches. +// Its methods run on the caller's tracked opening worker. +type FolderAuthorizer struct { + mu sync.Mutex + prefs fyne.Preferences + choose func(context.Context, string) (fyne.URI, error) + acquire func(context.Context, fyne.URI) (fyne.URI, func(), error) +} + +func NewFolderAuthorizer(prefs fyne.Preferences) *FolderAuthorizer { + return &FolderAuthorizer{prefs: prefs, choose: chooseSiblingFolderDarwin, acquire: fileaccess.Acquire} +} + +// AuthorizeSiblingFolder is only for admitted single-image sibling discovery. +func (a *FolderAuthorizer) AuthorizeSiblingFolder(ctx context.Context, files []fyne.URI) ([]fyne.URI, error) { + if err := ctx.Err(); err != nil { + return nil, err + } + // The channel is fixed at build time, independently of the host inspector. + //goland:noinspection GoBoolExpressions + if !distribution.AppleAppStore { + return files, nil + } + return a.authorize(ctx, files) +} + +func (a *FolderAuthorizer) authorize(ctx context.Context, files []fyne.URI) ([]fyne.URI, error) { + if err := ctx.Err(); err != nil { + return nil, err + } + var approved fyne.URI + permitted, err := authorizeSiblingFolder(files, func(directory string) (fyne.URI, error) { + if folder, err := a.recall(ctx, directory); folder != nil || err != nil { + return folder, err + } + var err error + approved, err = a.choose(ctx, directory) + return approved, err + }) + if cancelled := ctx.Err(); cancelled != nil { + return nil, cancelled + } + if err != nil { + return nil, err + } + if approved != nil { + // The policy above validated the explicitly selected folder. + a.remember(ctx, "", fileaccess.Snapshot(approved)) + } + return permitted, nil +} + +func (a *FolderAuthorizer) recall(ctx context.Context, directory string) (fyne.URI, error) { + a.mu.Lock() + records := a.load() + a.mu.Unlock() + for _, record := range records { + if err := ctx.Err(); err != nil { + return nil, err + } + // Paths are indexes only: native resolution is the authority and can move. + if !record.Directory || record.Relative != "" || len(record.Bookmark) == 0 { + continue + } + saved, err := fileaccess.FromRecord(record) + if err != nil { + continue + } + resolved, release, err := a.acquire(ctx, saved) + if err != nil { + continue + } + directoryExists, statErr := storage.CanList(resolved) + matches := statErr == nil && directoryExists && filepath.Clean(resolved.Path()) == filepath.Clean(directory) + refreshed := fileaccess.Snapshot(resolved) + release() + if err := ctx.Err(); err != nil { + return nil, err + } + if matches { + a.remember(ctx, record.URI, refreshed) + return resolved, nil + } + } + return nil, ctx.Err() +} + +// load and remember serialize preference updates only; native calls never hold mu. +func (a *FolderAuthorizer) load() []fileaccess.Record { + raw := a.prefs.String(keySiblingFolderGrants) + if raw == "" { + return nil + } + var records []fileaccess.Record + if err := json.Unmarshal([]byte(raw), &records); err != nil { + fyne.LogError("Could not read saved folder access", err) + return nil + } + return records +} + +func (a *FolderAuthorizer) remember(ctx context.Context, previous string, record fileaccess.Record) { + a.mu.Lock() + defer a.mu.Unlock() + if ctx.Err() != nil { + return + } + records := a.load() + updated := make([]fileaccess.Record, 0, len(records)+1) + for _, saved := range records { + if saved.URI != previous && saved.URI != record.URI { + updated = append(updated, saved) + } + } + updated = append(updated, record) + encoded, err := json.Marshal(updated) + if err != nil { + fyne.LogError("Could not save folder access", err) + return + } + if string(encoded) != a.prefs.String(keySiblingFolderGrants) { + a.prefs.SetString(keySiblingFolderGrants, string(encoded)) + } +} diff --git a/internal/filepicker/folders_test.go b/internal/filepicker/folders_test.go new file mode 100644 index 00000000..ae303ca2 --- /dev/null +++ b/internal/filepicker/folders_test.go @@ -0,0 +1,165 @@ +package filepicker + +import ( + "context" + "encoding/json" + "errors" + "path/filepath" + "slices" + "testing" + + "fyne.io/fyne/v2" + "fyne.io/fyne/v2/storage" + "fyne.io/fyne/v2/test" + + "github.com/frathe/picfetch/internal/fileaccess" +) + +func folderFixture(t *testing.T, name string, directory bool) fyne.URI { + t.Helper() + uri, err := fileaccess.FromRecord(fileaccess.Record{URI: storage.NewFileURI(name).String(), Bookmark: []byte(name), Directory: directory}) + if err != nil { + t.Fatal(err) + } + return uri +} + +func TestFolderApprovalSurvivesNewAuthorizer(t *testing.T) { + app := test.NewApp() + t.Cleanup(app.Quit) + root := t.TempDir() + folder := folderFixture(t, root, true) + first := folderFixture(t, filepath.Join(root, "first.jpg"), false) + second := folderFixture(t, filepath.Join(root, "second.jpg"), false) + prompts, active := 0, 0 + for _, file := range []fyne.URI{first, second} { + authorizer := NewFolderAuthorizer(app.Preferences()) + authorizer.choose = func(_ context.Context, _ string) (fyne.URI, error) { + prompts++ + return folder, nil + } + authorizer.acquire = func(_ context.Context, uri fyne.URI) (fyne.URI, func(), error) { + active++ + return uri, func() { active-- }, nil + } + got, err := authorizer.authorize(context.Background(), []fyne.URI{file}) + if err != nil || len(got) != 1 { + t.Fatalf("result %v, %v", got, err) + } + record := fileaccess.Snapshot(got[0]) + if got[0].String() != file.String() || !record.Directory || record.Relative != file.Name() { + t.Fatalf("lost selected image or folder authority: %+v", record) + } + if active != 0 { + t.Fatal("native scope retained after authorization") + } + } + if prompts != 1 { + t.Fatalf("folder prompts across launches = %d, want 1", prompts) + } +} + +func TestSavedFolderApprovalValidation(t *testing.T) { + for _, scenario := range []string{"renewed", "moved", "wrong-folder", "revoked", "offline", "malformed", "file-grant", "cancel-lookup", "cancel-panel", "multiple"} { + t.Run(scenario, func(t *testing.T) { + app := test.NewApp() + t.Cleanup(app.Quit) + root, other := t.TempDir(), t.TempDir() + folder := folderFixture(t, root, true) + file := folderFixture(t, filepath.Join(root, "image.jpg"), false) + record := fileaccess.Snapshot(folder) + if scenario == "moved" { + record.URI = storage.NewFileURI(filepath.Join(other, "old")).String() + } + if scenario == "file-grant" { + record = fileaccess.Snapshot(file) + } + encoded, err := json.Marshal([]fileaccess.Record{record}) + if err != nil { + t.Fatal(err) + } + original := string(encoded) + if scenario == "malformed" { + original = "invalid json" + } + app.Preferences().SetString(keySiblingFolderGrants, original) + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + authorizer := NewFolderAuthorizer(app.Preferences()) + prompts, acquired, released := 0, 0, 0 + authorizer.choose = func(_ context.Context, _ string) (fyne.URI, error) { + prompts++ + if scenario == "cancel-panel" { + cancel() + return folder, nil + } + return nil, nil + } + authorizer.acquire = func(_ context.Context, _ fyne.URI) (fyne.URI, func(), error) { + if scenario == "revoked" || scenario == "cancel-panel" { + return nil, nil, errors.New("revoked") + } + acquired++ + resolved := fileaccess.Snapshot(folder) + resolved.Bookmark = []byte("renewed") + if scenario == "wrong-folder" { + resolved.URI = storage.NewFileURI(other).String() + } + if scenario == "offline" { + resolved.URI = storage.NewFileURI(filepath.Join(other, "missing")).String() + } + uri, err := fileaccess.FromRecord(resolved) + if err != nil { + t.Fatal(err) + } + if scenario == "cancel-lookup" { + cancel() + } + return uri, func() { released++ }, nil + } + inputs := []fyne.URI{file} + if scenario == "multiple" { + inputs = append(inputs, folderFixture(t, filepath.Join(root, "second.jpg"), false)) + } + got, err := authorizer.authorize(ctx, inputs) + if acquired != released { + t.Fatalf("scopes acquired %d released %d", acquired, released) + } + switch scenario { + case "renewed", "moved": + if err != nil || len(got) != 1 || prompts != 0 || string(fileaccess.Snapshot(got[0]).Bookmark) != "renewed" { + t.Fatalf("saved access not reused: result=%v prompts=%d err=%v", got, prompts, err) + } + var saved []fileaccess.Record + if err := json.Unmarshal([]byte(app.Preferences().String(keySiblingFolderGrants)), &saved); err != nil { + t.Fatal(err) + } + if len(saved) != 1 || saved[0].URI != folder.String() || string(saved[0].Bookmark) != "renewed" || saved[0].Relative != "" { + t.Fatalf("renewal not persisted: %+v", saved) + } + case "cancel-lookup", "cancel-panel": + if !errors.Is(err, context.Canceled) || got != nil { + t.Fatalf("cancelled operation delivered %v, %v", got, err) + } + wantPrompts := 0 + if scenario == "cancel-panel" { + wantPrompts = 1 + } + if prompts != wantPrompts { + t.Fatalf("prompts %d, want %d", prompts, wantPrompts) + } + default: + wantPrompts := 1 + if scenario == "multiple" { + wantPrompts = 0 + } + if err != nil || !slices.Equal(got, inputs) || prompts != wantPrompts { + t.Fatalf("selection=%v prompts=%d err=%v", got, prompts, err) + } + } + if scenario != "renewed" && scenario != "moved" && app.Preferences().String(keySiblingFolderGrants) != original { + t.Fatal("unsuccessful or cancelled authorization changed saved grants") + } + }) + } +} diff --git a/internal/ui/build.go b/internal/ui/build.go index 2d83fe96..943d2463 100644 --- a/internal/ui/build.go +++ b/internal/ui/build.go @@ -112,7 +112,7 @@ func buildViewer(application fyne.App, startup startupState) (*viewer, fyne.Wind updateExecutable: os.Executable, } - view.authorizeSiblingFolder = filepicker.AuthorizeSiblingFolder + view.authorizeSiblingFolder = filepicker.NewFolderAuthorizer(application.Preferences()).AuthorizeSiblingFolder view.chooserUI = fyneChooserQueue{} view.scanUI = fyneChooserQueue{} view.osInputs.queue = fyneChooserQueue{} diff --git a/packaging/apple-app-store/README.md b/packaging/apple-app-store/README.md index a472ba6a..81a755fe 100644 --- a/packaging/apple-app-store/README.md +++ b/packaging/apple-app-store/README.md @@ -68,13 +68,14 @@ Run the read-only prerequisite check: APPLE_STORE_TEAM_ID=YOURTEAMID \ APPLE_STORE_TESTFLIGHT=1 \ APPLE_STORE_PROFILE=/absolute/path/PicFetch.provisionprofile \ +APPLE_STORE_WORKER_PROFILE=/absolute/path/PicFetchWorker.provisionprofile \ make apple-store-preflight ``` It uses `/Applications/Xcode.app/Contents/Developer` without changing the system developer-directory selection. Override `APPLE_STORE_DEVELOPER_DIR` for another full Xcode installation. It checks for Store application and installer signing -identities with private keys. `APPLE_STORE_TESTFLIGHT=1` requires a profile file; +identities with private keys. `APPLE_STORE_TESTFLIGHT=1` requires app and XPC worker profile files for this route; without TestFlight, an app using only unrestricted macOS entitlements may omit a profile, per [Apple TN3125](https://developer.apple.com/documentation/technotes/tn3125-inside-code-signing-provisioning-profiles). It does not install credentials, access an Apple account, upload a package or @@ -122,7 +123,7 @@ Required inputs: authorization separately; this preparation does not publish anything. The local target below is explicitly an ad-hoc qualification route; distribution -signing, installer packaging and submission validation are still separate work. A successful prerequisite check cannot +real distribution signing and submission validation are still separate qualification work. A successful prerequisite check cannot substitute for signed sandbox or submission evidence. @@ -199,8 +200,64 @@ Opening a single image in the Store build offers a native folder permission pane when sibling discovery needs a wider grant. This shared path covers the Open dialog, window drops and Open With/Dock delivery. Confirm **Allow Folder Access** for the containing folder to enable Left/Right browsing. The selected image stays -on screen initially; siblings retain the confirmed directory bookmark. +on screen initially; siblings retain the confirmed directory bookmark. The folder +approval is also saved in app preferences and reused for fresh single-image opens, +including after quitting and relaunching. Saved bookmarks are resolved and checked +before use; moved/stale bookmarks are refreshed. An unavailable or unusable grant +falls back to the permission panel. Earlier test builds did not keep this separate +approval history, so approve each folder once in the new build. Cancel opens only the selected image. Existing directory grants, folders, multiple-file selections, saved-collection replay and merge additions do not request this extra permission. A replaced or cancelled opening discards late permission results. Native presentation is tracked without blocking shutdown. + + +## Distribution-signed candidate packaging + +`make apple-store-package-signed` consumes a qualified local app and creates a +fresh output directory containing a signed app, `PicFetch.pkg` and provenance +manifest. It does not upload or install anything and does not modify the input +app. This tooling is implemented; a real Store-signed run still needs credentials +and remains unverified. Continue using `apple-store-package-local` for direct E2E +runs: Store distribution signatures are intended for Apple's distribution path. + +```sh +APPLE_STORE_APP=/absolute/path/local-output/PicFetch.app \ +APPLE_STORE_SIGNED_OUTPUT_DIR=/absolute/path/fresh-store-candidate \ +APPLE_STORE_TEAM_ID=YOURTEAMID \ +APPLE_STORE_APP_IDENTITY=APPLICATION_CERTIFICATE_SHA1 \ +APPLE_STORE_INSTALLER_IDENTITY=INSTALLER_CERTIFICATE_SHA1 \ +APPLE_STORE_TESTFLIGHT=1 \ +APPLE_STORE_PROFILE=/absolute/path/PicFetch.provisionprofile \ +APPLE_STORE_WORKER_PROFILE=/absolute/path/PicFetchWorker.provisionprofile \ +make apple-store-package-signed +``` + +Identity selectors are full uppercase SHA-1 certificate fingerprints from +Keychain's valid identity inventory. The application identity must be Apple +Distribution or 3rd Party Mac Developer Application; the installer identity must +be 3rd Party Mac Developer Installer, both for the chosen Team. Private keys stay +in Keychain; only public certificate bytes are read for fingerprint comparison. +No private-key export or Apple-account login is performed. + +This route requires explicit app (`io.github.frathe.picfetch`) and XPC worker +(`io.github.frathe.picfetch.worker`) profiles when TestFlight is requested. +Without TestFlight, these bundles' unrestricted sandbox claims may omit profiles. +Inherited helpers retain only sandbox/inherit and have no profiles. Supplied +profiles are copied as immutable bytes, decoded with `security cms`, and checked +for macOS, exact identifier/prefix, Team, validity, distribution scope and the +selected certificate. These checks are diagnostics; profile plist structure is +not a stable API or authoritative proof of CMS/DER acceptance. Apple's validation +remains required. See [TN3125](https://developer.apple.com/documentation/technotes/tn3125-inside-code-signing-provisioning-profiles) +and [manual distribution signing](https://developer.apple.com/documentation/xcode/creating-distribution-signed-code-for-the-mac/). + +Code is signed inside-out and checked against the Apple anchor, selected leaf +certificate, Team and code identifier. The installer uses `productbuild +--component` targeting `/Applications`. `pkgutil` checks its signature and the selected leaf certificate's SHA-256 fingerprint, then +expands it without installation; the sole app payload must match the verified +signed app in bytes and executable permissions. A missing or mismatched input +qualification manifest, existing output directory or failed validation stops +publication. Portable policy tests run in CI; the native installer roundtrip +uses an explicitly unsigned disposable installer and does not prove distribution +signing. The remaining SDK/privacy, real certificate/profile, TestFlight and +App Store submission gates remain open. diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index f02df5c5..8c2de5ed 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -1277,8 +1277,138 @@ Ronin subsequently reported multiple-file Open-dialog/drop navigation still failing. A new focused diagnostic subcase in the existing Left/Right test opens two distinct selected-file bookmarks through picker/drop/OS; all three retain two images and navigate Right to b.jpg then Left to a.jpg (UI 0.793s). This does -not reproduce the packaged-app report. Clarification is pending whether selected -images themselves cannot be navigated or unselected folder siblings are missing. -Do not widen multi-selection to whole-folder discovery without resolving that -ambiguity. Latest E2E app remains unchanged; no fix is claimed. GoLand inspected -the diagnostic test: only the existing exact-excluded duplicate fixture remains. +not reproduce the packaged-app report. Ronin clarified that the reported behavior +was reaching the selection boundaries and confirmed it is intentional: multiple +inputs must browse only the selected images, while a single-file open/drop may +discover folder siblings. Close this report as expected behavior; preserve the +selection-size and Left/Right coverage. No production change or rebuilt artifact +is needed for this report. Focused navigation race tests also passed (UI 6.128s). +GoLand inspected the diagnostic test: only the existing exact-excluded duplicate +fixture remains. This clarification supersedes the earlier pending diagnosis. + +### Persistent sibling-folder approval — implementation contract + +Request: reuse explicit folder approvals across fresh single-image opens and app +launches. Preserve multiple-selection/merge/replay behavior. Deep continuation, +T0 inline, zero spawns, two reviews; no dependency, entitlement or UI string change. +Apple's Accessing files from the macOS App Sandbox documentation confirms stored +security-scoped bookmarks require explicit resolution/start/stop on later use. +Existing fileaccess.Acquire already renews stale bookmarks and balances scopes. + +Task: add per-viewer filepicker.FolderAuthorizer backed by app preferences; +resolve saved directory bookmarks on the existing scan worker before prompting, +require the resolved directory to match the requested parent, save renewed data, +and persist only validated explicit folder consent. Invalid/missing/offline grants +fall back to the existing panel; cancellation must not publish a new grant. +Never infer permission from a matching pathname alone. Keep native scopes bounded +and no native work under the preference lock. Filepicker owns the grant preference +as part of native input authorization, independently of transient session data. + +Files: filepicker folder authorizer + tests, UI build wiring, architecture index, +exact Qodana test exclusion, evidence/todos/docs. Tests: a new authorizer instance +with persisted preferences suppresses the second prompt; unrelated/failed/moved +bookmarks, renewal, cancelled consent and multi-selection retain their contracts. +Verify focused picker/UI race tests, Store compile, GoLand changed files, +make verify-build and fresh universal Store test package/13 artifact guards. +Physical relaunch confirmation remains a user E2E check. Full native-amd64 race +suite remains CI-owned because the local Docker daemon is ARM. + +Persistent approval verification: the new-authorizer test first failed with two +prompts instead of one, then passed after persistence/reuse. Tests cover distinct +images across authorizer lifetimes, explicit selection identity, balanced scopes, +moved and renewed bookmarks, wrong resolved folders, revoked/offline/malformed +records, file-only grants, cancellation during lookup/panel, and multi-selection. +Renewal tests exposed duplicate trailing-slash records from deriving a folder +back from its child; fixed by persisting the validated native folder directly. +Isolated Go overlay mutations disabling folder matching and renewal persistence +both fail their targeted guards. Overlays are scratch-only .fixture files. + +Focused race results: filepicker 1.546s; root navigation/consent lifecycle 8.518s. +Store-tagged filepicker 0.364s. make verify-build passed formatting, notice checks, +vet and build. GoLand inspected all four changed code files including weak +warnings; corrected import ordering and reinspected the two new files cleanly. +The exact folders_test.go Qodana exclusion is added; no root top-level tests were +added, so the existing shard manifest remains unchanged. Two inline reviews, +zero agents. No new runtime/model dependency or entitlement; existing notices +and SDK/privacy release gates remain unchanged. + +Fresh universal bundle: bin/apple-store-e2e-persistent-folders/PicFetch.app; +ZIP/SHA256 beside the output directory, TESTING.md and source manifest inside it. +All 13 signed-artifact guards pass (3.179s); all 24 ZIP payload hashes match the +manifest. Existing E2E bundles preserved. Native relaunch/moved-folder UI evidence +remains pending Ronin; unit tests simulate native resolution. Previous builds do +not have a separate stored approval history, so one initial approval per folder +is expected in this build. Current runtime change remains uncommitted; previous +Secretive documentation signing was refused, and the earlier b98d848 push ended +with an SSH disconnect. Latest remote remains 75356d9; no fresh CI pass is claimed +for this change. The completed local package includes the uncommitted sources +and records their hashes explicitly. + +User E2E confirmation (2026-09-30): Ronin tested the persistent-folder build +and reported "Awesome yes it works perfectly!" in response to the requested +quit/relaunch check. Persistent approval reuse across an actual app relaunch is +confirmed. This does not establish moved-folder behavior, which remains a +separate live qualification item. No further runtime changes were needed. + +### Distribution packaging tooling — contract and limits + +Deep continuation, T0 inline, zero spawns/two reviews. Add a separate +make apple-store-package-signed route consuming an already-qualified local app +and writing only a fresh output directory. Never change the E2E bundle, upload, +install credentials, or install the output. Reuse existing native/entitlement/ +privacy verification and retain the input manifest/hash evidence. + +Select explicit valid application/installer identities for the requested Team; +reject wrong-role, absent or ambiguous identities. Decode supplied profiles with +security cms and diagnose macOS platform, exact app ID/prefix, Team, expiry, +distribution-only scope and allowed signing certificate. Profile plist inspection +is a local diagnostic, not proof of Apple's CMS/DER acceptance. Require app and +XPC profiles for this route's TestFlight option; keep inherited helpers unprofiled +with their exact sandbox/inherit entitlements. Sign inside-out, verify every code +item against the Apple anchor, selected leaf fingerprint and Team, then create a +productbuild --component /Applications installer. Validate installer signature +and expand it to check that its payload matches the signed app exactly. + +Primary references (2026-09-30): Apple TN3125 (profiles, macOS exceptions, profile +location) and Creating distribution-signed code for macOS (manual signing and +nested code), plus installed productbuild/pkgutil/codesign manuals. These are +version-sensitive build diagnostics. Existing SDK privacy gaps remain release +gates regardless of signing success. + +Files: scripts/macstorepackage distribution module/tests, narrow package.py +verification parameter, Make target, CI portable-test discovery, architecture, +README/todos/evidence. Verify portable policy tests, real native unsigned installer +payload roundtrip and negative guards, existing 13 artifact guards, GoLand +inspections and Make checks. Missing certificates/profiles prevent a true signed +artifact test; record that boundary rather than substitute ad-hoc evidence. + +Distribution-tooling verification (2026-09-30): policy tests initially failed with +13 missing-validation assertions. Completed policy/assembly tests cover wrong +identity role/Team/certificate, wildcard/wrong app IDs, legacy App ID prefixes, +expired/future/development profiles, exact helper/service claims, immutable +profile snapshots, TestFlight profile requirements, installer leaf fingerprint, +wrong destinations/scripts/extra or changed payloads and signing order. New +portable tests are included in CI discovery. No Apple credentials are read by +those tests; command assembly uses explicit mocks and is not signing evidence. + +Actual native productbuild --component/pkgutil --expand-full roundtrip of the +persistent-folder test app passed; extracted files and executable permissions +match and codesign --verify --deep --strict still passes. The disposable installer +was unsigned and never installed. A real ad-hoc artifact fails the new +Apple-anchored distribution identity requirement as expected. Full native/portable +packaging suite: 24 tests, 10.002s, all passed. Python scripts parse with the 3.11 +grammar; make fmt-check and Make target dry-run pass. Prior unchanged Go runtime +build/vet/race evidence carries forward from the persistent-approval section. + +GoLand inspected distribution.py, test_distribution.py, package.py, Makefile and +CI YAML with no findings; final changed Python files were reinspected cleanly. +Review added exact installer certificate SHA-256 comparison after pkgutil trust +validation, immutable profile decoding/embedding, and input-to-staged file +comparison before signing. Two reviews, zero agents. Preflight's TestFlight +check was aligned to require separate app/worker profiles; shell syntax passes. +Xcode 27.0 is available, but this invocation has no configured Team ID or either +TestFlight profile. Certificate availability is not established without the Team +input. Therefore real distribution signing, profile CMS/DER acceptance, installer +trust output against actual certificates, TestFlight and Apple SDK/privacy review +remain unverified. No distribution artifact, upload or installation was performed. +The user-confirmed local E2E app is unchanged by this tooling work. diff --git a/qodana.yaml b/qodana.yaml index 19f7c7f1..82c8698c 100644 --- a/qodana.yaml +++ b/qodana.yaml @@ -137,6 +137,7 @@ exclude: - "internal/filemanager/filemanager_test.go" - "internal/filepicker/darwin_test.go" - "internal/filepicker/filepicker_test.go" + - "internal/filepicker/folders_test.go" - "internal/filepicker/windows_test.go" - "internal/filescan/authority_test.go" - "internal/filescan/filescan_test.go" diff --git a/scripts/apple-store-preflight.sh b/scripts/apple-store-preflight.sh index daa6dd80..ce28bc4d 100644 --- a/scripts/apple-store-preflight.sh +++ b/scripts/apple-store-preflight.sh @@ -63,14 +63,24 @@ else fi fi -profile="${APPLE_STORE_PROFILE:-}" -if [ -z "$profile" ] && [ "${APPLE_STORE_TESTFLIGHT:-0}" != 1 ]; then - printf 'No profile supplied; unrestricted Mac App Store entitlements may omit it. TestFlight always requires one.\n' -elif [ -z "$profile" ] || [ ! -f "$profile" ]; then - fail 'Set APPLE_STORE_PROFILE to the Mac App Store Connect provisioning profile for io.github.frathe.picfetch.' -else - printf 'Provisioning profile supplied; its signature, identity, expiry and entitlements still require artifact validation.\n' -fi +for role in app worker; do + if [ "$role" = app ]; then + profile="${APPLE_STORE_PROFILE:-}" + variable=APPLE_STORE_PROFILE + identifier=io.github.frathe.picfetch + else + profile="${APPLE_STORE_WORKER_PROFILE:-}" + variable=APPLE_STORE_WORKER_PROFILE + identifier=io.github.frathe.picfetch.worker + fi + if [ -z "$profile" ] && [ "${APPLE_STORE_TESTFLIGHT:-0}" != 1 ]; then + printf 'No %s profile supplied; unrestricted Mac App Store entitlements may omit it. This TestFlight route requires both bundle profiles.\n' "$role" + elif [ -z "$profile" ] || [ ! -f "$profile" ]; then + fail "Set $variable to the Mac App Store Connect provisioning profile for $identifier." + else + printf '%s provisioning profile supplied; identity, expiry, entitlements and Apple acceptance remain to be validated.\n' "$role" + fi +done if [ "$failures" -ne 0 ]; then printf '%s preparation inputs remain unresolved.\n' "$failures" >&2 diff --git a/scripts/macstorepackage/distribution.py b/scripts/macstorepackage/distribution.py new file mode 100644 index 00000000..1ca63846 --- /dev/null +++ b/scripts/macstorepackage/distribution.py @@ -0,0 +1,237 @@ +"""Prepare a distribution-signed Store candidate without uploading or installing it. + +Profile plist checks are diagnostics. Apple's profile/SDK/submission acceptance +remains a separate gate; this tool does not claim to authenticate a CMS profile. +""" +import argparse +import copy +from datetime import datetime, timezone +import hashlib +import json +import os +from pathlib import Path +import plistlib +import re +import shutil +import ssl +import tempfile +import xml.etree.ElementTree as ET + +from package import APP_ID, ENTITLEMENTS, digest, run, verify, write_plist + + +def select_identity(output, fingerprint, team, role): + if not re.fullmatch(r"[A-F0-9]{40}", fingerprint) or not re.fullmatch(r"[A-Z0-9]{10}", team): + raise ValueError("explicit SHA-1 identity fingerprint and 10-character Team ID required") + identities = re.findall(r'^\s*\d+\) ([A-F0-9]{40}) "([^"\n]+)"\s*$', output, re.M) + selected = [name for key, name in identities if key == fingerprint] + prefixes = {"app": ("Apple Distribution: ", "3rd Party Mac Developer Application: "), + "installer": ("3rd Party Mac Developer Installer: ",)} + if (len(selected) != 1 or not selected[0].startswith(prefixes[role]) or + not selected[0].endswith("(" + team + ")") or + sum(name == selected[0] for _, name in identities) != 1): + raise ValueError(f"missing, ambiguous or wrong-team {role} Store identity") + return selected[0] + + +def profile_entitlements(profile, team, identifier, certificate, role, now): + if role not in ("app", "service") or not isinstance(profile, dict): + raise ValueError("profile requires an application or XPC service") + claims = profile.get("Entitlements", {}) + prefix = profile.get("ApplicationIdentifierPrefix", []) + certificates = profile.get("DeveloperCertificates", []) + if (not isinstance(claims, dict) or profile.get("TeamIdentifier") != [team] or + profile.get("Platform") != ["OSX"] or not isinstance(prefix, list) or + len(prefix) != 1 or not isinstance(prefix[0], str) or + not re.fullmatch(r"[A-Z0-9]{10}", prefix[0]) or + claims.get("com.apple.application-identifier") != prefix[0] + "." + identifier or + claims.get("com.apple.developer.team-identifier") != team): + raise ValueError("profile platform, Team or explicit application identifier mismatch") + if ("ProvisionedDevices" in profile or profile.get("ProvisionsAllDevices", False) is not False or + claims.get("get-task-allow", False) is not False or + claims.get("com.apple.security.get-task-allow", False) is not False): + raise ValueError("profile is not restricted to Store distribution") + created, expires = profile.get("CreationDate"), profile.get("ExpirationDate") + if not isinstance(created, datetime) or not isinstance(expires, datetime): + raise ValueError("profile lacks validity dates") + created = created.replace(tzinfo=timezone.utc) if created.tzinfo is None else created + expires = expires.replace(tzinfo=timezone.utc) if expires.tzinfo is None else expires + if not created <= now < expires: + raise ValueError("profile is expired or not yet valid") + if not isinstance(certificates, list) or certificate not in { + hashlib.sha1(cert).hexdigest().upper() for cert in certificates if isinstance(cert, bytes)}: + raise ValueError("profile does not authorize the selected signing certificate") + result = copy.deepcopy(ENTITLEMENTS[role]) + result.update({"com.apple.application-identifier": claims["com.apple.application-identifier"], + "com.apple.developer.team-identifier": team}) + if "beta-reports-active" in claims: + if claims["beta-reports-active"] is not True: + raise ValueError("unexpected beta-reports-active profile claim") + result["beta-reports-active"] = True + return result + + +def bundle_files(app): + result = {} + for path in sorted(app.rglob("*")): + if path.is_symlink(): + raise ValueError("unexpected bundle symlink") + if path.is_file(): + result[str(path.relative_to(app))] = {"sha256": digest(path), + "executable": bool(path.stat().st_mode & 0o111)} + return result + + +def verify_payload(expanded, app): + infos = list(expanded.rglob("PackageInfo")) + if len(infos) != 1: + raise ValueError("installer must contain exactly one component") + info = ET.parse(infos[0]).getroot() + if (info.get("identifier") != APP_ID or info.get("install-location") != "/Applications" or + info.find("scripts") is not None): + raise ValueError("unexpected installer identity, destination or scripts") + payload = infos[0].parent / "Payload" + if not payload.is_dir() or {p.name for p in payload.iterdir()} != {"PicFetch.app"}: + raise ValueError("unexpected installer payload") + if bundle_files(payload / "PicFetch.app") != bundle_files(app): + raise ValueError("installer payload differs from the verified signed app") + + +def verify_installer_report(report, name, fingerprint): + # pkgutil must also have exited successfully; parsing adds exact leaf selection. + leaf = re.findall(r"^\s*1\.\s+(.+?)\s*$", report, re.M) + status = re.findall(r"^\s*Status:\s*(.+?)\s*$", report, re.M) + accepted = ("signed by a developer certificate issued by Apple", + "signed by a certificate trusted by macOS", + "signed by a certificate trusted by Mac OS X") + if leaf != [name] or len(status) != 1 or not status[0].startswith(accepted): + raise ValueError("installer signature does not match the selected trusted identity") + leaf_block = re.search(r"^\s*1\..*?\n(.*?)(?=^\s*2\.|\Z)", report, re.M | re.S) + fingerprints = re.findall(r"SHA256 Fingerprint:\s*((?:[0-9A-Fa-f]{2}[ \t\r\n]*){32})", + leaf_block[1] if leaf_block else "") + if len(fingerprints) != 1 or re.sub(r"\s", "", fingerprints[0]).upper() != fingerprint: + raise ValueError("installer leaf certificate fingerprint mismatch") + + +def installer_certificate(name, fingerprint, env): + certificates = run("security", "find-certificate", "-a", "-c", name, "-p", capture=True, env=env).decode() + selected = [] + for pem in re.findall(r"-----BEGIN CERTIFICATE-----.*?-----END CERTIFICATE-----", certificates, re.S): + data = ssl.PEM_cert_to_DER_cert(pem) + if hashlib.sha1(data).hexdigest().upper() == fingerprint: + selected.append(data) + if len(selected) != 1: + raise ValueError("installer public certificate missing or ambiguous") + return hashlib.sha256(selected[0]).hexdigest().upper() + + +def code_items(app): + contents = app / "Contents" + service = contents / "XPCServices" / (APP_ID + ".worker.xpc") + return ([(p, None, p.name) for p in sorted((contents / "Frameworks").iterdir())] + + [(contents / "MacOS/picfetch-worker-client", "helper", APP_ID + ".worker-client"), + (service / "Contents/MacOS/picfetch-image-worker", "helper", APP_ID + ".image-worker"), + (service, "service", APP_ID + ".worker"), (app, "app", APP_ID)]) + + +def verify_code_identities(app, team, fingerprint, env): + for path, _, identifier in code_items(app): + requirement = (f'=anchor apple generic and certificate leaf = H"{fingerprint}" ' + f'and certificate leaf[subject.OU] = "{team}" and identifier "{identifier}"') + run("codesign", "--verify", "--strict", "--all-architectures", "-R", requirement, path, env=env) + + +def read_profile(path, env): + # Decode exactly the bytes later embedded, even if the source file changes. + data = path.read_bytes() + with tempfile.TemporaryDirectory(prefix="picfetch-profile-") as temporary: + snapshot = Path(temporary) / "profile.provisionprofile" + snapshot.write_bytes(data) + decoded = run("security", "cms", "-D", "-i", snapshot, capture=True, env=env) + return data, plistlib.loads(decoded) + + +def sign_candidate(args): + if not re.fullmatch(r"[A-Z0-9]{10}", args.team): + raise ValueError("10-character developer Team ID required") + if args.testflight and not (args.profile and args.worker_profile): + raise ValueError("TestFlight route requires explicit app and XPC worker profiles") + app, out = Path(args.app).resolve(), Path(args.out).absolute() + if out.exists() or out.is_symlink() or out.is_relative_to(app): + raise ValueError("choose a fresh output directory outside the input app") + env = os.environ | {"DEVELOPER_DIR": os.environ.get("APPLE_STORE_DEVELOPER_DIR", "/Applications/Xcode.app/Contents/Developer"), + "LC_ALL": "C", "LANG": "C"} + # Inspect public identity metadata only; private keys remain in the Keychain. + app_identities = run("security", "find-identity", "-v", "-p", "codesigning", capture=True, env=env).decode() + identities = run("security", "find-identity", "-v", capture=True, env=env).decode() + select_identity(app_identities, args.app_identity, args.team, "app") + installer_name = select_identity(identities, args.installer_identity, args.team, "installer") + installer_fingerprint = installer_certificate(installer_name, args.installer_identity, env) + verify(app, env) + source_manifest = json.loads((app.parent / "manifest.json").read_text()) + original = bundle_files(app) + if {name: value["sha256"] for name, value in original.items()} != source_manifest.get("files"): + raise ValueError("input app does not match its local qualification manifest") + entitlements = copy.deepcopy(ENTITLEMENTS) + profiles = {} + for role, value in (("app", args.profile), ("service", args.worker_profile)): + if value: + path = Path(value).resolve() + data, profile = read_profile(path, env) + entitlements[role] = profile_entitlements(profile, args.team, + APP_ID if role == "app" else APP_ID + ".worker", + args.app_identity, role, datetime.now(timezone.utc)) + profiles[role] = data + out.parent.mkdir(parents=True, exist_ok=True) + with tempfile.TemporaryDirectory(prefix=".picfetch-distribution-", dir=out.parent) as temporary: + stage = Path(temporary) + candidate = stage / "PicFetch.app" + run("ditto", app, candidate, env=env) + if bundle_files(candidate) != original: + raise ValueError("staged app differs from qualified input") + for role, data in profiles.items(): + bundle = candidate if role == "app" else candidate / "Contents/XPCServices" / (APP_ID + ".worker.xpc") + (bundle / "Contents/embedded.provisionprofile").write_bytes(data) + for role, values in entitlements.items(): + write_plist(stage / (role + ".plist"), values) + for path, role, identifier in code_items(candidate): + claims = ["--entitlements", stage / (role + ".plist")] if role else [] + run("codesign", "--force", "--sign", args.app_identity, "--timestamp", "--identifier", identifier, + *claims, path, env=env) + verify(candidate, env, expected_entitlements=entitlements) + verify_code_identities(candidate, args.team, args.app_identity, env) + installer = stage / "PicFetch.pkg" + run("productbuild", "--component", candidate, "/Applications", "--sign", installer_name, + "--timestamp", installer, env=env) + report = run("pkgutil", "--check-signature", installer, capture=True, env=env).decode() + verify_installer_report(report, installer_name, installer_fingerprint) + expanded = stage / "expanded" + run("pkgutil", "--expand-full", installer, expanded, env=env) + verify_payload(expanded, candidate) + manifest = {"purpose": "distribution-signed candidate; Apple submission validation still required", + "team": args.team, "app_identity": args.app_identity, + "installer_identity": args.installer_identity, "installer_certificate_sha256": installer_fingerprint, + "testflight": args.testflight, + "profiles": {role: hashlib.sha256(data).hexdigest() for role, data in profiles.items()}, + "input": source_manifest, "files": bundle_files(candidate), "installer_sha256": digest(installer)} + if bundle_files(app) != original: + raise ValueError("input app changed during packaging") + out.mkdir() + shutil.move(candidate, out / candidate.name) + shutil.move(installer, out / installer.name) + (out / "manifest.json").write_text(json.dumps(manifest, indent=2) + "\n") + print(f"Distribution candidate: {out}; not uploaded or installed", flush=True) + + +if __name__ == "__main__": + parser = argparse.ArgumentParser(description=__doc__) + for flag, variable in (("app", "APPLE_STORE_APP"), ("out", "APPLE_STORE_SIGNED_OUTPUT_DIR"), + ("team", "APPLE_STORE_TEAM_ID"), ("app-identity", "APPLE_STORE_APP_IDENTITY"), + ("installer-identity", "APPLE_STORE_INSTALLER_IDENTITY"), + ("profile", "APPLE_STORE_PROFILE"), ("worker-profile", "APPLE_STORE_WORKER_PROFILE")): + parser.add_argument("--" + flag, default=os.environ.get(variable, "")) + parser.add_argument("--testflight", action="store_true", default=os.environ.get("APPLE_STORE_TESTFLIGHT") == "1") + options = parser.parse_args() + if not all((options.app, options.out, options.team, options.app_identity, options.installer_identity)): + parser.error("input app, fresh output, Team and both identity SHA-1 fingerprints are required") + sign_candidate(options) diff --git a/scripts/macstorepackage/package.py b/scripts/macstorepackage/package.py index ae1922f5..dba89319 100644 --- a/scripts/macstorepackage/package.py +++ b/scripts/macstorepackage/package.py @@ -1,7 +1,8 @@ """Build and validate an ad-hoc, universal Apple Store qualification app. -This route has no distribution signing or upload mode. Apple account credentials -are unnecessary; the result cannot be submitted as an App Store release. +This local route needs no Apple account credentials and cannot be submitted as +an App Store release. distribution.py consumes a qualified app for separate +distribution signing; neither route uploads artifacts. """ import argparse import hashlib @@ -50,8 +51,8 @@ def check_dependencies(paths): raise ValueError(f"non-system native dependency: {path}") -def check_entitlements(role, actual): - if actual != ENTITLEMENTS[role]: +def check_entitlements(role, actual, expected=None): + if actual != (ENTITLEMENTS if expected is None else expected)[role]: raise ValueError(f"unexpected {role} entitlements: {actual}") @@ -135,7 +136,7 @@ def native_check(path, architectures, minima, env): check_dependencies("\n".join(dependencies)) -def verify(app, env): +def verify(app, env, expected_entitlements=None): attributes = run("xattr", "-r", app, capture=True, env=env) if any(line.rsplit(b": ", 1)[-1] == b"com.apple.quarantine" for line in attributes.splitlines()): raise ValueError("bundle contains com.apple.quarantine; rebuild from unquarantined inputs") @@ -146,7 +147,7 @@ def verify(app, env): service / "Contents/MacOS/picfetch-image-worker": "helper"} for path, role in roles.items(): data = run("codesign", "-d", "--entitlements", ":-", path, capture=True, env=env) - check_entitlements(role, plistlib.loads(data)) + check_entitlements(role, plistlib.loads(data), expected_entitlements) run("codesign", "--verify", "--strict", path, env=env) plist = plistlib.loads((contents / "Info.plist").read_bytes()) if plist.get("CFBundleIdentifier") != APP_ID or plist.get("CFBundleExecutable") != "PicFetch": diff --git a/scripts/macstorepackage/test_distribution.py b/scripts/macstorepackage/test_distribution.py new file mode 100644 index 00000000..09303120 --- /dev/null +++ b/scripts/macstorepackage/test_distribution.py @@ -0,0 +1,245 @@ +"""Portable signing-policy guards; native installer tests are opt-in.""" +from argparse import Namespace +from datetime import datetime, timedelta, timezone +import hashlib +import json +import os +from pathlib import Path +import shutil +import ssl +import subprocess +import sys +import tempfile +import unittest +from unittest.mock import patch + +sys.path.insert(0, str(Path(__file__).parent)) +import distribution +from package import APP_ID, run + + +class DistributionPolicy(unittest.TestCase): + def setUp(self): + self.now = datetime(2026, 9, 30, tzinfo=timezone.utc) + self.team = "ABCDEFGHIJ" + self.certificate = b"synthetic certificate" + self.fingerprint = hashlib.sha1(self.certificate).hexdigest().upper() + self.profile = { + "TeamIdentifier": [self.team], "ApplicationIdentifierPrefix": ["LEGACYPFX1"], + "Platform": ["OSX"], "CreationDate": self.now - timedelta(days=1), + "ExpirationDate": self.now + timedelta(days=30), + "DeveloperCertificates": [self.certificate], + "Entitlements": {"com.apple.application-identifier": "LEGACYPFX1." + APP_ID, + "com.apple.developer.team-identifier": self.team}, + } + + def check(self, profile): + return distribution.profile_entitlements(profile, self.team, APP_ID, + self.fingerprint, "app", self.now) + + def test_rejects_wrong_profile_authority(self): + changes = [ + {"TeamIdentifier": ["OTHERTEAM1"]}, {"Platform": ["iOS"]}, + {"ExpirationDate": self.now}, {"CreationDate": self.now + timedelta(days=1)}, + {"DeveloperCertificates": [b"other"]}, {"ProvisionedDevices": []}, + {"ProvisionsAllDevices": True}, {"ApplicationIdentifierPrefix": ["WRONGPREFX"]}, + {"Entitlements": {"com.apple.application-identifier": "LEGACYPFX1.*"}}, + {"Entitlements": self.profile["Entitlements"] | {"com.apple.security.get-task-allow": True}}, + {"Entitlements": self.profile["Entitlements"] | {"com.apple.developer.team-identifier": "OTHERTEAM1"}}, + ] + for change in changes: + with self.subTest(change=change), self.assertRaises(ValueError): + self.check(self.profile | change) + + def test_exact_claims_preserve_legacy_prefix(self): + claims = self.check(self.profile) + self.assertEqual(claims, distribution.ENTITLEMENTS["app"] | self.profile["Entitlements"]) + self.assertNotIn("get-task-allow", claims) + self.assertNotIn("com.apple.security.network.server", claims) + self.assertEqual(self.profile["Entitlements"]["com.apple.application-identifier"], "LEGACYPFX1." + APP_ID) + + def test_profile_does_not_broaden_capabilities_or_apply_to_helpers(self): + profile = self.profile | {"Entitlements": self.profile["Entitlements"] | { + "beta-reports-active": True, "com.apple.security.network.server": True}} + claims = self.check(profile) + self.assertIs(claims["beta-reports-active"], True) + self.assertNotIn("com.apple.security.network.server", claims) + with self.assertRaises(ValueError): + distribution.profile_entitlements(self.profile, self.team, APP_ID, + self.fingerprint, "helper", self.now) + worker = dict(self.profile) + worker["Entitlements"] = self.profile["Entitlements"] | {"com.apple.application-identifier": "LEGACYPFX1." + APP_ID + ".worker"} + claims = distribution.profile_entitlements(worker, self.team, APP_ID + ".worker", + self.fingerprint, "service", self.now) + self.assertNotIn("com.apple.security.network.client", claims) + self.assertEqual(claims["com.apple.application-identifier"], "LEGACYPFX1." + APP_ID + ".worker") + + def test_identity_requires_role_team_and_unique_fingerprint(self): + name = "Apple Distribution: Fixture (ABCDEFGHIJ)" + output = f' 1) {self.fingerprint} "{name}"\n 1 valid identities found\n' + self.assertEqual(distribution.select_identity(output, self.fingerprint, self.team, "app"), name) + for text, fingerprint, team, role in [ + (output, self.fingerprint, self.team, "installer"), + (output, self.fingerprint, "OTHERTEAM1", "app"), + (output, "-", self.team, "app"), + (output, "0" * 40, self.team, "app"), + (output + output, self.fingerprint, self.team, "app"), + (output.replace("Apple Distribution:", "Developer ID Application:"), self.fingerprint, self.team, "app"), + ]: + with self.subTest(text=text, role=role), self.assertRaises(ValueError): + distribution.select_identity(text, fingerprint, team, role) + + +class InstallerPolicy(unittest.TestCase): + def test_installer_report_requires_selected_trusted_leaf(self): + name = "3rd Party Mac Developer Installer: Fixture (ABCDEFGHIJ)" + report = f"Package: PicFetch.pkg\n Status: signed by a certificate trusted by macOS\n Certificate Chain:\n 1. {name}\n SHA256 Fingerprint: {'12' * 32}\n 2. Apple Root CA\n" + distribution.verify_installer_report(report, name, "12" * 32) + for changed in (report.replace(name, "Developer ID Installer: Fixture (ABCDEFGHIJ)"), + report.replace("12" * 32, "34" * 32), + report.replace("trusted by macOS", "not trusted by macOS"), + report.replace("signed by a certificate trusted by macOS", "no signature")): + with self.assertRaises(ValueError): + distribution.verify_installer_report(changed, name, "12" * 32) + + def test_payload_requires_exact_app_and_install_location(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + app = root / "PicFetch.app" + app.mkdir() + (app / "fixture").write_bytes(b"signed-content") + expanded = root / "expanded" + component = expanded / "component.pkg" + payload = component / "Payload" + payload.mkdir(parents=True) + shutil.copytree(app, payload / app.name) + info = component / "PackageInfo" + original = f'' + info.write_text(original) + distribution.verify_payload(expanded, app) + for xml in (original.replace("/Applications", "/Library"), + original.replace(APP_ID, "other.app"), + original.replace('/>', '>')): + info.write_text(xml) + with self.assertRaises(ValueError): + distribution.verify_payload(expanded, app) + info.write_text(original) + (payload / app.name / "fixture").write_bytes(b"tampered") + with self.assertRaisesRegex(ValueError, "differs"): + distribution.verify_payload(expanded, app) + (payload / app.name / "fixture").write_bytes(b"signed-content") + (payload / "extra").write_text("extra payload") + with self.assertRaisesRegex(ValueError, "unexpected installer payload"): + distribution.verify_payload(expanded, app) + + +@unittest.skipUnless(os.environ.get("PICFETCH_TEST_MAC_APP_BUNDLE"), "requires a built signed app") +class NativeInstallerGuards(unittest.TestCase): + def test_ad_hoc_app_cannot_pass_distribution_identity_check(self): + app = Path(os.environ["PICFETCH_TEST_MAC_APP_BUNDLE"]).resolve() + with self.assertRaises(subprocess.CalledProcessError): + distribution.verify_code_identities(app, "ABCDEFGHIJ", "A" * 40, os.environ.copy()) + + def test_productbuild_roundtrip_preserves_signed_app(self): + # Deliberately unsigned installer: this qualifies payload assembly only. + app = Path(os.environ["PICFETCH_TEST_MAC_APP_BUNDLE"]).resolve() + before = distribution.bundle_files(app) + with tempfile.TemporaryDirectory(prefix="picfetch-installer-guard-") as temporary: + root = Path(temporary) + installer = root / "PicFetch.pkg" + run("productbuild", "--component", app, "/Applications", installer) + expanded = root / "expanded" + run("pkgutil", "--expand-full", installer, expanded) + distribution.verify_payload(expanded, app) + extracted = next(expanded.rglob("Payload")) / "PicFetch.app" + run("codesign", "--verify", "--deep", "--strict", extracted) + self.assertEqual(distribution.bundle_files(app), before) + + +class DistributionAssembly(unittest.TestCase): + def test_signs_inside_out_and_publishes_verified_payload_only(self): + with tempfile.TemporaryDirectory() as temporary: + root = Path(temporary) + source = root / "input" / "PicFetch.app" + for relative in ("Contents/MacOS/PicFetch", "Contents/Frameworks/runtime.dylib", + "Contents/MacOS/picfetch-worker-client", + "Contents/XPCServices/" + APP_ID + ".worker.xpc/Contents/MacOS/worker", + "Contents/XPCServices/" + APP_ID + ".worker.xpc/Contents/MacOS/picfetch-image-worker"): + path = source / relative + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text("fixture") + original = distribution.bundle_files(source) + (source.parent / "manifest.json").write_text(json.dumps({"files": {name: value["sha256"] for name, value in original.items()}})) + app_key = "A" * 40 + installer_certificate = b"fixture installer certificate" + installer_key = hashlib.sha1(installer_certificate).hexdigest().upper() + installer_sha256 = hashlib.sha256(installer_certificate).hexdigest().upper() + installer_name = "3rd Party Mac Developer Installer: Fixture (ABCDEFGHIJ)" + listing = f' 1) {app_key} "Apple Distribution: Fixture (ABCDEFGHIJ)"\n 2) {installer_key} "{installer_name}"\n' + commands = [] + candidate = None + + def fake_run(*args, **_): + nonlocal candidate + args = list(map(str, args)) + commands.append(args) + if args[:2] == ["security", "find-identity"]: + return listing.encode() + if args[:2] == ["security", "find-certificate"]: + return ssl.DER_cert_to_PEM_cert(installer_certificate).encode() + if args[0] == "ditto": + shutil.copytree(args[1], args[2]) + candidate = Path(args[2]) + elif args[0] == "productbuild": + self.assertEqual(args[1:4], ["--component", str(candidate), "/Applications"]) + self.assertEqual(args[4:7], ["--sign", installer_name, "--timestamp"]) + Path(args[-1]).write_bytes(b"package") + elif args[:2] == ["pkgutil", "--check-signature"]: + return f"Status: signed by a certificate trusted by macOS\n 1. {installer_name}\n SHA256 Fingerprint: {installer_sha256}\n".encode() + elif args[:2] == ["pkgutil", "--expand-full"]: + component = Path(args[-1]) / "component.pkg" + payload = component / "Payload" + payload.mkdir(parents=True) + shutil.copytree(candidate, payload / "PicFetch.app") + (component / "PackageInfo").write_text(f'') + return None + + args = Namespace(app=str(source), out=str(root / "output"), team="ABCDEFGHIJ", + app_identity=app_key, installer_identity=installer_key, + profile="", worker_profile="", testflight=False) + with patch.object(distribution, "run", side_effect=fake_run), patch.object(distribution, "verify") as verify: + distribution.sign_candidate(args) + self.assertEqual(verify.call_count, 2) + signed = [cmd for cmd in commands if cmd[:2] == ["codesign", "--force"]] + self.assertEqual([cmd[-1].split("/")[-1] for cmd in signed], + ["runtime.dylib", "picfetch-worker-client", "picfetch-image-worker", APP_ID + ".worker.xpc", "PicFetch.app"]) + self.assertTrue(all(cmd[2:4] == ["--sign", app_key] for cmd in signed)) + validations = [cmd for cmd in commands if cmd[:2] == ["codesign", "--verify"]] + self.assertEqual(len(validations), 5) + self.assertTrue(all('anchor apple generic' in cmd[-2] and app_key in cmd[-2] for cmd in validations)) + self.assertEqual(distribution.bundle_files(source), original) + self.assertTrue((Path(args.out) / "PicFetch.pkg").is_file()) + self.assertTrue((Path(args.out) / "manifest.json").is_file()) + + def test_profile_snapshot_is_the_same_data_decoded_and_embedded(self): + with tempfile.TemporaryDirectory() as temporary: + source = Path(temporary) / "profile" + source.write_bytes(b"original CMS") + + def decode(*args, **_): + snapshot = Path(args[-1]) + self.assertEqual(snapshot.read_bytes(), b"original CMS") + source.write_bytes(b"changed after snapshot") + return distribution.plistlib.dumps({"fixture": True}) + + with patch.object(distribution, "run", side_effect=decode): + data, profile = distribution.read_profile(source, {}) + self.assertEqual(data, b"original CMS") + self.assertEqual(profile, {"fixture": True}) + + def test_testflight_requires_both_bundle_profiles_before_side_effects(self): + for app_profile, worker_profile in (("", ""), ("app", ""), ("", "worker")): + args = Namespace(team="ABCDEFGHIJ", testflight=True, profile=app_profile, worker_profile=worker_profile) + with patch.object(distribution, "run") as run_tool, self.assertRaisesRegex(ValueError, "app and XPC"): + distribution.sign_candidate(args) + run_tool.assert_not_called() diff --git a/todos.md b/todos.md index f49e20b1..9735172a 100644 --- a/todos.md +++ b/todos.md @@ -32,10 +32,14 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Add explicit Store Open-dialog folder consent for single-image sibling navigation; preserve cancellation and the initially selected image. Latest rebuilt E2E app is in `bin/apple-store-e2e-drop-navigation/`. - [x] Ronin confirmed native Open-dialog folder consent fixes navigation; extend the same consent to shared drop/Open With discovery with cancellation and replacement coverage. - [ ] Confirm the rebuilt shared-consent app with a single-file window drop and Open With. -- [ ] Diagnose Ronin's multiple-file picker/drop navigation report; selected-file navigation passes the automated harness, packaged symptom needs clarification. +- [x] Ronin confirmed multiple-file picker/drop navigation must remain within the selected images; folder sibling discovery applies only to single-file inputs. Automated selection coverage passes. +- [x] Persist explicit sibling-folder approvals in app preferences and validate/reuse bookmarks on fresh single-image opens; keep renewal, cancellation and selection guards. +- [x] Ronin confirmed persistent folder approvals work across real quit/relaunch in `bin/apple-store-e2e-persistent-folders/PicFetch.app`. +- [ ] Confirm moved-folder bookmark reuse in the persistent-folder test app; automated coverage passes, live moved-folder testing remains open. - [x] Validate copyright/category metadata and reject quarantine attributes in local Store packages. - [ ] Finish GUI sandbox workflows, moved-folder relaunch, and physical Intel/minimum-OS validation. -- [ ] Add the signed Store packaging route and validate the final artifact on Intel and Apple Silicon. +- [x] Implement a separate Store distribution-signing/installer route with identity, profile and payload guards. +- [ ] Run the signed route with real Apple certificates/profiles and validate the resulting candidate on the required distribution path and platform matrix. - [x] Prepare [listing/review notes and human-input checklist](packaging/apple-app-store/submission-draft.md); inventory privacy/network/export facts. - [x] Preserve exact Abseil privacy manifests/license/provenance; verify original Microsoft runtime signatures before local re-signing. - [ ] Resolve remaining [Protobuf/ONNX privacy coverage and SDK validation](packaging/apple-app-store/privacy-dependency-audit.md), confirm final privacy/export answers and validate Apple submission. From da2df0ccea411195789cd4b9cac973b8b5dd028b Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 12:37:26 +0200 Subject: [PATCH 14/63] Document App Store handoff checklist and qualification evidence --- packaging/apple-app-store/README.md | 2 + packaging/apple-app-store/ronin-checklist.md | 164 ++++++++++++++++++ packaging/apple-app-store/submission-draft.md | 22 ++- plans/2026-09-29-apple-app-store.md | 51 ++++++ todos.md | 9 +- 5 files changed, 237 insertions(+), 11 deletions(-) create mode 100644 packaging/apple-app-store/ronin-checklist.md diff --git a/packaging/apple-app-store/README.md b/packaging/apple-app-store/README.md index 81a755fe..145c3f81 100644 --- a/packaging/apple-app-store/README.md +++ b/packaging/apple-app-store/README.md @@ -11,6 +11,8 @@ and evidence. [Apple requirements](../../docs/apple-app-store-requirements-2026- records current primary sources and distinguishes native macOS requirements from mobile-platform privacy rules. +For the remaining owner actions, follow [Ronin's ordered checklist](ronin-checklist.md). + ## Build policy Use `no_emoji,nodynamic,appleappstore` for this macOS channel. Do not combine diff --git a/packaging/apple-app-store/ronin-checklist.md b/packaging/apple-app-store/ronin-checklist.md new file mode 100644 index 00000000..72aee693 --- /dev/null +++ b/packaging/apple-app-store/ronin-checklist.md @@ -0,0 +1,164 @@ +# Ronin's Mac App Store checklist + +Updated 2026-09-30. Work through the numbered steps in order; stop after step 4 +and hand the signing inputs back to Pico. Steps 5–7 can then proceed while Pico +qualifies the signed candidate. No upload or release is authorized by this list. + +**Already done:** Xcode is installed and licensed; the local universal Store test +app is built; you confirmed folder approvals survive quit/relaunch. All platform +and race CI jobs and CodeQL passed for `62661a1`. Qodana completed with eight +reviewed unused-function false positives and no actionable findings. You +reauthorized committing and pushing this documentation after your lunch break. + +Current test app: `bin/apple-store-e2e-persistent-folders/PicFetch.app`. +Its adjacent `TESTING.md` and `manifest.json` record the build. Keep this app for +local E2E; Pico will make the distribution candidate in a separate directory. + +## 1. Confirm the Apple developer team + +- [ ] Sign in to [Apple Developer](https://developer.apple.com/account/) and + confirm the intended paid Developer Program team is active. +- [ ] Have the Account Holder review any pending agreements in + [App Store Connect](https://appstoreconnect.apple.com/). Confirm Account Holder + or Admin access for the certificate/profile work below. +- [ ] Give Pico the selected **10-character Team ID** and whether the app will + be published by you personally or an organization. Passwords and recovery + codes are not needed. + +Apple limits distribution-certificate creation to the Account Holder or Admin; +see its [certificate overview](https://developer.apple.com/help/account/create-certificates/certificates-overview). +Pending agreements can prevent creating an app record; see +[adding an app](https://developer.apple.com/help/app-store-connect/create-an-app-record/add-a-new-app). + +## 2. Register or confirm both explicit identifiers + +In Certificates, Identifiers & Profiles, register these explicit App IDs on the +chosen team, or confirm existing matching entries belong to it: + +| Purpose | Bundle identifier | Suggested description | +|---|---|---| +| PicFetch app | `io.github.frathe.picfetch` | PicFetch | +| Embedded worker | `io.github.frathe.picfetch.worker` | PicFetch Worker | + +- [ ] Both explicit IDs are available on the correct team. + +Use Apple's [App ID registration guide](https://developer.apple.com/help/account/identifiers/register-an-app-id/). +If either identifier is unavailable, tell Pico before choosing another: changing +the app identity affects saved settings and sandbox data. No extra portal +capabilities are requested by the current packaging route. + +## 3. Put both distribution signing identities in this Mac's Keychain + +The read-only Keychain check on 2026-09-30 found **zero valid Store application +identities and zero valid Store installer identities with private keys**. + +- [ ] Make an **Apple Distribution** application signing identity available. +- [ ] Make a **Mac Installer Distribution** identity available. Keychain normally + displays this as `3rd Party Mac Developer Installer: ...`. +- [ ] Confirm each certificate has its matching private key in Keychain Access. + +If the team already has suitable identities, import them securely from their +owner instead of unnecessarily replacing or revoking them. For new certificates, +create the request on the Mac that will retain the private key using Apple's +[CSR guide](https://developer.apple.com/help/account/certificates/create-a-certificate-signing-request/), +then choose the relevant certificate type in the developer portal. Apple's +[certificate overview](https://developer.apple.com/help/account/create-certificates/certificates-overview) +distinguishes the app and installer roles. Pico only needs Keychain access to +sign, not private keys pasted into chat or committed to the repository. + +## 4. Download two distribution provisioning profiles + +Our TestFlight packaging route requires a separate profile for the app and its +XPC worker. For **each** identifier from step 2, create a **Mac App Store Connect** +profile using the application distribution certificate from step 3. Download +both files. Follow Apple's +[profile guide](https://developer.apple.com/help/account/provisioning-profiles/create-an-app-store-provisioning-profile/). + +- [ ] App profile downloaded for `io.github.frathe.picfetch`. +- [ ] Worker profile downloaded for `io.github.frathe.picfetch.worker`. +- [ ] Tell Pico the two absolute file paths and confirm the intended Team ID. + +**Handoff:** Pico can now identify the certificate fingerprints, run preflight, +run `make apple-store-package-signed`, and check nested signatures, profiles, +installer trust and extracted payloads. The exact invocation is in the +[packaging README](README.md). You do not need to assemble that command yourself. +Keychain may ask you to approve signing. This work does not upload the app. + +## 5. Create the App Store Connect app record + +- [ ] In App Store Connect, use Apps -> + -> New App, platform **macOS**. +- [ ] Select `io.github.frathe.picfetch`, choose the primary language, and + confirm **PicFetch** as the available name. +- [ ] Choose your internal SKU; `picfetch-macos` is a suggested value. +- [ ] Tell Pico that the record exists and provide its Apple ID/app link. + +Use Apple's [new-app guide](https://developer.apple.com/help/app-store-connect/create-an-app-record/add-a-new-app). +Only the main app needs an App Store Connect listing. Registering the embedded +worker for signing does not mean creating a second store product. + +## 6. Finish the remaining hands-on checks + +Use disposable copies of images for writing, overwriting and Trash tests. +Record the build, macOS version, architecture, action and actual result for any +failure. The existing local app can catch regressions now; repeat acceptance +checks on the eventual TestFlight-delivered build. + +- [ ] Drop one image onto the window and open one through Finder's Open With: + after folder approval, Left/Right reaches its siblings. Canceling a new folder + request still leaves the selected image usable. +- [ ] Check a saved session and a saved Favorite after quitting and relaunching. +- [ ] Grant a disposable folder, save a Favorite, quit, move/rename the folder + in Finder, relaunch, and reopen the Favorite. Report whether images resolve + and whether another grant is requested; then try a single image from the new + location. Automated bookmark tests pass; this native behavior is still open. +- [ ] If available, repeat with an external volume disconnected, then + reconnected. The app should remain responsive and recover after access returns. +- [ ] Exercise Save/export to a chosen destination, overwrite a disposable copy, + Trash, clipboard, Reveal in Finder and wallpaper. Check the actual output/effect. +- [ ] Exercise HEIC, Similarity Explorer/visual search, cancel/reopen, and maps + in the GUI. Confirm ordinary browsing still works when model setup is declined. +- [ ] Arrange access to physical Intel hardware and the declared minimum systems: + macOS **13.4 Intel** and **14.0 Apple Silicon**. Tell Pico what machines/testers + are available so a specific qualification run can be prepared. + +Single-file browsing expands into the folder. Multiple selected/dropped files +intentionally stay within that selection. You already confirmed this UX and +persistent folder approvals; neither is an outstanding redesign request. +Rosetta tests and current Intel CI do not establish minimum-OS GUI support. + +## 7. Review the listing and owner decisions + +Use the [submission draft](submission-draft.md) and +[privacy/network inventory](privacy-dependency-audit.md) as prepared inputs. + +- [ ] Confirm description, category, primary language, price and territories. +- [ ] Supply legal/copyright owner, review contact and any required seller/trader + details; complete the current age-rating questionnaire. +- [ ] Choose the public support and privacy URLs and contact details. Pico will + check that their final content is accessible without signing in. +- [ ] Review the privacy and encryption/export answers against the technical + inventory. Local image analysis alone does not settle all network disclosures. +- [ ] Select or approve nonprivate, licensed/synthetic example images and review + screenshots captured from the final viewer. Pico can prepare the screenshots + when desktop access is available; generated mockups are not the deliverable. + +## 8. Approve TestFlight upload once technical gates are resolved + +- [ ] Review Pico's signed-candidate report and authorize an upload to App Store + Connect/TestFlight, including who should be invited to test. +- [ ] Install the delivered build and confirm the final E2E results. + +Before asking for this approval, Pico must finish the Protobuf/ONNX privacy and +SDK qualification, verify the actual signed installer, and report any remaining +platform gaps. CI passing does not settle those gates. These are technical tasks +owned by Pico; you are not being asked to invent dependency declarations. + +## 9. Decide on App Review and release + +- [ ] Approve the final listing, screenshots, review notes and questionnaires. +- [ ] Separately authorize submission for App Review. +- [ ] Choose release timing and authorize release after approval. + +Creating certificates, signing a candidate, uploading a beta, submitting for +review and releasing are distinct steps. None has been silently selected by +this checklist. diff --git a/packaging/apple-app-store/submission-draft.md b/packaging/apple-app-store/submission-draft.md index a3855564..3184c789 100644 --- a/packaging/apple-app-store/submission-draft.md +++ b/packaging/apple-app-store/submission-draft.md @@ -100,7 +100,7 @@ export questionnaire has been assessed for the actual shipped closure. Apple requires the developer to determine the applicable documentation; see [export compliance](https://developer.apple.com/help/app-store-connect/manage-app-information/overview-of-export-compliance/). -## Technical gates before account work +## Technical gates before submission - Resolve the remaining Protobuf/ONNX privacy manifest coverage and final SDK validation. Abseil declarations and pre-sign Microsoft identity checks are now @@ -114,16 +114,24 @@ requires the developer to determine the applicable documentation; see both declared minimum OS versions. - Run the full native Linux/amd64 CI suite, CodeQL and fresh post-suppression Qodana analysis. ARM Docker emulation cannot replace the isolation gate. -- Complete the Store certificate signing and productbuild route; validate the - profile when supplied and inspect the final installer payload. Ad-hoc tests - cannot establish distribution trust or App Store acceptance. +- Run the implemented `make apple-store-package-signed` route with real Apple + identities/profiles and qualify its candidate. Portable policy and native + unsigned-installer payload tests pass; they cannot establish distribution + trust or App Store acceptance. ## Human inputs last -1. Signed Git commits and pushing the preparation branch were reauthorized by - Ronin after the overnight work; submission/release still require a separate decision. +Follow [Ronin's ordered checklist](ronin-checklist.md) for account setup, +signing inputs, remaining E2E checks and submission decisions. Account setup can +proceed while Pico resolves the technical gates above. + +1. Signed commit `62661a1` was pushed before Ronin's lunch break. He + reauthorized committing and pushing the current documentation after returning; + submission/release still require a separate decision. 2. Supply the developer Team ID and Store app/installer signing identities with - their private keys. TestFlight also needs a matching provisioning profile. + their private keys in Keychain. This TestFlight route also needs separate + profiles for `io.github.frathe.picfetch` and `io.github.frathe.picfetch.worker`; + see the packaging README for the exact command and input names. 3. Create or select the App Store Connect record for `io.github.frathe.picfetch`. 4. Confirm legal contact, seller/trader status, age rating, pricing, availability, privacy questionnaire and encryption/export answers from the prepared facts. diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index 8c2de5ed..05e25ac6 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -1412,3 +1412,54 @@ input. Therefore real distribution signing, profile CMS/DER acceptance, installe trust output against actual certificates, TestFlight and Apple SDK/privacy review remain unverified. No distribution artifact, upload or installation was performed. The user-confirmed local E2E app is unchanged by this tooling work. + +### Lunch-break handoff and latest CI — 2026-09-30 + +Signed commit 62661a19291e5f936dc02751b88914158c50beaa was pushed before +Ronin requested the lunch-break commit pause. This supersedes earlier local-only +status in this chronological record. No further commits are authorized until +Ronin returns and authorizes them. The existing E2E app remains unchanged. + +All three runs completed successfully on exactly that revision: + +- [CI 36697931823](https://github.com/frathe/picfetch/actions/runs/36697931823): + validation, all four native Linux/amd64 race shards, Linux native guards, + Windows tests, and both macOS architecture native guards passed. +- [CodeQL 36697931680](https://github.com/frathe/picfetch/actions/runs/36697931680): + Go and Actions analysis passed. +- [Qodana 36697931732](https://github.com/frathe/picfetch/actions/runs/36697931732): + scan completed; post-suppression /qodana.sarif.json contains eight findings. + Reviewed the actual report in .scratch/apple-store-package/qodana-62661a1/, + not the pre-suppression CSV or start baseline. All are known + GoUnusedExportedFunction false positives: fileaccess.Stat, NeedsCapture, + HasScope, Export, Import; macbundle.CurrentRuntimeDirectory; + openwith.InstallWindowDrop and Stop. Rechecked current callers in ui, + favthumbs, favstore and similarity. The former global folder-authorizer warning + disappeared after the per-instance change. No actionable finding or broad + suppression added. The Qodana for Go annotation check is neutral, not a report + with zero findings. + +This is platform/native-guard evidence, not physical minimum-OS GUI or Apple +submission acceptance. No fresh GitHub Codex code review was requested, so no +new clean bot-review claim is made. Local changed-code GoLand evidence above +still applies; this continuation changes documentation only. + +A separate read-only security find-identity inventory found zero valid Store +application and zero valid Store installer identities with private keys in the +accessible Keychain. Only aggregate counts were retained. This establishes the +local credential gap, not the state of Ronin's Apple membership or identities +held elsewhere. Xcode installation/license work is already complete. + +Created packaging/apple-app-store/ronin-checklist.md with ordered account, +identifier, certificate and two-profile setup; a handoff back to Pico for actual +signing; remaining live E2E/platform checks; owner listing decisions; and separate +upload/review/release decisions. Checked Apple primary guidance for current +portal terminology and linked it in the checklist. Updated README, submission +draft and todos to point to it and reflect the commit pause. SDK privacy closure +remains an explicitly technical responsibility; no declaration or Apple approval +was invented to make the handoff appear complete. + +Ronin returned from lunch and explicitly authorized committing and pushing the +current state. Updated the checklist/submission/todo status to end the pause. +This handoff changes documentation only; local Markdown targets resolve and +git diff --check passes. Runtime inspection/CI evidence remains at 62661a1. diff --git a/todos.md b/todos.md index 9735172a..20321500 100644 --- a/todos.md +++ b/todos.md @@ -29,7 +29,7 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Qualify retained visual search through production ARM/Intel-Rosetta workers; keep scopes through clipboard/Finder/Trash calls and source metadata reads. - [x] Capture native window-drop URLs before GLFW flattens paths; refresh restored moved/stale bookmarks with immutable persistence coverage. - [x] Verify production workers deny an ungranted private source while an allowed control image still completes inference, on ARM and Intel under Rosetta. -- [x] Add explicit Store Open-dialog folder consent for single-image sibling navigation; preserve cancellation and the initially selected image. Latest rebuilt E2E app is in `bin/apple-store-e2e-drop-navigation/`. +- [x] Add explicit Store Open-dialog folder consent for single-image sibling navigation; preserve cancellation and the initially selected image. Latest E2E app is in `bin/apple-store-e2e-persistent-folders/`. - [x] Ronin confirmed native Open-dialog folder consent fixes navigation; extend the same consent to shared drop/Open With discovery with cancellation and replacement coverage. - [ ] Confirm the rebuilt shared-consent app with a single-file window drop and Open With. - [x] Ronin confirmed multiple-file picker/drop navigation must remain within the selected images; folder sibling discovery applies only to single-file inputs. Automated selection coverage passes. @@ -40,12 +40,13 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [ ] Finish GUI sandbox workflows, moved-folder relaunch, and physical Intel/minimum-OS validation. - [x] Implement a separate Store distribution-signing/installer route with identity, profile and payload guards. - [ ] Run the signed route with real Apple certificates/profiles and validate the resulting candidate on the required distribution path and platform matrix. -- [x] Prepare [listing/review notes and human-input checklist](packaging/apple-app-store/submission-draft.md); inventory privacy/network/export facts. +- [x] Prepare [listing/review notes](packaging/apple-app-store/submission-draft.md) and [ordered owner checklist](packaging/apple-app-store/ronin-checklist.md); inventory privacy/network/export facts. - [x] Preserve exact Abseil privacy manifests/license/provenance; verify original Microsoft runtime signatures before local re-signing. - [ ] Resolve remaining [Protobuf/ONNX privacy coverage and SDK validation](packaging/apple-app-store/privacy-dependency-audit.md), confirm final privacy/export answers and validate Apple submission. - [x] Enable the full Xcode compiler after local license acceptance. -- [ ] Human inputs last: developer Team ID, Store application/installer certificates, TestFlight provisioning profile, App Store Connect record and final submission decisions. -- [x] Ronin reauthorized signed commits and pushing the preparation branch after the overnight work. +- [ ] Human inputs last: developer Team ID, Store application/installer certificates, separate app/worker TestFlight provisioning profiles, App Store Connect record and final submission decisions. +- [x] Signed and pushed `62661a1`; full platform/race CI and CodeQL passed. Fresh Qodana SARIF has eight reviewed false positives and no actionable findings. +- [x] Ronin returned from lunch on 2026-09-30 and explicitly reauthorized committing and pushing the current state. ## Deferred From 710ba5c989d2b29c86204b84a1cee7f08b52f19c Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 13:05:48 +0200 Subject: [PATCH 15/63] Prepare manually approved Apple Store CI signing --- .github/workflows/apple-store.yml | 128 ++++++++++++++ ARCHITECTURE.md | 7 + packaging/apple-app-store/README.md | 72 ++++++++ packaging/apple-app-store/ronin-checklist.md | 17 ++ plans/2026-09-29-apple-app-store.md | 90 ++++++++++ scripts/macstorepackage/ci_sign.py | 156 ++++++++++++++++++ scripts/macstorepackage/test_ci_sign.py | 165 +++++++++++++++++++ todos.md | 3 + 8 files changed, 638 insertions(+) create mode 100644 .github/workflows/apple-store.yml create mode 100644 scripts/macstorepackage/ci_sign.py create mode 100644 scripts/macstorepackage/test_ci_sign.py diff --git a/.github/workflows/apple-store.yml b/.github/workflows/apple-store.yml new file mode 100644 index 00000000..7c99600b --- /dev/null +++ b/.github/workflows/apple-store.yml @@ -0,0 +1,128 @@ +name: Apple Store candidate + +# Signing qualification only. Upload/TestFlight/release automation is separate. +on: + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: apple-store-candidate + cancel-in-progress: false + +jobs: + test: + if: github.repository == 'frathe/picfetch' && github.ref == 'refs/heads/main' + uses: ./.github/workflows/ci.yml + + build: + needs: test + runs-on: macos-15 + timeout-minutes: 60 + outputs: + artifact-id: ${{ steps.qualified.outputs.artifact-id }} + steps: + - uses: actions/checkout@v7 + with: + ref: ${{ github.sha }} + persist-credentials: false + - uses: actions/setup-go@v7 + with: + go-version-file: go.mod + cache: false + - uses: actions/setup-python@v7 + with: + python-version: '3.12' + - name: Download pinned runtime archives + # The packager checks exact size/hash and Microsoft code signatures. + run: | + curl --fail --location --retry 3 --proto '=https' --proto-redir '=https' \ + https://github.com/microsoft/onnxruntime/releases/download/v1.29.0/onnxruntime-osx-arm64-1.29.0.tgz \ + --output "$RUNNER_TEMP/onnx-arm64.tgz" + curl --fail --location --retry 3 --proto '=https' --proto-redir '=https' \ + https://github.com/microsoft/onnxruntime/releases/download/v1.23.2/onnxruntime-osx-x86_64-1.23.2.tgz \ + --output "$RUNNER_TEMP/onnx-amd64.tgz" + - name: Build and inspect universal sandbox app + env: + APPLE_STORE_OUTPUT_DIR: ${{ runner.temp }}/apple-store-qualified + APPLE_STORE_ARM64_ARCHIVE: ${{ runner.temp }}/onnx-arm64.tgz + APPLE_STORE_AMD64_ARCHIVE: ${{ runner.temp }}/onnx-amd64.tgz + run: make apple-store-package-local + - name: Run native packaging guards + env: + PICFETCH_TEST_MAC_APP_BUNDLE: ${{ runner.temp }}/apple-store-qualified/PicFetch.app + run: python3 -m unittest discover -s scripts/macstorepackage + - name: Archive qualification artifact and approval summary + env: + SOURCE_SHA: ${{ github.sha }} + run: | + tar -czf "$RUNNER_TEMP/apple-store-qualified.tgz" -C "$RUNNER_TEMP/apple-store-qualified" PicFetch.app manifest.json + { + echo '### Apple Store signing approval' + echo "Source revision: $SOURCE_SHA" + echo 'Full CI and native packaging guards passed for this candidate.' + echo 'Approve the apple-store-signing environment to sign this artifact.' + echo 'The signed installer is a qualification candidate. SDK/privacy and Apple acceptance remain open.' + echo 'This workflow does not upload to Apple or release the app.' + shasum -a 256 "$RUNNER_TEMP/apple-store-qualified.tgz" + } >> "$GITHUB_STEP_SUMMARY" + - uses: actions/upload-artifact@v7 + id: qualified + with: + name: apple-store-qualified-${{ github.sha }}-${{ github.run_attempt }} + path: ${{ runner.temp }}/apple-store-qualified.tgz + retention-days: 7 + if-no-files-found: error + + sign: + needs: build + runs-on: macos-15 + timeout-minutes: 30 + # Required reviewer is Ronin (frathe); environment allows main only. + # Configure the environment before storing any signing secrets. + environment: apple-store-signing + env: + APPLE_STORE_TEAM_ID: ${{ vars.APPLE_STORE_TEAM_ID }} + APPLE_STORE_APP_IDENTITY: ${{ vars.APPLE_STORE_APP_IDENTITY }} + APPLE_STORE_INSTALLER_IDENTITY: ${{ vars.APPLE_STORE_INSTALLER_IDENTITY }} + steps: + - uses: actions/checkout@v7 + with: + ref: ${{ github.sha }} + persist-credentials: false + - uses: actions/setup-python@v7 + with: + python-version: '3.12' + - uses: actions/download-artifact@v8 + with: + artifact-ids: ${{ needs.build.outputs.artifact-id }} + merge-multiple: true + path: ${{ runner.temp }}/apple-store-download + - name: Restore same-run qualification artifact + run: | + mkdir "$RUNNER_TEMP/apple-store-qualified" + tar -xzf "$RUNNER_TEMP/apple-store-download/apple-store-qualified.tgz" -C "$RUNNER_TEMP/apple-store-qualified" + - name: Sign with temporary Keychain + env: + APPLE_STORE_APP: ${{ runner.temp }}/apple-store-qualified/PicFetch.app + APPLE_STORE_SIGNED_OUTPUT_DIR: ${{ runner.temp }}/apple-store-signed + APPLE_STORE_APP_P12_BASE64: ${{ secrets.APPLE_STORE_APP_P12_BASE64 }} + APPLE_STORE_APP_P12_PASSWORD: ${{ secrets.APPLE_STORE_APP_P12_PASSWORD }} + APPLE_STORE_INSTALLER_P12_BASE64: ${{ secrets.APPLE_STORE_INSTALLER_P12_BASE64 }} + APPLE_STORE_INSTALLER_P12_PASSWORD: ${{ secrets.APPLE_STORE_INSTALLER_P12_PASSWORD }} + APPLE_STORE_PROFILE_BASE64: ${{ secrets.APPLE_STORE_PROFILE_BASE64 }} + APPLE_STORE_WORKER_PROFILE_BASE64: ${{ secrets.APPLE_STORE_WORKER_PROFILE_BASE64 }} + run: python3 scripts/macstorepackage/ci_sign.py + - name: Clean temporary credentials + if: ${{ always() }} + run: python3 scripts/macstorepackage/ci_sign.py --cleanup + - name: Upload verified candidate and provenance + uses: actions/upload-artifact@v7 + with: + name: apple-store-signed-${{ github.sha }}-${{ github.run_attempt }} + path: | + ${{ runner.temp }}/apple-store-signed/PicFetch.pkg + ${{ runner.temp }}/apple-store-signed/manifest.json + retention-days: 14 + if-no-files-found: error diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 92e52549..ddd7e43d 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -317,6 +317,13 @@ payload. Profile field checks do not establish Apple's CMS/DER or submission acceptance. `test_distribution.py` covers policy and assembly, with opt-in native unsigned installer roundtrip and ad-hoc signature rejection guards. Neither route uploads or installs the result. +`ci_sign.py` wraps distribution signing on GitHub-hosted Macs with temporary +Keychain/profile ownership, sanitized subprocess boundaries and cleanup on +failure/cancellation. `.github/workflows/apple-store.yml` gates a main-only manual +candidate on reusable CI and native package checks, then requires the protected +`apple-store-signing` environment before credentials enter a separate signing job. +Its same-run artifact carries the exact clean source revision; only the installer +and provenance are retained from signing. Account upload/release is separate. `packaging/tools.mk` owns reviewed CLI versions and multiarchitecture image digests consumed by Makefile and the release/Store workflows. diff --git a/packaging/apple-app-store/README.md b/packaging/apple-app-store/README.md index 145c3f81..a8353b2f 100644 --- a/packaging/apple-app-store/README.md +++ b/packaging/apple-app-store/README.md @@ -263,3 +263,75 @@ publication. Portable policy tests run in CI; the native installer roundtrip uses an explicitly unsigned disposable installer and does not prove distribution signing. The remaining SDK/privacy, real certificate/profile, TestFlight and App Store submission gates remain open. + +## CI candidate signing and manual approval + +`.github/workflows/apple-store.yml` prepares signed candidates on demand. It runs +only for `main` in `frathe/picfetch`, pins both checkouts to the dispatch revision, +and requires the existing full CI gate. A credential-free hosted Mac builds the +universal app from pinned runtime archives and runs native packaging guards. +A separate hosted Mac downloads that same-run artifact and signs it only after +the **apple-store-signing** environment is approved. No Apple upload, TestFlight +invitation, App Review submission or automatic release is part of this workflow. +The workflow must reach the default branch before GitHub exposes manual dispatch. + +The environment was configured on 2026-09-30 with **frathe** as required reviewer, +a single `main` branch policy, and administrator bypass disabled. Self-review is +allowed so Ronin can approve a run he started, matching the existing Microsoft +signing workflow. Recheck these settings before adding credentials; merely +naming an environment in YAML does not create its approval rules. GitHub documents +[environment protection](https://docs.github.com/en/actions/how-tos/deploy/configure-and-manage-deployments/manage-environments). + +In repository Settings -> Environments -> **apple-store-signing**, add these +**environment secrets** (not repository-wide secrets): + +| Secret | Value | +|---|---| +| `APPLE_STORE_APP_P12_BASE64` | Base64 of a password-protected `.p12` containing the Apple Distribution certificate and its private key | +| `APPLE_STORE_APP_P12_PASSWORD` | Password for that application `.p12` | +| `APPLE_STORE_INSTALLER_P12_BASE64` | Base64 of a password-protected `.p12` containing the Mac Installer Distribution certificate and its private key | +| `APPLE_STORE_INSTALLER_P12_PASSWORD` | Password for that installer `.p12` | +| `APPLE_STORE_PROFILE_BASE64` | Base64 of the app's Mac App Store Connect `.provisionprofile` | +| `APPLE_STORE_WORKER_PROFILE_BASE64` | Base64 of the worker's Mac App Store Connect `.provisionprofile` | + +Add these **environment variables** in the same environment: + +| Variable | Value | +|---|---| +| `APPLE_STORE_TEAM_ID` | The 10-character Apple developer Team ID | +| `APPLE_STORE_APP_IDENTITY` | Full uppercase 40-character SHA-1 of the application certificate | +| `APPLE_STORE_INSTALLER_IDENTITY` | Full uppercase 40-character SHA-1 of the installer certificate | + +Export only the intended certificate/private-key pair for each `.p12` from +Keychain Access. For hosted CI, those private keys must be available on the +runner; the local-only signing route keeps them on your Mac instead. GitHub's +[Apple signing guide](https://docs.github.com/en/actions/how-tos/deploy/deploy-to-third-party-platforms/sign-xcode-applications) +describes password-protected certificate export and Base64 secrets. Base64 is an +encoding, not encryption. Keep exports outside the repository and enter passwords +through the secret UI. For example, upload one encoded file without printing it: + +```sh +base64 -i /absolute/private/path/application.p12 | \ + gh secret set APPLE_STORE_APP_P12_BASE64 --repo frathe/picfetch --env apple-store-signing +``` + +The wrapper generates its temporary Keychain password; no extra Keychain-password +secret is needed. It imports only during the signing step, restricts key access +to Apple signing tools, uses only that Keychain for identity lookup, and removes +raw exports before calling the packager. It strips encoded credentials/passwords +from the packager environment. Success, failure and handled cancellation restore +the old search list and remove temporary credentials; `always()` retries cleanup. +Hosted runner destruction covers an uncatchable kill. No private-key files or +credential directory is uploaded or cached. + +After merging and configuring the inputs, select Actions -> **Apple Store +candidate** -> Run workflow -> **main**. Review the build summary's source SHA +and artifact hash, then approve the signing environment. Download +`apple-store-signed--` for `PicFetch.pkg` and its provenance manifest. +Signing refuses an artifact whose recorded source differs from the dispatch SHA +or whose source was dirty. Installer payload checks then verify the actual files. + +Credential-free policy/lifecycle tests and workflow linting are available now. +The first approved hosted run with real identities/profiles remains required to +qualify noninteractive Keychain access, profile acceptance and distribution trust. +The SDK/privacy, minimum-OS GUI and Apple submission gates remain unchanged. diff --git a/packaging/apple-app-store/ronin-checklist.md b/packaging/apple-app-store/ronin-checklist.md index 72aee693..7f728cd5 100644 --- a/packaging/apple-app-store/ronin-checklist.md +++ b/packaging/apple-app-store/ronin-checklist.md @@ -84,6 +84,23 @@ installer trust and extracted payloads. The exact invocation is in the [packaging README](README.md). You do not need to assemble that command yourself. Keychain may ask you to approve signing. This work does not upload the app. +## 4a. Enable CI signing with your approval gate + +- [ ] After the workflow reaches `main`, review the **apple-store-signing** + GitHub environment: required reviewer **frathe**, only the `main` branch, + administrator bypass disabled. Pico configured and checked these settings. +- [ ] Export password-protected application and installer `.p12` files from + Keychain Access, including each matching private key, outside this repository. +- [ ] Add the six environment secrets and three environment variables listed + in [CI signing setup](README.md#ci-candidate-signing-and-manual-approval). + This is the hosted-CI alternative to leaving all keys only on your Mac. +- [ ] Start **Apple Store candidate** on `main`, review its build summary, and + approve the signing job. Pico can inspect the resulting candidate and logs. + +No App Store Connect API key is needed for this signing-only workflow. Upload +credentials and release automation are later work. The first real approved CI +signing run is still unverified; current tests use synthetic credential boundaries. + ## 5. Create the App Store Connect app record - [ ] In App Store Connect, use Apps -> + -> New App, platform **macOS**. diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index 05e25ac6..74c6a2f0 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -1463,3 +1463,93 @@ Ronin returned from lunch and explicitly authorized committing and pushing the current state. Updated the checklist/submission/todo status to end the pause. This handoff changes documentation only; local Markdown targets resolve and git diff --check passes. Runtime inspection/CI evidence remains at 62661a1. + +### CI signing preparation — contract (2026-09-30) + +Ronin requested preparing CI signing now. Deep continuation, T0 inline; zero +spawns, two review rounds. Deliver a manually dispatched main-only workflow +that gates on the existing CI, builds a universal qualification artifact without +credentials, then signs it in a separate GitHub-hosted macOS job using the +apple-store-signing environment. Upload only candidate installer/provenance; +App Store upload, release triggers and account changes are out of scope. + +Task graph: credential lifecycle tests -> lifecycle implementation -> workflow +and documentation -> final verification. Files: scripts/macstorepackage/ci_sign.py +and test_ci_sign.py; .github/workflows/apple-store.yml; packaging README/checklist, +architecture, todos and this evidence record. No runtime or dependency change. + +AC1: Reject missing/malformed secrets, non-hosted/non-main contexts and mismatched +source provenance before importing keys. AC2: restrict imported keys to Apple +signing tools; use a private temporary directory/Keychain, suppress secret-bearing +command/error output, strip secrets from child packaging environment. AC3: restore +Keychain search list and delete temporary secrets on success, partial import +failure, signing failure and handled cancellation; retain a workflow always() +cleanup fallback. Verify AC1–3 with python3 -m unittest discover -s +scripts/macstorepackage; fake security/subprocess boundary, no actual private keys. +Native keychain/signing acceptance remains a credential-dependent gate. + +AC4: workflow_dispatch only, fixed main revision, read-only repository permission, +full reusable CI gate, separate build/sign jobs, environment-scoped secrets and +explicit artifact paths. Verify actionlint plus focused workflow contract checks. +AC5: list exact secrets/variables and owner setup order, distinguish candidate +signing from SDK/TestFlight/release approval. Verify local documentation links, +git diff --check, make fmt-check and GoLand inspections of all changed code/YAML. + +GitHub primary references checked: Installing an Apple certificate on macOS +runners for Xcode development; Managing environments for deployment. Use hosted +runner destruction as the final boundary after an uncatchable kill; restore and +remove transient credentials proactively while the process can still run. +No new dependencies or changes to the shipped dependency/license closure. + +CI signing verification and review (2026-09-30): Ronin explicitly requested +manual approval like Microsoft signing, plus commit/push on completion. Created +and read back GitHub environment apple-store-signing: required reviewer frathe +(user 5739279), prevent_self_review=false to permit approving his own dispatch, +can_admins_bypass=false, and exactly one deployment policy (branch main). No +secrets/variables were installed and no signing run/upload was started. Existing +Microsoft environments were read for reference and left unchanged. + +Lifecycle tests first failed on missing validation/import/cleanup; implementation +then passed. Seven CI tests cover input/source rejection, private temporary +storage, two tool-scoped imports, secret-free packaging environment, partial +import/partition/signing failure, KeyboardInterrupt/SIGTERM cleanup, restore +retry, sanitized native errors and the workflow approval/artifact contract. +Full packaging suite with the existing native E2E artifact: 31 tests passed in +9.833s. Later SIGTERM and artifact-ID additions pass the seven focused CI tests. +Isolated mutations removing provenance, secret filtering, cleanup or manual +approval, or enabling unrestricted key access, each fail the expected guard. +No real private keys were used by these tests. + +A real tar create/extract roundtrip preserves every qualified app file hash, +executable bit and manifest byte; the restored app passes native package and +nested/outer signature verification. Existing test apps are unchanged. The +workflow's build job emits the immutable artifact ID; signing downloads that ID, +so failed-job reruns reuse the correct artifact and full reruns get distinct names. +Both checkouts use github.sha and disable persisted Git credentials. Build/sign +use separate hosted runners; credentials enter only the approved signing step. + +Official actionlint 1.7.12 (MIT, development-only; no shipped dependency change) +was obtained from rhysd/actionlint's tagged release and matched its published +SHA-256. The Go proxy route timed out; the release archive worked. Lint first +caught runner.temp at job-env scope; moved those values to step-env scope and +reran cleanly. ShellCheck was unavailable, so actionlint ran with -shellcheck=; +all three multiline workflow shell blocks pass bash -n. Python setup uses the official +MIT-licensed actions/setup-python v7 action with Python 3.12. Its execution/runtime +dependencies stay on CI; no Python runtime or action code is packaged in PicFetch. + +GoLand inspected both Python files and the workflow, including weak warnings. +Two initial python-version schema warnings were checked against upstream v7 +action.yml, which declares that input; a fresh GoLand scan then returned no +findings without suppression. Final workflow and Python inspections are clean. +Python sources parse under the 3.11 grammar; make fmt-check, local Markdown +link checks and git diff --check pass. Two inline reviews, zero agents. Unchanged +Go-runtime build/vet/race evidence carries forward from 62661a1; fresh CI follows +the authorized push. Complete local Docker races remain unavailable on ARM. + +Limits: actual certificate/private-key import and noninteractive signing, +App Store profile acceptance and hosted universal packaging remain unverified +until the workflow reaches main and Ronin supplies the six environment secrets +and three variables. The checklist now includes these inputs and the manual +approval step. The workflow deliberately only retains a candidate .pkg and +provenance; SDK privacy qualification, TestFlight upload and release automation +remain open. No claim of release readiness follows from these tests. diff --git a/scripts/macstorepackage/ci_sign.py b/scripts/macstorepackage/ci_sign.py new file mode 100644 index 00000000..2f8a4da0 --- /dev/null +++ b/scripts/macstorepackage/ci_sign.py @@ -0,0 +1,156 @@ +"""Sign a CI candidate with temporary credentials on a GitHub-hosted Mac. + +The workflow's protected environment supplies approval. This runner guard is +additional validation, not a replacement for GitHub environment protection. +""" +import argparse +import base64 +import binascii +import hashlib +import json +import os +from pathlib import Path +import re +import secrets +import shlex +import shutil +import signal +import subprocess +import sys + + +SECRET_NAMES = tuple('APPLE_STORE_' + name for name in ( + 'APP_P12_BASE64', 'APP_P12_PASSWORD', 'INSTALLER_P12_BASE64', + 'INSTALLER_P12_PASSWORD', 'PROFILE_BASE64', 'WORKER_PROFILE_BASE64')) + + +def public_environment(env): + return {key: value for key, value in env.items() if key not in SECRET_NAMES} + + +def security(*args): + # Do not use package.run: it prints arguments, including Keychain passwords. + try: + result = subprocess.run(['/usr/bin/security', *map(str, args)], check=True, + stdout=subprocess.PIPE, stderr=subprocess.PIPE, + env=public_environment(os.environ)) + except (OSError, subprocess.CalledProcessError): + raise RuntimeError('Apple Keychain operation failed: ' + args[0]) from None + return result.stdout.decode() + + +def work_directory(env): + if (env.get('GITHUB_ACTIONS') != 'true' or env.get('RUNNER_ENVIRONMENT') != 'github-hosted' or + env.get('RUNNER_OS') != 'macOS' or env.get('GITHUB_REF') != 'refs/heads/main' or + env.get('GITHUB_REPOSITORY') != 'frathe/picfetch'): + raise ValueError('CI signing requires the main branch on a GitHub-hosted macOS runner') + root = Path(env.get('RUNNER_TEMP', '')) + if not root.is_absolute() or not root.is_dir(): + raise ValueError('absolute RUNNER_TEMP directory required') + work = root / 'picfetch-apple-signing' + if work.is_symlink(): + raise ValueError('unexpected signing directory symlink') + return work + + +def inputs(env): + for name in (*SECRET_NAMES, 'APPLE_STORE_APP', 'APPLE_STORE_SIGNED_OUTPUT_DIR'): + if not env.get(name): + raise ValueError('missing CI signing input: ' + name) + for name, pattern in (('APPLE_STORE_TEAM_ID', r'[A-Z0-9]{10}'), + ('APPLE_STORE_APP_IDENTITY', r'[A-F0-9]{40}'), + ('APPLE_STORE_INSTALLER_IDENTITY', r'[A-F0-9]{40}'), + ('GITHUB_SHA', r'[a-f0-9]{40}')): + if not re.fullmatch(pattern, env.get(name, '')): + raise ValueError('invalid CI signing input: ' + name) + decoded = {} + for name in SECRET_NAMES: + if name.endswith('_BASE64'): + try: + decoded[name] = base64.b64decode(''.join(env[name].split()), validate=True) + except (ValueError, binascii.Error): + raise ValueError('invalid base64 input: ' + name) from None + if not decoded[name]: + raise ValueError('empty decoded input: ' + name) + manifest = json.loads((Path(env['APPLE_STORE_APP']).parent / 'manifest.json').read_text()) + if (manifest.get('source') != env['GITHUB_SHA'] or manifest.get('dirty') is not False or + manifest.get('source_diff_sha256') != hashlib.sha256(b'').hexdigest() or + manifest.get('untracked_sources') != {}): + raise ValueError('qualification artifact must match the clean workflow revision') + return decoded + + +def cleanup(env): + work = work_directory(env) + if not work.exists(): + return + errors = [] + saved = work / 'search-list.json' + if saved.exists(): + try: + previous = json.loads(saved.read_text()) + security('list-keychains', '-d', 'user', '-s', *previous) + except (OSError, ValueError, RuntimeError) as error: + errors.append(error) + keychain = work / 'signing.keychain-db' + if keychain.exists(): + try: + security('delete-keychain', keychain) + except RuntimeError as error: + errors.append(error) + # Remove raw credentials even if restoring Keychain state needs a retry. + for pattern in ('*.p12', '*.provisionprofile'): + for path in work.glob(pattern): + path.unlink() + if errors: + raise RuntimeError('CI credential cleanup incomplete; always() step must retry') from None + shutil.rmtree(work) + + +def sign(env): + work = work_directory(env) + decoded = inputs(env) + work.mkdir(mode=0o700) # Never reuse another run's credential directory. + try: + previous = shlex.split(security('list-keychains', '-d', 'user')) + (work / 'search-list.json').write_text(json.dumps(previous)) + keychain = work / 'signing.keychain-db' + password = secrets.token_urlsafe(32) + security('create-keychain', '-p', password, keychain) + security('set-keychain-settings', '-lut', '3600', keychain) + security('unlock-keychain', '-p', password, keychain) + for role in ('APP', 'INSTALLER'): + p12 = work / (role.lower() + '.p12') + p12.write_bytes(decoded['APPLE_STORE_' + role + '_P12_BASE64']) + security('import', p12, '-P', env['APPLE_STORE_' + role + '_P12_PASSWORD'], + '-t', 'agg', '-f', 'pkcs12', '-k', keychain, + '-T', '/usr/bin/codesign', '-T', '/usr/bin/productbuild', '-T', '/usr/bin/productsign') + p12.unlink() + security('set-key-partition-list', '-S', 'apple-tool:,apple:,codesign:', '-k', password, keychain) + security('list-keychains', '-d', 'user', '-s', keychain) + child = public_environment(env) | {'APPLE_STORE_TESTFLIGHT': '1'} + for role in ('PROFILE', 'WORKER_PROFILE'): + profile = work / (role.lower() + '.provisionprofile') + profile.write_bytes(decoded['APPLE_STORE_' + role + '_BASE64']) + child['APPLE_STORE_' + role] = str(profile) + subprocess.run([sys.executable, str(Path(__file__).with_name('distribution.py'))], + check=True, env=child) + finally: + cleanup(env) + + +def cancelled(number, _): + raise SystemExit(128 + number) + + +if __name__ == '__main__': + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--cleanup', action='store_true') + options = parser.parse_args() + signal.signal(signal.SIGTERM, cancelled) + signal.signal(signal.SIGINT, cancelled) + try: + (cleanup if options.cleanup else sign)(os.environ) + except (ValueError, RuntimeError, OSError, subprocess.CalledProcessError) as failure: + print('CI signing failed: ' + str(failure), file=sys.stderr) + sys.exit(1) diff --git a/scripts/macstorepackage/test_ci_sign.py b/scripts/macstorepackage/test_ci_sign.py new file mode 100644 index 00000000..f947f61f --- /dev/null +++ b/scripts/macstorepackage/test_ci_sign.py @@ -0,0 +1,165 @@ +"""CI credential lifetime tests: no real Keychain or private keys are used.""" +import base64 +import hashlib +import json +from pathlib import Path +import subprocess +import sys +import tempfile +import unittest +from unittest.mock import patch + +sys.path.insert(0, str(Path(__file__).parent)) +import ci_sign + + +class CICredentials(unittest.TestCase): + def setUp(self): + self.temporary = tempfile.TemporaryDirectory() + self.addCleanup(self.temporary.cleanup) + self.root = Path(self.temporary.name) + self.app = self.root / 'qualified/PicFetch.app' + self.app.mkdir(parents=True) + self.env = {'GITHUB_ACTIONS': 'true', 'RUNNER_ENVIRONMENT': 'github-hosted', + 'RUNNER_OS': 'macOS', 'GITHUB_REF': 'refs/heads/main', + 'GITHUB_REPOSITORY': 'frathe/picfetch', 'GITHUB_SHA': 'a' * 40, + 'RUNNER_TEMP': str(self.root), 'APPLE_STORE_APP': str(self.app), + 'APPLE_STORE_SIGNED_OUTPUT_DIR': str(self.root / 'signed'), + 'APPLE_STORE_TEAM_ID': 'ABCDEFGHIJ', 'APPLE_STORE_APP_IDENTITY': 'A' * 40, + 'APPLE_STORE_INSTALLER_IDENTITY': 'B' * 40} + for name in ('APP_P12', 'INSTALLER_P12', 'PROFILE', 'WORKER_PROFILE'): + self.env['APPLE_STORE_' + name + '_BASE64'] = base64.b64encode(name.encode()).decode() + self.env['APPLE_STORE_APP_P12_PASSWORD'] = 'app-secret' + self.env['APPLE_STORE_INSTALLER_P12_PASSWORD'] = 'installer-secret' + self.manifest = {'source': self.env['GITHUB_SHA'], 'dirty': False, + 'source_diff_sha256': hashlib.sha256(b'').hexdigest(), 'untracked_sources': {}} + self.write_manifest() + self.calls = [] + self.fail_operation = None + + def write_manifest(self): + (self.app.parent / 'manifest.json').write_text(json.dumps(self.manifest)) + + def security(self, *args): + self.calls.append(args) + if args[0] == self.fail_operation: + raise RuntimeError('synthetic keychain failure') + if args == ('list-keychains', '-d', 'user'): + return '"/Users/runner/Library/Keychains/login.keychain-db"\n' + if args[0] == 'create-keychain': + Path(args[-1]).touch() + return '' + + def assert_clean(self): + self.assertFalse((self.root / 'picfetch-apple-signing').exists()) + self.assertIn(('list-keychains', '-d', 'user', '-s', + '/Users/runner/Library/Keychains/login.keychain-db'), self.calls) + self.assertTrue(any(call[0] == 'delete-keychain' for call in self.calls)) + + def test_signs_with_scoped_keys_and_no_secret_environment(self): + def package(*args, **kwargs): + child = kwargs['env'] + self.assertFalse(any('BASE64' in key or 'PASSWORD' in key for key in child)) + self.assertEqual(child['APPLE_STORE_TESTFLIGHT'], '1') + self.assertEqual(Path(child['APPLE_STORE_PROFILE']).read_bytes(), b'PROFILE') + self.assertEqual(Path(child['APPLE_STORE_WORKER_PROFILE']).read_bytes(), b'WORKER_PROFILE') + self.assertEqual((self.root / 'picfetch-apple-signing').stat().st_mode & 0o777, 0o700) + self.assertFalse(list((self.root / 'picfetch-apple-signing').glob('*.p12'))) + with patch.object(ci_sign, 'security', self.security), patch.object(ci_sign.subprocess, 'run', package): + ci_sign.sign(self.env) + imports = [call for call in self.calls if call[0] == 'import'] + self.assertEqual(len(imports), 2) + for call in imports: + self.assertNotIn('-A', call) + self.assertIn('/usr/bin/codesign', call) + self.assertIn('/usr/bin/productbuild', call) + self.assert_clean() + + def test_rejects_missing_or_bad_inputs_before_keychain(self): + for key, value in [('APPLE_STORE_APP_P12_PASSWORD', ''), ('APPLE_STORE_PROFILE_BASE64', '!'), + ('APPLE_STORE_WORKER_PROFILE_BASE64', ''), ('GITHUB_REF', 'refs/heads/topic'), + ('RUNNER_ENVIRONMENT', 'self-hosted'), ('RUNNER_OS', 'Linux'), + ('GITHUB_REPOSITORY', 'someone/fork'), ('APPLE_STORE_TEAM_ID', 'bad'), + ('APPLE_STORE_APP_IDENTITY', 'name'), ('GITHUB_SHA', '')]: + with self.subTest(key=key), patch.object(ci_sign, 'security', self.security): + with self.assertRaises(ValueError): + ci_sign.sign(self.env | {key: value}) + self.assertEqual(self.calls, []) + + def test_rejects_unqualified_source(self): + for key, value in [('source', 'b' * 40), ('dirty', True), + ('source_diff_sha256', 'wrong'), ('untracked_sources', {'extra': 'hash'})]: + with self.subTest(key=key), patch.object(ci_sign, 'security', self.security): + original = self.manifest.copy() + self.manifest[key] = value + self.write_manifest() + with self.assertRaises(ValueError): + ci_sign.sign(self.env) + self.assertEqual(self.calls, []) + self.manifest = original + + def test_cleans_partial_import_and_signing_failure_or_cancellation(self): + for failure in ('import', 'set-key-partition-list', 'package', 'cancel', 'sigterm'): + with self.subTest(failure=failure): + self.calls = [] + self.fail_operation = failure + error = KeyboardInterrupt() if failure == 'cancel' else RuntimeError('failed package') + if failure == 'sigterm': + error = lambda *_args, **_kwargs: ci_sign.cancelled(15, None) + with patch.object(ci_sign, 'security', self.security), \ + patch.object(ci_sign.subprocess, 'run', side_effect=error): + with self.assertRaises((RuntimeError, KeyboardInterrupt, SystemExit)): + ci_sign.sign(self.env) + self.assert_clean() + + def test_cleanup_retries_failed_search_list_restore(self): + work = self.root / 'picfetch-apple-signing' + work.mkdir() + (work / 'search-list.json').write_text(json.dumps(['/original.keychain-db'])) + (work / 'signing.keychain-db').touch() + self.fail_operation = 'list-keychains' + with patch.object(ci_sign, 'security', self.security): + with self.assertRaises(RuntimeError): + ci_sign.cleanup(self.env) + self.assertTrue((work / 'search-list.json').exists()) + self.assertTrue(any(call[0] == 'delete-keychain' for call in self.calls)) + self.fail_operation = None + ci_sign.cleanup(self.env) + self.assertFalse(work.exists()) + + def test_keychain_failure_does_not_disclose_command_or_output(self): + error = subprocess.CalledProcessError(1, ['security', 'import', 'private-secret'], + output=b'private-secret', stderr=b'private-secret') + with patch.object(ci_sign.subprocess, 'run', side_effect=error): + with self.assertRaises(RuntimeError) as caught: + ci_sign.security('import', 'private-secret') + self.assertNotIn('private-secret', str(caught.exception)) + self.assertTrue(caught.exception.__suppress_context__) + + +class CIWorkflow(unittest.TestCase): + def test_signing_requires_main_ci_build_and_environment_approval(self): + workflow = (Path(__file__).resolve().parents[2] / '.github/workflows/apple-store.yml').read_text() + self.assertEqual(workflow.split('on:\n', 1)[1].split('\npermissions:', 1)[0].strip(), + 'workflow_dispatch:') + self.assertIn("github.repository == 'frathe/picfetch' && github.ref == 'refs/heads/main'", workflow) + self.assertIn('uses: ./.github/workflows/ci.yml', workflow) + build, signing = workflow.split(' build:\n', 1)[1].split(' sign:\n', 1) + self.assertIn('needs: test', build) + self.assertNotIn('secrets.', build) + self.assertIn('needs: build', signing) + self.assertIn('artifact-ids: ${{ needs.build.outputs.artifact-id }}', signing) + self.assertIn('artifact-id: ${{ steps.qualified.outputs.artifact-id }}', workflow) + self.assertIn('environment: apple-store-signing', signing) + self.assertIn('if: ${{ always() }}', signing) + self.assertIn('ci_sign.py --cleanup', signing) + self.assertEqual(workflow.count('ref: ${{ github.sha }}'), 2) + self.assertEqual(workflow.count('persist-credentials: false'), 2) + self.assertNotIn('secrets: inherit', workflow) + self.assertNotIn('contents: write', workflow) + for name in ci_sign.SECRET_NAMES: + self.assertIn('${{ secrets.' + name + ' }}', signing) + + +if __name__ == '__main__': + unittest.main() diff --git a/todos.md b/todos.md index 20321500..8c718af4 100644 --- a/todos.md +++ b/todos.md @@ -39,6 +39,9 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Validate copyright/category metadata and reject quarantine attributes in local Store packages. - [ ] Finish GUI sandbox workflows, moved-folder relaunch, and physical Intel/minimum-OS validation. - [x] Implement a separate Store distribution-signing/installer route with identity, profile and payload guards. +- [x] Prepare a main-only CI candidate signing workflow with a separate temporary-Keychain signing job and Ronin's required approval environment. +- [ ] Add the documented Apple CI environment secrets/variables and qualify the first approved hosted signing run after the workflow reaches main. +- [ ] Prepare App Store Connect upload/TestFlight and release automation after signed-candidate and SDK/privacy qualification; keep submission/release decisions separate. - [ ] Run the signed route with real Apple certificates/profiles and validate the resulting candidate on the required distribution path and platform matrix. - [x] Prepare [listing/review notes](packaging/apple-app-store/submission-draft.md) and [ordered owner checklist](packaging/apple-app-store/ronin-checklist.md); inventory privacy/network/export facts. - [x] Preserve exact Abseil privacy manifests/license/provenance; verify original Microsoft runtime signatures before local re-signing. From 71fd5777b18c679e88561b5ee3a09df05f3b04f4 Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 13:28:51 +0200 Subject: [PATCH 16/63] Preserve scoped metadata sorting and moved analysis sources --- ARCHITECTURE.md | 8 +- internal/filesort/filesort.go | 21 +-- internal/filesort/filesort_test.go | 25 ++++ internal/similarity/analyze.go | 13 +- internal/similarity/client.go | 1 + internal/similarity/search_worker.go | 16 +- internal/similarity/worker_access.go | 17 +++ internal/similarity/worker_access_test.go | 169 ++++++++++++++++++++++ plans/2026-09-29-apple-app-store.md | 56 +++++++ todos.md | 2 + 10 files changed, 302 insertions(+), 26 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index ddd7e43d..0a795ac5 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -856,8 +856,9 @@ actual TCP/UDP denial, pipe transfer, cancellation, broker crash, group exit and implicit bookmark transfer from a separate app container. Similarity `worker_access*.go` captures app/model/cache and exact source grants in the existing bounded request stream, retains parent scopes through process -Wait and releases receiving scopes at worker exit. Explorer/search capture -authority before leaving UI. Final Store signing remains qualification work. +Wait and releases receiving scopes at worker exit. Its request-owned source-path +lookup maps file I/O to moved bookmark locations while keeping collection/query, +result and cache identities unchanged. Explorer/search capture authority before leaving UI. Final Store signing remains qualification work. ### `internal/heic` @@ -1237,7 +1238,8 @@ against the `preferences.SortBy*` vocabulary. No Fyne import. ### `internal/filesort` Five orderings the S key cycles, plus `Label` (`lang.L`) and preference -string translation (`FromPref` / `PrefValue`). +string translation (`FromPref` / `PrefValue`). Metadata keys use operation-scoped +`fileaccess.Stat`, including the EXIF-to-mtime fallback. | File | Responsibility | |------|----------------| diff --git a/internal/filesort/filesort.go b/internal/filesort/filesort.go index b6929d90..b912107c 100644 --- a/internal/filesort/filesort.go +++ b/internal/filesort/filesort.go @@ -24,6 +24,7 @@ import ( "fyne.io/fyne/v2" "fyne.io/fyne/v2/lang" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/imaging" "github.com/frathe/picfetch/internal/preferences" ) @@ -104,9 +105,9 @@ func Order(ctx context.Context, m Mode, raw []fyne.URI) []fyne.URI { return date.UnixNano(), err }) case ByModTime: - sortByInt64Key(ctx, ordered, func(u fyne.URI) (int64, error) { return modTimeOf(u).UnixNano(), nil }) + sortByInt64Key(ctx, ordered, func(u fyne.URI) (int64, error) { return modTimeOf(ctx, u).UnixNano(), nil }) case BySize: - sortByInt64Key(ctx, ordered, func(u fyne.URI) (int64, error) { return fileSizeOf(u), nil }) + sortByInt64Key(ctx, ordered, func(u fyne.URI) (int64, error) { return fileSizeOf(ctx, u), nil }) default: // ByName if ctx.Err() == nil { sort.SliceStable(ordered, func(i, j int) bool { @@ -162,12 +163,12 @@ func sortByInt64Key(ctx context.Context, files []fyne.URI, key func(fyne.URI) (i } } -// statFile stats u's filesystem path, reporting ok=false for anything that +// statFile holds u's authority while reading metadata, reporting ok=false for anything that // can't be stat'd (a broken symlink, a permissions error, a file removed // since the scan) so the mtime/size sort keys below have a well-defined // fallback instead of erroring the whole sort out. -func statFile(u fyne.URI) (os.FileInfo, bool) { - info, err := os.Stat(u.Path()) +func statFile(ctx context.Context, u fyne.URI) (os.FileInfo, bool) { + info, err := fileaccess.Stat(ctx, u) if err != nil { return nil, false } @@ -177,16 +178,16 @@ func statFile(u fyne.URI) (os.FileInfo, bool) { // modTimeOf returns u's filesystem modification time, or the zero time if // it can't be stat'd - which sorts first, same as an unreadable capture // date or size does for their own modes. -func modTimeOf(u fyne.URI) time.Time { - if info, ok := statFile(u); ok { +func modTimeOf(ctx context.Context, u fyne.URI) time.Time { + if info, ok := statFile(ctx, u); ok { return info.ModTime() } return time.Time{} } // fileSizeOf returns u's size in bytes, or 0 if it can't be stat'd. -func fileSizeOf(u fyne.URI) int64 { - if info, ok := statFile(u); ok { +func fileSizeOf(ctx context.Context, u fyne.URI) int64 { + if info, ok := statFile(ctx, u); ok { return info.Size() } return 0 @@ -210,7 +211,7 @@ func captureOrModTime(ctx context.Context, u fyne.URI) (time.Time, error) { if ok { return date, nil } - return modTimeOf(u), nil + return modTimeOf(ctx, u), nil } // Label returns the window-title prefix for m, or "" for the diff --git a/internal/filesort/filesort_test.go b/internal/filesort/filesort_test.go index 8e583dd9..36b942fc 100644 --- a/internal/filesort/filesort_test.go +++ b/internal/filesort/filesort_test.go @@ -14,6 +14,7 @@ import ( "fyne.io/fyne/v2/storage" "fyne.io/fyne/v2/test" + "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/uitest" ) @@ -306,3 +307,27 @@ func TestOrder_CaptureDateReadErrorsPreserveFallbackExceptCancellation(t *testin }) } } + +func TestOrderMetadataHonorsRetiredAuthority(t *testing.T) { + for _, mode := range []Mode{ByModTime, BySize, ByCaptureDate} { + t.Run(mode.PrefValue(), func(t *testing.T) { + allowed := storage.NewFileURI(uitest.WriteTempFile(t, "allowed.dat", []byte("a"))) + blocked := storage.NewFileURI(uitest.WriteTempFile(t, "blocked.dat", []byte("larger"))) + base := time.Unix(1700000000, 0) + for i, uri := range []fyne.URI{allowed, blocked} { + modified := base.Add(time.Duration(i) * time.Hour) + if err := os.Chtimes(uri.Path(), modified, modified); err != nil { + t.Fatal(err) + } + } + // The file remains readable to this test process, but its capability + // has retired. A bare path stat would incorrectly bypass that denial. + retired := fileaccess.NewDestination(blocked, func() {}) + fileaccess.ReleaseDestination(retired) + got := Order(context.Background(), mode, []fyne.URI{allowed, retired}) + if !slices.Equal(got, []fyne.URI{retired, allowed}) { + t.Fatalf("metadata sort bypassed retired authority: %v", got) + } + }) + } +} diff --git a/internal/similarity/analyze.go b/internal/similarity/analyze.go index b0558313..b1741e17 100644 --- a/internal/similarity/analyze.go +++ b/internal/similarity/analyze.go @@ -89,13 +89,14 @@ func analyzeLocal(ctx context.Context, req request, controls <-chan Control, emi return err } item := Item{Path: path} + sourcePath := req.SourcePaths.resolve(path) var sourceErr error if !filepath.IsAbs(path) { sourceErr = fmt.Errorf("source must be an absolute local path") } var before os.FileInfo if sourceErr == nil { - before, sourceErr = os.Stat(path) + before, sourceErr = os.Stat(sourcePath) } if sourceErr == nil { item.Size, item.ModifiedNS = before.Size(), before.ModTime().UnixNano() @@ -122,13 +123,13 @@ func analyzeLocal(ctx context.Context, req request, controls <-chan Control, emi } if sourceErr == nil && reused && (item.Facts.Version != FactsVersion || item.Facts.Width <= 0 || item.Facts.Height <= 0) { factsStart := time.Now() - data, bounds, readErr := imaging.ReadAndProbe(ctx, storage.NewFileURI(path)) + data, bounds, readErr := imaging.ReadAndProbe(ctx, storage.NewFileURI(sourcePath)) sourceErr = readErr if sourceErr == nil && fmt.Sprintf("%x", sha256.Sum256(data)) != item.SHA256 { sourceErr = fmt.Errorf("source changed since cached representation") } if sourceErr == nil { - item.Facts, sourceErr = imageFacts(ctx, path, data, bounds) + item.Facts, sourceErr = imageFacts(ctx, sourcePath, data, bounds) backfilled = sourceErr == nil } event.Measurements.DecodeSeconds += time.Since(factsStart).Seconds() @@ -144,10 +145,10 @@ func analyzeLocal(ctx context.Context, req request, controls <-chan Control, emi } } decodeStart := time.Now() - data, bounds, readErr := imaging.ReadAndProbe(ctx, storage.NewFileURI(path)) + data, bounds, readErr := imaging.ReadAndProbe(ctx, storage.NewFileURI(sourcePath)) sourceErr = readErr if sourceErr == nil { - item.Facts, sourceErr = imageFacts(ctx, path, data, bounds) + item.Facts, sourceErr = imageFacts(ctx, sourcePath, data, bounds) } if sourceErr == nil { item.SHA256 = fmt.Sprintf("%x", sha256.Sum256(data)) @@ -175,7 +176,7 @@ func analyzeLocal(ctx context.Context, req request, controls <-chan Control, emi return ctx.Err() } if sourceErr == nil { - after, statErr := os.Stat(path) + after, statErr := os.Stat(sourcePath) if statErr != nil { sourceErr = statErr } else if !os.SameFile(before, after) || after.Size() != item.Size || after.ModTime().UnixNano() != item.ModifiedNS { diff --git a/internal/similarity/client.go b/internal/similarity/client.go index 66e699d9..5fe93b14 100644 --- a/internal/similarity/client.go +++ b/internal/similarity/client.go @@ -102,6 +102,7 @@ type Client struct { } type request struct { + SourcePaths workerSourcePaths `json:",omitempty"` Access []fileaccess.Transfer `json:",omitempty"` Search *SearchRequest `json:",omitempty"` Assets string diff --git a/internal/similarity/search_worker.go b/internal/similarity/search_worker.go index 7cfcc0ac..3cb1ddb9 100644 --- a/internal/similarity/search_worker.go +++ b/internal/similarity/search_worker.go @@ -15,6 +15,7 @@ import ( ) type searchPreparer struct { + sources workerSourcePaths assets string cache *representationStore encoder *Encoder @@ -25,7 +26,8 @@ type searchPreparer struct { func (p *searchPreparer) prepare(ctx context.Context, path string) (Item, bool, error) { item := Item{Path: path} - before, err := os.Stat(path) + sourcePath := p.sources.resolve(path) + before, err := os.Stat(sourcePath) if err != nil { return item, false, err } @@ -43,7 +45,7 @@ func (p *searchPreparer) prepare(ctx context.Context, path string) (Item, bool, reused = true } if !reused { - data, bounds, err := imaging.ReadAndProbe(ctx, storage.NewFileURI(path)) + data, bounds, err := imaging.ReadAndProbe(ctx, storage.NewFileURI(sourcePath)) if err != nil { return item, false, err } @@ -61,7 +63,7 @@ func (p *searchPreparer) prepare(ctx context.Context, path string) (Item, bool, if err != nil { return item, false, err } - item.Facts, err = imageFacts(ctx, path, data, bounds) + item.Facts, err = imageFacts(ctx, sourcePath, data, bounds) if err != nil { return item, false, err } @@ -72,7 +74,7 @@ func (p *searchPreparer) prepare(ctx context.Context, path string) (Item, bool, } item.Preview = preview.Bytes() } - after, err := os.Stat(path) + after, err := os.Stat(p.sources.resolve(path)) if err != nil { return item, false, err } @@ -110,7 +112,7 @@ func searchLocal(ctx context.Context, req request, queries <-chan SearchQuery, e if cache != nil { defer cache.close() } - p := searchPreparer{assets: req.Assets, cache: cache, versions: map[string]os.FileInfo{}} + p := searchPreparer{assets: req.Assets, cache: cache, versions: map[string]os.FileInfo{}, sources: req.SourcePaths} if cacheErr != nil { p.warning = cacheErr.Error() } @@ -141,7 +143,7 @@ func (p *searchPreparer) validate(ctx context.Context, reference Item, matches [ return err } before := p.versions[path] - after, err := os.Stat(path) + after, err := os.Stat(p.sources.resolve(path)) if err != nil || before == nil || !sameVersion(before, after) { return fmt.Errorf("search source changed: %s", path) } @@ -175,7 +177,7 @@ func (p *searchPreparer) refreshFavorites(ctx context.Context, items []Item) { } func (p *searchPreparer) completePreview(ctx context.Context, item Item) (Item, error) { - data, _, err := imaging.ReadAndProbe(ctx, storage.NewFileURI(item.Path)) + data, _, err := imaging.ReadAndProbe(ctx, storage.NewFileURI(p.sources.resolve(item.Path))) if err != nil { return item, err } diff --git a/internal/similarity/worker_access.go b/internal/similarity/worker_access.go index 6dee8669..122331a2 100644 --- a/internal/similarity/worker_access.go +++ b/internal/similarity/worker_access.go @@ -14,6 +14,17 @@ import ( "github.com/frathe/picfetch/internal/heic" ) +// workerSourcePaths keeps collection/cache identities separate from the locations +// resolved while exporting authority. It is immutable after request admission. +type workerSourcePaths map[string]string + +func (paths workerSourcePaths) resolve(path string) string { + if resolved := paths[path]; resolved != "" { + return resolved + } + return path +} + type transferExport func(context.Context, fyne.URI) (fileaccess.Transfer, func(), error) // captureWorkerAccess retains exact source authority through process exit. @@ -48,6 +59,12 @@ func captureWorkerAccess(ctx context.Context, req *request, app string, export t } releases = append(releases, stop) req.Access = append(req.Access, grant) + if grant.Path != path { + if req.SourcePaths == nil { + req.SourcePaths = make(workerSourcePaths) + } + req.SourcePaths[path] = grant.Path + } return nil } for _, path := range []string{app, filepath.Join(req.Assets, "vision_model.onnx"), filepath.Join(req.Assets, "preprocessor_config.json")} { diff --git a/internal/similarity/worker_access_test.go b/internal/similarity/worker_access_test.go index ee6db392..026b289e 100644 --- a/internal/similarity/worker_access_test.go +++ b/internal/similarity/worker_access_test.go @@ -2,13 +2,22 @@ package similarity import ( "context" + "crypto/sha256" + "encoding/json" "errors" + "fmt" + "image/color" + "os" + "path/filepath" + "slices" "testing" + "time" "fyne.io/fyne/v2" "fyne.io/fyne/v2/storage" "github.com/frathe/picfetch/internal/fileaccess" + "github.com/frathe/picfetch/internal/uitest" ) func TestWorkerAccessCapturesExactPathsUntilRetirement(t *testing.T) { @@ -66,3 +75,163 @@ func TestWorkerAccessFailureRetiresPriorGrants(t *testing.T) { t.Fatalf("err=%v retired=%d", err, retired) } } + +func TestWorkerAccessMovedSourcesKeepCollectionIdentity(t *testing.T) { + checkScans := RegisterLocalFiles() + t.Cleanup(func() { + if err := checkScans(); err != nil { + t.Error(err) + } + }) + for _, search := range []bool{false, true} { + t.Run(fmt.Sprintf("search=%t", search), func(t *testing.T) { + ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second) + defer cancel() + policy := cacheTestPolicy(t) + items := []Item{cacheFixtureItem(t, "one.jpg"), cacheFixtureItem(t, "two.jpg")} + store := cacheTestStore(t, policy) + moved := map[string]string{} + var paths []string + for i := range items { + item := &items[i] + data := uitest.EncodeJPEG(t, 32, 24, color.White) + if err := os.WriteFile(item.Path, data, 0600); err != nil { + t.Fatal(err) + } + info, err := os.Stat(item.Path) + if err != nil { + t.Fatal(err) + } + item.Path = storage.NewFileURI(item.Path).Path() + item.Size, item.ModifiedNS = info.Size(), info.ModTime().UnixNano() + item.SHA256 = fmt.Sprintf("%x", sha256.Sum256(data)) + if err := store.write(ctx, *item); err != nil { + t.Fatal(err) + } + paths = append(paths, item.Path) + newPath := storage.NewFileURI(filepath.Join(t.TempDir(), "moved.jpg")).Path() + if err := os.Rename(item.Path, newPath); err != nil { + t.Fatal(err) + } + moved[item.Path] = newPath + } + req := request{Assets: t.TempDir(), Paths: slices.Clone(paths), + GeneralAnalysisDir: policy.Roots.GeneralDir, GeneralAnalysisLimitBytes: policy.GeneralLimitBytes} + if search { + req.Search = &SearchRequest{SessionID: 7, Paths: slices.Clone(paths), Cache: policy} + } + active := 0 + release, err := captureWorkerAccess(ctx, &req, t.TempDir(), func(_ context.Context, uri fyne.URI) (fileaccess.Transfer, func(), error) { + path := uri.Path() + if resolved := moved[path]; resolved != "" { + path = resolved + } + active++ + return fileaccess.Transfer{Path: path, Bookmark: []byte("test grant")}, func() { active-- }, nil + }) + if err != nil { + t.Fatal(err) + } + defer release() + if !slices.Equal(req.Paths, paths) || search && !slices.Equal(req.Search.Paths, paths) { + t.Fatal("capture changed collection identities") + } + // Exercise the actual serialized boundary, not an in-process-only map. + wire, err := json.Marshal(req) + if err != nil { + t.Fatal(err) + } + var decoded request + if err := json.Unmarshal(wire, &decoded); err != nil { + t.Fatal(err) + } + for original, resolved := range moved { + if decoded.SourcePaths.resolve(original) != resolved { + t.Fatalf("worker still reads stale source %s", original) + } + } + preparer := searchPreparer{cache: store, versions: map[string]os.FileInfo{}, sources: decoded.SourcePaths} + for _, item := range items { + got, reused, err := preparer.prepare(ctx, item.Path) + if err != nil || !reused || got.Path != item.Path { + t.Fatalf("moved preparation: %+v, reused=%t, %v", got, reused, err) + } + preview, err := preparer.completePreview(ctx, got) + if err != nil || len(preview.Preview) == 0 || preview.Path != item.Path { + t.Fatalf("moved Favorite preview: %v", err) + } + } + if err := preparer.validate(ctx, items[0], []Match{{Path: items[1].Path}}, true); err != nil { + t.Fatal(err) + } + queries := make(chan SearchQuery, 1) + queries <- SearchQuery{ID: 1, ReferencePath: paths[0]} + var matches []Match + err = preparer.run(ctx, SearchRequest{SessionID: 7, Paths: paths, Cache: policy}, queries, func(event SearchEvent) error { + if event.Kind == SearchFinal { + matches = event.Matches + } + if event.Kind == SearchReady { + close(queries) + } + return nil + }) + if err != nil || len(matches) != 1 || matches[0].Path != paths[1] { + t.Fatalf("original query/result identity: %v, %v", matches, err) + } + if active == 0 { + t.Fatal("grants retired before worker work") + } + release() + if active != 0 { + t.Fatal("grants leaked") + } + if err := os.WriteFile(moved[paths[0]], []byte("changed"), 0600); err != nil { + t.Fatal(err) + } + if err := preparer.validate(ctx, items[0], nil, false); err == nil { + t.Fatal("moved source change missed") + } + }) + } +} + +func TestWorkerMovedSourceAnalyzer(t *testing.T) { + assets := os.Getenv("PICFETCH_SIMILARITY_ASSETS") + if assets == "" { + t.Skip("requires pinned native analysis assets") + } + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + item := cacheFixtureItem(t, "original.jpg") + policy := cacheTestPolicy(t) + store := cacheTestStore(t, policy) + if err := store.write(ctx, item); err != nil { + t.Fatal(err) + } + store.close() + moved := filepath.Join(t.TempDir(), "moved.jpg") + if err := os.Rename(item.Path, moved); err != nil { + t.Fatal(err) + } + req := request{Assets: assets, Paths: []string{item.Path}, SourcePaths: workerSourcePaths{item.Path: moved}, + GeneralAnalysisDir: policy.Roots.GeneralDir, GeneralAnalysisLimitBytes: policy.GeneralLimitBytes} + executable, err := os.Executable() + if err != nil { + t.Fatal(err) + } + cmd := (Client{}).workerCommand(ctx, executable) + cmd.Env = append(os.Environ(), workerEnvironment+"=1") + var final Event + err = analyzeCommand(ctx, cmd, req, nil, func(event Event) { + if event.Complete { + final = event + } + }) + if err != nil { + t.Fatal(err) + } + if final.Successful != 1 || final.Reused != 1 || final.Failed != 0 || len(final.Items) != 1 || final.Items[0].Path != item.Path { + t.Fatalf("moved analyzer source lost identity or reuse: %+v", final) + } +} diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index 74c6a2f0..b791b35c 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -1553,3 +1553,59 @@ and three variables. The checklist now includes these inputs and the manual approval step. The workflow deliberately only retains a candidate .pkg and provenance; SDK privacy qualification, TestFlight upload and release automation remain open. No claim of release readiness follows from these tests. + +### PR #75 review loop — first code findings + +Ronin invoked the GitHub review loop. At 710ba5c, GoLand/actionlint and focused +packaging checks are clean; fresh Qodana SARIF has the same eight validated +unused-function false positives. CodeQL passes. Earlier da2df0c Windows native +clipboard test failed at its 30s child deadline (multiple case 30.03s, single +3.87s); current 710ba5c passed that guard unchanged. Do not infer a product fix +or relax the guard from this observation. Captured events retained in scratch. + +Codex reviewed da2df0c and reported two P2 findings, confirmed in current code: +4143950052: filesort mtime/size/EXIF-fallback stats bypass fileaccess acquisition. +4143950062: analysis/search transfers grant a resolved moved bookmark path but +worker I/O still uses the original path. Security review is still running. + +Deep continuation, both fixes T0 inline, zero spawns/two reviews. First reproduce +sort authority bypass through a closed destination capability with a still-existing +file; verify all three sort modes honor the denied scope, while ordinary sorting +and cancellation remain covered. Route stat through fileaccess.Stat with the +operation context, preserving the existing unavailable-key fallback. + +Keep analysis/search public paths, queries, results and cache membership unchanged. +Capture a request-owned original-to-resolved source lookup alongside grants; use +that lookup for worker stat/decode/version checks and Favorite preview reads. +Cloning/mutating public source lists or translating result identities is unnecessary. +Verify moved files with real rename and cached representations, retained-search +queries/validation/preview reads, and analyzer output when pinned assets exist. +Run focused race tests, native Store build checks and GoLand on changed files; +use latest native CI for full cross-platform verification. These changes do not +alter entitlements, runtime dependencies or the network boundary. + +First findings verification: the metadata-capability regression failed all three +modes before the fix, then filesort race tests passed (1.499s). Worker capture +regressions failed both Analyze/Search request forms before recording the resolved +lookup. The real isolated analyzer with pinned local assets failed with "no images +were represented successfully" on a renamed source before the I/O fix. Focused +analysis/search/cache/protocol race checks now pass (10.121s), including cached +moved-file reuse, unchanged query/result identities, preview regeneration, source +change detection and grant retirement. Test setup registers the same read-only +file repository production uses; no UI or desktop integration is touched. + +The native analyzer test is opt-in via PICFETCH_SIMILARITY_ASSETS and runs in the +real isolated child; ordinary package runs without pinned assets skip that one +case. Both hosted macOS native jobs already install assets and run the complete +similarity package, so they also execute it. Store-tagged focused filesort and +worker-access tests pass (0.327s/0.324s). make verify-build passes notice/generated +checks, vet and build. GoLand inspected all seven changed code/test files, +including weak warnings, with no findings. Existing exact Qodana exclusions cover +both test files; no new root UI tests or shard changes. Two reviews, zero agents. + +CI/CodeQL at 710ba5c completed successfully across every required platform/race +job. Its Qodana report was read and the eight false positives rechecked against +current callers. Codex's security review at da2df0c completed without findings; +the code review's two P2 findings are addressed by this pending fix. After push, +reply and resolve both threads, then request another code/security review of the +new head. Older clean gates do not count as completion of this requested loop. diff --git a/todos.md b/todos.md index 8c718af4..09bd1e9a 100644 --- a/todos.md +++ b/todos.md @@ -36,6 +36,8 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Persist explicit sibling-folder approvals in app preferences and validate/reuse bookmarks on fresh single-image opens; keep renewal, cancellation and selection guards. - [x] Ronin confirmed persistent folder approvals work across real quit/relaunch in `bin/apple-store-e2e-persistent-folders/PicFetch.app`. - [ ] Confirm moved-folder bookmark reuse in the persistent-folder test app; automated coverage passes, live moved-folder testing remains open. +- [x] Fix PR #75 findings: acquire source authority for metadata sorting and map analysis/search file I/O to resolved bookmark locations while preserving collection/cache identities. +- [ ] Finish the requested PR #75 review loop: fresh clean Codex code/security review and native CI on the latest fix commit. - [x] Validate copyright/category metadata and reject quarantine attributes in local Store packages. - [ ] Finish GUI sandbox workflows, moved-folder relaunch, and physical Intel/minimum-OS validation. - [x] Implement a separate Store distribution-signing/installer route with identity, profile and payload guards. From 0ecfaf69c9927f495e5cc1e4d66cf760aad7fff8 Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 13:47:08 +0200 Subject: [PATCH 17/63] Scope development worker fixture to ordinary builds --- internal/similarity/worker_access_test.go | 7 +++++ plans/2026-09-29-apple-app-store.md | 32 +++++++++++++++++++++++ todos.md | 4 +-- 3 files changed, 41 insertions(+), 2 deletions(-) diff --git a/internal/similarity/worker_access_test.go b/internal/similarity/worker_access_test.go index 026b289e..606ff0dd 100644 --- a/internal/similarity/worker_access_test.go +++ b/internal/similarity/worker_access_test.go @@ -16,6 +16,7 @@ import ( "fyne.io/fyne/v2" "fyne.io/fyne/v2/storage" + "github.com/frathe/picfetch/internal/distribution" "github.com/frathe/picfetch/internal/fileaccess" "github.com/frathe/picfetch/internal/uitest" ) @@ -197,6 +198,12 @@ func TestWorkerAccessMovedSourcesKeepCollectionIdentity(t *testing.T) { } func TestWorkerMovedSourceAnalyzer(t *testing.T) { + // This fixture launches the Go test executable with development assets. + // Store workers require an installed package's runtime and native broker. + //goland:noinspection GoBoolExpressions + if distribution.StoreManaged { + t.Skip("requires an ordinary worker executable; Store workers need a packaged fixture") + } assets := os.Getenv("PICFETCH_SIMILARITY_ASSETS") if assets == "" { t.Skip("requires pinned native analysis assets") diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index b791b35c..335babf8 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -1609,3 +1609,35 @@ current callers. Codex's security review at da2df0c completed without findings; the code review's two P2 findings are addressed by this pending fix. After push, reply and resolve both threads, then request another code/security review of the new head. Older clean gates do not count as completion of this requested loop. + +### PR #75 follow-up — development worker fixture admission + +Signed/pushed 71fd577 fixes both confirmed review findings. Both threads were +answered with regression evidence and resolved; the fresh code review on that +commit reports no findings. Security review is pending. Its CodeQL checks pass; +Qodana's post-suppression SARIF identifies exactly eight unused-export false +positives, each verified against production callers (including filesort's scoped Stat call). +Both macOS architectures and Linux native guards pass. Latest gate status and +final review evidence are retained on PR #75; this record does not predeclare a +clean final round. + +Windows Store CI exposed a test-fixture mismatch in TestWorkerMovedSourceAnalyzer: +the unbundled Go test executable cannot load native code from development assets +under Store policy. The error names the missing runtime beside the test executable; +it is unrelated to source mapping. Restrict this opt-in subprocess fixture to +ordinary builds, where it can launch its actual worker with verified pinned assets. +Store-tagged TestWorkerAccessMovedSourcesKeepCollectionIdentity still exercises +both Analyze/Search serialization, resolved reads, preview, retained queries, +version validation and grant lifetime. Existing packaged worker/network guards +and Store runtime-admission tests remain unchanged. + +Focused ordinary worker/access race tests with pinned assets pass (2.695s), +including the actual analyzer child. With the same development environment set, +Apple Store-tagged access and download-policy tests pass (0.329s); only the +unbundled subprocess fixture skips explicitly. GoLand inspected the changed test +including weak warnings with no findings. This is a test-only follow-up with no +application change: the fresh bin/apple-store-e2e-review-fixes/PicFetch.app and +adjacent testing notes remain valid. Its 31 package guards passed (9.612s), and +all 24 ZIP payload hashes match the manifest. Full Windows confirmation and a +fresh code/security review follow this correction's push. Lead-only Thin fix, +zero agents; one inline review. No dependency or permission changes. diff --git a/todos.md b/todos.md index 09bd1e9a..bd90ca42 100644 --- a/todos.md +++ b/todos.md @@ -29,7 +29,7 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Qualify retained visual search through production ARM/Intel-Rosetta workers; keep scopes through clipboard/Finder/Trash calls and source metadata reads. - [x] Capture native window-drop URLs before GLFW flattens paths; refresh restored moved/stale bookmarks with immutable persistence coverage. - [x] Verify production workers deny an ungranted private source while an allowed control image still completes inference, on ARM and Intel under Rosetta. -- [x] Add explicit Store Open-dialog folder consent for single-image sibling navigation; preserve cancellation and the initially selected image. Latest E2E app is in `bin/apple-store-e2e-persistent-folders/`. +- [x] Add explicit Store Open-dialog folder consent for single-image sibling navigation; preserve cancellation and the initially selected image. Latest E2E app is in `bin/apple-store-e2e-review-fixes/`. - [x] Ronin confirmed native Open-dialog folder consent fixes navigation; extend the same consent to shared drop/Open With discovery with cancellation and replacement coverage. - [ ] Confirm the rebuilt shared-consent app with a single-file window drop and Open With. - [x] Ronin confirmed multiple-file picker/drop navigation must remain within the selected images; folder sibling discovery applies only to single-file inputs. Automated selection coverage passes. @@ -37,7 +37,7 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Ronin confirmed persistent folder approvals work across real quit/relaunch in `bin/apple-store-e2e-persistent-folders/PicFetch.app`. - [ ] Confirm moved-folder bookmark reuse in the persistent-folder test app; automated coverage passes, live moved-folder testing remains open. - [x] Fix PR #75 findings: acquire source authority for metadata sorting and map analysis/search file I/O to resolved bookmark locations while preserving collection/cache identities. -- [ ] Finish the requested PR #75 review loop: fresh clean Codex code/security review and native CI on the latest fix commit. +- [ ] Finish the requested [PR #75 review loop](https://github.com/frathe/picfetch/pull/75): final review/CI evidence lives on the PR. Code review is clean at `71fd577`; verify the subsequent development-fixture correction on Windows and obtain fresh code/security review. - [x] Validate copyright/category metadata and reject quarantine attributes in local Store packages. - [ ] Finish GUI sandbox workflows, moved-folder relaunch, and physical Intel/minimum-OS validation. - [x] Implement a separate Store distribution-signing/installer route with identity, profile and payload guards. From 0a670e64ccc10a4e4a9ee6a262906fb5546f2f08 Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 14:17:32 +0200 Subject: [PATCH 18/63] Pin Apple signing workflow actions to verified commits --- .github/workflows/apple-store.yml | 16 ++++----- plans/2026-09-29-apple-app-store.md | 44 +++++++++++++++++++++++++ scripts/macstorepackage/test_ci_sign.py | 11 +++++++ todos.md | 2 +- 4 files changed, 64 insertions(+), 9 deletions(-) diff --git a/.github/workflows/apple-store.yml b/.github/workflows/apple-store.yml index 7c99600b..a52bed9d 100644 --- a/.github/workflows/apple-store.yml +++ b/.github/workflows/apple-store.yml @@ -23,15 +23,15 @@ jobs: outputs: artifact-id: ${{ steps.qualified.outputs.artifact-id }} steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 with: ref: ${{ github.sha }} persist-credentials: false - - uses: actions/setup-go@v7 + - uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7 with: go-version-file: go.mod cache: false - - uses: actions/setup-python@v7 + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7 with: python-version: '3.12' - name: Download pinned runtime archives @@ -67,7 +67,7 @@ jobs: echo 'This workflow does not upload to Apple or release the app.' shasum -a 256 "$RUNNER_TEMP/apple-store-qualified.tgz" } >> "$GITHUB_STEP_SUMMARY" - - uses: actions/upload-artifact@v7 + - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 id: qualified with: name: apple-store-qualified-${{ github.sha }}-${{ github.run_attempt }} @@ -87,14 +87,14 @@ jobs: APPLE_STORE_APP_IDENTITY: ${{ vars.APPLE_STORE_APP_IDENTITY }} APPLE_STORE_INSTALLER_IDENTITY: ${{ vars.APPLE_STORE_INSTALLER_IDENTITY }} steps: - - uses: actions/checkout@v7 + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 with: ref: ${{ github.sha }} persist-credentials: false - - uses: actions/setup-python@v7 + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7 with: python-version: '3.12' - - uses: actions/download-artifact@v8 + - uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8 with: artifact-ids: ${{ needs.build.outputs.artifact-id }} merge-multiple: true @@ -118,7 +118,7 @@ jobs: if: ${{ always() }} run: python3 scripts/macstorepackage/ci_sign.py --cleanup - name: Upload verified candidate and provenance - uses: actions/upload-artifact@v7 + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7 with: name: apple-store-signed-${{ github.sha }}-${{ github.run_attempt }} path: | diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index 335babf8..f86e385a 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -1641,3 +1641,47 @@ adjacent testing notes remain valid. Its 31 package guards passed (9.612s), and all 24 ZIP payload hashes match the manifest. Full Windows confirmation and a fresh code/security review follow this correction's push. Lead-only Thin fix, zero agents; one inline review. No dependency or permission changes. + +### PR #75 security follow-up — immutable action references + +Security review at 71fd577 reported advisory P2 4144350716: mutable action tags +in the signing job permit future retargeting before the secret-consuming step. +Confirmed. Pin every external action in both candidate-build and signing jobs; +the credential-free build also supplies the executable artifact that gets signed. +Keep the main-only gate, environment approval, exact artifact ID and cleanup. +The same-repository reusable CI workflow is already bound to this source commit. +Existing weekly GitHub Actions Dependabot configuration covers pin updates. + +On 2026-09-30 the official actions repositories resolved the existing major tags +to these commits, each with GitHub API commit verification reported valid: + +| Action | Existing tag | Immutable commit | +| --- | --- | --- | +| actions/checkout | v7 | 3d3c42e5aac5ba805825da76410c181273ba90b1 | +| actions/setup-go | v7 | b7ad1dad31e06c5925ef5d2fc7ad053ef454303e | +| actions/setup-python | v7 | 5fda3b95a4ea91299a34e894583c3862153e4b97 | +| actions/upload-artifact | v7 | 043fb46d1a93c77aae656e7c1c64a875d1fc6a0a | +| actions/download-artifact | v8 | 3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c | + +Exact upstream sources are https://github.com/actions//commit/ +from this table. Each pinned LICENSE is MIT; action.yml uses bundled Node 24 +JavaScript entrypoints, not composite actions with further mutable action refs. +These unchanged CI-only dependencies and their bundled dependencies stay on +hosted runners; no action code enters the app or adds shipped notice obligations. +Pinning freezes the selected release; it is not an exhaustive upstream code audit. + +A workflow policy regression first rejected all eight floating references, then +passed after pinning. All eight CI tests and all 32 native/portable packaging +checks pass (9.560s). actionlint 1.7.12 passes with ShellCheck unavailable as +previously recorded; shell bodies are unchanged. GoLand inspected the test and +workflow including weak warnings. Initial action-input warnings were checked +against the exact pinned action.yml files; a fresh inspection is clean without +suppression. Standard continuation, lead-only fix/review, zero agents. Full CI, +CodeQL, fresh Qodana SARIF and another code/security review follow the push. + +At 0ecfaf6 both native macOS guard artifacts explicitly report the moved-source +analyzer test passing (ARM 0.44s, Intel 1.49s). All Linux race partitions, native +Linux/macOS, validation and CodeQL pass. Fresh Qodana SARIF contains the same +eight validated false positives. Windows first attempt hit the unchanged +clipboard PowerShell test deadline at 30.01s; only the failed Windows job was +rerun with original assertions/timeouts. Its result remains on the PR CI run. diff --git a/scripts/macstorepackage/test_ci_sign.py b/scripts/macstorepackage/test_ci_sign.py index f947f61f..984796f1 100644 --- a/scripts/macstorepackage/test_ci_sign.py +++ b/scripts/macstorepackage/test_ci_sign.py @@ -3,6 +3,7 @@ import hashlib import json from pathlib import Path +import re import subprocess import sys import tempfile @@ -138,6 +139,16 @@ def test_keychain_failure_does_not_disclose_command_or_output(self): class CIWorkflow(unittest.TestCase): + def test_candidate_actions_use_immutable_commits(self): + workflow = (Path(__file__).resolve().parents[2] / '.github/workflows/apple-store.yml').read_text() + actions = re.findall(r'^\s*(?:-\s+)?uses:\s*(\S+)', workflow, re.MULTILINE) + self.assertTrue(actions) + for action in actions: + if action == './.github/workflows/ci.yml': + continue + with self.subTest(action=action): + self.assertRegex(action, r'^actions/[a-z-]+@[0-9a-f]{40}$') + def test_signing_requires_main_ci_build_and_environment_approval(self): workflow = (Path(__file__).resolve().parents[2] / '.github/workflows/apple-store.yml').read_text() self.assertEqual(workflow.split('on:\n', 1)[1].split('\npermissions:', 1)[0].strip(), diff --git a/todos.md b/todos.md index bd90ca42..03659800 100644 --- a/todos.md +++ b/todos.md @@ -37,7 +37,7 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Ronin confirmed persistent folder approvals work across real quit/relaunch in `bin/apple-store-e2e-persistent-folders/PicFetch.app`. - [ ] Confirm moved-folder bookmark reuse in the persistent-folder test app; automated coverage passes, live moved-folder testing remains open. - [x] Fix PR #75 findings: acquire source authority for metadata sorting and map analysis/search file I/O to resolved bookmark locations while preserving collection/cache identities. -- [ ] Finish the requested [PR #75 review loop](https://github.com/frathe/picfetch/pull/75): final review/CI evidence lives on the PR. Code review is clean at `71fd577`; verify the subsequent development-fixture correction on Windows and obtain fresh code/security review. +- [ ] Finish the requested [PR #75 review loop](https://github.com/frathe/picfetch/pull/75): final review/CI evidence lives on the PR. Code review is clean at `71fd577`; pin candidate/signing actions for the confirmed security advisory and obtain fresh code/security review plus passing CI on the final head. - [x] Validate copyright/category metadata and reject quarantine attributes in local Store packages. - [ ] Finish GUI sandbox workflows, moved-folder relaunch, and physical Intel/minimum-OS validation. - [x] Implement a separate Store distribution-signing/installer route with identity, profile and payload guards. From 4a8048405bef8a1eff0a00190a2452f8b12742df Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 14:41:00 +0200 Subject: [PATCH 19/63] Record clean Apple Store preparation review evidence --- plans/2026-09-29-apple-app-store.md | 32 +++++++++++++++++++++++++++++ todos.md | 2 +- 2 files changed, 33 insertions(+), 1 deletion(-) diff --git a/plans/2026-09-29-apple-app-store.md b/plans/2026-09-29-apple-app-store.md index f86e385a..84fc1ce6 100644 --- a/plans/2026-09-29-apple-app-store.md +++ b/plans/2026-09-29-apple-app-store.md @@ -1685,3 +1685,35 @@ Linux/macOS, validation and CodeQL pass. Fresh Qodana SARIF contains the same eight validated false positives. Windows first attempt hit the unchanged clipboard PowerShell test deadline at 30.01s; only the failed Windows job was rerun with original assertions/timeouts. Its result remains on the PR CI run. + +### PR #75 accepted review outcome — application/CI revision 0a670e6 + +All three confirmed findings are fixed and their threads answered/resolved: +metadata-sort authority and moved analysis/search sources in 71fd577, immutable +candidate/signing action pins in 0a670e6. The intermediate test-only 0ecfaf6 +restricts the unbundled analyzer fixture to ordinary builds while preserving +Store-tagged source mapping and installed-runtime policy guards. + +On 2026-09-30, Codex's fresh code review completed at 12:26 UTC with no findings; +its fresh security review completed at 12:39 UTC explicitly reporting no security +issues, both naming 0a670e6. All three review threads are resolved and pagination +is exhausted. The summary's older action-pin advisory is retained history; the +latest security comment reports a clean result. No merge or release occurred. + +Full CI run 36713820517 passes: all four Linux race partitions, Linux native, +Windows on its first attempt, both macOS architectures and validation. CodeQL +run 36713820552 passes Go and Actions analysis. Qodana run 36713820387 completed; +its post-suppression /qodana.sarif.json identifies 0a670e6 and the same eight +unused-export false positives, with production callers verified and no actionable +findings. GoLand evidence for changed code/workflow/test files, including weak +warnings, remains the clean inspections recorded above. The 32 local package +checks and actionlint pass. Full local Docker races remain unavailable on ARM; +the complete native-amd64 CI suite supplies that gate. + +This final documentation-only record carries unchanged-code inspection evidence +from 0a670e6. A final fresh code/security review and CI run for its documentation +commit are tracked directly on PR #75. The PR is the live source for those final +checks, avoiding further status-only commits that would invalidate their head. +Apple certificates/profiles, SDK/privacy qualification and hands-on distribution +validation remain open in the owner checklist; a clean code review does not +establish App Store release readiness. diff --git a/todos.md b/todos.md index 03659800..074fad06 100644 --- a/todos.md +++ b/todos.md @@ -37,7 +37,7 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 - [x] Ronin confirmed persistent folder approvals work across real quit/relaunch in `bin/apple-store-e2e-persistent-folders/PicFetch.app`. - [ ] Confirm moved-folder bookmark reuse in the persistent-folder test app; automated coverage passes, live moved-folder testing remains open. - [x] Fix PR #75 findings: acquire source authority for metadata sorting and map analysis/search file I/O to resolved bookmark locations while preserving collection/cache identities. -- [ ] Finish the requested [PR #75 review loop](https://github.com/frathe/picfetch/pull/75): final review/CI evidence lives on the PR. Code review is clean at `71fd577`; pin candidate/signing actions for the confirmed security advisory and obtain fresh code/security review plus passing CI on the final head. +- [x] Complete the application/CI review fixes in [PR #75](https://github.com/frathe/picfetch/pull/75): `0a670e6` has fresh clean code/security reviews, passing platform/race CI and CodeQL, and no actionable local/Qodana findings. All three threads are resolved. Final documentation-commit review/CI evidence lives on the PR. - [x] Validate copyright/category metadata and reject quarantine attributes in local Store packages. - [ ] Finish GUI sandbox workflows, moved-folder relaunch, and physical Intel/minimum-OS validation. - [x] Implement a separate Store distribution-signing/installer route with identity, profile and payload guards. From 8533342bf662723dbda67a43ca96a9fab58761ef Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 15:51:51 +0200 Subject: [PATCH 20/63] feat: animate Trane in console help --- ARCHITECTURE.md | 13 ++ Makefile | 4 + .../2026-09-30-trane-ascii.md | 85 ++++++++ go.mod | 2 +- internal/consolehelp/help.go | 141 ++++++++++++++ internal/consolehelp/help_test.go | 182 ++++++++++++++++++ internal/consolehelp/render.go | 104 ++++++++++ internal/consolehelp/terminal_other.go | 9 + internal/consolehelp/terminal_windows.go | 21 ++ internal/launch/launch.go | 2 +- main.go | 6 +- main_test.go | 4 +- qodana.yaml | 1 + scripts/traneascii/README.md | 32 +++ scripts/traneascii/main.go | 18 ++ todos.md | 4 + 16 files changed, 623 insertions(+), 5 deletions(-) create mode 100644 finished_refactorings/2026-09-30-trane-ascii.md create mode 100644 internal/consolehelp/help.go create mode 100644 internal/consolehelp/help_test.go create mode 100644 internal/consolehelp/render.go create mode 100644 internal/consolehelp/terminal_other.go create mode 100644 internal/consolehelp/terminal_windows.go create mode 100644 scripts/traneascii/README.md create mode 100644 scripts/traneascii/main.go diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 0a795ac5..f2b67318 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -27,6 +27,19 @@ policy enters UI composition before preferences/session access. It loads embedde Help's offline Licenses and Privacy policy windows before startup. `main_darwin_test.go` asserts the graft landed — this is the only test binary that links the Cocoa driver. +### `scripts/traneascii` + +Developer preview of the production console help, reached through `make trane`. +`main.go` joins `launch.Usage` to `consolehelp.Write` without linking the desktop. + +### `internal/consolehelp` + +`Write` renders one eased, three-second ASCII Trane turn above terminal help, +then leaves a front portrait and PicFetch wordmark. `render.go` owns the original +ellipsoid sculpture and depth-tested lighting; `help.go` owns fitting, wrapping, +signals and terminal restoration. Build-tagged terminal files enable/restore +Windows VT mode. Pipes, small terminals and `TERM=dumb` receive plain usage. + ### `scripts/historymovie` Development-only Git history films, reached through `make movie` and diff --git a/Makefile b/Makefile index 8a92821c..4136c8da 100644 --- a/Makefile +++ b/Makefile @@ -178,6 +178,10 @@ location-map-qualify: build ## Collect native macOS Location Map evidence from e location-map-check-evidence: build ## Validate native evidence against this build and the explicit expected image count go run ./scripts/locationmapqualify check -evidence "$(LOCATION_MAP_EVIDENCE)" -images "$(LOCATION_MAP_EXPECTED_IMAGES)" -binary "$(BIN_DIR)/$(BIN_NAME)" +.PHONY: trane +trane: ## Preview the app's animated Trane console help (one turn; Ctrl+C skips) + go run -tags "$(APP_TAGS)" ./scripts/traneascii + MOVIE_SECONDS ?= 180 MOVIE_DIR ?= .scratch/history-movies export MOVIE_SECONDS MOVIE_DIR diff --git a/finished_refactorings/2026-09-30-trane-ascii.md b/finished_refactorings/2026-09-30-trane-ascii.md new file mode 100644 index 00000000..e3b1320e --- /dev/null +++ b/finished_refactorings/2026-09-30-trane-ascii.md @@ -0,0 +1,85 @@ +# Trane ASCII turntable + +Deliverable: `picfetch --help` plays one three-second, eased 360-degree ASCII +Trane turn above stationary help, reveals an ASCII PicFetch wordmark beneath +the final front portrait, then exits successfully. `make trane` previews the +same code. Ronin selected one turn then exit, requested the wordmark, and set a +four-second maximum. The underlying angle renderer is seamlessly loopable. + +Scope expanded from a standalone art demo to application startup integration; +use the Deep route for the Windows console boundary. All work remains inline +with Pico: no delegated tasks or reviews. Ronin accepted the work and authorized +committing and pushing the Trane changes after verification. + +## Decisions and limits + +- Native Go, original analytic ellipsoid geometry inspired by + `assets/trane/trane_security_superhero.png`, monochrome ASCII with ANSI cursor + control. No images, downloaded mesh, Python runtime or Fyne startup. +- Synchronous bounded rendering at 30 FPS; elapsed-time easing; a local signal + context and stopped ticker own cancellation. Ctrl+C/SIGTERM restore the + terminal and leave plain help. Normal completion leaves portrait/wordmark/help + in scrollback. No renderer goroutines or mutable global seams. +- All help remains available. Wrapping and adaptive artwork preserve spare + terminal margins. Pipes, `TERM=dumb`, unavailable console support and small + terminals get immediate plain help. Shrinking below the minimum exits cleanly. +- Windows enables/restores VT output with the existing `x/sys/windows` API. +- Existing `golang.org/x/term` v0.46.0 becomes direct (unchanged version), source + `https://proxy.golang.org/golang.org/x/term/@v/v0.46.0.zip`, BSD-3-Clause. Its + license is already shipped in `THIRD-PARTY-NOTICES.md` and the reviewed notice + manifest; existing x/sys notices remain unchanged. No new shipped dependency. +- Honest limits: stylized sculpture, not a recovered Trane mesh; a generous + terminal (recommended 100x55+) is needed to show art and full help together. + Native Windows/Linux console behavior remains unverified on this Mac. + +## Tasks and verification + +1. Renderer and presentation, Pico inline: `internal/consolehelp/{render,help}.go`, + build-tagged terminal files and `help_test.go`. Contract: `Write(io.Writer, + usage string) error` keeps nonterminal help unchanged. Verify periodic ASCII + frames, different viewpoints, cancellation, deterministic three-second + completion, wordmark placement, wrapping, resizing and partial-write cleanup + with `go test -race ./internal/consolehelp`. +2. Integration, depends on 1, Pico inline: `main.go`, existing `main_test.go`, + `internal/launch/launch.go`, preview command/README, Makefile, go.mod, exact + Qodana exclusion, architecture and todos. Verify early startup exit using + `go test -tags no_emoji,nodynamic -run 'TestLaunchArgs|TestLaunchStartupContract' .`; + test parser separately and run the built binary in a real PTY. +3. Gate, depends on 1/2, Pico inline: GoLand per-file inspections including weak + warnings, `make verify`, Windows/Linux helper cross-compilation and actual + terminal capture. Budget: zero spawns; three visual passes; one complete gate. + +## Evidence + +- Renderer test first failed with `front view has no sculpture`; implementing + the renderer made it pass. Cancellation first failed because no animation was + emitted, then passed after terminal lifecycle implementation. +- `go test -race -tags no_emoji,nodynamic ./internal/consolehelp ./internal/launch` + passed. Added resize and partial-write regressions also pass under `-race`. +- Root CLI tests passed, including the final strengthened plain-output and + `TestLaunchStartupContract` run. +- `make verify-build` passed: formatting, TUF, Qodana exact test exclusions, + generated vectors/assets, both notice checks, full tagged vet and build. + `make build` produced `bin/picfetch`. Native linker emitted only its existing + duplicate `-lobjc` warning. +- `make verify` stopped at its platform guard: daemon is `linux/aarch64`, not + native Linux/amd64. Full Docker race suite remains unverified; no isolation + policy or tests were weakened. +- Console-help test binaries cross-compiled for Windows/amd64 and Linux/amd64. +- Actual built `bin/picfetch --help` in a 120x65 PTY: 90 frames, exit 0 in + **3.436 seconds**, including process startup. SIGINT: exit 0 in 0.026 seconds + with cursor/screen restored. 80x24 and `TERM=dumb`: plain help, exit 0 in + 0.022/0.021 seconds. Piped help has no ANSI and exits 0. +- Full front portrait, wordmark and every help flag inspected in retained + `.scratch/trane-ascii/final.txt`; raw PTY streams are beside it. No image + preview was generated (Pillow unavailable). +- GoLand `get_file_problems(errorsOnly=false)` returned no findings for renderer, + help, tests, both platform adapters, main, launch and preview command. This is + the IDE inspection fallback, not a fresh Qodana SARIF pass. Final reruns on + help, renderer, expanded tests and root tests also returned no findings. + Analyzed scope is these working-tree changes atop `4a80484`, using the IDE's + active profile (not an asserted equivalent of `qodana.starter`). Final + `make fmt-check` and `git diff --check` passed. + +Cost: zero spawns, three visual passes (front/profile/back; front bridge; +integrated portrait/wordmark), one full gate attempt. diff --git a/go.mod b/go.mod index ca1fcf7b..ea5a6607 100644 --- a/go.mod +++ b/go.mod @@ -20,6 +20,7 @@ require ( golang.org/x/mod v0.41.0 golang.org/x/net v0.59.0 golang.org/x/sys v0.48.0 + golang.org/x/term v0.46.0 ) // Intel macOS's last official native runtime is 1.23.2. Keep its API 23 binding @@ -107,7 +108,6 @@ require ( golang.org/x/crypto v0.57.0 // indirect golang.org/x/sync v0.23.0 // indirect golang.org/x/telemetry v0.0.0-20260811182544-a038080d80e5 // indirect - golang.org/x/term v0.46.0 // indirect golang.org/x/text v0.42.0 // indirect golang.org/x/tools v0.49.0 // indirect golang.org/x/vuln v1.7.0 // indirect diff --git a/internal/consolehelp/help.go b/internal/consolehelp/help.go new file mode 100644 index 00000000..30e4acdd --- /dev/null +++ b/internal/consolehelp/help.go @@ -0,0 +1,141 @@ +// Package consolehelp presents terminal-only ASCII artwork above command help. +package consolehelp + +import ( + "context" + "errors" + "io" + "os" + "os/signal" + "strings" + "syscall" + "time" + + "golang.org/x/term" +) + +// Write prints usage, animating Trane when the output terminal has room. +func Write(out io.Writer, usage string) error { + file, ok := out.(*os.File) + if ok && os.Getenv("TERM") != "dumb" && term.IsTerminal(int(file.Fd())) { + restore, err := enableANSI(file) + if err == nil { + defer restore() + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) + defer stop() + return animate(ctx, out, usage, func() (int, int, error) { + return term.GetSize(int(file.Fd())) + }) + } + } + _, err := io.WriteString(out, usage) + return err +} + +const turnDuration = 3 * time.Second + +func animate(ctx context.Context, out io.Writer, usage string, size func() (int, int, error)) error { + ticker := time.NewTicker(time.Second / 30) + defer ticker.Stop() + return play(ctx, out, usage, size, ticker.C, time.Now()) +} + +func play(ctx context.Context, out io.Writer, usage string, size func() (int, int, error), ticks <-chan time.Time, start time.Time) (err error) { + active := false + final := usage + defer func() { + if active { + _, restoreErr := io.WriteString(out, "\x1b[?25h\x1b[?1049l"+final) + err = errors.Join(err, restoreErr) + } + }() + elapsed := time.Duration(0) + lastColumns, lastRows := 0, 0 + for { + columns, rows, sizeErr := size() + width, height, text := layout(columns, rows, usage) + if sizeErr != nil || height == 0 { + if !active { + _, err = io.WriteString(out, usage) + } + return err + } + var frame strings.Builder + if !active { + // Mark active before the write, so even a partial write gets cleanup. + active = true + frame.WriteString("\x1b[?1049h\x1b[?25l") + } + resized := columns != lastColumns || rows != lastRows + if resized { + frame.WriteString("\x1b[2J") + } + frame.WriteString("\x1b[H") + progress := elapsed.Seconds() / turnDuration.Seconds() + // Smooth acceleration and deceleration settle the full turn at the front. + angle := 360 * progress * progress * (3 - 2*progress) + frame.WriteString(render(angle, width, height)) + if resized { + frame.WriteString("\n\n" + centered("Trane / Ctrl+C to skip", width) + "\n\n\n\n\n" + text) + } + if _, err = io.WriteString(out, frame.String()); err != nil { + return err + } + lastColumns, lastRows = columns, rows + select { + case <-ctx.Done(): + return nil + case now := <-ticks: + elapsed = now.Sub(start) + if elapsed >= turnDuration { + final = render(0, width, height) + "\n" + wordmark(width) + "\n\n" + text + return nil + } + } + } +} + +// Keep a spare column/row to avoid automatic wrapping or terminal scrolling. +// If artwork cannot fit, ordinary help remains usable in scrollback and pipes. +func layout(columns, rows int, usage string) (width, height int, text string) { + width = min(columns-1, 120) + if width < 48 || rows < 16 { + return 0, 0, usage + } + var lines []string + for _, line := range strings.Split(strings.TrimSuffix(usage, "\n"), "\n") { + for len(line) > width { + cut := strings.LastIndexByte(line[:width+1], ' ') + if cut <= 4 { + cut = width + } + lines = append(lines, line[:cut]) + line = " " + strings.TrimLeft(line[cut:], " ") + } + lines = append(lines, line) + } + height = min(32, rows-len(lines)-9) + if height < 10 { + return 0, 0, usage + } + return width, height, strings.Join(lines, "\n") + "\n" +} + +func centered(text string, width int) string { + return strings.Repeat(" ", max(0, (width-len(text))/2)) + text +} + +func wordmark(width int) string { + // Hand-lettered ASCII PicFetch, kept small enough for an ordinary terminal. + const lettering = ` ____ _ _____ _ _ +| _ \(_) ___| ___|__| |_ ___| |__ +| |_) | |/ __| |_ / _ \ __/ __| '_ \ +| __/| | (__| _| __/ || (__| | | | +|_| |_|\___|_| \___|\__\___|_| |_|` + lines := strings.Split(lettering, "\n") + indent := strings.Repeat(" ", max(0, (width-36)/2)) + for i := range lines { + lines[i] = indent + lines[i] + } + return strings.Join(lines, "\n") +} diff --git a/internal/consolehelp/help_test.go b/internal/consolehelp/help_test.go new file mode 100644 index 00000000..31cd9238 --- /dev/null +++ b/internal/consolehelp/help_test.go @@ -0,0 +1,182 @@ +package consolehelp + +import ( + "context" + "errors" + "strings" + "testing" + "time" +) + +func TestOneTurnExitsAndLeavesPortraitAboveHelp(t *testing.T) { + var out strings.Builder + start := time.Unix(0, 0) + ticks := make(chan time.Time, 2) + ticks <- start.Add(1500 * time.Millisecond) + ticks <- start.Add(3 * time.Second) + const usage = "Usage:\n picfetch [flags]\n" + if err := play(context.Background(), &out, usage, func() (int, int, error) { return 100, 50, nil }, ticks, start); err != nil { + t.Fatal(err) + } + width, height, text := layout(100, 50, usage) + if !strings.Contains(out.String(), render(180, width, height)) { + t.Fatal("turn never displayed the back of the sculpture") + } + want := "\x1b[?25h\x1b[?1049l" + render(0, width, height) + "\n" + wordmark(width) + "\n\n" + text + if !strings.HasSuffix(out.String(), want) { + t.Fatal("completed turn did not leave front portrait above help") + } +} + +func TestAnimationCancellationRestoresTerminal(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + var output strings.Builder + out := callbackWriter{out: &output, after: cancel} + const usage = "Usage:\n picfetch [flags]\n" + if err := animate(ctx, out, usage, func() (int, int, error) { return 100, 50, nil }); err != nil { + t.Fatal(err) + } + result := output.String() + if !strings.Contains(result, "\x1b[?1049h") || !strings.Contains(result, usage) { + t.Fatal("animation did not display artwork and help") + } + if !strings.HasSuffix(result, "\x1b[?25h\x1b[?1049l"+usage) { + t.Fatal("cancel did not restore cursor/screen and leave ordinary help") + } +} + +type callbackWriter struct { + out *strings.Builder + after func() +} + +func (w callbackWriter) Write(p []byte) (int, error) { + n, err := w.out.Write(p) + w.after() + return n, err +} + +func TestSmallTerminalKeepsCompleteHelp(t *testing.T) { + var out strings.Builder + const usage = "Usage:\n picfetch [flags]\n" + if err := animate(context.Background(), &out, usage, func() (int, int, error) { return 20, 10, nil }); err != nil { + t.Fatal(err) + } + if out.String() != usage { + t.Fatalf("small terminal must get unchanged help, got %q", out.String()) + } +} + +func TestResizeAndOutputFailureRestoreTerminal(t *testing.T) { + const usage = "Usage:\n picfetch [flags]\n" + start := time.Unix(0, 0) + t.Run("resize", func(t *testing.T) { + var out strings.Builder + ticks := make(chan time.Time, 1) + ticks <- start.Add(time.Second) + calls := 0 + size := func() (int, int, error) { + calls++ + if calls > 1 { + return 40, 10, nil + } + return 100, 50, nil + } + if err := play(context.Background(), &out, usage, size, ticks, start); err != nil { + t.Fatal(err) + } + if !strings.HasSuffix(out.String(), "\x1b[?25h\x1b[?1049l"+usage) { + t.Fatal("shrinking the terminal lost help or terminal restoration") + } + }) + t.Run("partial_write", func(t *testing.T) { + out := &partialWriter{} + err := animate(context.Background(), out, usage, func() (int, int, error) { return 100, 50, nil }) + if !errors.Is(err, errOutput) || !strings.Contains(out.String(), "\x1b[?25h\x1b[?1049l") { + t.Fatalf("partial write: error=%v, cleanup missing=%v", err, out.String()) + } + }) +} + +var errOutput = errors.New("output failed") + +type partialWriter struct{ strings.Builder } + +func (w *partialWriter) Write(p []byte) (int, error) { + if w.Len() == 0 { + n, _ := w.Builder.Write(p[:5]) + return n, errOutput + } + return w.Builder.Write(p) +} + +// Prevent io.WriteString from selecting the embedded Builder's WriteString. +func (w *partialWriter) WriteString(s string) (int, error) { return w.Write([]byte(s)) } + +func TestLayoutPreservesHelpAndFits(t *testing.T) { + usage := "Usage:\n picfetch [flags]\n\n --sort=MODE " + strings.Repeat("description ", 18) + "\n" + width, height, wrapped := layout(80, 60, usage) + if width == 0 || height < 10 { + t.Fatal("large terminal unexpectedly declined animation") + } + if strings.Join(strings.Fields(wrapped), " ") != strings.Join(strings.Fields(usage), " ") { + t.Fatal("wrapping changed help content") + } + for _, line := range strings.Split(wrapped, "\n") { + if len(line) >= 80 { + t.Fatalf("help line reaches terminal wrap column: %q", line) + } + } + if height+strings.Count(wrapped, "\n")+9 > 60 { + t.Fatal("art, branding and help exceed the viewport") + } +} + +func TestWritePlainHelp(t *testing.T) { + var out strings.Builder + const usage = "Usage:\n picfetch [flags]\n" + if err := Write(&out, usage); err != nil || out.String() != usage { + t.Fatalf("Write = %q, %v; want unchanged usage", out.String(), err) + } + want := errors.New("closed output") + if err := Write(failingWriter{want}, usage); !errors.Is(err, want) { + t.Fatalf("Write error = %v; want %v", err, want) + } +} + +type failingWriter struct{ err error } + +func (w failingWriter) Write(_ []byte) (int, error) { return 0, w.err } + +func TestTurntable(t *testing.T) { + front := render(0, 80, 32) + if front == "" || strings.TrimSpace(front) == "" { + t.Fatal("front view has no sculpture") + } + if front != render(360, 80, 32) { + t.Fatal("one full turn must return to exactly the first frame") + } + views := make(map[string]bool) + for _, angle := range []float64{0, 45, 90, 180, 270} { + frame := render(angle, 80, 32) + rows := strings.Split(frame, "\n") + if len(rows) != 32 { + t.Fatalf("angle %v: got %d rows", angle, len(rows)) + } + for _, row := range rows { + if len(row) != 80 { + t.Fatalf("angle %v: row width %d", angle, len(row)) + } + for _, c := range row { + if c < ' ' || c > '~' { + t.Fatalf("art contains non-ASCII character %q", c) + } + } + } + views[frame] = true + } + if len(views) != 5 { + t.Fatal("front, angled, side and back views must be distinct") + } +} diff --git a/internal/consolehelp/render.go b/internal/consolehelp/render.go new file mode 100644 index 00000000..3f247819 --- /dev/null +++ b/internal/consolehelp/render.go @@ -0,0 +1,104 @@ +package consolehelp + +import ( + "math" + "strings" +) + +type vector struct{ x, y, z float64 } +type ellipsoid struct { + center, radius vector + albedo float64 +} + +// trane is an original analytic sculpture based on the retained mascot artwork. +// All details are surfaces in the same 3D space, including the eyes and tongue. +func trane() []ellipsoid { + return []ellipsoid{ + {vector{0, 0.15, 0}, vector{0.92, 1.02, 0.78}, 0.88}, // skull + {vector{0, -0.65, -0.12}, vector{0.64, 0.58, 0.57}, 0.8}, // neck ruff + {vector{0, -0.4, 0.55}, vector{0.68, 0.67, 0.5}, 0.95}, // jaw + {vector{0, 0.13, 0.86}, vector{0.32, 0.30, 0.36}, 0.9}, // nose bridge + {vector{0, -0.45, 0.99}, vector{0.44, 0.46, 0.13}, 0.1}, // open smile + {vector{0, -0.69, 1.12}, vector{0.24, 0.3, 0.1}, 0.58}, // tongue + {vector{0, -0.67, 1.219}, vector{0.018, 0.17, 0.007}, 0.22}, + {vector{0, 0.04, 1.26}, vector{0.3, 0.2, 0.2}, 0.12}, + {vector{-0.09, 0.12, 1.43}, vector{0.085, 0.035, 0.014}, 0.92}, + {vector{-0.19, 0.99, -0.05}, vector{0.38, 0.23, 0.38}, 0.9}, + {vector{-0.91, -0.05, -0.04}, vector{0.4, 0.89, 0.4}, 0.68}, + {vector{-1.05, -0.49, 0.02}, vector{0.34, 0.46, 0.34}, 0.72}, + {vector{-0.42, 0.43, 0.64}, vector{0.3, 0.37, 0.22}, 0.54}, + {vector{-0.42, 0.43, 0.78}, vector{0.235, 0.29, 0.12}, 1.0}, + {vector{-0.4, 0.42, 0.875}, vector{0.145, 0.2, 0.067}, 0.13}, + {vector{-0.445, 0.51, 0.933}, vector{0.052, 0.066, 0.02}, 1.0}, + {vector{-0.42, 0.76, 0.66}, vector{0.29, 0.1, 0.16}, 0.76}, + {vector{-0.27, -0.17, 0.95}, vector{0.39, 0.3, 0.34}, 1.0}, + {vector{0.91, -0.05, -0.04}, vector{0.4, 0.89, 0.4}, 0.68}, + {vector{1.05, -0.49, 0.02}, vector{0.34, 0.46, 0.34}, 0.72}, + {vector{0.42, 0.43, 0.64}, vector{0.3, 0.37, 0.22}, 0.54}, + {vector{0.42, 0.43, 0.78}, vector{0.235, 0.29, 0.12}, 1.0}, + {vector{0.4, 0.42, 0.875}, vector{0.145, 0.2, 0.067}, 0.13}, + {vector{0.355, 0.51, 0.933}, vector{0.052, 0.066, 0.02}, 1.0}, + {vector{0.42, 0.76, 0.66}, vector{0.29, 0.1, 0.16}, 0.76}, + {vector{0.27, -0.17, 0.95}, vector{0.39, 0.3, 0.34}, 1.0}, + } +} + +// render uses orthographic ray/ellipsoid intersections and a depth buffer. +// Terminal cells are assumed to be twice as tall as they are wide. +func render(degrees float64, width, height int) string { + const ramp = " .,:;irsXA253hMHGS#9B&@" + angle := math.Mod(degrees, 360) * math.Pi / 180 + sine, cosine := math.Sincos(angle) + scale := math.Min(float64(width)/3.5, float64(height)*2/2.9) + pixels := []byte(strings.Repeat(" ", width*height)) + depths := make([]float64, len(pixels)) + for i := range depths { + depths[i] = math.Inf(-1) + } + for _, shape := range trane() { + cx, cy, cz := shape.center.x, shape.center.y, shape.center.z + rx, ry, rz := shape.radius.x, shape.radius.y, shape.radius.z + centerX, centerZ := cosine*cx+sine*cz, -sine*cx+cosine*cz + extentX := math.Hypot(cosine*rx, sine*rz) + left := max(0, int(math.Floor(float64(width)/2+(centerX-extentX)*scale))) + right := min(width, int(math.Ceil(float64(width)/2+(centerX+extentX)*scale))) + top := max(0, int(math.Floor(float64(height)/2-(cy+ry)*scale/2))) + bottom := min(height, int(math.Ceil(float64(height)/2-(cy-ry)*scale/2))) + ix, iy, iz := 1/(rx*rx), 1/(ry*ry), 1/(rz*rz) + quadratic := sine*sine*ix + cosine*cosine*iz + for row := top; row < bottom; row++ { + y := (float64(height)/2-float64(row)-0.5)*2/scale - cy + for col := left; col < right; col++ { + x := (float64(col)+0.5-float64(width)/2)/scale - centerX + localX, localZ := cosine*x, sine*x + linear := -sine*localX*ix + cosine*localZ*iz + constant := localX*localX*ix + y*y*iy + localZ*localZ*iz - 1 + discriminant := linear*linear - quadratic*constant + if discriminant < 0 { + continue + } + z := (-linear + math.Sqrt(discriminant)) / quadratic + index := row*width + col + if z+centerZ <= depths[index] { + continue + } + depths[index] = z + centerZ + localX -= sine * z + localZ += cosine * z + nx, ny, nz := localX*ix, y*iy, localZ*iz + worldX, worldZ := cosine*nx+sine*nz, -sine*nx+cosine*nz + length := math.Sqrt(nx*nx + ny*ny + nz*nz) + light := max(0, (-0.45*worldX+0.65*ny+0.61*worldZ)/length) + brightness := shape.albedo * (0.24 + 0.76*light) + shade := max(1, min(len(ramp)-1, int(math.Round(brightness*float64(len(ramp)-1))))) + pixels[index] = ramp[shade] + } + } + } + rows := make([]string, height) + for row := range rows { + rows[row] = string(pixels[row*width : (row+1)*width]) + } + return strings.Join(rows, "\n") +} diff --git a/internal/consolehelp/terminal_other.go b/internal/consolehelp/terminal_other.go new file mode 100644 index 00000000..3ad9f743 --- /dev/null +++ b/internal/consolehelp/terminal_other.go @@ -0,0 +1,9 @@ +//go:build !windows + +package consolehelp + +import "os" + +func enableANSI(_ *os.File) (func(), error) { + return func() {}, nil +} diff --git a/internal/consolehelp/terminal_windows.go b/internal/consolehelp/terminal_windows.go new file mode 100644 index 00000000..cba702f0 --- /dev/null +++ b/internal/consolehelp/terminal_windows.go @@ -0,0 +1,21 @@ +//go:build windows + +package consolehelp + +import ( + "os" + + "golang.org/x/sys/windows" +) + +func enableANSI(file *os.File) (func(), error) { + handle := windows.Handle(file.Fd()) + var mode uint32 + if err := windows.GetConsoleMode(handle, &mode); err != nil { + return nil, err + } + if err := windows.SetConsoleMode(handle, mode|windows.ENABLE_VIRTUAL_TERMINAL_PROCESSING); err != nil { + return nil, err + } + return func() { _ = windows.SetConsoleMode(handle, mode) }, nil +} diff --git a/internal/launch/launch.go b/internal/launch/launch.go index 7157006f..a6691872 100644 --- a/internal/launch/launch.go +++ b/internal/launch/launch.go @@ -280,7 +280,7 @@ func Usage() string { for _, s := range flagSpecs { b.WriteString(fmt.Sprintf(" %-*s %s\n", width, flagLabel(s), s.help)) } - b.WriteString(fmt.Sprintf(" %-*s %s\n", width, "--help", "print this help and exit")) + b.WriteString(fmt.Sprintf(" %-*s %s\n", width, "--help", "show help and exit (Ctrl+C skips terminal animation)")) b.WriteString("\nFlags may appear anywhere among the paths, and -flag reads the same as\n") b.WriteString("--flag. Use -- to end flag parsing, for a path that starts with a dash.\n") diff --git a/main.go b/main.go index 6befa596..58ecdc7b 100644 --- a/main.go +++ b/main.go @@ -20,6 +20,7 @@ import ( "fyne.io/fyne/v2/lang" "fyne.io/fyne/v2/storage" + "github.com/frathe/picfetch/internal/consolehelp" "github.com/frathe/picfetch/internal/distribution" "github.com/frathe/picfetch/internal/heic" "github.com/frathe/picfetch/internal/launch" @@ -97,7 +98,10 @@ func launchArgs(args []string, stdout, stderr io.Writer) (paths []string, opts l paths, opts, err := launch.Parse(args) switch { case errors.Is(err, launch.ErrHelp): - _, _ = io.WriteString(stdout, launch.Usage()) + if err := consolehelp.Write(stdout, launch.Usage()); err != nil { + _, _ = fmt.Fprintf(stderr, "picfetch: %v\n", err) + return nil, launch.Options{}, 1 + } return nil, launch.Options{}, 0 case err != nil: diff --git a/main_test.go b/main_test.go index fd8cb36e..19cdc708 100644 --- a/main_test.go +++ b/main_test.go @@ -315,8 +315,8 @@ func TestLaunchArgs_HelpPrintsUsageAndExits(t *testing.T) { if len(paths) != 0 { t.Errorf("paths = %q, want none", paths) } - if !strings.Contains(stdout.String(), "--slideshow") { - t.Errorf("stdout = %q, want the usage text", stdout.String()) + if stdout.String() != launch.Usage() { + t.Errorf("stdout = %q, want unchanged plain usage without terminal controls", stdout.String()) } if stderr.String() != "" { t.Errorf("stderr = %q, want nothing", stderr.String()) diff --git a/qodana.yaml b/qodana.yaml index 82c8698c..df08926e 100644 --- a/qodana.yaml +++ b/qodana.yaml @@ -106,6 +106,7 @@ exclude: - "internal/clipboard/copyfiles_test.go" - "internal/clipboard/copyfiles_windows_test.go" - "internal/completion/completion_test.go" + - "internal/consolehelp/help_test.go" - "internal/decodepool/decodepool_test.go" - "internal/displays/displays_test.go" - "internal/displays/xrandr_test.go" diff --git a/scripts/traneascii/README.md b/scripts/traneascii/README.md new file mode 100644 index 00000000..1c73c189 --- /dev/null +++ b/scripts/traneascii/README.md @@ -0,0 +1,32 @@ +# Trane in the terminal + +Run from the repository root without linking or starting the desktop app: + +```sh +make trane +``` + +A shaded, stylized 3D Trane head makes one smooth three-second turn above the +app's help text, then reveals an ASCII PicFetch wordmark and exits. This is the +same renderer used by `picfetch --help`. Only ASCII characters form the picture; +ANSI sequences move the cursor. Ctrl+C skips the animation and prints plain help. + +```sh +./bin/picfetch --help +./bin/picfetch --help > /tmp/picfetch-help.txt +``` + +Use a dark terminal with a light monospace font, preferably at least 100 columns +by 55 rows. Artwork scales to the available space. If the complete wrapped help, +wordmark and at least ten rows of artwork will not fit, plain help prints +immediately. Piped output and `TERM=dumb` also print ordinary help immediately. +Resizing to a smaller terminal ends animation cleanly, preserving the full help. + +Rotation uses elapsed time at up to 30 FPS, with smooth acceleration and braking. +The camera stays fixed; depth-tested ellipsoids form the head, ears, muzzle, eyes +and tongue. Frame 360 equals frame 0; the underlying turn is seamlessly loopable. +Normal completion leaves the portrait, wordmark and help in terminal scrollback. + +This is original procedural geometry inspired by the retained Trane artwork, +not a recovered mesh. It uses Go and the existing `golang.org/x/term` dependency, +starts no render workers, and returns before desktop startup or storage access. diff --git a/scripts/traneascii/main.go b/scripts/traneascii/main.go new file mode 100644 index 00000000..ca3f88fd --- /dev/null +++ b/scripts/traneascii/main.go @@ -0,0 +1,18 @@ +// Command traneascii previews the same animated help as picfetch --help, +// without starting or linking the desktop application. +package main + +import ( + "fmt" + "os" + + "github.com/frathe/picfetch/internal/consolehelp" + "github.com/frathe/picfetch/internal/launch" +) + +func main() { + if err := consolehelp.Write(os.Stdout, launch.Usage()); err != nil { + _, _ = fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } +} diff --git a/todos.md b/todos.md index 074fad06..242d64a5 100644 --- a/todos.md +++ b/todos.md @@ -10,6 +10,10 @@ #### Internal +- [x] Add a three-second ASCII Trane turn and PicFetch wordmark to terminal + `--help`, with plain piped output and a `make trane` developer preview. + Verification and limitations: [turntable record](finished_refactorings/2026-09-30-trane-ascii.md). + ## Open ### Mac App Store preparation From bb6ce5dd86d4adf0f3271fbee50fb68692df15ff Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 15:53:46 +0200 Subject: [PATCH 21/63] Fix Windows image path transport into PowerShell --- internal/filepicker/filepicker.go | 9 +- internal/filepicker/filepicker_test.go | 35 ++++- internal/filepicker/windows_test.go | 41 ++++++ internal/trash/trash.go | 16 +-- internal/trash/trash_test.go | 145 ++++++++++++++++----- plans/2026-09-30-windows-path-transport.md | 110 ++++++++++++++++ scripts/nativeguards/main.go | 3 +- todos.md | 7 + 8 files changed, 311 insertions(+), 55 deletions(-) create mode 100644 plans/2026-09-30-windows-path-transport.md diff --git a/internal/filepicker/filepicker.go b/internal/filepicker/filepicker.go index b1f84df1..7cb7ce6f 100644 --- a/internal/filepicker/filepicker.go +++ b/internal/filepicker/filepicker.go @@ -184,17 +184,16 @@ func chooseSaveWindows(suggestedPath string) ([]byte, error) { } // buildPowerShellSaveCmd builds that dialog's script. The suggested path is -// embedded whole and split by PowerShell's own [System.IO.Path], not by Go's +// passed as environment data and split by PowerShell's [System.IO.Path], not Go's // filepath: this function's test runs on whatever machine builds the module, // where a `C:\...` path is one long base name to filepath.Dir. Letting the // platform that will actually run the script do its own splitting keeps the // two from disagreeing. func buildPowerShellSaveCmd(suggestedPath string) *exec.Cmd { - escaped := powerShellEscape(suggestedPath) script := `Add-Type -AssemblyName System.Windows.Forms $dlg = New-Object System.Windows.Forms.SaveFileDialog -$dlg.FileName = "` + escaped + `" -$dlg.InitialDirectory = [System.IO.Path]::GetDirectoryName("` + escaped + `") +$dlg.FileName = $env:PICFETCH_SAVE_PATH +$dlg.InitialDirectory = [System.IO.Path]::GetDirectoryName($env:PICFETCH_SAVE_PATH) $dlg.OverwritePrompt = $true $dlg.Title = "` + powerShellEscape(lang.L("Export image")) + `" if ($dlg.ShowDialog() -eq [System.Windows.Forms.DialogResult]::OK) { @@ -203,6 +202,8 @@ if ($dlg.ShowDialog() -eq [System.Windows.Forms.DialogResult]::OK) { script = powerShellPickerScript(script) cmd := exec.Command("powershell", "-NoProfile", "-NonInteractive", "-Command", script) + // Path values are data, including PowerShell smart-quote delimiters. + cmd.Env = append(cmd.Environ(), "PICFETCH_SAVE_PATH="+suggestedPath) hideConsoleWindow(cmd) return cmd } diff --git a/internal/filepicker/filepicker_test.go b/internal/filepicker/filepicker_test.go index d5fda6f1..da47c3ee 100644 --- a/internal/filepicker/filepicker_test.go +++ b/internal/filepicker/filepicker_test.go @@ -166,8 +166,7 @@ func TestBuildPowerShellSaveCmd(t *testing.T) { for _, want := range []string{ "SaveFileDialog", - "holiday.png", - `C:\photos`, + "$env:PICFETCH_SAVE_PATH", "$dlg.OverwritePrompt = $true", } { if !strings.Contains(script, want) { @@ -176,6 +175,38 @@ func TestBuildPowerShellSaveCmd(t *testing.T) { } } +// Path bytes must survive transport without becoming executable script text. +func TestBuildPowerShellSaveCmd_PathIsData(t *testing.T) { + t.Setenv("PICFETCH_SAVE_PATH", "inherited stale suggestion") + t.Setenv("PICFETCH_PICKER_ENV_CONTROL", "preserved") + baseline := buildPowerShellSaveCmd(`C:\photos\ordinary.jpg`).Args + for _, path := range []string{ + `C:\photos\ordinary.jpg`, + `C:\photos\“;Write-Output INJECTED;#”.jpg`, + `C:\photos\”;Write-Output INJECTED;#“.jpg`, + "C:\\photos\\$dollar`tick; (brackets) 'single' ‘curly’.jpg", + `C:\photos\café 東京 😀 [1]=100%.jpg`, + `\\server\share\ spaced name .jpg`, + } { + t.Run(path, func(t *testing.T) { + cmd := buildPowerShellSaveCmd(path) + if !slices.Equal(cmd.Args, baseline) || strings.Contains(strings.Join(cmd.Args, "\n"), path) { + t.Fatal("suggested path changed executable PowerShell source") + } + env := cmd.Environ() + if !slices.Contains(env, "PICFETCH_SAVE_PATH="+path) || slices.Contains(env, "PICFETCH_SAVE_PATH=inherited stale suggestion") { + t.Fatal("child environment lost or replaced the exact suggested path") + } + if !slices.Contains(env, "PICFETCH_PICKER_ENV_CONTROL=preserved") { + t.Fatal("unrelated inherited environment was lost") + } + if os.Getenv("PICFETCH_SAVE_PATH") != "inherited stale suggestion" { + t.Fatal("parent environment changed") + } + }) + } +} + func TestChooseSaveLinux_ReturnsErrorWhenZenityMissing(t *testing.T) { origLookup := lookupZenity t.Cleanup(func() { lookupZenity = origLookup }) diff --git a/internal/filepicker/windows_test.go b/internal/filepicker/windows_test.go index d22bd052..f743603c 100644 --- a/internal/filepicker/windows_test.go +++ b/internal/filepicker/windows_test.go @@ -8,6 +8,7 @@ import ( "os" "os/exec" "path/filepath" + "strings" "testing" "time" ) @@ -41,3 +42,43 @@ func TestWindowsPickerTransport_EmitsUTF8PathArrays(t *testing.T) { } } } + +// Exercise the actual WinForms properties and PowerShell parser without showing +// a native dialog. Only the modal call is replaced with its synthetic result. +func TestWindowsSaveTransport_PathIsData(t *testing.T) { + for _, name := range []string{"ordinary.jpg", "“;Write-Output INJECTED;#” $dollar`tick ‘quote’ 東京.jpg"} { + for _, cancelDialog := range []bool{false, true} { + path := filepath.Join(t.TempDir(), name) + cmd := buildPowerShellSaveCmd(path) + script := cmd.Args[len(cmd.Args)-1] + result := "[System.Windows.Forms.DialogResult]::OK" + if cancelDialog { + result = "[System.Windows.Forms.DialogResult]::Cancel" + } + if strings.Count(script, "$dlg.ShowDialog()") != 1 { + t.Fatal("missing modal seam") + } + script = strings.Replace(script, "$dlg.ShowDialog()", result, 1) + // Check the real save-panel fields independently of its serialized result. + script = strings.Replace(script, "if ("+result, `if ($dlg.FileName -cne $env:PICFETCH_SAVE_PATH -or $dlg.InitialDirectory -cne $env:PICFETCH_SAVE_TEST_DIR -or -not $dlg.OverwritePrompt) { throw 'save properties changed' } +if (`+result, 1) + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + run := exec.CommandContext(ctx, cmd.Path, append(cmd.Args[1:len(cmd.Args)-1], script)...) + run.Env = append(cmd.Environ(), "PICFETCH_SAVE_TEST_DIR="+filepath.Dir(path)) + run.SysProcAttr = cmd.SysProcAttr + out, err := run.Output() + cancel() + selected, err := decodePickedPaths(out, err) + if err != nil { + t.Fatalf("save transport: %v (output %q)", err, out) + } + if cancelDialog { + if selected != nil { + t.Fatalf("cancel selected paths: %v", selected) + } + } else if len(selected) != 1 || filepath.FromSlash(selected[0].Path()) != path { + t.Fatalf("selected %v, want exact path %q", selected, path) + } + } + } +} diff --git a/internal/trash/trash.go b/internal/trash/trash.go index d4982f23..e411dfba 100644 --- a/internal/trash/trash.go +++ b/internal/trash/trash.go @@ -125,26 +125,16 @@ func moveWindows(path string) error { } script := `Add-Type -AssemblyName Microsoft.VisualBasic try { - [Microsoft.VisualBasic.FileIO.FileSystem]::` + method + `("` + escapePowerShellPath(path) + `", 'OnlyErrorDialogs', 'SendToRecycleBin') + [Microsoft.VisualBasic.FileIO.FileSystem]::` + method + `($env:PICFETCH_TRASH_PATH, 'OnlyErrorDialogs', 'SendToRecycleBin') } catch { [Console]::Error.WriteLine($_.Exception.Message) exit 1 }` cmd := exec.Command("powershell", "-NoProfile", "-NonInteractive", "-Command", script) + // Keep the selected path out of PowerShell source, including smart quotes. + cmd.Env = append(cmd.Environ(), "PICFETCH_TRASH_PATH="+path) hideConsoleWindow(cmd) _, err = runTrashCommand(cmd) return err } - -// escapePowerShellPath escapes path for embedding inside a double-quoted -// PowerShell string literal. Windows paths can't contain a literal " (an -// illegal filename character there), so unlike filepicker's -// powerShellEscape this only needs to guard PowerShell's own -// metacharacters: ` (its escape character) and $ (variable interpolation) - -// both of which are legal in a Windows filename, unlike the app-generated -// temp paths clipboard.go/filepicker.go embed the same way. -func escapePowerShellPath(path string) string { - path = strings.ReplaceAll(path, "`", "``") - return strings.ReplaceAll(path, "$", "`$") -} diff --git a/internal/trash/trash_test.go b/internal/trash/trash_test.go index 9059f5c8..baf4aadd 100644 --- a/internal/trash/trash_test.go +++ b/internal/trash/trash_test.go @@ -1,13 +1,16 @@ package trash import ( + "context" "errors" "os" "os/exec" "path/filepath" + "runtime" "slices" "strings" "testing" + "time" ) func TestMoveLinux_PrefersGio(t *testing.T) { @@ -201,7 +204,7 @@ func TestMoveWindows_BuildsExpectedScript(t *testing.T) { "SendToRecycleBin", "catch", "exit 1", - path, + "$env:PICFETCH_TRASH_PATH", } { if !strings.Contains(gotScript, want) { t.Errorf("script does not contain %q:\n%s", want, gotScript) @@ -214,39 +217,57 @@ func TestMoveWindows_BuildsExpectedScript(t *testing.T) { } } -// TestMoveWindows_EscapesPathMetacharacters guards a real difference from -// clipboard.go's copyImageWindows/filepicker.go's chooseFilesWindows: those -// embed only app-generated temp paths or a UI label, but this path is an -// arbitrary dropped file's path, which - unlike a Windows path with a -// literal " (illegal, so never a concern) - can legally contain ` and $, -// both special inside a PowerShell double-quoted string. -func TestMoveWindows_EscapesPathMetacharacters(t *testing.T) { +// Exercise real file/directory admission, but intercept the OS mutation. +func TestMoveWindows_PathIsData(t *testing.T) { origRun := runTrashCommand t.Cleanup(func() { runTrashCommand = origRun }) - - var gotScript string - runTrashCommand = func(cmd *exec.Cmd) ([]byte, error) { - for i, a := range cmd.Args { - if a == "-Command" && i+1 < len(cmd.Args) { - gotScript = cmd.Args[i+1] + t.Setenv("PICFETCH_TRASH_PATH", "inherited stale target") + t.Setenv("PICFETCH_TRASH_ENV_CONTROL", "preserved") + for _, directory := range []bool{false, true} { + var baseline []string + for _, name := range []string{ + "ordinary.jpg", + "“;Write-Output INJECTED;#”.jpg", + "”);Write-Output INJECTED;#“.jpg", + "$dollar`tick; (brackets) 'single' ‘curly’.jpg", + "café 東京 😀 [1]=100%.jpg", + } { + path := filepath.Join(t.TempDir(), name) + if directory { + if err := os.Mkdir(path, 0o700); err != nil { + t.Fatal(err) + } + } else if err := os.WriteFile(path, nil, 0o600); err != nil { + t.Fatal(err) + } + calls := 0 + runTrashCommand = func(cmd *exec.Cmd) ([]byte, error) { + calls++ + if baseline == nil { + baseline = slices.Clone(cmd.Args) + } + if !slices.Equal(cmd.Args, baseline) || strings.Contains(strings.Join(cmd.Args, "\n"), path) { + t.Error("target path changed executable PowerShell source") + } + env := cmd.Environ() + if !slices.Contains(env, "PICFETCH_TRASH_PATH="+path) || slices.Contains(env, "PICFETCH_TRASH_PATH=inherited stale target") { + t.Error("child environment lost or replaced the exact target path") + } + if !slices.Contains(env, "PICFETCH_TRASH_ENV_CONTROL=preserved") { + t.Error("unrelated environment was lost") + } + return nil, nil + } + if err := moveWindows(path); err != nil { + t.Fatal(err) + } + if calls != 1 { + t.Fatalf("command calls = %d, want 1", calls) + } + if os.Getenv("PICFETCH_TRASH_PATH") != "inherited stale target" { + t.Fatal("parent environment changed") } } - return nil, nil - } - - path := filepath.Join(t.TempDir(), "$weird`file.jpg") - if err := os.WriteFile(path, nil, 0o644); err != nil { - t.Fatal(err) - } - if err := moveWindows(path); err != nil { - t.Fatalf("moveWindows() error = %v", err) - } - - if !strings.Contains(gotScript, "`$weird") { - t.Errorf("script does not escape $ in the path:\n%s", gotScript) - } - if !strings.Contains(gotScript, "``file") { - t.Errorf("script does not escape ` in the path:\n%s", gotScript) } } @@ -274,10 +295,64 @@ func TestMoveWindows_ReturnsCommandError(t *testing.T) { } } -func TestEscapePowerShellPath(t *testing.T) { - got := escapePowerShellPath("C:\\a$b`c") - want := "C:\\a`$b``c" - if got != want { - t.Errorf("escapePowerShellPath() = %q, want %q", got, want) +// Replace only the native deletion sink with a recorder. PowerShell parses the +// production command and receives the real child environment; no file is moved. +func TestWindowsTrashTransport_PathIsData(t *testing.T) { + if runtime.GOOS != "windows" { + t.Skip("requires Windows PowerShell") + } + origRun := runTrashCommand + t.Cleanup(func() { runTrashCommand = origRun }) + for _, directory := range []bool{false, true} { + for _, name := range []string{"ordinary.jpg", "”);Write-Output INJECTED;#“ $dollar`tick ‘quote’ 東京.jpg"} { + path := filepath.Join(t.TempDir(), name) + if directory { + if err := os.Mkdir(path, 0o700); err != nil { + t.Fatal(err) + } + } else if err := os.WriteFile(path, nil, 0o600); err != nil { + t.Fatal(err) + } + runTrashCommand = func(cmd *exec.Cmd) ([]byte, error) { + script := cmd.Args[len(cmd.Args)-1] + if strings.Count(script, "[Microsoft.VisualBasic.FileIO.FileSystem]::") != 1 { + t.Fatal("missing recycle seam") + } + script = strings.Replace(script, "[Microsoft.VisualBasic.FileIO.FileSystem]::", "[PicFetchTrashProbe]::", 1) + prefix := `[Console]::OutputEncoding = New-Object System.Text.UTF8Encoding($false) +Add-Type @' +using System; +public class PicFetchTrashProbe { + public static void DeleteFile(string path, string ui, string recycle) { Record(path, ui, recycle, "file"); } + public static void DeleteDirectory(string path, string ui, string recycle) { Record(path, ui, recycle, "directory"); } + private static void Record(string path, string ui, string recycle, string kind) { + if (ui != "OnlyErrorDialogs" || recycle != "SendToRecycleBin" || kind != Environment.GetEnvironmentVariable("PICFETCH_TRASH_TEST_KIND")) { throw new Exception("recycle contract changed"); } + Console.Write(path); + } +} +'@ +` + kind := "file" + if directory { + kind = "directory" + } + ctx, cancel := context.WithTimeout(context.Background(), time.Minute) + defer cancel() + run := exec.CommandContext(ctx, cmd.Path, append(cmd.Args[1:len(cmd.Args)-1], prefix+script)...) + run.Env = append(cmd.Environ(), "PICFETCH_TRASH_TEST_KIND="+kind) + run.SysProcAttr = cmd.SysProcAttr + out, err := run.Output() + if err == nil && string(out) != path { + t.Errorf("recycle received %q, want %q", out, path) + } + return out, err + } + if err := moveWindows(path); err != nil { + t.Fatal(err) + } + if _, err := os.Stat(path); err != nil { + t.Fatalf("test touched native trash: %v", err) + } + } } } diff --git a/plans/2026-09-30-windows-path-transport.md b/plans/2026-09-30-windows-path-transport.md new file mode 100644 index 00000000..6a0d21d4 --- /dev/null +++ b/plans/2026-09-30-windows-path-transport.md @@ -0,0 +1,110 @@ +# Windows image paths remain data in PowerShell + +## Contract and scope + +Fix the first medium finding in `.scratch/sec_todo.md`: Export suggestions and +Trash targets must never enter PowerShell source. Preserve exact Unicode paths, +initial save directory/name, picker cancellation/JSON, file/directory recycling, +error propagation, inherited environment and hidden-console behavior. + +Route: Deep (Windows security boundary). Lead implements and reviews inline. +Repository process sections 7/8 prohibit delegated review and hot-context fixes; +those instructions override the skill's fresh-agent investigation/review defaults. +Separate pre-patch and candidate-review passes remain required. The initial +implementation made no commits; Ronin subsequently authorized commit, push and +the GitHub review loop on 2026-09-30. Other security findings stay separate. + +## Investigation + +Export: `suggestedExportPath` -> `buildPowerShellSaveCmd` -> `powershell -Command`. +Trash: `Move` -> `moveWindows` -> the same script parser. Current escaping misses +PowerShell's smart-quote delimiters. The narrow boundary is command construction: +use per-child environment data, following `clipboard.copyImageWindows`, rather +than extend an incomplete character escape list. No dependency/API changes. + +Sibling pass: clipboard image/file transports already use environment data. +Wallpaper uses a separate cache-generated PNG path (`ui.writeWallpaperFile`), +not the selected image filename; its legacy string helper is outside this fix. +Picker title interpolation takes shipped translations, not source paths. + +## Tasks, file map and acceptance gates + +1. Lead: change existing `internal/filepicker/filepicker_test.go` and + `internal/trash/trash_test.go` to guard constant script text and exact path + transport, including ordinary names, smart quotes, shell punctuation, + non-ASCII text and inherited-variable collisions. Observe the old code fail. + Command: `go test -tags no_emoji,nodynamic ./internal/filepicker ./internal/trash`. +2. Lead, depends on 1: update command construction in + `internal/filepicker/filepicker.go` and `internal/trash/trash.go`. + Run the same package tests; verify no loss of native error/cancellation behavior. +3. Lead, depends on 2: add native Windows regression coverage without opening + panels or touching the recycle bin, using existing test files; wire required + guards into `scripts/nativeguards/main.go` as needed. Cross-compile tests on + macOS; actual Windows execution is a separate gate if unavailable locally. +4. Lead: inspect final candidate for bypasses/regressions, run focused race tests, + vet and Windows cross-builds, GoLand inspections including weak warnings, and + the required final `make verify` once. Record unavailable gates honestly. +5. Lead: update this evidence record, `todos.md` and the scratch finding status. + +Task graph: 1 -> 2 -> 3 -> 4 -> 5. All inline: delegation gate G5 fails (existing +context), and repository section 8 reserves review for the lead. Budget: zero +spawns, one candidate review round, one full-suite attempt at final verification. +No new packages or moved files; no architecture map change needed. + +## Evidence + +Analyzed revision: `4a8048405bef8a1eff0a00190a2452f8b12742df` plus this +working-tree patch on 2026-09-30. + +- Red: package tests failed on the original implementation because path bytes + changed PowerShell source and no exact child-environment value existed. +- Green: `go test -tags no_emoji,nodynamic ./internal/filepicker ./internal/trash + ./scripts/nativeguards` passed after the transport changes. This is the + strongest local substitute for the original Windows exploit: script arguments + stay identical across malicious/ordinary paths and child values round-trip + unchanged. Both file/directory dispatch, missing-path errors and child-process + failures retain existing passing coverage. +- Focused race: `go test -race -tags no_emoji,nodynamic ./internal/filepicker + ./internal/trash ./scripts/nativeguards` passed. +- Native Windows coverage: `TestWindowsSaveTransport_PathIsData` exercises actual + WinForms properties/JSON and synthetic OK/Cancel results; the modal call is + replaced, so no panel opens. `TestWindowsTrashTransport_PathIsData` replaces + only the native deletion sink with a recorder and checks exact path/method/ + recycle options; nothing is moved. Both are required by `nativeguards`' Windows + suite. Native Windows execution is unavailable on this macOS host and remains + unverified; the Trash native test skips here and the picker native test is + Windows-build-selected. +- Cross-platform: host and Windows/amd64 focused vet passed; both Windows/amd64 + package test binaries compiled; Windows/arm64 package builds passed. Initial + sandbox cache-access failures were resolved by authorized cache access. +- Local analysis: GoLand MCP `get_file_problems(errorsOnly=false)` inspected all + six changed Go files with the IDE profile, including weak warnings. No + actionable findings. The two existing duplicate-code weak warnings in + `filepicker_test.go` (open/save command assertions) are covered by that file's + existing exact `qodana.yaml` DuplicatedCode exclusion. No new test file or UI + shard assignment was introduced. This is the IDE inspection fallback, not a + fresh Qodana scan; hosted Qodana/CodeQL remain unverified for this candidate. +- Candidate review (lead, one round): all direct callers still feed the fixed + command builders. Source-controlled filenames reach only environment values; + no evaluation/interpolation is applied to those values. Method selection uses + fixed file/directory constants. Inherited collisions are overridden in the + child, and the parent environment remains unchanged. Ordinary Unicode/UNC + suggestions and real admitted file/directory paths pass the portable guards. + No confirmed bypass or compatibility regression found in the changed boundary. +- Full gate: `make verify` stopped at its platform guard: Docker reports + `linux/aarch64`, while complete isolation/race tests require native Linux/amd64. + No isolation test was skipped or policy relaxed to claim a pass. +- `make verify-build` passed (exit 0): formatting, TUF, exact Qodana test + exclusions, generated assets/notices, full host vet and full host build. + +## Handoff + +Implementation is complete; final verification is blocked on native Windows +execution and the native Linux/amd64 full suite. Keep the original finding open +until those gates pass. Run the existing Windows CI native-guard suite after the +patch is committed/pushed. Ronin authorized this step and the GitHub review +loop on 2026-09-30. Review and CI evidence will be recorded on +[PR #75](https://github.com/frathe/picfetch/pull/75). + +Ledger: zero spawns, one inline candidate review, one full-suite attempt. +Unrelated Trane files and edits were preserved. diff --git a/scripts/nativeguards/main.go b/scripts/nativeguards/main.go index d642adc3..ad5153b8 100644 --- a/scripts/nativeguards/main.go +++ b/scripts/nativeguards/main.go @@ -87,7 +87,8 @@ func suiteFor(name, hostOS string) (suite, error) { s.goos = "windows" s.require("internal/wallpaper", "TestSetWindows_TargetPreservesOpaqueIDAndUnicodePath", "TestSetWindows_TargetValidationFailsBeforeMutation") s.require("internal/clipboard", "TestCopyFilesWindows_DecodesUTF8WithNonUTF8Default") - s.require("internal/filepicker", "TestWindowsPickerTransport_EmitsUTF8PathArrays") + s.require("internal/filepicker", "TestWindowsPickerTransport_EmitsUTF8PathArrays", "TestWindowsSaveTransport_PathIsData") + s.require("internal/trash", "TestWindowsTrashTransport_PathIsData") s.require("internal/update", "TestApplyWindows_ReplacesDestAndKeepsOld", "TestApplyWindows_MissingStagedBinaryRestoresDest", "TestWindowsRelaunchCommand_PassesThePIDInTheInheritedEnvironment", "TestClassifyApplyError_WindowsErrno", "TestWaitMilliseconds_NeverConvertsToAnUnboundedWait") s.require("internal/ui/autoupdate", "TestUpdater_AutomaticAndManualShareCompleteTransaction", "TestApplyStagedUpdate_SuccessRemovesTheStageOnEveryPlatform") s.require("internal/distribution", "TestStoreManaged_DefaultBuildIsFalse") diff --git a/todos.md b/todos.md index 242d64a5..d5d3ad48 100644 --- a/todos.md +++ b/todos.md @@ -1,5 +1,12 @@ # PicFetch — TODOs +## Security verification pending + +- [ ] Qualify the Windows Export/Trash path-transport fix with native Windows + guards and the native Linux/amd64 full suite. The implementation, portable + regressions, focused race tests and local inspections are complete; see + [verification record](plans/2026-09-30-windows-path-transport.md). + ## Done ### What's Changed From 44b04e0555ade9c64757f585e2e7f4a18e3ea812 Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 16:10:04 +0200 Subject: [PATCH 22/63] Record Windows path security qualification --- .../2026-09-30-windows-path-transport.md | 50 +++++++++++++++---- todos.md | 13 +++-- 2 files changed, 47 insertions(+), 16 deletions(-) rename {plans => finished_refactorings}/2026-09-30-windows-path-transport.md (69%) diff --git a/plans/2026-09-30-windows-path-transport.md b/finished_refactorings/2026-09-30-windows-path-transport.md similarity index 69% rename from plans/2026-09-30-windows-path-transport.md rename to finished_refactorings/2026-09-30-windows-path-transport.md index 6a0d21d4..e68d5895 100644 --- a/plans/2026-09-30-windows-path-transport.md +++ b/finished_refactorings/2026-09-30-windows-path-transport.md @@ -97,14 +97,46 @@ working-tree patch on 2026-09-30. - `make verify-build` passed (exit 0): formatting, TUF, exact Qodana test exclusions, generated assets/notices, full host vet and full host build. -## Handoff +## Hosted qualification and review -Implementation is complete; final verification is blocked on native Windows -execution and the native Linux/amd64 full suite. Keep the original finding open -until those gates pass. Run the existing Windows CI native-guard suite after the -patch is committed/pushed. Ronin authorized this step and the GitHub review -loop on 2026-09-30. Review and CI evidence will be recorded on -[PR #75](https://github.com/frathe/picfetch/pull/75). +Signed fix commit `bb6ce5dd86d4adf0f3271fbee50fb68692df15ff` was pushed to +[PR #75](https://github.com/frathe/picfetch/pull/75) with Ronin's authorization. +The earlier Trane commit `8533342` was already on the branch; its changes were +not included in the security-fix commit. -Ledger: zero spawns, one inline candidate review, one full-suite attempt. -Unrelated Trane files and edits were preserved. +- [CI 36725097510](https://github.com/frathe/picfetch/actions/runs/36725097510) + passed on the first attempt: validation, all four native Linux/amd64 race + partitions, Linux native guards, Windows tests, and both macOS architectures. + The local full-suite platform limitation is covered by this hosted run. +- The downloaded Windows event artifact confirms + `TestWindowsSaveTransport_PathIsData` passed (2.35 seconds) and + `TestWindowsTrashTransport_PathIsData` passed (7.03 seconds), with run/pass + events and no skips. The production PowerShell commands received exact + ordinary/malicious Unicode paths at the instrumented sinks; picker OK/Cancel + and file/directory recycle semantics remained intact. This closes the native + verification gap without opening a dialog or moving a real file. +- [CodeQL 36725097446](https://github.com/frathe/picfetch/actions/runs/36725097446) + passed Go and Actions analysis. +- [Qodana 36725097549](https://github.com/frathe/picfetch/actions/runs/36725097549) + completed. Its fresh post-suppression `/qodana.sarif.json` contains eight + `GoUnusedExportedFunction` false positives, all checked against production + callers: fileaccess.Stat, NeedsCapture, HasScope, Export, Import; + macbundle.CurrentRuntimeDirectory; openwith.InstallWindowDrop and Stop. + No actionable result or new broad suppression. The neutral annotation check + is not described as an empty report. Downloaded evidence is retained locally + under `.scratch/windows-path-review/bb6ce5d/`. +- [Fresh code review](https://github.com/frathe/picfetch/pull/75#issuecomment-5912827831) + and [fresh security review](https://github.com/frathe/picfetch/pull/75#issuecomment-5912966500) + report no findings on `bb6ce5d`. All three historical threads remain resolved; + this round introduced none. + +The first medium finding is fixed and qualified. This documentation-only +completion record carries forward unchanged-code local inspection/test evidence +from `bb6ce5d`; it does not claim a review of an older commit covers a new head. +Fresh code/security reviews and hosted CI for the final documentation commit +will be retained on PR #75, without another status-only commit. No merge or +release is authorized by this review loop. + +Ledger: zero spawns, one inline candidate review, one local full-suite attempt; +hosted full-suite and Windows qualification passed. The GitHub review loop is +additional user-authorized work. Unrelated Trane changes were preserved. diff --git a/todos.md b/todos.md index d5d3ad48..f8035ae2 100644 --- a/todos.md +++ b/todos.md @@ -1,12 +1,5 @@ # PicFetch — TODOs -## Security verification pending - -- [ ] Qualify the Windows Export/Trash path-transport fix with native Windows - guards and the native Linux/amd64 full suite. The implementation, portable - regressions, focused race tests and local inspections are complete; see - [verification record](plans/2026-09-30-windows-path-transport.md). - ## Done ### What's Changed @@ -15,6 +8,12 @@ #### Bugfix +- [x] Keep Windows Export/Trash filenames out of PowerShell source. Qualified + with native Windows regression events, the full Linux/amd64 race suite, + macOS checks, CodeQL, reviewed Qodana SARIF and clean code/security reviews + on `bb6ce5d`. See the [verification record](finished_refactorings/2026-09-30-windows-path-transport.md); + final documentation-head review/CI evidence is retained on PR #75. + #### Internal - [x] Add a three-second ASCII Trane turn and PicFetch wordmark to terminal From 6146a9e8ca28b5ffc0ff8cb5b8fbfa6546c3299e Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 16:42:18 +0200 Subject: [PATCH 23/63] Restrict release write authority to publication --- .github/workflows/release.yml | 10 ++- docs/release-signing.md | 8 ++ plans/2026-09-30-release-permissions.md | 108 ++++++++++++++++++++++++ scripts/msixstage/msixstage_test.go | 70 +++++++++++++++ todos.md | 6 ++ 5 files changed, 201 insertions(+), 1 deletion(-) create mode 100644 plans/2026-09-30-release-permissions.md diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index e21abb75..bbcbfd2d 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -6,7 +6,7 @@ on: - "v*" permissions: - contents: write + contents: read jobs: # The gate: a tag push runs the full CI suite (format, vet, build, test) @@ -29,6 +29,8 @@ jobs: steps: - name: Check out code uses: actions/checkout@v7 + with: + persist-credentials: false - name: Set up Go uses: actions/setup-go@v7 @@ -60,6 +62,8 @@ jobs: steps: - name: Check out code uses: actions/checkout@v7 + with: + persist-credentials: false - name: Set up Go uses: actions/setup-go@v7 @@ -219,10 +223,14 @@ jobs: release: needs: [build-macos, build-cross, sign-windows] + permissions: + contents: write runs-on: ubuntu-latest steps: - name: Check out code uses: actions/checkout@v7 + with: + persist-credentials: false - name: Download macOS artifacts uses: actions/download-artifact@v8 diff --git a/docs/release-signing.md b/docs/release-signing.md index 9701ac9d..07261047 100644 --- a/docs/release-signing.md +++ b/docs/release-signing.md @@ -53,6 +53,14 @@ a missing local installer fails instead of fetching another copy. ## Release flow +The release workflow defaults to `contents: read`; only the final `release` job +has `contents: write` to publish the approved artifacts. Build jobs and the +reusable CI gate therefore cannot use their job token to modify repository +contents or publish a release. All three checkouts in the release workflow use +`persist-credentials: false`; publication authenticates with its job token +through the release action. The protected signing environment remains the +manual approval boundary for Certum credentials. + 1. A v* tag runs the normal reusable CI test gate. 2. The Linux cross-build produces the two unsigned Windows ZIP artifacts. 3. The sign-windows job waits for the protected release-signing environment, diff --git a/plans/2026-09-30-release-permissions.md b/plans/2026-09-30-release-permissions.md new file mode 100644 index 00000000..fc1da9bf --- /dev/null +++ b/plans/2026-09-30-release-permissions.md @@ -0,0 +1,108 @@ +# Restrict ordinary release write authority + +## Contract + +Address the next medium item in `.scratch/sec_todo.md`. Ordinary tag-release +build tools must not receive repository-write authority. Only the final +publication job needs `contents: write`; checkout must not persist credentials. +Preserve the tag trigger, reusable CI gate, build outputs, manual Windows signing +approval, signed-artifact dependency and publisher authentication. + +Route: Standard (one CI hardening change). Lead owns investigation, tests, patch +and review. No delegation: the existing repository agreement reserves reviews +and hot-context fixes for the lead. No new dependency, package, UI string or +runtime behavior. The separate mutable publisher-tag finding stays open. + +## Source-backed investigation + +`release.yml` grants workflow-wide contents write. `build-macos` and `build-cross` +inherit it while executing external tools. `sign-windows` overrides to read and +uses the protected `release-signing` environment. The reusable `ci.yml` already +sets read permissions. The final `release` job needs write for action-gh-release; +its checkout and both build checkouts retain credentials by default. + +Use the existing job-level permission mechanism and checkout input. Artifact +upload/download in the same workflow uses the Actions runtime, not contents +write. Publisher action authentication remains its job token. No changes to +signing secrets or approval policy are needed. + +## Tasks and proof + +1. Add semantic YAML policy coverage beside the existing standalone release + checks in `scripts/msixstage/msixstage_test.go`. Reject workflow-wide write, + any non-publication job write, and persisted checkout credentials. Require + publication contents write and the existing signing approval/dependency. + Red/green command: `go test -tags no_emoji,nodynamic ./scripts/msixstage + -run TestReleaseWorkflowPermissionBoundary -count=1`. +2. Change `.github/workflows/release.yml` and explain the authority boundary in + `docs/release-signing.md`. Run the focused policy test, owning package and + adjacent workflow tests, actionlint and GoLand inspections including weak + warnings; challenge guards against unsafe variants. +3. Run local build/consistency verification, update `todos.md` and the scratch + checklist with observed results. A real tag-triggered release publishes + artifacts and requires signing approval; do not create a release for testing. + +Graph: 1 -> 2 -> 3. Budget: zero spawns, one inline candidate review, no repeated +broad race suite for this workflow-only change. No new test file, Qodana +exclusion, package map entry or UI shard is needed. + +## Evidence + +Base: `44b04e0555ade9c64757f585e2e7f4a18e3ea812`, working-tree patch reviewed +on 2026-09-30. The prior Windows fix/review loop is complete at that commit; +[final evidence](https://github.com/frathe/picfetch/pull/75#issuecomment-5913430773) +is on PR #75. Its hosted results do not cover this new workflow patch. + +- Red: `TestReleaseWorkflowPermissionBoundary` failed on workflow-wide write, + inherited build authority, persisted checkouts and missing explicit publisher + authority. The reusable `test` caller's permission ceiling was also write; + its called CI jobs already narrowed their own effective permissions to read. +- Green: the same focused command passes after making workflow contents read, + granting explicit contents write only to `release`, and disabling persistence + on the three checkouts. +- Negative verification: independently injected a build-job write override, + re-enabled checkout credential persistence, and removed the signing dependency. + Every variant failed for its expected policy diagnostic. The intended file was + restored in `finally` and the final control passed. +- `go test -tags no_emoji,nodynamic ./scripts/msixstage ./scripts/linuxdesktop + ./scripts/updaternotices ./scripts/wingettag`: all four packages passed. +- `/tmp/picfetch-actionlint-1.7.12/actionlint .github/workflows/release.yml`: + passed. Existing validated local tool; no shipped dependency added. +- `make verify-build`: passed (exit 0), including formatting, exact Qodana + exclusions, TUF/assets/notices, full host vet and build. +- GoLand MCP `get_file_problems(errorsOnly=false)` reviewed the workflow and + modified Go test, including weak warnings. The test has no findings. The + workflow has only the existing HTTP timestamp warning at its unchanged + Certum `/tr` endpoint. The [vendor's signing guide](https://files.certum.eu/documents/manual_en/CS-Code_Signing_in_the_Cloud_Signtool_jarsigner_signing.pdf) + documents this endpoint; timestamp authenticity relies on the signed RFC-3161 + response and the existing SignTool verification, not HTTP transport secrecy. + This is the same disposition recorded in the earlier PR 30 review; no broad + suppression or unrelated signing change was added. IDE analysis is a fallback, + not a new hosted Qodana SARIF result. +- Candidate review (lead, one round): no extra permission grants or authenticated + Git operations in the affected build/publication steps. Checkout still uses + its read-capable job token to clone, but does not retain credentials afterward. + The [publisher action's token input](https://github.com/softprops/action-gh-release/blob/v3/action.yml) + defaults to `github.token`; its explicit contents write permission remains. + Tag triggers, CI/build dependencies, the protected signing environment and + signed-artifact consumption are unchanged. Unspecified job permission scopes + remain denied by GitHub's existing permission semantics. +- `git diff --check`: passed. Existing test file remains exactly excluded from + Qodana duplicate-code checks; no new test file, UI test or package map change. + +## Review and release qualification + +The change is implemented and locally verified. Ronin subsequently authorized +commit/push and a fresh GitHub review loop on 2026-09-30. The final code/security +reviews, CI, CodeQL and fresh Qodana SARIF dispositions for the pushed head are +recorded on [PR #75](https://github.com/frathe/picfetch/pull/75). That mutable +record will hold the final outcome without another status-only commit; this +local evidence does not predeclare a passed hosted gate. +No release tag, publisher run or signing approval was initiated for testing. +The current Docker daemon is ARM64; the native Linux/amd64 full suite remains +unavailable locally. No broad race rerun or relaxed isolation policy was used +for this workflow-only patch. The floating publisher tag is the separate next +medium finding and remains unchanged. + +Ledger: zero spawns; one inline candidate review; focused owning/adjacent tests, +three negative guard probes and one local build/consistency gate. diff --git a/scripts/msixstage/msixstage_test.go b/scripts/msixstage/msixstage_test.go index 59284819..6dda16fe 100644 --- a/scripts/msixstage/msixstage_test.go +++ b/scripts/msixstage/msixstage_test.go @@ -20,6 +20,8 @@ import ( "testing" "github.com/frathe/picfetch/internal/imaging" + + "go.yaml.in/yaml/v3" ) func TestHEICStaticDeclarations(t *testing.T) { @@ -320,6 +322,74 @@ func TestStoreListingAssets(t *testing.T) { } } +func TestReleaseWorkflowPermissionBoundary(t *testing.T) { + raw, err := os.ReadFile(filepath.Join("..", "..", ".github", "workflows", "release.yml")) + if err != nil { + t.Fatal(err) + } + var workflow struct { + Permissions map[string]string `yaml:"permissions"` + Jobs map[string]struct { + Permissions map[string]string `yaml:"permissions"` + Environment string `yaml:"environment"` + Needs yaml.Node `yaml:"needs"` + Steps []struct { + Uses string `yaml:"uses"` + With map[string]any `yaml:"with"` + } `yaml:"steps"` + } `yaml:"jobs"` + } + if err := yaml.Unmarshal(raw, &workflow); err != nil { + t.Fatal(err) + } + if workflow.Permissions["contents"] != "read" { + t.Error("release workflow must default to contents read") + } + for scope, access := range workflow.Permissions { + if access == "write" { + t.Errorf("workflow default grants %s write", scope) + } + } + for name, job := range workflow.Jobs { + permissions := job.Permissions + if permissions == nil { + permissions = workflow.Permissions + } + for scope, access := range permissions { + if access == "write" && (name != "release" || scope != "contents") { + t.Errorf("job %s has unnecessary %s write authority", name, scope) + } + } + for _, step := range job.Steps { + action, _, _ := strings.Cut(step.Uses, "@") + if !strings.EqualFold(action, "actions/checkout") { + continue + } + // Both YAML booleans and literal string inputs are accepted by Actions. + persist := step.With["persist-credentials"] + if persist != false && persist != "false" { + t.Errorf("job %s checkout must explicitly disable persisted credentials", name) + } + } + } + publication, ok := workflow.Jobs["release"] + if !ok || publication.Permissions["contents"] != "write" { + t.Fatal("publication job must retain its explicit contents write grant") + } + var needs []string + if err := publication.Needs.Decode(&needs); err != nil { + t.Fatal(err) + } + for _, job := range []string{"build-macos", "build-cross", "sign-windows"} { + if !slices.Contains(needs, job) { + t.Errorf("publication lost required gate %s", job) + } + } + if workflow.Jobs["sign-windows"].Environment != "release-signing" { + t.Error("Windows signing lost its manual approval environment") + } +} + func TestStandaloneArchivesRetainNotices(t *testing.T) { root := filepath.Clean(filepath.Join("..", "..")) workflow, err := os.ReadFile(filepath.Join(root, ".github", "workflows", "release.yml")) diff --git a/todos.md b/todos.md index f8035ae2..e26e07e4 100644 --- a/todos.md +++ b/todos.md @@ -16,6 +16,12 @@ #### Internal +- [x] Restrict release-workflow contents write to publication and disable + credential persistence on release checkouts. Semantic policy guards preserve + signing approval and reject permission/credential regressions. Local checks + pass; final hosted review/CI evidence is retained on PR #75 and linked from + the [verification record](plans/2026-09-30-release-permissions.md). + - [x] Add a three-second ASCII Trane turn and PicFetch wordmark to terminal `--help`, with plain piped output and a `make trane` developer preview. Verification and limitations: [turntable record](finished_refactorings/2026-09-30-trane-ascii.md). From 8505c434f4df247a5450b8c118b4e9b6b40886d9 Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 17:25:29 +0200 Subject: [PATCH 24/63] Preserve cancellation during broker startup and terminal help --- internal/consolehelp/help.go | 21 ++++-- internal/consolehelp/help_test.go | 32 +++++++-- internal/macworker/native/client.m | 30 ++++++++- plans/2026-09-30-review-cancellation.md | 89 +++++++++++++++++++++++++ scripts/macworkerqualify/check.py | 21 ++++++ scripts/macworkerqualify/driver.m | 4 ++ scripts/macworkerqualify/run.sh | 13 ++++ todos.md | 5 ++ 8 files changed, 200 insertions(+), 15 deletions(-) create mode 100644 plans/2026-09-30-review-cancellation.md diff --git a/internal/consolehelp/help.go b/internal/consolehelp/help.go index 30e4acdd..667b6740 100644 --- a/internal/consolehelp/help.go +++ b/internal/consolehelp/help.go @@ -4,6 +4,7 @@ package consolehelp import ( "context" "errors" + "fmt" "io" "os" "os/signal" @@ -21,11 +22,12 @@ func Write(out io.Writer, usage string) error { restore, err := enableANSI(file) if err == nil { defer restore() - ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) - defer stop() - return animate(ctx, out, usage, func() (int, int, error) { + notices := make(chan os.Signal, 2) + signal.Notify(notices, os.Interrupt, syscall.SIGTERM) + defer signal.Stop(notices) + return animate(context.Background(), out, usage, func() (int, int, error) { return term.GetSize(int(file.Fd())) - }) + }, notices) } } _, err := io.WriteString(out, usage) @@ -34,13 +36,13 @@ func Write(out io.Writer, usage string) error { const turnDuration = 3 * time.Second -func animate(ctx context.Context, out io.Writer, usage string, size func() (int, int, error)) error { +func animate(ctx context.Context, out io.Writer, usage string, size func() (int, int, error), notices <-chan os.Signal) error { ticker := time.NewTicker(time.Second / 30) defer ticker.Stop() - return play(ctx, out, usage, size, ticker.C, time.Now()) + return play(ctx, out, usage, size, ticker.C, time.Now(), notices) } -func play(ctx context.Context, out io.Writer, usage string, size func() (int, int, error), ticks <-chan time.Time, start time.Time) (err error) { +func play(ctx context.Context, out io.Writer, usage string, size func() (int, int, error), ticks <-chan time.Time, start time.Time, notices <-chan os.Signal) (err error) { active := false final := usage defer func() { @@ -85,6 +87,11 @@ func play(ctx context.Context, out io.Writer, usage string, size func() (int, in select { case <-ctx.Done(): return nil + case notice := <-notices: + if notice == os.Interrupt { + return nil + } + return fmt.Errorf("help interrupted by %s", notice) case now := <-ticks: elapsed = now.Sub(start) if elapsed >= turnDuration { diff --git a/internal/consolehelp/help_test.go b/internal/consolehelp/help_test.go index 31cd9238..c9b4a20c 100644 --- a/internal/consolehelp/help_test.go +++ b/internal/consolehelp/help_test.go @@ -3,7 +3,9 @@ package consolehelp import ( "context" "errors" + "os" "strings" + "syscall" "testing" "time" ) @@ -15,7 +17,7 @@ func TestOneTurnExitsAndLeavesPortraitAboveHelp(t *testing.T) { ticks <- start.Add(1500 * time.Millisecond) ticks <- start.Add(3 * time.Second) const usage = "Usage:\n picfetch [flags]\n" - if err := play(context.Background(), &out, usage, func() (int, int, error) { return 100, 50, nil }, ticks, start); err != nil { + if err := play(context.Background(), &out, usage, func() (int, int, error) { return 100, 50, nil }, ticks, start, nil); err != nil { t.Fatal(err) } width, height, text := layout(100, 50, usage) @@ -34,7 +36,7 @@ func TestAnimationCancellationRestoresTerminal(t *testing.T) { var output strings.Builder out := callbackWriter{out: &output, after: cancel} const usage = "Usage:\n picfetch [flags]\n" - if err := animate(ctx, out, usage, func() (int, int, error) { return 100, 50, nil }); err != nil { + if err := animate(ctx, out, usage, func() (int, int, error) { return 100, 50, nil }, nil); err != nil { t.Fatal(err) } result := output.String() @@ -60,7 +62,7 @@ func (w callbackWriter) Write(p []byte) (int, error) { func TestSmallTerminalKeepsCompleteHelp(t *testing.T) { var out strings.Builder const usage = "Usage:\n picfetch [flags]\n" - if err := animate(context.Background(), &out, usage, func() (int, int, error) { return 20, 10, nil }); err != nil { + if err := animate(context.Background(), &out, usage, func() (int, int, error) { return 20, 10, nil }, nil); err != nil { t.Fatal(err) } if out.String() != usage { @@ -83,7 +85,7 @@ func TestResizeAndOutputFailureRestoreTerminal(t *testing.T) { } return 100, 50, nil } - if err := play(context.Background(), &out, usage, size, ticks, start); err != nil { + if err := play(context.Background(), &out, usage, size, ticks, start, nil); err != nil { t.Fatal(err) } if !strings.HasSuffix(out.String(), "\x1b[?25h\x1b[?1049l"+usage) { @@ -92,7 +94,7 @@ func TestResizeAndOutputFailureRestoreTerminal(t *testing.T) { }) t.Run("partial_write", func(t *testing.T) { out := &partialWriter{} - err := animate(context.Background(), out, usage, func() (int, int, error) { return 100, 50, nil }) + err := animate(context.Background(), out, usage, func() (int, int, error) { return 100, 50, nil }, nil) if !errors.Is(err, errOutput) || !strings.Contains(out.String(), "\x1b[?25h\x1b[?1049l") { t.Fatalf("partial write: error=%v, cleanup missing=%v", err, out.String()) } @@ -180,3 +182,23 @@ func TestTurntable(t *testing.T) { t.Fatal("front, angled, side and back views must be distinct") } } + +func TestAnimationSignalsRestoreTerminal(t *testing.T) { + for _, notice := range []os.Signal{os.Interrupt, syscall.SIGTERM} { + t.Run(notice.String(), func(t *testing.T) { + var out strings.Builder + notices := make(chan os.Signal, 1) + notices <- notice + const usage = "Usage: picfetch\n" + err := play(context.Background(), &out, usage, func() (int, int, error) { + return 100, 50, nil + }, nil, time.Unix(0, 0), notices) + if (err == nil) != (notice == os.Interrupt) { + t.Fatalf("signal %v returned %v; only Ctrl+C is successful", notice, err) + } + if !strings.HasSuffix(out.String(), "\x1b[?25h\x1b[?1049l"+usage) { + t.Fatal("signal did not restore cursor/screen and ordinary help") + } + }) + } +} diff --git a/internal/macworker/native/client.m b/internal/macworker/native/client.m index 383fe6c3..1085d140 100644 --- a/internal/macworker/native/client.m +++ b/internal/macworker/native/client.m @@ -5,7 +5,28 @@ #include #include +// Retain a cancellation that arrives before dispatch registers its signal source. +// The handler only records a signal-safe flag; XPC work stays on dispatch. +static volatile sig_atomic_t cancellationRequested = 0; +static void retainCancellation(int number) { + (void)number; + cancellationRequested = 1; +} + int main(int argc, const char **argv) { + if (signal(SIGTERM, retainCancellation) == SIG_ERR) { + perror("worker cancellation setup"); + return 1; + } + // A launcher may leave SIGTERM blocked across exec. Once the retaining + // handler is installed it is safe to deliver that pending cancellation. + sigset_t cancellationMask; + sigemptyset(&cancellationMask); + sigaddset(&cancellationMask, SIGTERM); + if (sigprocmask(SIG_UNBLOCK, &cancellationMask, NULL) < 0) { + perror("worker cancellation mask"); + return 1; + } @autoreleasepool { if (argc != 2 || (strcmp(argv[1], "heic") && strcmp(argv[1], "similarity"))) { fprintf(stderr, "usage: picfetch-worker-client heic|similarity\n"); @@ -33,17 +54,20 @@ int main(int argc, const char **argv) { // Cancellation must allow the service to kill and reap its children // before the caller's Wait returns. A killed broker also invalidates // the connection; the service independently retires its process group. - signal(SIGTERM, SIG_IGN); dispatch_source_t cancellation = dispatch_source_create(DISPATCH_SOURCE_TYPE_SIGNAL, SIGTERM, 0, dispatch_get_global_queue(QOS_CLASS_DEFAULT, 0)); - dispatch_source_set_event_handler(cancellation, ^{ + void (^cancel)(void) = ^{ [[connection remoteObjectProxyWithErrorHandler:^(NSError *error) { fprintf(stderr, "worker cancellation: %s\n", error.description.UTF8String); finish(1); }] cancel]; + }; + dispatch_source_set_event_handler(cancellation, cancel); + dispatch_source_set_registration_handler(cancellation, ^{ + if (cancellationRequested) cancel(); }); - dispatch_resume(cancellation); [connection resume]; + dispatch_resume(cancellation); [[connection remoteObjectProxyWithErrorHandler:^(NSError *error) { fprintf(stderr, "worker service: %s\n", error.description.UTF8String); finish(1); diff --git a/plans/2026-09-30-review-cancellation.md b/plans/2026-09-30-review-cancellation.md new file mode 100644 index 00000000..f22d48dd --- /dev/null +++ b/plans/2026-09-30-review-cancellation.md @@ -0,0 +1,89 @@ +# PR 75 cancellation review fixes + +Base: `6146a9e8ca28b5ffc0ff8cb5b8fbfa6546c3299e`. Its security review, +CodeQL, Qodana (eight confirmed unused-export false positives) and full CI passed. +The code review raised two confirmed cancellation defects; a fresh clean round +is required after their disposition. + +Route: Deep for the native broker boundary, implemented and reviewed by the +lead. Zero delegates; focused local tests and native qualification, full CI in +GitHub. No dependency, package, UI string or public API change. + +## Contracts and tasks + +1. Broker SIGTERM received during handler installation must survive until the + XPC cancel path can retire and reap the child. Preserve normal mode/pipe, + network isolation, bookmark and crash behavior. Add deterministic early + SIGTERM injection to the existing signed qualification fixture; demonstrate + failure before fixing and pass afterward with + `sh scripts/macworkerqualify/run.sh`. Files: native/client.m, qualification + run.sh, driver.m and check.py. Guard both HEIC and similarity. +2. Animated terminal help must restore cursor/screen on both SIGINT and SIGTERM; + only SIGINT means successful skip. Existing launchArgs error handling must + return nonzero for SIGTERM. Add signal cases to consolehelp/help_test.go, + then fix help.go. Run `go test -race ./internal/consolehelp` and a real PTY + subprocess probe; preserve redirected/small-terminal/resize/output-error paths. +3. Inspect every changed code file including weak warnings, run focused native + and Go regressions plus `make verify-build`, update this record and todos, + sign/push, reply/resolve both threads and request one fresh review round. + +Graph: each regression -> its fix -> shared qualification/review. One candidate +review per fix. No broad local race suite. Existing test files/exclusions stay. + +## Investigation + +- Broker changes SIGTERM to SIG_IGN before dispatch registration. A signal in + that gap is lost. A native Darwin probe also proved SIG_BLOCK plus SIG_IGN + discards it (`sigpending` returned zero), so the suggested mask alone is + insufficient. The broker now installs a minimal handler that records a + `volatile sig_atomic_t` flag before Foundation setup, explicitly unblocks + inherited SIGTERM, and replays a recorded cancellation in dispatch's + registration handler. Later signals use the ordinary event handler. Resume + the XPC connection before activating the source; both routes preserve the + service's existing cleanup acknowledgment. No XPC work runs in a signal handler. +- consolehelp.Write subscribes both signals to a context whose cancellation + play treats as success. launchArgs already returns 1 for Write errors; no + root startup change is required. Pass notifications into the existing finite + animation loop, avoiding another goroutine or mutable test seam. + +## Verification + +- Native red: injecting SIGTERM immediately after the production signal + disposition call made the signed broker's early-HEIC case time out while its + worker remained active. A mask-only candidate also failed and was replaced. +- Native green: `sh scripts/macworkerqualify/run.sh` passes all 12 cases: both + worker modes with blocked/unblocked inherited masks, normal mode/pipe/reaping, + invalid mode, held/descendant cancellation, broker crash, normal descendant + cleanup and private-source bookmark transfer. The injection is compile-time + only; production has no test environment or argument seam. Clang uses + `-Wall -Wextra -Werror`. The app/helper/service fixtures are ad-hoc signed. +- Console red: a real PTY sent SIGTERM after the first rendered frame; original + help restored the terminal but returned 0. The same probe now observes SIGINT + exit 0 and SIGTERM exit 1, both with cursor/alternate-screen restoration. +- `TestAnimationSignalsRestoreTerminal` covers both signals. A deliberately + restored SIGTERM-as-success branch failed its exact expected diagnostic; + the intended source was restored immediately afterward. +- `go test -race -tags no_emoji,nodynamic ./internal/consolehelp + ./internal/macworker` passes, including existing resize/plain/output-error + help checks. `make verify-build` completes formatting, asset/TUF/notice and + exact-exclusion checks, full host vet and build without errors; only the + existing native duplicate `-lobjc` linker warning appears. +- GoLand `get_file_problems(errorsOnly=false)` reports no findings in all six + changed Go/native/Python/shell code files. Reinspected the Python fixture + after adding both inherited signal masks. This is the IDE-local fallback, + not a replacement for fresh hosted Qodana SARIF assessment. +- Root `go test -tags no_emoji,nodynamic -run + 'TestLaunchArgs|TestLaunchStartupContract' .` passes (0.571s). Console-help + tests cross-compile for Windows/amd64 with CGO disabled (exit 0). +- Candidate review (lead): early and later signal paths converge on the same + idempotent XPC cancel operation; normal completion still waits for the reply + after service reaping. Terminal cleanup remains deferred on every return, + and existing launchArgs/preview error handling returns nonzero. No new Go + worker or public interface, dependency, package, test file or shard entry. + +Final hosted outcomes will be recorded in the PR after the fresh review; +no release or merge is authorized. Native qualification here covers Apple +Silicon, not a physical Intel host. Full native amd64 suites run in hosted CI. + +Ledger: zero spawns; one final candidate review of each fix; targeted race, +native qualification, real PTY and one local build/static gate. diff --git a/scripts/macworkerqualify/check.py b/scripts/macworkerqualify/check.py index b391145d..306a0c4d 100644 --- a/scripts/macworkerqualify/check.py +++ b/scripts/macworkerqualify/check.py @@ -1,5 +1,6 @@ """Exercise the actual signed broker/service and transferred process pipes.""" import errno +from functools import partial import json import os from pathlib import Path @@ -30,6 +31,26 @@ def assert_reaped(pid): raise AssertionError(f"worker {pid} survived broker completion") +for mask in (signal.SIG_BLOCK, signal.SIG_UNBLOCK): + for mode in ("heic", "similarity"): + process = subprocess.Popen(command("early-" + mode), stdin=subprocess.PIPE, + stdout=subprocess.PIPE, stderr=subprocess.PIPE, + text=True, preexec_fn=partial(signal.pthread_sigmask, mask, {signal.SIGTERM})) + try: + output, error = process.communicate("hold\n", timeout=10) + # Cancellation may beat admission or retire a worker already submitted. + assert process.returncode in (2, 128 + signal.SIGKILL), (output, error, process.returncode) + for line in output.splitlines(): + assert_reaped(json.loads(line)["pid"]) + print(f"PASS {mode}/{mask.name}: SIGTERM during handler installation preserved; cleanup acknowledged") + except subprocess.TimeoutExpired as error: + raise AssertionError(f"early {mode} cancellation timed out: {error.stdout!r}; {error.stderr!r}") from error + finally: + if process.poll() is None: + process.kill() + process.communicate(timeout=15) + + for mode in ("heic", "similarity"): result = subprocess.run(command(mode), input="echo\n", capture_output=True, text=True, timeout=15) diff --git a/scripts/macworkerqualify/driver.m b/scripts/macworkerqualify/driver.m index 4b419cc7..260139dc 100644 --- a/scripts/macworkerqualify/driver.m +++ b/scripts/macworkerqualify/driver.m @@ -11,6 +11,10 @@ int main(int argc, char **argv) { NSString *broker = [NSBundle.mainBundle.bundlePath stringByAppendingPathComponent:@"Contents/MacOS/picfetch-worker-client"]; const char *mode = argv[1]; + if (!strncmp(mode, "early-", 6)) { + broker = [broker stringByAppendingString:@"-early"]; + mode += 6; + } if (!strcmp(mode, "grant")) { NSURL *directory = [NSFileManager.defaultManager URLsForDirectory:NSDocumentDirectory inDomains:NSUserDomainMask].firstObject; diff --git a/scripts/macworkerqualify/run.sh b/scripts/macworkerqualify/run.sh index 7f1f66e1..2628e5f9 100755 --- a/scripts/macworkerqualify/run.sh +++ b/scripts/macworkerqualify/run.sh @@ -33,11 +33,24 @@ PLIST esac echo '' >> "$stage/$kind.plist" done +# Inject at the production signal-disposition boundary, without a runtime seam. +cat > "$stage/early-cancel.h" <<'HEADER' +#import +#include +static inline void (*qualify_signal(int number, void (*handler)(int)))(int) { + void (*previous)(int) = signal(number, handler); + if (number == SIGTERM && previous != SIG_ERR) raise(SIGTERM); + return previous; +} +#define signal qualify_signal +HEADER xcrun clang -fobjc-arc -Wall -Wextra -Werror -framework Foundation "$repo/internal/macworker/native/client.m" -o "$app/Contents/MacOS/picfetch-worker-client" +xcrun clang -fobjc-arc -Wall -Wextra -Werror -framework Foundation -include "$stage/early-cancel.h" "$repo/internal/macworker/native/client.m" -o "$app/Contents/MacOS/picfetch-worker-client-early" xcrun clang -fobjc-arc -Wall -Wextra -Werror -framework Foundation "$repo/internal/macworker/native/service.m" -o "$service/Contents/MacOS/worker" xcrun clang -fobjc-arc -Wall -Wextra -Werror -framework Foundation "$repo/scripts/macworkerqualify/driver.m" -o "$app/Contents/MacOS/qualification" xcrun clang -fobjc-arc -Wall -Wextra -Werror -framework Foundation "$repo/scripts/macworkerqualify/fixture.m" -o "$service/Contents/MacOS/picfetch-image-worker" codesign --force --sign - --entitlements "$stage/helper.plist" "$app/Contents/MacOS/picfetch-worker-client" +codesign --force --sign - --entitlements "$stage/helper.plist" "$app/Contents/MacOS/picfetch-worker-client-early" codesign --force --sign - --entitlements "$stage/helper.plist" "$service/Contents/MacOS/picfetch-image-worker" codesign --force --sign - --entitlements "$stage/service.plist" "$service" codesign --force --sign - --entitlements "$stage/app.plist" "$app" diff --git a/todos.md b/todos.md index e26e07e4..65353478 100644 --- a/todos.md +++ b/todos.md @@ -8,6 +8,11 @@ #### Bugfix +- [x] Preserve broker cancellation during signal-source startup and retain a + nonzero exit status when SIGTERM interrupts animated terminal help. Native + XPC and terminal regressions pass; fresh hosted review evidence is retained + on PR #75. See the [verification record](plans/2026-09-30-review-cancellation.md). + - [x] Keep Windows Export/Trash filenames out of PowerShell source. Qualified with native Windows regression events, the full Linux/amd64 race suite, macOS checks, CodeQL, reviewed Qodana SARIF and clean code/security reviews From 91b88053a5abc4fc901eca570ed5543593aaec58 Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 17:49:29 +0200 Subject: [PATCH 25/63] Pin release publication actions to reviewed commits --- .github/workflows/release.yml | 17 +++-- plans/2026-09-30-release-action-pins.md | 98 +++++++++++++++++++++++++ scripts/msixstage/msixstage_test.go | 38 ++++++++++ todos.md | 6 ++ 4 files changed, 152 insertions(+), 7 deletions(-) create mode 100644 plans/2026-09-30-release-action-pins.md diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index bbcbfd2d..85a6d5a8 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -228,36 +228,36 @@ jobs: runs-on: ubuntu-latest steps: - name: Check out code - uses: actions/checkout@v7 + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7 with: persist-credentials: false - name: Download macOS artifacts - uses: actions/download-artifact@v8 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8 with: name: picfetch-macos-arm64 path: dist - name: Download Intel macOS artifacts - uses: actions/download-artifact@v8 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8 with: name: picfetch-macos-x86_64 path: dist - name: Download Linux artifacts - uses: actions/download-artifact@v8 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8 with: name: picfetch-linux path: dist - name: Download signed Windows artifacts - uses: actions/download-artifact@v8 + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8 with: name: picfetch-windows-signed path: dist - name: Set up Go for final notice inspection - uses: actions/setup-go@v7 + uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7 with: go-version-file: go.mod cache-dependency-path: go.sum @@ -269,7 +269,10 @@ jobs: go run ./scripts/updaternotices -artifact dist/picfetch-linux-amd64.tar.gz -artifact dist/picfetch-linux-arm64.tar.gz - name: Create GitHub release - uses: softprops/action-gh-release@v3 + uses: softprops/action-gh-release@efb35369e0ad2afab669f228072c1b0d510eae64 # v3.0.3 with: + # Both inputs are declared in this pinned action's action.yml. + # noinspection UndefinedParamsPresent files: dist/* + # noinspection UndefinedParamsPresent body_path: .github/release-notes.md diff --git a/plans/2026-09-30-release-action-pins.md b/plans/2026-09-30-release-action-pins.md new file mode 100644 index 00000000..ce9c99b1 --- /dev/null +++ b/plans/2026-09-30-release-action-pins.md @@ -0,0 +1,98 @@ +# Immutable ordinary release publication actions + +## Contract and scope + +Address the last medium finding in `.scratch/sec_todo.md`, after the completed +PR 75 review loop at `8505c434f4df247a5450b8c118b4e9b6b40886d9`. +The ordinary release publisher must execute a reviewed immutable action commit. +Every peer action in the same `contents: write` job shares that authority, so +pin the entire publication job. Preserve versions, inputs, artifact names, +notice checks, manual signing approval, permission boundary and tag trigger. + +Route: Standard, lead inline; zero delegates. Files: release.yml, existing +msixstage_test.go, this plan and todos.md. No application code, dependencies, +packages, test files or UI strings are added. Other jobs' build-tool trust is +outside this publication-action finding. + +## Source and compatibility review + +The previous floating `softprops/action-gh-release@v3` resolves to the same +commit selected below (3.0.3). Official GitHub commit APIs report `verified=true` +and `reason=valid` for all four revisions, checked 2026-09-30. The publisher's +immutable action.yml and src/run.ts/src/util.ts retain tag detection, `dist/*` +file expansion, body_path reading and the default `github.token` input. +The peer actions' metadata retains all current inputs and Node 24 entry points. +No runtime version upgrade is part of this change. + +| Action | Existing ref | Selected full commit | +|---|---|---| +| softprops/action-gh-release | v3 (3.0.3) | `efb35369e0ad2afab669f228072c1b0d510eae64` | +| actions/checkout | v7 (7.0.1) | `3d3c42e5aac5ba805825da76410c181273ba90b1` | +| actions/download-artifact | v8 | `3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c` | +| actions/setup-go | v7 | `b7ad1dad31e06c5925ef5d2fc7ad053ef454303e` | + +Sources are the corresponding official GitHub repositories at these revisions. +Each LICENSE was read and is MIT: retain copyright/license when copying or +redistributing their software. These actions and their bundled Node dependency +closures execute only on CI runners; they are not copied into PicFetch or its +release archives. The same selected revisions already back the current floating +refs. No shipped dependency or third-party notice change is needed. Runner action +checkouts retain upstream license files. Existing weekly GitHub Actions Dependabot +coverage keeps future pin updates reviewable; version comments remain beside pins. + +Publisher metadata: https://github.com/softprops/action-gh-release/blob/efb35369e0ad2afab669f228072c1b0d510eae64/action.yml +Publisher license: https://github.com/softprops/action-gh-release/blob/efb35369e0ad2afab669f228072c1b0d510eae64/LICENSE + +## Tasks and verification + +1. Add semantic YAML coverage for all publication action refs and unchanged + publisher files/body inputs. Red: `go test -tags no_emoji,nodynamic + ./scripts/msixstage -run TestReleasePublicationActionsPinned -count=1`. +2. Pin all seven action steps (four unique upstream actions) in the publication + job. Green with the same command. Independently restore a mutable publisher + tag and a mutable peer-action tag; require both variants to fail, then restore. +3. Run owning/adjacent workflow package tests, actionlint, GoLand inspections of + both code/config files including weak warnings and `make verify-build`. + Review the final diff and record evidence. Full suite remains hosted CI; + a real release upload requires a separately authorized tag/signing run. + +Graph: regression -> pins -> qualification. Budget: zero spawns, one inline +candidate review, one build/static gate, no broad local race rerun. + +## Evidence and remaining qualification + +- Red: `TestReleasePublicationActionsPinned` rejected all seven floating refs. +- Green: both publication-pin and release-permission guards pass. Five negative + probes reject a mutable publisher, mutable peer, abbreviated SHA, missing + publisher and changed artifact input. The final workflow was restored after + each run and passed as the control. +- `go test -tags no_emoji,nodynamic ./scripts/msixstage ./scripts/linuxdesktop + ./scripts/updaternotices ./scripts/wingettag` passed (owning package 3.512s). +- Existing actionlint 1.7.12 passes. `make verify-build` exited 0, including + format/TUF/generated-input/notice/exclusion checks and full host vet/build. + `git diff --check` passes. +- GoLand inspected both changed code/config files including weak warnings. The + Go test is clear. At the new immutable publisher ref, its metadata inspection + incorrectly reported `files` and `body_path` undefined; the pinned upstream + action.yml explicitly declares both. Two input-local `UndefinedParamsPresent` + suppressions were added and reinspection confirms those warnings are gone. + The [inspection documentation](https://www.jetbrains.com/help/inspectopedia/UndefinedParamsPresent.html) + identifies this exact suppression. The unchanged Certum HTTP timestamp weak + warning retains its source-backed disposition from the permission plan. + Policy tests and actionlint passed again after the comment-only suppression. +- Candidate review (lead): all seven publication action steps use full SHAs; + no tag or abbreviated ref remains in that write-enabled job. Selected refs + resolve to the current upstream versions; Node entry points, publisher inputs, + notice checks, artifact names and protected signing dependency are unchanged. + The earlier permission guard still passes. Existing exact test exclusion + applies; no new test file or shard entry is required. + +Implementation and local verification are complete. Hosted results for the +pushed head will be recorded on PR 75. Pinning fixes mutable resolution; it does +not prove all upstream software safe or remove other build-tool trust. Actual +publication remains qualification for the next separately authorized tag release; +no release was created for testing. Full native Linux/amd64 suites run in hosted +CI because the local Docker daemon is ARM64. + +Ledger: zero spawns, one inline candidate review, five negative probes, one +owning/adjacent package run and one local build/static gate. diff --git a/scripts/msixstage/msixstage_test.go b/scripts/msixstage/msixstage_test.go index 6dda16fe..d1653b21 100644 --- a/scripts/msixstage/msixstage_test.go +++ b/scripts/msixstage/msixstage_test.go @@ -390,6 +390,44 @@ func TestReleaseWorkflowPermissionBoundary(t *testing.T) { } } +func TestReleasePublicationActionsPinned(t *testing.T) { + raw, err := os.ReadFile(filepath.Join("..", "..", ".github", "workflows", "release.yml")) + if err != nil { + t.Fatal(err) + } + var workflow struct { + Jobs map[string]struct { + Steps []struct { + Uses string `yaml:"uses"` + With map[string]string `yaml:"with"` + } `yaml:"steps"` + } `yaml:"jobs"` + } + if err := yaml.Unmarshal(raw, &workflow); err != nil { + t.Fatal(err) + } + pinned := regexp.MustCompile(`^[^@]+@[0-9a-f]{40}$`) + publishers := 0 + for _, step := range workflow.Jobs["release"].Steps { + if step.Uses == "" { + continue + } + if !pinned.MatchString(step.Uses) { + t.Errorf("publication action must use an immutable full commit SHA: %s", step.Uses) + } + action, _, _ := strings.Cut(step.Uses, "@") + if strings.EqualFold(action, "softprops/action-gh-release") { + publishers++ + if step.With["files"] != "dist/*" || step.With["body_path"] != ".github/release-notes.md" { + t.Error("release publisher lost its archive or release-note inputs") + } + } + } + if publishers != 1 { + t.Errorf("want exactly one release publisher, got %d", publishers) + } +} + func TestStandaloneArchivesRetainNotices(t *testing.T) { root := filepath.Clean(filepath.Join("..", "..")) workflow, err := os.ReadFile(filepath.Join(root, ".github", "workflows", "release.yml")) diff --git a/todos.md b/todos.md index 65353478..ef0288cb 100644 --- a/todos.md +++ b/todos.md @@ -21,6 +21,12 @@ #### Internal +- [x] Pin every action in the write-enabled release publication job to a + reviewed full commit. Preserve publisher inputs and signing gates; semantic + regressions reject mutable refs. Local validation and provenance are in the + [verification record](plans/2026-09-30-release-action-pins.md); hosted results + are retained on PR #75. + - [x] Restrict release-workflow contents write to publication and disable credential persistence on release checkouts. Semantic policy guards preserve signing approval and reject permission/credential regressions. Local checks From 382a40d8ea195613b1990ced2581c966445c682c Mon Sep 17 00:00:00 2001 From: frathe Date: Wed, 30 Sep 2026 19:15:09 +0200 Subject: [PATCH 26/63] Bound secondary JPEG source reads during Save and Export --- internal/imaging/mutations.go | 28 ++++- internal/imaging/mutations_test.go | 111 ++++++++++++++++++ internal/imaging/save.go | 2 +- .../2026-09-30-secondary-jpeg-read-budget.md | 81 +++++++++++++ todos.md | 6 + 5 files changed, 226 insertions(+), 2 deletions(-) create mode 100644 plans/2026-09-30-secondary-jpeg-read-budget.md diff --git a/internal/imaging/mutations.go b/internal/imaging/mutations.go index ec389c74..d7beb577 100644 --- a/internal/imaging/mutations.go +++ b/internal/imaging/mutations.go @@ -170,7 +170,33 @@ func readFileContext(ctx context.Context, path string) ([]byte, error) { return nil, err } defer func() { _ = f.Close() }() - return io.ReadAll(contextRead{ctx: ctx, in: f}) + return readMutationSource(ctx, f) +} + +// Secondary reads obey the same encoded-input policy as initial image loads. +// Stat rejects known oversized files cheaply; the stream bound also covers +// regular files that grow after that observation. +func readMutationSource(ctx context.Context, f *os.File) ([]byte, error) { + limit := MaxEncodedBytes() + info, err := f.Stat() + if err != nil { + return nil, err + } + if info.Size() > limit { + return nil, &InputTooLargeError{limit: limit} + } + return readMutationBytes(ctx, f, limit) +} + +func readMutationBytes(ctx context.Context, in io.Reader, limit int64) ([]byte, error) { + data, err := io.ReadAll(contextRead{ctx: ctx, in: io.LimitReader(in, limit+1)}) + if err != nil { + return nil, err + } + if int64(len(data)) > limit { + return nil, &InputTooLargeError{limit: limit} + } + return data, nil } type contextRead struct { diff --git a/internal/imaging/mutations_test.go b/internal/imaging/mutations_test.go index cb0bea54..e283c02e 100644 --- a/internal/imaging/mutations_test.go +++ b/internal/imaging/mutations_test.go @@ -561,3 +561,114 @@ func TestExportDoesNotReadMetadataThroughClosedSourceAuthority(t *testing.T) { t.Fatal("export read metadata outside source access") } } + +func TestSecondaryJPEGReadsRespectInputLimit(t *testing.T) { + for _, change := range []string{"grow", "replace"} { + for _, operation := range []string{"save", "export", "mislabeled export"} { + t.Run(change+"/"+operation, func(t *testing.T) { + source := uitest.TempGPSJPEGURI(t, "source.jpg", 8, 4, 48.858222, 2.2945) + original := mustRead(t, source.Path()) + withMaxEncodedBytes(t, int64(len(original))) + loaded, err := LoadImage(source, 0) + if err != nil { + t.Fatal(err) + } + oversized := append(bytes.Clone(original), 0) + if change == "grow" { + f, err := os.OpenFile(source.Path(), os.O_APPEND|os.O_WRONLY, 0) + if err != nil { + t.Fatal(err) + } + _, writeErr := f.Write([]byte{0}) + closeErr := f.Close() + if err := errors.Join(writeErr, closeErr); err != nil { + t.Fatal(err) + } + } else { + replacement := writeTempFile(t, "replacement.jpg", oversized) + if err := os.Rename(replacement, source.Path()); err != nil { + t.Fatal(err) + } + } + if operation == "save" { + result, err := SaveRotatedContext(context.Background(), source, loaded.Frames[0]) + var tooLarge *InputTooLargeError + if !errors.As(err, &tooLarge) || result.Committed { + t.Fatalf("oversized save = %+v, %v; want uncommitted InputTooLargeError", result, err) + } + } else { + if operation == "mislabeled export" { + path := filepath.Join(filepath.Dir(source.Path()), "source.png") + if err := os.Rename(source.Path(), path); err != nil { + t.Fatal(err) + } + source = storage.NewFileURI(path) + } + data, err := jpegSourceBytesContext(context.Background(), source) + var tooLarge *InputTooLargeError + if !errors.As(err, &tooLarge) || data != nil { + t.Fatalf("oversized metadata read = %d bytes, %v", len(data), err) + } + dest := storage.NewFileURI(filepath.Join(t.TempDir(), "copy.jpg")) + result, err := ExportContext(context.Background(), dest, loaded.Frames[0], source, ExportOptions{}) + if err != nil || !result.Committed { + t.Fatalf("pixel-only export = %+v, %v", result, err) + } + if len(jpegMetadataSegments(mustRead(t, dest.Path()))) != 0 { + t.Fatal("export retained metadata from oversized source") + } + } + if !bytes.Equal(mustRead(t, source.Path()), oversized) { + t.Fatal("operation changed oversized source") + } + }) + } + } +} + +func TestSecondaryJPEGReadsAtExactLimitPreserveMetadata(t *testing.T) { + for _, operation := range []string{"save", "export"} { + t.Run(operation, func(t *testing.T) { + source := uitest.TempGPSJPEGURI(t, "source.jpg", 8, 4, 48.858222, 2.2945) + original := mustRead(t, source.Path()) + withMaxEncodedBytes(t, int64(len(original))) + loaded, err := LoadImage(source, 0) + if err != nil { + t.Fatal(err) + } + dest := source + if operation == "save" { + err = SaveRotated(source, loaded.Frames[0]) + } else { + dest = storage.NewFileURI(filepath.Join(t.TempDir(), "copy.jpg")) + err = Export(dest, loaded.Frames[0], source, ExportOptions{}) + } + if err != nil { + t.Fatal(err) + } + if !ReadMetadata(mustRead(t, dest.Path())).HasGPS { + t.Fatal("exact-limit source lost GPS metadata") + } + }) + } +} + +func TestReadMutationSourceBoundsStream(t *testing.T) { + const limit = 64 + in := bytes.NewReader(bytes.Repeat([]byte{0xff}, 4096)) + data, err := readMutationBytes(context.Background(), in, limit) + var tooLarge *InputTooLargeError + if !errors.As(err, &tooLarge) || data != nil { + t.Fatalf("growing stream = %d bytes, %v", len(data), err) + } + if consumed := in.Size() - int64(in.Len()); consumed != limit+1 { + t.Fatalf("read consumed %d bytes, want %d", consumed, limit+1) + } + + ctx, cancel := context.WithCancel(context.Background()) + cancel() + in = bytes.NewReader([]byte{0xff, 0xd8}) + if _, err := readMutationBytes(ctx, in, limit); !errors.Is(err, context.Canceled) || in.Len() != 2 { + t.Fatalf("cancelled stream = %v, remaining %d bytes", err, in.Len()) + } +} diff --git a/internal/imaging/save.go b/internal/imaging/save.go index 4ef4e8ee..e2159de3 100644 --- a/internal/imaging/save.go +++ b/internal/imaging/save.go @@ -311,7 +311,7 @@ func jpegFileBytesContext(ctx context.Context, path string) ([]byte, error) { if _, err := f.Seek(0, io.SeekStart); err != nil { return nil, err } - return io.ReadAll(contextRead{ctx: ctx, in: f}) + return readMutationSource(ctx, f) } // writeEncodedContext encodes img into a same-volume staging file and diff --git a/plans/2026-09-30-secondary-jpeg-read-budget.md b/plans/2026-09-30-secondary-jpeg-read-budget.md new file mode 100644 index 00000000..9af0ec5a --- /dev/null +++ b/plans/2026-09-30-secondary-jpeg-read-budget.md @@ -0,0 +1,81 @@ +# Bound Save and Export JPEG rereads + +Route: Standard, lead inline, zero delegates. The repository agreement keeps +hot-context investigation, fixes and every review with the lead; the security +skill's investigation and candidate review are separate inline passes. + +## Contract + +After display admission, an external writer can grow or replace a regular JPEG. +Save Changes rereads it through `readFileContext`; JPEG Export acquires source +authority and rereads through `jpegFileBytesContext`. Both currently allocate +without a byte limit. Every complete reread must capture `MaxEncodedBytes`, +reject an oversized stat before allocation, and enforce the bound while reading +even when the file grows after stat. Cancellation remains observable. + +Save rejects oversized metadata input without replacing the source. Export +keeps its existing unreadable-source behavior: write displayed pixels without +source metadata. Exact-limit JPEGs, mislabeled JPEGs, non-JPEG fast rejection, +permissions, atomic writes and URI authority remain supported. Metadata-removal +limits and the separate source-identity findings are outside this patch. + +## Tasks and acceptance commands + +1. Extend existing `mutations_test.go` with replacement/growth-after-display, + exact-limit metadata preservation and bounded streaming regressions. + Red/green: `go test -tags no_emoji,nodynamic ./internal/imaging -run + 'TestSecondaryJPEG|TestReadMutationSource' -count=1`. +2. Add a shared private bounded reader in `mutations.go`; apply it to Save's + full reread and Export's magic-confirmed reread in `save.go`. Inspect all + direct callers and challenge alternate filenames and growth during reads. +3. Run imaging race tests, focused root Save/Export regressions, Apple Store + tagged coverage, GoLand inspections including weak warnings and + `make verify-build`. Commit/push this fix, complete fresh GitHub code/security + reviews, full CI, CodeQL and post-suppression Qodana assessment before the + next issue. No broad local race suite; the Docker daemon is ARM64. + +Graph: regression -> bounded reads -> qualification -> commit/review loop. +Budget: zero spawns, one inline candidate review, focused tests and one local +build/static gate. No dependency, package, UI string or test-file addition. + +## Evidence + +- Red: all six growth/replacement cases reproduced unbounded rereads after a + successful display load. Save committed; normal and mislabeled Export sources + returned bytes past the configured ceiling. Exact-limit controls passed. +- Green: focused Save/Export, magic detection and cancellation tests pass. + Shared stream coverage consumes exactly limit+1 bytes from a larger producer, + returns no partial bytes on overflow and reads nothing after cancellation. + Removing the stream limiter (while retaining stat admission) made this guard + fail with 4096 bytes consumed instead of 65. Source restored afterward. +- `go test -race -tags no_emoji,nodynamic ./internal/imaging`: PASS (33.404s). + Focused root Save/Export race regressions: PASS (34.226s). Apple Store-tagged + bounded read and live-authority tests using the actual `appleappstore` tag: + PASS (0.492s). The initial `applestore` spelling selected ordinary files; + that run is not counted as Store verification. +- `make verify-build`: PASS, including full host vet/build, format, generated + inputs, notices, TUF and exact test exclusion checks. Initial sandboxed race + and build attempts could not access Go cache entries; authorized reruns passed. +- GoLand inspected all three changed Go files with weak warnings included. + Production files are clear. The existing missing-source test at line 152 has + an invalid nullable-result warning: `WriteResult` is a nonpointer value whose + zero `Committed` field intentionally remains observable alongside an error. + No suppression or unrelated test change is needed. Final helper reinspection + after the negative probe is clear. The restored final helper also passed + its focused race rerun (1.314s) and `make verify-build` passed again. +- Lead candidate review: both complete rereads share the helper; no extension + alias bypasses Export's content check. Both outcomes preserve existing error + behavior and acquired URI authority; non-JPEG sources still stop after two + bytes. Stat is only an early optimization, never the sole memory guard. + Metadata-removal's tighter independent budget is unchanged. No new test file, + UI test or package move requires exclusions, shard or architecture changes. + +Hosted code/security reviews, full native Linux/amd64 CI, CodeQL and a fresh +post-suppression Qodana assessment are pending for the pushed fix. Final results +will be retained on PR 75 before proceeding to the recursive-scan finding. +On resumption, the lead rechecked the complete candidate and direct callers; +the focused security race regressions passed again (1.644s), and +`git diff --check HEAD` was clear. Ronin explicitly authorized commit/push and +a separate completed GitHub review loop after each security fix. +Ledger: zero spawns, one inline candidate review, one negative streaming probe, +focused regression/race checks and local build/static gate. diff --git a/todos.md b/todos.md index ef0288cb..35e760a5 100644 --- a/todos.md +++ b/todos.md @@ -8,6 +8,12 @@ #### Bugfix +- [x] Bound Save Changes and JPEG Export secondary source reads by the encoded + input limit, including growth after stat. Save leaves oversized sources intact; + Export preserves pixel-only fallback. Local race, Store-tagged and static + checks pass; hosted qualification is pending on PR #75. See the + [verification record](plans/2026-09-30-secondary-jpeg-read-budget.md). + - [x] Preserve broker cancellation during signal-source startup and retain a nonzero exit status when SIGTERM interrupts animated terminal help. Native XPC and terminal regressions pass; fresh hosted review evidence is retained From 1439746bb77cf8dbc2dafc4078ebba2e9d3a6012 Mon Sep 17 00:00:00 2001 From: frathe Date: Thu, 1 Oct 2026 09:03:37 +0200 Subject: [PATCH 27/63] Add fixed pixel screenshot window mode with terminal picker --- .github/testshards/internal-ui.tsv | 3 +- ARCHITECTURE.md | 21 +++- README.md | 20 ++++ internal/consolehelp/help_test.go | 40 ++++++++ internal/consolehelp/resolution.go | 112 +++++++++++++++++++++ internal/consolehelp/resolution_other.go | 12 +++ internal/consolehelp/resolution_unix.go | 32 ++++++ internal/consolehelp/resolution_windows.go | 30 ++++++ internal/launch/launch.go | 31 ++++++ internal/launch/launch_test.go | 42 ++++++++ internal/screenshots/app.go | 108 ++++++++++++++++++++ internal/screenshots/app_test.go | 41 ++++++++ internal/ui/harness_test.go | 7 +- internal/ui/launchoptions.go | 8 ++ internal/ui/launchoptions_test.go | 62 ++++++++++++ internal/ui/run.go | 2 +- internal/ui/startup.go | 11 ++ internal/winpos/darwin.go | 25 +++++ internal/winpos/linux.go | 37 +++++++ internal/winpos/other.go | 6 ++ internal/winpos/windows.go | 42 ++++++++ internal/winpos/winpos.go | 16 +++ internal/winpos/winpos_test.go | 24 +++++ main.go | 7 ++ main_startup.go | 8 ++ main_startup_test.go | 36 +++++++ plans/2026-10-01-fixed-size-mode.md | 100 ++++++++++++++++++ qodana.yaml | 1 + todos.md | 5 + 29 files changed, 882 insertions(+), 7 deletions(-) create mode 100644 internal/consolehelp/resolution.go create mode 100644 internal/consolehelp/resolution_other.go create mode 100644 internal/consolehelp/resolution_unix.go create mode 100644 internal/consolehelp/resolution_windows.go create mode 100644 internal/screenshots/app.go create mode 100644 internal/screenshots/app_test.go create mode 100644 plans/2026-10-01-fixed-size-mode.md diff --git a/.github/testshards/internal-ui.tsv b/.github/testshards/internal-ui.tsv index 087545de..ee5c8217 100644 --- a/.github/testshards/internal-ui.tsv +++ b/.github/testshards/internal-ui.tsv @@ -5,7 +5,7 @@ # baseline-run: 34627413846 # baseline-attempts: 1 # weights: single CI attempt; projected test sums, not measured shard runtimes -# ui-1: 271 entries, 470.520s projected weight sum +# ui-1: 272 entries, 470.520s projected weight sum # ui-2: 255 entries, 470.530s projected weight sum # ui-3: 224 entries, 504.870s projected weight sum # rebalance: 27 whole-test moves; slowest projected sum within 5% of mean @@ -170,6 +170,7 @@ TestInvalidateLoad_CancelsPriorLoadContext ui-1 TestKey0YieldsCopySelectionAndResetsRotation ui-1 TestLastUpdateCheckDay_ConcurrentWithCurrentPreferences ui-1 TestLaunchOptions_AppliesEverySetting ui-1 +TestLaunchOptions_FixedSizeGeometryAndModes ui-1 TestLaunchOptions_PictureFrameSpentWhenLaunchCancelled ui-1 TestLaunchOptions_PictureFrameSpentWhenResetBeforeScan ui-1 TestLaunchOptions_PictureFrameSpentWhenTheLaunchLoadsNothing ui-1 diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index f2b67318..14072f78 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -37,7 +37,9 @@ Developer preview of the production console help, reached through `make trane`. `Write` renders one eased, three-second ASCII Trane turn above terminal help, then leaves a front portrait and PicFetch wordmark. `render.go` owns the original ellipsoid sculpture and depth-tested lighting; `help.go` owns fitting, wrapping, -signals and terminal restoration. Build-tagged terminal files enable/restore +signals and terminal restoration. `resolution.go` owns the fixed-size screenshot picker; its build-tagged readers +poll without detached stdin workers, so interruption restores raw mode. +Build-tagged terminal files enable/restore Windows VT mode. Pipes, small terminals and `TERM=dumb` receive plain usage. ### `scripts/historymovie` @@ -1066,6 +1068,17 @@ on screen). `realdrag_test.go` replays recorded title-bar drags. | `detector.go` | Ring buffer, idle gap, one-shot `armed` latch. | | `analyse.go` | Centroid, accumulated angle, sign consistency, radius-vs-angle fit. | +### `internal/screenshots` + +`app.go` decorates the application before viewer/feature construction for a +fixed-size launch. Every app-owned window contains natural content minimums and +suppresses programmatic resize/fullscreen requests, while forwarding native and +desktop position/topmost operations. Show converts whole-window physical pixels +through winpos's native chrome/backing metrics into Fyne content coordinates. +`app_test.go` covers main and secondary windows, content replacement and reopen. +`startup.go` captures original persisted geometry in the launch override so these +windows never replace standing geometry during screenshot sessions. + ### `internal/winpos` Fyne has no position getter and no move event. `Get` reads the native @@ -1079,10 +1092,10 @@ pollers across close/reopen for `WaitForTracking`. | File | Responsibility | |------|----------------| -| `winpos.go` | `Get`, `Set`, `Maximize`, `Unmaximize`. | +| `winpos.go` | `Get`, `Set`, `Maximize`, `Unmaximize`, `ScreenshotContentSize`; fixed windows reject maximize/restore. | | `poll.go` | `PollAt` / `Poll` / `Poller` / `PollInterval` / `GestureInterval`; per-call native-read/dispatch seam and cancellation-aware acknowledgement. | | `tracker.go` | `Tracker` atomics: `Store` / `Get` / `Capture` / `Restore`. | -| `darwin.go` / `windows.go` / `linux.go` / `other.go` | Platform position + maximize. Linux/Wayland: `Get` reports `ok=false`. | +| `darwin.go` / `windows.go` / `linux.go` / `other.go` | Platform position, maximize and screenshot chrome/backing metrics. Linux/Wayland: `Get` reports `ok=false`. | OS integrations (`clipboard`, `displays`, `filemanager`, `filepicker`, `trash`, `wallpaper`) use @@ -1244,7 +1257,7 @@ against the `preferences.SortBy*` vocabulary. No Fyne import. | File | Responsibility | |------|----------------| -| `launch.go` | `Options`, `Parse`, `Usage`, `ErrHelp`; the `flagSpecs` table every flag is declared in. | +| `launch.go` | `Options`, `Resolution`, `ScreenshotResolutions`, `Parse`, `Usage`, `ErrHelp`; the `flagSpecs` table every flag is declared in. | | `policy.go` | Explicit immutable `Policy`, `UpdatePermission` and storage selection, independent of trial resources and offline prerequisites. | | `preparation.go` | Per-launch prerequisite/reservation operations and `Prepared`, the single evidence owner; partial acquisition and finalization preserve joined errors and retained evidence. | diff --git a/README.md b/README.md index dac2d646..6f7fc728 100644 --- a/README.md +++ b/README.md @@ -226,6 +226,7 @@ set up the launch. Run `picfetch --help` for the same list. | Flag | Effect | |------|--------| +| `--fixed-size-mode` | Pick 1280 x 800, 1440 x 900, 2560 x 1600 or 2880 x 1800 with Up/Down and Return; constrain app windows for screenshots. | | `--slideshow` | Start in picture-frame mode once the given files have loaded. | | `--shuffle` | Advance to a random file in picture-frame mode. | | `--interval=DURATION` | Picture-frame auto-advance interval, e.g. `8s` or `1m30s`. | @@ -250,6 +251,25 @@ A photo frame that starts with the machine is then just: picfetch --slideshow --shuffle --interval=20s /srv/photos ``` +For screenshot sessions, run the executable directly from a terminal: + +```sh +picfetch -fixed-size-mode +# Optional preset for scripts: +picfetch -fixed-size-mode=1280x800 ~/photos +# Packaged macOS application: +/path/to/PicFetch.app/Contents/MacOS/picfetch -fixed-size-mode +``` + +Dimensions are physical pixels of the whole window, including its title bar. +Main and secondary PicFetch windows keep that size; image fitting, Grid, Explorer, +Location Map, picture-frame and Spiral cannot enlarge them or enter fullscreen. +Saved window geometry is preserved. Native OS file dialogs retain their OS sizing. +On macOS use a window capture without a shadow: Cmd+Shift+4, Space, then hold +Option while clicking the window. Shadows add pixels outside the selected size. +The exact sizes have been measured on Retina macOS; native Windows/X11 captures +still need qualification, and unsupported native geometry is logged. + ## Building All build tasks are defined in the [Makefile](Makefile). Run `make help` to diff --git a/internal/consolehelp/help_test.go b/internal/consolehelp/help_test.go index c9b4a20c..783736c8 100644 --- a/internal/consolehelp/help_test.go +++ b/internal/consolehelp/help_test.go @@ -1,13 +1,17 @@ package consolehelp import ( + "bytes" "context" "errors" + "fmt" "os" "strings" "syscall" "testing" "time" + + "github.com/frathe/picfetch/internal/launch" ) func TestOneTurnExitsAndLeavesPortraitAboveHelp(t *testing.T) { @@ -202,3 +206,39 @@ func TestAnimationSignalsRestoreTerminal(t *testing.T) { }) } } + +func TestResolutionPicker(t *testing.T) { + for _, tc := range []struct { + keys string + index int + cancelled bool + }{ + {keys: "\r", index: 0}, {keys: "\x1b[B\n", index: 1}, + {keys: "\x1b[B\x1b[B\r", index: 2}, {keys: "\x1b[A\r", index: 3}, + {keys: "\x1bOB\x1bOA\r", index: 0}, {keys: "\x03", cancelled: true}, + {keys: "\x04", cancelled: true}, + } { + t.Run(fmt.Sprintf("%q", tc.keys), func(t *testing.T) { + var out bytes.Buffer + input := strings.NewReader(tc.keys) + size, err := pickResolution(&out, func() (byte, bool, error) { b, err := input.ReadByte(); return b, err == nil, err }, nil) + if tc.cancelled { + if err == nil { + t.Fatal("cancelled picker launched") + } + return + } + if err != nil { + t.Fatal(err) + } + if want := launch.ScreenshotResolutions()[tc.index]; size != want { + t.Fatalf("size=%v want=%v", size, want) + } + for _, label := range []string{"1280 x 800px", "1440 x 900px", "2560 x 1600px", "2880 x 1800px"} { + if !strings.Contains(out.String(), label) { + t.Fatalf("missing %s", label) + } + } + }) + } +} diff --git a/internal/consolehelp/resolution.go b/internal/consolehelp/resolution.go new file mode 100644 index 00000000..f9f84bf9 --- /dev/null +++ b/internal/consolehelp/resolution.go @@ -0,0 +1,112 @@ +package consolehelp + +import ( + "errors" + "fmt" + "io" + "os" + "os/signal" + "syscall" + + "golang.org/x/term" + + "github.com/frathe/picfetch/internal/launch" +) + +// SelectResolution offers a keyboard picker before creating the desktop app. +// A missing terminal is an error; scripts can supply an explicit flag value. +func SelectResolution(in *os.File, out io.Writer) (size launch.Resolution, err error) { + output, ok := out.(*os.File) + if !ok || !term.IsTerminal(int(in.Fd())) || !term.IsTerminal(int(output.Fd())) || os.Getenv("TERM") == "dumb" { + return size, errors.New("fixed-size-mode needs an interactive terminal; use -fixed-size-mode=1280x800 for a scripted launch") + } + restoreANSI, err := enableANSI(output) + if err != nil { + return size, err + } + defer restoreANSI() + state, err := term.MakeRaw(int(in.Fd())) + if err != nil { + return size, err + } + defer func() { err = errors.Join(err, term.Restore(int(in.Fd()), state)) }() + notices := make(chan os.Signal, 2) + signal.Notify(notices, os.Interrupt, syscall.SIGTERM) + defer signal.Stop(notices) + defer func() { + _, restoreErr := io.WriteString(out, "\x1b[?25h\x1b[?1049l") + err = errors.Join(err, restoreErr) + }() + if _, err = io.WriteString(out, "\x1b[?1049h\x1b[?25l"); err != nil { + return size, err + } + size, err = pickResolution(out, func() (byte, bool, error) { return readTerminalByte(in) }, notices) + return size, err +} + +func pickResolution(out io.Writer, read func() (byte, bool, error), notices <-chan os.Signal) (launch.Resolution, error) { + sizes := launch.ScreenshotResolutions() + selected, escape := 0, 0 + draw := func() error { + if _, err := io.WriteString(out, "\x1b[H\x1b[2JChoose screenshot size (Up/Down, Return; Ctrl+C cancels):\r\n\r\n"); err != nil { + return err + } + for i, size := range sizes { + marker := " " + if i == selected { + marker = "> " + } + if _, err := fmt.Fprintf(out, "%s%d x %dpx\r\n", marker, size.Width, size.Height); err != nil { + return err + } + } + return nil + } + if err := draw(); err != nil { + return launch.Resolution{}, err + } + for { + select { + case <-notices: + return launch.Resolution{}, errors.New("fixed-size selection interrupted") + default: + } + key, ready, err := read() + if err != nil { + return launch.Resolution{}, fmt.Errorf("fixed-size selection: %w", err) + } + if !ready { + continue + } + if key == 3 || key == 4 { + return launch.Resolution{}, errors.New("fixed-size selection cancelled") + } + if escape == 1 { + if key == '[' || key == 'O' { + escape = 2 + continue + } + escape = 0 + } else if escape == 2 { + escape = 0 + switch key { + case 'A': + selected = (selected + len(sizes) - 1) % len(sizes) + case 'B': + selected = (selected + 1) % len(sizes) + default: + continue + } + if err := draw(); err != nil { + return launch.Resolution{}, err + } + continue + } + switch key { + case 27: + escape = 1 + case '\r', '\n': + return sizes[selected], nil + } + } +} diff --git a/internal/consolehelp/resolution_other.go b/internal/consolehelp/resolution_other.go new file mode 100644 index 00000000..5add6b36 --- /dev/null +++ b/internal/consolehelp/resolution_other.go @@ -0,0 +1,12 @@ +//go:build !darwin && !linux && !freebsd && !openbsd && !netbsd && !dragonfly && !windows + +package consolehelp + +import ( + "errors" + "os" +) + +func readTerminalByte(_ *os.File) (byte, bool, error) { + return 0, false, errors.New("interactive resolution selection is unsupported on this platform") +} diff --git a/internal/consolehelp/resolution_unix.go b/internal/consolehelp/resolution_unix.go new file mode 100644 index 00000000..526c142a --- /dev/null +++ b/internal/consolehelp/resolution_unix.go @@ -0,0 +1,32 @@ +//go:build darwin || linux || freebsd || openbsd || netbsd || dragonfly + +package consolehelp + +import ( + "errors" + "io" + "os" + + "golang.org/x/sys/unix" +) + +// Polling bounds signal response without leaving an unjoinable stdin worker. +func readTerminalByte(file *os.File) (byte, bool, error) { + events := []unix.PollFd{{Fd: int32(file.Fd()), Events: unix.POLLIN}} + count, err := unix.Poll(events, 100) + if errors.Is(err, unix.EINTR) { + return 0, false, nil + } + if err != nil || count == 0 { + return 0, false, err + } + if events[0].Revents&unix.POLLNVAL != 0 { + return 0, false, os.ErrClosed + } + var data [1]byte + n, err := file.Read(data[:]) + if n == 0 && err == nil { + err = io.EOF + } + return data[0], n != 0, err +} diff --git a/internal/consolehelp/resolution_windows.go b/internal/consolehelp/resolution_windows.go new file mode 100644 index 00000000..cbf2ea39 --- /dev/null +++ b/internal/consolehelp/resolution_windows.go @@ -0,0 +1,30 @@ +//go:build windows + +package consolehelp + +import ( + "fmt" + "io" + "os" + + "golang.org/x/sys/windows" +) + +func readTerminalByte(file *os.File) (byte, bool, error) { + status, err := windows.WaitForSingleObject(windows.Handle(file.Fd()), 100) + if err != nil { + return 0, false, err + } + if status == uint32(windows.WAIT_TIMEOUT) { + return 0, false, nil + } + if status != windows.WAIT_OBJECT_0 { + return 0, false, fmt.Errorf("terminal wait status %d", status) + } + var data [1]byte + n, err := file.Read(data[:]) + if n == 0 && err == nil { + err = io.EOF + } + return data[0], n != 0, err +} diff --git a/internal/launch/launch.go b/internal/launch/launch.go index a6691872..57e0ca09 100644 --- a/internal/launch/launch.go +++ b/internal/launch/launch.go @@ -38,6 +38,10 @@ type Options struct { // setting: there is no saved value to override and nothing to restore. PictureFrame bool + // FixedSizeMode requests the terminal picker unless FixedSize is supplied. + FixedSizeMode bool + FixedSize *Resolution + // ExplorerTrial requests an isolated native collection in a new directory. ExplorerTrial string LocationMapTrial string @@ -56,6 +60,16 @@ type Options struct { MaxFiles *int } +// Resolution measures the entire screenshot window in physical pixels. +type Resolution struct { + Width, Height int +} + +// ScreenshotResolutions returns a fresh value, so callers cannot alter presets. +func ScreenshotResolutions() [4]Resolution { + return [4]Resolution{{1280, 800}, {1440, 900}, {2560, 1600}, {2880, 1800}} +} + // sortModes is every value --sort accepts, in the order the usage text lists // them. Parse validates against this rather than handing the string straight // to filesort.FromPref, which maps anything unrecognized to ByName - so @@ -79,6 +93,23 @@ type spec struct { } var flagSpecs = []spec{ + { + name: "fixed-size-mode", + help: "pick a fixed screenshot size with Up/Down and Return (or =1280x800, =1440x900, =2560x1600, =2880x1800)", + set: func(o *Options, raw string) error { + if enabled, err := strconv.ParseBool(raw); err == nil { + o.FixedSizeMode, o.FixedSize = enabled, nil + return nil + } + for _, size := range ScreenshotResolutions() { + if raw == fmt.Sprintf("%dx%d", size.Width, size.Height) { + o.FixedSizeMode, o.FixedSize = true, &size + return nil + } + } + return errors.New("want true, false, 1280x800, 1440x900, 2560x1600 or 2880x1800") + }, + }, { name: "location-map-trial", arg: "DIR", diff --git a/internal/launch/launch_test.go b/internal/launch/launch_test.go index 1d0110e5..bf6608ed 100644 --- a/internal/launch/launch_test.go +++ b/internal/launch/launch_test.go @@ -310,3 +310,45 @@ func TestTrialPreparationHonorsCancellation(t *testing.T) { t.Fatalf("cancelled trial owner=%v err=%v", owner, err) } } + +func TestFixedSizeMode(t *testing.T) { + for _, tc := range []struct { + flag string + enabled bool + size *Resolution + invalid bool + }{ + {flag: "-fixed-size-mode", enabled: true}, + {flag: "--fixed-size-mode=true", enabled: true}, + {flag: "--fixed-size-mode=false"}, + {flag: "--fixed-size-mode=1280x800", enabled: true, size: &Resolution{1280, 800}}, + {flag: "--fixed-size-mode=1440x900", enabled: true, size: &Resolution{1440, 900}}, + {flag: "--fixed-size-mode=2560x1600", enabled: true, size: &Resolution{2560, 1600}}, + {flag: "--fixed-size-mode=2880x1800", enabled: true, size: &Resolution{2880, 1800}}, + {flag: "--fixed-size-mode=1280x900", invalid: true}, + {flag: "--fixed-size-mode=", invalid: true}, + } { + t.Run(tc.flag, func(t *testing.T) { + paths, opts, err := Parse([]string{"photo.jpg", tc.flag}) + if tc.invalid { + if err == nil { + t.Fatal("invalid resolution accepted") + } + return + } + if err != nil { + t.Fatal(err) + } + if len(paths) != 1 || paths[0] != "photo.jpg" || opts.FixedSizeMode != tc.enabled { + t.Fatalf("paths=%v options=%+v", paths, opts) + } + if tc.size == nil { + if opts.FixedSize != nil { + t.Fatal("unexpected selection") + } + } else if opts.FixedSize == nil || *opts.FixedSize != *tc.size { + t.Fatalf("size=%v want %v", opts.FixedSize, tc.size) + } + }) + } +} diff --git a/internal/screenshots/app.go b/internal/screenshots/app.go new file mode 100644 index 00000000..c026a62c --- /dev/null +++ b/internal/screenshots/app.go @@ -0,0 +1,108 @@ +// Package screenshots constrains PicFetch-owned windows for a screenshot launch. +package screenshots + +import ( + "math" + + "fyne.io/fyne/v2" + "fyne.io/fyne/v2/container" + "fyne.io/fyne/v2/driver" + "fyne.io/fyne/v2/driver/desktop" + + "github.com/frathe/picfetch/internal/winpos" +) + +type application struct { + fyne.App + width, height int +} + +// App returns an application whose windows share physical screenshot dimensions. +// The caller supplies a validated launch preset. The native driver remains owned +// by the original application; lifecycle and persistence operations pass through. +func App(app fyne.App, width, height int) fyne.App { + return &application{App: app, width: width, height: height} +} + +func (a *application) NewWindow(title string) fyne.Window { + w := &window{Window: a.App.NewWindow(title), app: a.App, width: a.width, height: a.height} + w.Window.SetContent(container.New(fixedLayout{})) + w.Window.SetPadded(false) + w.Window.Resize(fyne.NewSize(float32(a.width), float32(a.height))) + w.Window.SetFixedSize(true) + return w +} + +type window struct { + fyne.Window + app fyne.App + width, height int + content fyne.CanvasObject +} + +// fixedLayout keeps a panel's natural minimum from expanding the native window. +// Its children still get normal layout inside the available screenshot surface. +type fixedLayout struct{} + +func (fixedLayout) MinSize(_ []fyne.CanvasObject) fyne.Size { return fyne.Size{} } +func (fixedLayout) Layout(objects []fyne.CanvasObject, size fyne.Size) { + for _, object := range objects { + object.Move(fyne.Position{}) + object.Resize(size) + } +} + +func (w *window) Resize(_ fyne.Size) {} +func (w *window) SetFullScreen(_ bool) {} +func (w *window) SetFixedSize(_ bool) {} +func (w *window) RequestFullScreenSecondary() {} +func (w *window) Content() fyne.CanvasObject { return w.content } +func (w *window) SetContent(content fyne.CanvasObject) { + w.content = content + root := container.New(fixedLayout{}) + if content != nil { + root.Add(content) + } + w.Window.SetContent(root) +} +func (w *window) Show() { + w.Window.Show() + size, ok := winpos.ScreenshotContentSize(w.Window, w.width, w.height) + if !ok { + // The test driver has no chrome or backing scale. Native unsupported + // backends retain the bounded canvas size and report the missing metric. + if _, native := w.Window.(driver.NativeWindow); native { + fyne.LogError("screenshot window geometry unavailable", nil) + } + size = fyne.NewSize(float32(w.width), float32(w.height)) + } + scale := w.Window.Canvas().Scale() + if scale <= 0 { + scale = 1 + } + // Fyne rounds native coordinates upward. Bias by one float step so division + // by a fractional UI scale cannot add a physical pixel at that boundary. + logical := fyne.NewSize(math.Nextafter32(size.Width/scale, 0), math.Nextafter32(size.Height/scale, 0)) + if !ok { + logical = size + } // exact dimensions for the headless test driver + w.Window.SetFixedSize(false) + w.Window.Resize(logical) + w.Window.SetFixedSize(true) +} +func (w *window) ShowAndRun() { w.Show(); w.app.Run() } +func (w *window) RunNative(fn func(any)) { + if native, ok := w.Window.(driver.NativeWindow); ok { + native.RunNative(fn) + } +} +func (w *window) RequestPosition(x, y int) { + if native, ok := w.Window.(desktop.Window); ok { + native.RequestPosition(x, y) + } +} +func (w *window) RequestAlwaysOnTop() { + if native, ok := w.Window.(desktop.Window); ok { + native.RequestAlwaysOnTop() + } +} diff --git a/internal/screenshots/app_test.go b/internal/screenshots/app_test.go new file mode 100644 index 00000000..0e33370e --- /dev/null +++ b/internal/screenshots/app_test.go @@ -0,0 +1,41 @@ +package screenshots + +import ( + "testing" + + "fyne.io/fyne/v2" + "fyne.io/fyne/v2/canvas" + "fyne.io/fyne/v2/test" +) + +func TestWindowConstraint(t *testing.T) { + application := test.NewApp() + defer application.Quit() + constrained := App(application, 1280, 800) + for _, title := range []string{"main", "secondary"} { + w := constrained.NewWindow(title) + content := canvas.NewRectangle(nil) + content.SetMinSize(fyne.NewSize(4000, 3000)) + w.SetContent(content) + w.Resize(fyne.NewSize(6000, 5000)) + w.SetFullScreen(true) + w.SetFixedSize(false) + w.Show() + if got := w.Canvas().Size(); got != fyne.NewSize(1280, 800) { + t.Fatalf("%s size=%v", title, got) + } + if !w.FixedSize() || w.FullScreen() { + t.Fatalf("%s fixed=%v fullscreen=%v", title, w.FixedSize(), w.FullScreen()) + } + if w.Content() != content { + t.Fatal("content identity lost") + } + w.SetContent(canvas.NewRectangle(nil)) + w.Resize(fyne.NewSize(200, 200)) + w.Show() + if got := w.Canvas().Size(); got != fyne.NewSize(1280, 800) { + t.Fatalf("reopen size=%v", got) + } + w.Close() + } +} diff --git a/internal/ui/harness_test.go b/internal/ui/harness_test.go index 69d1c8b8..15df2409 100644 --- a/internal/ui/harness_test.go +++ b/internal/ui/harness_test.go @@ -113,6 +113,11 @@ func buildTestStartupViewer(t *testing.T, application fyne.App) (*viewer, fyne.W func newTestUIWithPolicy(t *testing.T, policy launch.Policy) (v *viewer, win fyne.Window, closed func() bool) { t.Helper() + return newTestUIWithLaunchOptions(t, policy, launch.Options{}) +} + +func newTestUIWithLaunchOptions(t *testing.T, policy launch.Policy, opts launch.Options) (v *viewer, win fyne.Window, closed func() bool) { + t.Helper() // Reassert the shared app as the current one before building: the // persistence tests construct their own app, and Fyne makes whichever @@ -136,7 +141,7 @@ func newTestUIWithPolicy(t *testing.T, policy launch.Policy) (v *viewer, win fyn } var err error - v, win, err = buildStartupViewer(testApp, policy, testLaunchStorage(t)) + v, win, err = buildStartupViewerForLaunch(testApp, policy, testLaunchStorage(t), opts) if err != nil { t.Fatal(err) } diff --git a/internal/ui/launchoptions.go b/internal/ui/launchoptions.go index 61cc3831..108f1b99 100644 --- a/internal/ui/launchoptions.go +++ b/internal/ui/launchoptions.go @@ -32,12 +32,20 @@ type launchOverride struct { shuffle *bool interval *time.Duration maxScan *int + geometry *preferences.State } // restore rewrites every overridden field of s back to its pre-flag value. // Called by currentPreferences, at the end, so it is the last word on what // shutdown writes. func (o launchOverride) restore(s *preferences.State) { + if o.geometry != nil { + saved := o.geometry + s.WindowSize = saved.WindowSize + s.WindowPosX, s.WindowPosY, s.WindowPositionSet = saved.WindowPosX, saved.WindowPosY, saved.WindowPositionSet + s.SettingsWindow, s.ExifWindow, s.MosaicWindow = saved.SettingsWindow, saved.ExifWindow, saved.MosaicWindow + } + if o.sortMode != nil { s.SortMode = *o.sortMode } diff --git a/internal/ui/launchoptions_test.go b/internal/ui/launchoptions_test.go index 68304f6d..5c6dd94a 100644 --- a/internal/ui/launchoptions_test.go +++ b/internal/ui/launchoptions_test.go @@ -18,6 +18,7 @@ import ( "fyne.io/fyne/v2/storage" "fyne.io/fyne/v2/test" + "github.com/frathe/picfetch/internal/distribution" "github.com/frathe/picfetch/internal/filesort" "github.com/frathe/picfetch/internal/imaging" "github.com/frathe/picfetch/internal/launch" @@ -342,3 +343,64 @@ func TestLaunchOptions_PictureFrameSpentWhenResetBeforeScan(t *testing.T) { }) } } + +func TestLaunchOptions_FixedSizeGeometryAndModes(t *testing.T) { + before := preferences.Load(testApp) + t.Cleanup(func() { + p := testApp.Preferences() + for _, key := range []string{"windowWidth", "windowHeight", "windowPosX", "windowPosY", "windowPosSet"} { + p.RemoveValue(key) + } + for _, prefix := range []string{"settingsWin", "exifWin", "mosaicWin"} { + for _, suffix := range []string{"PosX", "PosY", "PosSet", "Width", "Height"} { + p.RemoveValue(prefix + suffix) + } + } + preferences.Save(testApp, before) + }) + saved := before + saved.WindowSize = fyne.NewSize(700, 500) + saved.WindowPosX, saved.WindowPosY, saved.WindowPositionSet = 43, 59, true + saved.SettingsWindow = preferences.WindowGeometry{X: 13, Y: 17, PositionSet: true, Size: fyne.NewSize(640, 700)} + saved.ExifWindow = preferences.WindowGeometry{X: 23, Y: 29, PositionSet: true, Size: fyne.NewSize(600, 650)} + saved.MosaicWindow = preferences.WindowGeometry{X: 31, Y: 37, PositionSet: true, Size: fyne.NewSize(800, 600)} + preferences.Save(testApp, saved) + opts := launch.Options{FixedSizeMode: true, FixedSize: &launch.Resolution{Width: 1280, Height: 800}} + v, win, _ := newTestUIWithLaunchOptions(t, testLaunchPolicy(t, opts, distribution.StoreManaged), opts) + v.applyLaunchOptions(opts) + win.Show() + assertSize := func() { + t.Helper() + if got := win.Canvas().Size(); got != fyne.NewSize(1280, 800) { + t.Fatalf("size=%v", got) + } + } + assertSize() + win.Resize(fyne.NewSize(3000, 2000)) + assertSize() + dropAndWait(t, v, uitest.TempJPEGURI(t, "fixed.jpg", 80, 60, color.White)) + assertSize() + v.grid.Toggle() + assertSize() + v.grid.Toggle() + v.togglePictureFrameMode() + if !v.slides.Active() { + t.Fatal("picture-frame mode did not enter") + } + if win.FullScreen() { + t.Fatal("picture-frame resized into fullscreen") + } + assertSize() + v.togglePictureFrameMode() + assertSize() + v.settingsWin.Show(v.currentPreferences(), v.launchPolicy.Updates()) + v.SetSortMode(filesort.ByModTime) + waitForSort(t, v) + current := v.currentPreferences() + if current.WindowSize != saved.WindowSize || current.WindowPosX != saved.WindowPosX || current.WindowPosY != saved.WindowPosY || current.WindowPositionSet != saved.WindowPositionSet || current.SettingsWindow != saved.SettingsWindow || current.ExifWindow != saved.ExifWindow || current.MosaicWindow != saved.MosaicWindow { + t.Fatalf("screenshot geometry persisted: %+v", current) + } + if current.SortMode != preferences.SortByModTime { + t.Fatal("unrelated setting was held back") + } +} diff --git a/internal/ui/run.go b/internal/ui/run.go index 3bc7fd1f..a650bb67 100644 --- a/internal/ui/run.go +++ b/internal/ui/run.go @@ -42,7 +42,7 @@ func Run(application fyne.App, initial []fyne.URI, opts launch.Options, prepared return launch.ErrInvalidPolicy } trial := prepared.ExplorerTrial() - view, window, err := buildStartupViewer(application, policy, ordinaryLaunchStorage) + view, window, err := buildStartupViewerForLaunch(application, policy, ordinaryLaunchStorage, opts) if err != nil { return err } diff --git a/internal/ui/startup.go b/internal/ui/startup.go index af7f00ba..5f351c8b 100644 --- a/internal/ui/startup.go +++ b/internal/ui/startup.go @@ -15,6 +15,7 @@ import ( "github.com/frathe/picfetch/internal/imaging" "github.com/frathe/picfetch/internal/launch" "github.com/frathe/picfetch/internal/preferences" + "github.com/frathe/picfetch/internal/screenshots" "github.com/frathe/picfetch/internal/session" "github.com/frathe/picfetch/internal/ui/autoupdate" ) @@ -40,6 +41,10 @@ func loadStartupState(application fyne.App) startupState { // buildStartupViewer is the shared load, construct, then restore entry point. // It leaves noPollerStop installed for startViewerRuntime to replace. func buildStartupViewer(application fyne.App, policy launch.Policy, resolveOrdinary func(fyne.App) (launch.Storage, error)) (*viewer, fyne.Window, error) { + return buildStartupViewerForLaunch(application, policy, resolveOrdinary, launch.Options{}) +} + +func buildStartupViewerForLaunch(application fyne.App, policy launch.Policy, resolveOrdinary func(fyne.App) (launch.Storage, error), opts launch.Options) (*viewer, fyne.Window, error) { if !policy.Valid() { return nil, nil, launch.ErrInvalidPolicy } @@ -58,7 +63,13 @@ func buildStartupViewer(application fyne.App, policy launch.Policy, resolveOrdin startup := loadStartupState(application) startup.policy = policy startup.storage = selected + if opts.FixedSize != nil { + application = screenshots.App(application, opts.FixedSize.Width, opts.FixedSize.Height) + } view, window := buildViewer(application, startup) + if opts.FixedSize != nil { + view.launchOverride.geometry = &startup.prefs + } restoreStartupGeometry(view, window, startup) return view, window, nil } diff --git a/internal/winpos/darwin.go b/internal/winpos/darwin.go index b2c3071e..3c17bffc 100644 --- a/internal/winpos/darwin.go +++ b/internal/winpos/darwin.go @@ -48,13 +48,38 @@ static void unzoomIfZoomed(uintptr_t nsWindowPtr) { [window zoom:nil]; } } + +static void screenshotContentSize(uintptr_t nsWindowPtr, int width, int height, + double *contentWidth, double *contentHeight) { + NSWindow *window = (__bridge NSWindow *)(void *)nsWindowPtr; + window.styleMask &= ~NSWindowStyleMaskResizable; + window.collectionBehavior = (window.collectionBehavior & ~NSWindowCollectionBehaviorFullScreenPrimary) + | NSWindowCollectionBehaviorFullScreenNone; + [window standardWindowButton:NSWindowZoomButton].enabled = NO; + NSRect target = window.frame; + target.size = NSMakeSize(width / window.backingScaleFactor, height / window.backingScaleFactor); + NSRect content = [window contentRectForFrameRect:target]; + *contentWidth = content.size.width; + *contentHeight = content.size.height; +} */ import "C" import ( + "fyne.io/fyne/v2" "fyne.io/fyne/v2/driver" ) +func platformScreenshotContentSize(ctx any, width, height int) (fyne.Size, bool) { + mac, ok := ctx.(driver.MacWindowContext) + if !ok || mac.NSWindow == 0 { + return fyne.Size{}, false + } + var contentWidth, contentHeight C.double + C.screenshotContentSize(C.uintptr_t(mac.NSWindow), C.int(width), C.int(height), &contentWidth, &contentHeight) + return fyne.NewSize(float32(contentWidth), float32(contentHeight)), true +} + func platformPosition(ctx any) (x, y int, ok bool) { mac, isMac := ctx.(driver.MacWindowContext) if !isMac || mac.NSWindow == 0 { diff --git a/internal/winpos/linux.go b/internal/winpos/linux.go index 0f295535..9278ba0d 100644 --- a/internal/winpos/linux.go +++ b/internal/winpos/linux.go @@ -5,6 +5,7 @@ package winpos /* #cgo LDFLAGS: -lX11 #include +#include #include // translateToRoot resolves win's own (0,0) corner to root-window (i.e. @@ -74,10 +75,29 @@ static void unmaximizeWindow(Display *display, Window win) { SubstructureRedirectMask | SubstructureNotifyMask, &xev); XFlush(display); } +// EWMH frame extents describe decorations in X11 physical pixels. +static int screenshotExtents(Display *display, Window win, int *horizontal, int *vertical) { + Atom property = XInternAtom(display, "_NET_FRAME_EXTENTS", False); + Atom actualType; + int format; + unsigned long count, remaining; + unsigned char *data = NULL; + int result = XGetWindowProperty(display, win, property, 0, 4, False, XA_CARDINAL, + &actualType, &format, &count, &remaining, &data); + int valid = result == Success && actualType == XA_CARDINAL && format == 32 && count == 4; + if (valid) { + unsigned long *extents = (unsigned long *)data; + *horizontal = (int)(extents[0] + extents[1]); + *vertical = (int)(extents[2] + extents[3]); + } + if (data != NULL) XFree(data); + return valid; +} */ import "C" import ( + "fyne.io/fyne/v2" "fyne.io/fyne/v2/driver" ) @@ -144,3 +164,20 @@ func platformUnmaximize(ctx any) { C.unmaximizeWindow(display, C.Window(x11.WindowHandle)) } + +func platformScreenshotContentSize(ctx any, width, height int) (fyne.Size, bool) { + x11, ok := ctx.(driver.X11WindowContext) + if !ok || x11.WindowHandle == 0 { + return fyne.Size{}, false + } + display := C.XOpenDisplay(nil) + if display == nil { + return fyne.Size{}, false + } + defer C.XCloseDisplay(display) + var horizontal, vertical C.int + if C.screenshotExtents(display, C.Window(x11.WindowHandle), &horizontal, &vertical) == 0 { + return fyne.Size{}, false + } + return fyne.NewSize(float32(width)-float32(horizontal), float32(height)-float32(vertical)), true +} diff --git a/internal/winpos/other.go b/internal/winpos/other.go index b30720ae..d7d36e93 100644 --- a/internal/winpos/other.go +++ b/internal/winpos/other.go @@ -2,6 +2,12 @@ package winpos +import "fyne.io/fyne/v2" + +func platformScreenshotContentSize(_ any, _, _ int) (fyne.Size, bool) { + return fyne.Size{}, false +} + // platformPosition has no implementation on BSD, mobile, wasm, and any // other backend without a native handle Get can read a position out of; // Get's caller already treats ok=false as "nothing to save". diff --git a/internal/winpos/windows.go b/internal/winpos/windows.go index 91bd8fe2..e9b30cf4 100644 --- a/internal/winpos/windows.go +++ b/internal/winpos/windows.go @@ -3,9 +3,11 @@ package winpos import ( + "strconv" "syscall" "unsafe" + "fyne.io/fyne/v2" "fyne.io/fyne/v2/driver" ) @@ -73,3 +75,43 @@ func platformUnmaximize(ctx any) { } _, _, _ = procShowWindow.Call(win.HWND, uintptr(swRestore)) } + +// Windows screenshots exclude the invisible resize border: DWM reports the +// visible frame, while GetClientRect reports the Fyne content surface. +func platformScreenshotContentSize(ctx any, width, height int) (fyne.Size, bool) { + win, ok := ctx.(driver.WindowsWindowContext) + if !ok || win.HWND == 0 { + return fyne.Size{}, false + } + // Fixed GLFW limits alone do not remove the native maximize command. + // Keep close/minimize, but remove resizing and maximize from the title bar. + getLong, setLong := "GetWindowLongPtrW", "SetWindowLongPtrW" + if strconv.IntSize == 32 { + getLong, setLong = "GetWindowLongW", "SetWindowLongW" + } + styleIndex := ^uintptr(15) // GWL_STYLE (-16) + style, _, _ := user32.NewProc(getLong).Call(win.HWND, styleIndex) + if style == 0 { + return fyne.Size{}, false + } + style &^= 0x00040000 | 0x00010000 // WS_THICKFRAME | WS_MAXIMIZEBOX + result, _, _ := user32.NewProc(setLong).Call(win.HWND, styleIndex, style) + if result == 0 { + return fyne.Size{}, false + } + _, _, _ = user32.NewProc("SetWindowPos").Call(win.HWND, 0, 0, 0, 0, 0, 0x37) // frame changed; preserve position/size/z-order/activation + var frame, client struct{ left, top, right, bottom int32 } + getClientRect := user32.NewProc("GetClientRect") + getFrame := syscall.NewLazyDLL("dwmapi.dll").NewProc("DwmGetWindowAttribute") + result, _, _ = getFrame.Call(win.HWND, 9, uintptr(unsafe.Pointer(&frame)), unsafe.Sizeof(frame)) + if result != 0 { + return fyne.Size{}, false + } + result, _, _ = getClientRect.Call(win.HWND, uintptr(unsafe.Pointer(&client))) + if result == 0 { + return fyne.Size{}, false + } + chromeWidth := (frame.right - frame.left) - (client.right - client.left) + chromeHeight := (frame.bottom - frame.top) - (client.bottom - client.top) + return fyne.NewSize(float32(width)-float32(chromeWidth), float32(height)-float32(chromeHeight)), true +} diff --git a/internal/winpos/winpos.go b/internal/winpos/winpos.go index 8c9b749d..c8c2392b 100644 --- a/internal/winpos/winpos.go +++ b/internal/winpos/winpos.go @@ -61,6 +61,9 @@ func Set(win fyne.Window, x, y int) { // platform-specific files. A no-op wherever there's no native handle to // reach (the fyne test driver, Wayland, mobile, wasm). func Maximize(win fyne.Window) { + if win.FixedSize() { + return + } native, isNative := win.(driver.NativeWindow) if !isNative { return @@ -83,6 +86,9 @@ func Maximize(win fyne.Window) { // minimized windows stay untouched; this does not restore from the taskbar. // A no-op wherever there's no native handle to reach, same as Maximize. func Unmaximize(win fyne.Window) { + if win.FixedSize() { + return + } native, isNative := win.(driver.NativeWindow) if !isNative { return @@ -91,3 +97,13 @@ func Unmaximize(win fyne.Window) { platformUnmaximize(ctx) }) } + +// ScreenshotContentSize locks native resizing and converts whole-window pixel +// dimensions to native content coordinates, accounting for chrome and Retina. +// The caller must Show first and divide by Canvas.Scale before Fyne Resize. +func ScreenshotContentSize(win fyne.Window, width, height int) (size fyne.Size, ok bool) { + if native, supported := win.(driver.NativeWindow); supported { + native.RunNative(func(ctx any) { size, ok = platformScreenshotContentSize(ctx, width, height) }) + } + return size, ok +} diff --git a/internal/winpos/winpos_test.go b/internal/winpos/winpos_test.go index 30be6358..cb1e42d1 100644 --- a/internal/winpos/winpos_test.go +++ b/internal/winpos/winpos_test.go @@ -3,6 +3,7 @@ package winpos import ( "testing" + "fyne.io/fyne/v2" "fyne.io/fyne/v2/test" ) @@ -27,3 +28,26 @@ func TestSet_NonDesktopWindowDoesNotPanic(t *testing.T) { Set(win, 100, 200) } + +type fixedNativeWindow struct { + fyne.Window + calls int +} + +func (w *fixedNativeWindow) RunNative(_ func(any)) { w.calls++ } + +func TestFixedWindowSkipsNativeMaximize(t *testing.T) { + w := &fixedNativeWindow{Window: test.NewWindow(nil)} + defer w.Close() + w.SetFixedSize(true) + Maximize(w) + Unmaximize(w) + if w.calls != 0 { + t.Fatal("fixed window admitted a native geometry change") + } + w.SetFixedSize(false) + Maximize(w) + if w.calls != 1 { + t.Fatal("ordinary maximize was disabled") + } +} diff --git a/main.go b/main.go index 58ecdc7b..cda1efb5 100644 --- a/main.go +++ b/main.go @@ -135,6 +135,13 @@ func productionStartup() startupOps { return startupOps{ heicWorker: heic.WorkerMain, similarityWorker: similarity.WorkerMain, + selectResolution: func(out io.Writer) (launch.Resolution, error) { + size, err := consolehelp.SelectResolution(os.Stdin, out) + if err == nil { + _, err = fmt.Fprintf(out, "Screenshot size: %d x %dpx\n", size.Width, size.Height) + } + return size, err + }, installOpenWith: func() { // Cocoa caches delegate selectors during glfw.Init. Installation // belongs before app construction. False is normal off macOS. diff --git a/main_startup.go b/main_startup.go index 85b00bb7..7f662282 100644 --- a/main_startup.go +++ b/main_startup.go @@ -15,6 +15,7 @@ import ( type startupOps struct { heicWorker func() bool similarityWorker func() bool + selectResolution func(io.Writer) (launch.Resolution, error) installOpenWith func() cleanup func() capture func(launch.Options) (launch.Policy, error) @@ -31,6 +32,13 @@ func runStartup(args []string, stdout, stderr io.Writer, ops startupOps) (code i if ops.heicWorker() || ops.similarityWorker() { return 0, nil } + if opts.FixedSizeMode && opts.FixedSize == nil { + selected, selectionErr := ops.selectResolution(stdout) + if selectionErr != nil { + return 1, selectionErr + } + opts.FixedSize = &selected + } // The native delegate must be installed before Fyne/GLFW initialization. ops.installOpenWith() policy, err := ops.capture(opts) diff --git a/main_startup_test.go b/main_startup_test.go index aaa655d2..f56d3207 100644 --- a/main_startup_test.go +++ b/main_startup_test.go @@ -4,6 +4,7 @@ import ( "bytes" "context" "errors" + "fmt" "io" "os" "path/filepath" @@ -290,3 +291,38 @@ func startupObservations(calls *[]string) startupOps { }, } } + +func TestFixedSizeStartup(t *testing.T) { + for _, cancelled := range []bool{false, true} { + t.Run(fmt.Sprint(cancelled), func(t *testing.T) { + var calls []string + ops := startupObservations(&calls) + failure := errors.New("selection cancelled") + picked := launch.Resolution{Width: 1440, Height: 900} + ops.selectResolution = func(_ io.Writer) (launch.Resolution, error) { + calls = append(calls, "picker") + if cancelled { + return launch.Resolution{}, failure + } + return picked, nil + } + ops.run = func(_ fyne.App, _ []fyne.URI, opts launch.Options, _ *launch.Prepared) error { + calls = append(calls, "run") + if opts.FixedSize == nil || *opts.FixedSize != picked { + t.Fatalf("missing selected size: %+v", opts) + } + return nil + } + code, err := runStartup([]string{"-fixed-size-mode"}, io.Discard, io.Discard, ops) + if cancelled { + if code != 1 || !errors.Is(err, failure) || !slices.Equal(calls, []string{"heic", "similarity", "picker"}) { + t.Fatalf("cancel: %d %v %v", code, err, calls) + } + return + } + if code != 0 || err != nil || slices.Index(calls, "picker") != 2 || slices.Index(calls, "picker") > slices.Index(calls, "app") { + t.Fatalf("startup: %d %v %v", code, err, calls) + } + }) + } +} diff --git a/plans/2026-10-01-fixed-size-mode.md b/plans/2026-10-01-fixed-size-mode.md new file mode 100644 index 00000000..73f8123b --- /dev/null +++ b/plans/2026-10-01-fixed-size-mode.md @@ -0,0 +1,100 @@ +# Fixed screenshot dimensions + +Route: Deep (terminal input, launch, UI composition and native desktop geometry). +Owner: Pico, inline; zero delegates because the contracts share hot context. + +Ronin requests `-fixed-size-mode`, an Up/Down and Return console picker for +1280 x 800, 1440 x 900, 2560 x 1600 and 2880 x 1800. The selected dimensions +measure physical pixels of the entire window, including the title bar, excluding +the screenshot shadow. Every PicFetch window uses that size for this invocation. +Panels, image loads, maximize, picture-frame and Spiral cannot change it. +Saved main and secondary geometry must survive the session. Native OS panels +remain managed by the OS. Resolutions larger than a screen require a suitable +display for unobstructed screen captures. + +No dependencies are added: Fyne v2.8.0, x/term and x/sys are already shipped; +existing notice delivery remains unchanged. Mac frame/content conversion follows +[AppKit's window geometry contract](https://developer.apple.com/documentation/appkit/nswindow/contentrect%28forframerect%3A%29?language=objc). + +## Acceptance and tasks + +Task graph: 1 -> 2 -> 3 -> 4. All tasks owned by T0 inline. + +1. CLI and console: `launch.Resolution`, four value presets, flag parsing, + optional `=WIDTHxHEIGHT` for scripts, raw terminal selection with restoration + and cancellation before desktop effects. Files: internal/launch, consolehelp, + main startup and existing tests. Verify focused parser, picker and startup + tests with `go test -tags no_emoji,nodynamic ./internal/launch ./internal/consolehelp .`. +2. Window constraint: application decorator in internal/screenshots routes all + NewWindow calls to a constrained window, prevents resize/fullscreen, contains + content minimums, preserves native/desktop operations and measures native + decorations/backing scale. Native metrics stay in winpos build-tag files; + native maximize honors fixed windows. Verify screenshots and winpos tests; + native macOS capture measurements for all four presets. +3. UI: construct the decorator before feature windows, retain original persisted + geometry in launchOverride. Verify main/secondary geometry preservation and + mode transitions using existing root harness; maintain shard assignments, + exact Qodana exclusions and architecture map. Focused root UI tests plus + `make check-test-shards`. +4. Land: `make verify-build`, affected race regressions, GoLand inspections of + all changed code (weak warnings included), commit/push, fresh GitHub code and + security review, CI, CodeQL and post-suppression Qodana. Full race suite runs + on native Linux/amd64 CI. Then make a fresh local universal sandboxed App + Store package and deliver its path and console command. Never merge/upload. + +Budgets: zero spawns; one local lead assessment per task, further rounds only +for findings; focused suites locally, full suite once on hosted CI per revision. +Evidence and actual cost ledger will be appended as work completes. Active work +stays here until Ronin accepts it. + +## Local evidence, 2026-10-01 + +- Parser, picker, startup and constrained-window regressions were observed red + before implementation (unknown flag, zero selection, missing selected startup + size and unconstrained window respectively), then green. +- Root UI regression was observed red at saved 700 x 500 instead of the requested + 1280 x 800. It now covers image loading, Grid, picture-frame, secondary windows, + preservation of all saved geometry and persistence of unrelated settings. +- Deliberately removing original geometry restoration and fixed-window native + maximize guards made their respective tests fail. Both mutations were restored. +- Focused race command: `go test -race -tags no_emoji,nodynamic + ./internal/launch ./internal/consolehelp ./internal/screenshots ./internal/winpos + . ./internal/ui -run 'TestFixed|TestResolutionPicker|TestWindowConstraint|TestLaunchOptions' + -count=1` passes. The first race run exposed an unjoined sort in the new test; + the test now waits on the existing sort completion signal before reading state. +- Real executable tested with `.scratch/fixed-size-mode/terminalcheck.py` and a + pseudo-terminal: Down/Return selects 1440 x 900; Ctrl+C and SIGTERM abort with + status 1; raw terminal flags, cursor and alternate screen restore. Darwin's + transient PENDIN bit is excluded from comparison; all other fields match. +- Native AppKit frame measurements after resize/fullscreen/maximize and large + content attempts: exactly 1280 x 800, 1440 x 900, 2560 x 1600 and 2880 x 1800 + physical pixels on the 2x Retina display. A fractional Fyne scale (requested + 1.25, reported 1.3) also preserves exactly 1280 x 800. Measurement probe and + JSON records are retained locally under .scratch and /private/tmp. Direct + `screencapture` could not create a window image, so these are native frame + measurements, not PNG qualification claims. +- GoLand per-file inspections (Project profile, errorsOnly=false) covered every + changed Go file, including all platform variants and tests. Weak import-order + and unnamed-field warnings were fixed and affected files re-inspected. No + actionable findings or timeouts remain. This is the documented IDE fallback, + not a local Qodana scan; fresh hosted post-suppression SARIF remains required. +- Windows amd64 cross-compilation and vet pass for screenshots, winpos and + consolehelp. Native Windows/X11 rendering is unverified; no such pass is claimed. +- `make verify-build` passes. Docker is unavailable locally, so canonical shard + validation and the full native Linux/amd64 race suite are delegated to CI. + A direct macOS shard attempt sees the pre-existing Darwin-only menu test and + therefore cannot qualify the canonical Linux inventory. The added test is + assigned to ui-1, count updated, and the new test path excluded exactly in Qodana. + +| Task | Spawns budget/actual | Lead review rounds | Full suite | +|------|----------------------|--------------------|------------| +| CLI / picker | 0 / 0 | 1 | no | +| Window policy / native metrics | 0 / 0 | 1 | no | +| UI / persistence | 0 / 0 | 2 (fixture completion fix) | no | +| Land | 0 / 0 | pending GitHub loop | hosted CI pending | + +The existing PR now also contains fresh security-priority follow-up threads on +metadata removal/export identity. Those confirmed older findings must be addressed +in the requested review loop before the new App Store package is built. Recursive +scan limits are explicitly classified by the bot as deferrable availability +hardening and remain an open backlog item. diff --git a/qodana.yaml b/qodana.yaml index df08926e..d7bf613d 100644 --- a/qodana.yaml +++ b/qodana.yaml @@ -107,6 +107,7 @@ exclude: - "internal/clipboard/copyfiles_windows_test.go" - "internal/completion/completion_test.go" - "internal/consolehelp/help_test.go" + - "internal/screenshots/app_test.go" - "internal/decodepool/decodepool_test.go" - "internal/displays/displays_test.go" - "internal/displays/xrandr_test.go" diff --git a/todos.md b/todos.md index 35e760a5..2da48ee5 100644 --- a/todos.md +++ b/todos.md @@ -6,6 +6,11 @@ #### New Features +- [ ] Fixed screenshot window mode: four console-selected physical pixel sizes, + constrained mode transitions and preserved saved geometry. Local native Retina + measurements pass; review/CI and fresh App Store test package remain pending. + See [plan and evidence](plans/2026-10-01-fixed-size-mode.md). + #### Bugfix - [x] Bound Save Changes and JPEG Export secondary source reads by the encoded From d402f3866f9cc58646fa7ac1751a58f4e432b274 Mon Sep 17 00:00:00 2001 From: frathe Date: Thu, 1 Oct 2026 09:48:39 +0200 Subject: [PATCH 28/63] Restore responsive Store folder consent and current bookmark parents --- ARCHITECTURE.md | 4 +- internal/filepicker/darwin.go | 109 ++++++++++++------- internal/filepicker/folders.go | 11 ++ internal/filepicker/folders_test.go | 39 ++++++- plans/2026-10-01-folder-access-regression.md | 69 ++++++++++++ todos.md | 5 + 6 files changed, 196 insertions(+), 41 deletions(-) create mode 100644 plans/2026-10-01-folder-access-regression.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 14072f78..61dec3b3 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1169,8 +1169,8 @@ can pick folders; Windows is files-only. | File | Responsibility | |------|----------------| | `filepicker.go` | Typed `Choose` (URI list) / `ChooseSave` (one URI), strict native result decoding, explicit folder consent policy for shared collection discovery, canonical-path Zenity framing and UTF-8 JSON PowerShell transport. | -| `folders.go` | Per-viewer `FolderAuthorizer` stores explicit sibling-folder bookmarks in app preferences; resolves and validates saved grants on tracked opening workers, refreshes moved/stale authority, and falls back to native consent. Native scopes are bounded, and preference locks never cover native calls. | -| `darwin.go` / `other.go` | `NSOpenPanel` / `NSSavePanel`, including cancellable Store sibling-folder permission, with a shared NSURL-to-JSON transport / stubs; `darwin_test.go` exercises the actual native serializer. | +| `folders.go` | Per-viewer `FolderAuthorizer` stores explicit sibling-folder bookmarks in app preferences; resolves the selected image before suggesting its current parent, validates saved grants on tracked opening workers, refreshes moved/stale authority, and falls back to native consent. Native scopes are bounded, and preference locks never cover native calls. | +| `darwin.go` / `other.go` | Completion-based `NSOpenPanel` / `NSSavePanel`, including Store sibling-folder permission. Native setup returns immediately to Fyne; the existing chooser worker waits for completion and owns selected URLs through NSURL-to-JSON transport / stubs. `darwin_test.go` exercises the actual native serializer. | | `windows.go` / `notwindows.go` | `hideConsoleWindow` pair. | ### `internal/trash` diff --git a/internal/filepicker/darwin.go b/internal/filepicker/darwin.go index 090fbba6..f51823cb 100644 --- a/internal/filepicker/darwin.go +++ b/internal/filepicker/darwin.go @@ -28,7 +28,7 @@ static char *serializePanelURLs(NSArray *urls, int scopes, char **failu *failure = strdup(error ? error.description.UTF8String : "selected URL bookmark capture failed"); return NULL; } - [paths addObject:@{@"uri":url.absoluteString, + [paths addObject:@{@"uri":url.filePathURL.absoluteString, @"bookmark":[bookmark base64EncodedStringWithOptions:0], @"directory":directory}]; } else { [paths addObject:url.path]; @@ -51,12 +51,12 @@ static char *roundTripPanelPaths(const char *input) { } } -// runOpenPanel shows an app-modal NSOpenPanel that allows files, folders, +// beginOpenPanel shows an asynchronous NSOpenPanel that allows files, folders, // and multi-select all at once - a combination none of AppleScript's // Standard Additions pickers offer. Must be called on the main thread (an // AppKit requirement); chooseFilesDarwin below guarantees that. Returns // retained URLs or an explicit cancellation/failure state. Serialization runs -// on the chooser worker after the modal panel closes. +// on the chooser worker after the panel completes. typedef struct { int state; // 0 cancellation, 1 selected URLs, -1 native failure. void *urls; @@ -79,17 +79,57 @@ static char *serializeSelection(panelResult result, int scopes, char **failure) } } -static panelResult runOpenPanel(const char *message) { +// The Fyne dispatcher must return while the remote panel processes events. +// A tracked chooser worker waits for this request; UI never waits on its semaphore. +@interface PicFetchPanelRequest : NSObject +@property(nonatomic, strong) NSSavePanel *panel; +@property(nonatomic, strong) id panelDelegate; +@property(nonatomic, strong) NSArray *urls; +@property(nonatomic, strong) dispatch_semaphore_t done; +@property(nonatomic) int state; +@end +@implementation PicFetchPanelRequest +@end + +static void *beginPanel(NSSavePanel *panel, id delegate) { + PicFetchPanelRequest *request = [PicFetchPanelRequest new]; + request.panel = panel; + request.panelDelegate = delegate; + request.done = dispatch_semaphore_create(0); + void *owner = (__bridge_retained void *)request; + [panel beginWithCompletionHandler:^(NSModalResponse response) { + request.state = response == NSModalResponseCancel ? 0 : -1; + if (response == NSModalResponseOK) { + NSArray *urls = [panel isKindOfClass:NSOpenPanel.class] ? ((NSOpenPanel *)panel).URLs : (panel.URL ? @[panel.URL] : @[]); + request.urls = [urls copy]; + request.state = urls.count ? 1 : -1; + } + panel.delegate = nil; + request.panel = nil; + request.panelDelegate = nil; + dispatch_semaphore_signal(request.done); + }]; + return owner; +} + +static panelResult waitForPanel(void *owner) { + if (!owner) return (panelResult){0, NULL}; + @autoreleasepool { + PicFetchPanelRequest *request = (__bridge_transfer PicFetchPanelRequest *)owner; + dispatch_semaphore_wait(request.done, DISPATCH_TIME_FOREVER); + if (request.state != 1) return (panelResult){request.state, NULL}; + return retainSelection(request.urls); + } +} + +static void *beginOpenPanel(const char *message) { NSOpenPanel *panel = [NSOpenPanel openPanel]; panel.message = [NSString stringWithUTF8String:message]; panel.canChooseFiles = YES; panel.canChooseDirectories = YES; panel.allowsMultipleSelection = YES; - NSModalResponse response = [panel runModal]; - if (response == NSModalResponseCancel) return (panelResult){0, NULL}; - if (response != NSModalResponseOK) return (panelResult){-1, NULL}; - return retainSelection(panel.URLs); + return beginPanel(panel, nil); } // Only the matching folder can be confirmed. Navigating elsewhere never grants @@ -104,10 +144,10 @@ static panelResult runOpenPanel(const char *message) { } @end -static panelResult runSiblingFolderPanel(const char *message, const char *prompt, const char *directory) { +static void *beginSiblingFolderPanel(const char *message, const char *prompt, const char *directory) { NSOpenPanel *panel = [NSOpenPanel openPanel]; NSURL *folder = [NSURL fileURLWithPath:[NSString stringWithUTF8String:directory] isDirectory:YES]; - __attribute__((objc_precise_lifetime)) PicFetchSiblingFolderDelegate *delegate = [PicFetchSiblingFolderDelegate new]; + PicFetchSiblingFolderDelegate *delegate = [PicFetchSiblingFolderDelegate new]; delegate.folder = folder; panel.delegate = delegate; panel.message = [NSString stringWithUTF8String:message]; @@ -117,20 +157,16 @@ static panelResult runSiblingFolderPanel(const char *message, const char *prompt panel.allowsMultipleSelection = NO; panel.canCreateDirectories = NO; panel.directoryURL = folder; - NSModalResponse response = [panel runModal]; - panel.delegate = nil; - if (response == NSModalResponseCancel) return (panelResult){0, NULL}; - if (response != NSModalResponseOK) return (panelResult){-1, NULL}; - return retainSelection(panel.URLs); + return beginPanel(panel, delegate); } -// runSavePanel shows an app-modal NSSavePanel pre-filled with name, opened -// on dir. Same main-thread requirement as runOpenPanel above; +// beginSavePanel shows an asynchronous NSSavePanel pre-filled with name, opened +// on dir. Same main-thread requirement as beginOpenPanel above; // chooseSaveDarwin guarantees it the same way. No allowedContentTypes is // set: the format is already decided by which "Export as…" item the user // picked, and constraining the panel would only stop them naming the file -// whatever they want. Returns the same retained-URL contract as runOpenPanel. -static panelResult runSavePanel(const char *message, const char *dir, const char *name) { +// whatever they want. Returns the same retained-URL contract as beginOpenPanel. +static void *beginSavePanel(const char *message, const char *dir, const char *name) { NSSavePanel *panel = [NSSavePanel savePanel]; panel.message = [NSString stringWithUTF8String:message]; panel.nameFieldStringValue = [NSString stringWithUTF8String:name]; @@ -139,10 +175,7 @@ static panelResult runSavePanel(const char *message, const char *dir, const char panel.directoryURL = [NSURL fileURLWithPath:[NSString stringWithUTF8String:dir] isDirectory:YES]; } - NSModalResponse response = [panel runModal]; - if (response == NSModalResponseCancel) return (panelResult){0, NULL}; - if (response != NSModalResponseOK) return (panelResult){-1, NULL}; - return retainSelection(panel.URL ? @[panel.URL] : @[]); + return beginPanel(panel, nil); } // The destination may not exist; retaining the original URL preserves its // implicit save permission without inspecting or bookmarking a nonexistent file. @@ -198,20 +231,19 @@ import ( // auto-dismissed on the first click, when that click tried and failed to // activate osascript. A panel owned by this app - a regular, frontmost GUI // app - has neither problem, and is how every native app shows this -// dialog. AppKit requires the panel on the main thread, which on darwin is -// exactly Fyne's UI thread, hence the fyne.DoAndWait hop; the UI behind -// the panel freezes for the duration, which is what app-modal means. That -// also means this must never be called *from* the UI goroutine (DoAndWait -// would deadlock) - production only reaches it via the viewer's -// openFileDialog background goroutine. Cancellation emits JSON null; successful -// paths use structured transport, while other modal responses are errors. +// dialog. AppKit panel setup and completion run on the native main thread. +// Only setup uses fyne.DoAndWait: keeping a runModal call inside Fyne's +// dispatcher stalls remote-panel updates on current macOS. The tracked chooser +// worker waits for native completion, then serializes the retained native URLs. +// Never call this from UI: setup and worker completion must remain independent. +// Cancellation emits JSON null; successful paths use structured transport. func chooseFilesDarwin() ([]byte, error) { cMsg := C.CString(lang.L("Open images")) defer C.free(unsafe.Pointer(cMsg)) - var result C.panelResult - fyne.DoAndWait(func() { result = C.runOpenPanel(cMsg) }) - return decodeNativePanel(result, distribution.AppleAppStore) + var request unsafe.Pointer + fyne.DoAndWait(func() { request = C.beginOpenPanel(cMsg) }) + return decodeNativePanel(C.waitForPanel(request), distribution.AppleAppStore) } // chooseSaveDarwin is chooseFilesDarwin's save-panel twin, in-process and @@ -228,8 +260,9 @@ func chooseSaveDarwin(suggestedPath string) (fyne.URI, error) { cName := C.CString(filepath.Base(suggestedPath)) defer C.free(unsafe.Pointer(cName)) - var result C.panelResult - fyne.DoAndWait(func() { result = C.runSavePanel(cMsg, cDir, cName) }) + var request unsafe.Pointer + fyne.DoAndWait(func() { request = C.beginSavePanel(cMsg, cDir, cName) }) + result := C.waitForPanel(request) if distribution.AppleAppStore { return decodeNativeSavePanel(result) } @@ -310,13 +343,13 @@ func chooseSiblingFolderDarwin(ctx context.Context, directory string) (fyne.URI, defer C.free(unsafe.Pointer(prompt)) path := C.CString(directory) defer C.free(unsafe.Pointer(path)) - var result C.panelResult + var request unsafe.Pointer fyne.DoAndWait(func() { if ctx.Err() == nil { - result = C.runSiblingFolderPanel(message, prompt, path) + request = C.beginSiblingFolderPanel(message, prompt, path) } }) - out, err := decodeNativePanel(result, true) + out, err := decodeNativePanel(C.waitForPanel(request), true) if err != nil { return nil, err } diff --git a/internal/filepicker/folders.go b/internal/filepicker/folders.go index 17979419..e148fc12 100644 --- a/internal/filepicker/folders.go +++ b/internal/filepicker/folders.go @@ -45,6 +45,17 @@ func (a *FolderAuthorizer) authorize(ctx context.Context, files []fyne.URI) ([]f if err := ctx.Err(); err != nil { return nil, err } + // A persisted file bookmark can now name a different path. Resolve its + // current identity before suggesting or comparing a parent folder; the + // recorded path is an index, never the authority for a new grant. + if len(files) == 1 && fileaccess.HasScope(files[0]) && !fileaccess.Snapshot(files[0]).Directory { + resolved, release, err := a.acquire(ctx, files[0]) + if err != nil { + return nil, err + } + defer release() + files = []fyne.URI{resolved} + } var approved fyne.URI permitted, err := authorizeSiblingFolder(files, func(directory string) (fyne.URI, error) { if folder, err := a.recall(ctx, directory); folder != nil || err != nil { diff --git a/internal/filepicker/folders_test.go b/internal/filepicker/folders_test.go index ae303ca2..6dc6be38 100644 --- a/internal/filepicker/folders_test.go +++ b/internal/filepicker/folders_test.go @@ -95,7 +95,10 @@ func TestSavedFolderApprovalValidation(t *testing.T) { } return nil, nil } - authorizer.acquire = func(_ context.Context, _ fyne.URI) (fyne.URI, func(), error) { + authorizer.acquire = func(_ context.Context, uri fyne.URI) (fyne.URI, func(), error) { + if !fileaccess.Snapshot(uri).Directory { + return uri, func() {}, nil + } if scenario == "revoked" || scenario == "cancel-panel" { return nil, nil, errors.New("revoked") } @@ -163,3 +166,37 @@ func TestSavedFolderApprovalValidation(t *testing.T) { }) } } + +func TestSiblingApprovalUsesResolvedImageParent(t *testing.T) { + app := test.NewApp() + t.Cleanup(app.Quit) + old, current := t.TempDir(), t.TempDir() + source := folderFixture(t, filepath.Join(old, "photo.jpg"), false) + resolved := folderFixture(t, filepath.Join(current, "photo.jpg"), false) + folder := folderFixture(t, current, true) + authorizer := NewFolderAuthorizer(app.Preferences()) + active := 0 + authorizer.acquire = func(_ context.Context, uri fyne.URI) (fyne.URI, func(), error) { + if uri != source { + t.Fatal("unexpected source acquisition") + } + active++ + return resolved, func() { active-- }, nil + } + authorizer.choose = func(_ context.Context, directory string) (fyne.URI, error) { + if directory != current { + t.Fatalf("permission panel starts at stale folder %q; want %q", directory, current) + } + if active != 1 { + t.Fatal("source resolution authority ended before folder selection") + } + return folder, nil + } + got, err := authorizer.authorize(context.Background(), []fyne.URI{source}) + if err != nil || len(got) != 1 { + t.Fatalf("authorization %v, %v", got, err) + } + if got[0].Path() != resolved.Path() || fileaccess.Snapshot(got[0]).Relative != "photo.jpg" || active != 0 { + t.Fatalf("lost resolved selection or scope release: %v active=%d", got, active) + } +} diff --git a/plans/2026-10-01-folder-access-regression.md b/plans/2026-10-01-folder-access-regression.md new file mode 100644 index 00000000..83980181 --- /dev/null +++ b/plans/2026-10-01-folder-access-regression.md @@ -0,0 +1,69 @@ +# Store folder-access regression + +Route: Deep, platform-native panel lifetime plus captured bookmark resolution. +Owner: Pico inline; no delegation, no new dependency or notice obligations. + +Ronin reports wrong initial folder and an interactive confirmation that cannot +complete after selecting the correct parent. Preserve exact-parent authority, +file-only cancellation, sibling discovery and remembered explicit grants. + +Task graph: native reproduction -> presentation fix -> moved-source regression +-> focused race/native qualification -> inspection -> commit/push/review -> build. +All tasks are T0 inline, with zero spawns and focused local suites; full CI remains +the GitHub native Linux/amd64/platform gate. + +## Acceptance + +1. In the sandboxed PicFetch test app, select one controlled JPEG, choose its + exact parent and confirm. The panel must close and title report both fixture + images. Cancel a different folder request: only the selected image opens. + Verify through the Computer Use native fixture loop. +2. Native panels start on UI and complete without blocking Fyne's dispatcher; + the tracked chooser worker owns the retained native URLs until serialization. + Verify normal Open, folder permission, cancel and Save in the same app. +3. A moved file bookmark suggests its resolved current parent and attaches that + folder's scope, with balanced acquisition/release. Command: + `go test -race -tags no_emoji,nodynamic ./internal/filepicker`. +4. Wrong/broader folders, revoked grants, cancellation, directory/multiple + selections and persisted grant reuse retain existing boundaries. Same suite. +5. GoLand every changed code file including weak warnings, make verify-build, + fresh GitHub code/security review, CI, CodeQL and post-suppression Qodana. + +Files: internal/filepicker/darwin.go, folders.go and existing folders_test.go; +native completion coverage in existing darwin_test.go if needed after assessment. +No native OS operation runs in ordinary unit tests. + +## Diagnosis evidence + +- Existing packaged sandbox app at 382a40d: exact fixture Photos folder selected, + Return leaves permission panel open. AX reports disabled confirmation. Record + `.scratch/folder-access-regression/repro.md` contains the native red assertion. +- Ranked URL comparison, stale bookmark, restored panel directory and completion + hypotheses were tested. Outside-sandbox NSURL comparison passes every spelling. +- Native thread sample shows runModal on the correct macOS main thread; another + fresh Open dialog can also stop updating its confirmation button. +- Asynchronous native presentation with a temporary default-run-loop probe makes + Open and Allow Folder Access update and complete. Actual sandbox app loads the + selected second.jpg with `(2/2)`. Native trace shows file-reference selection + and expected path standardize equally. Temporary logging/probe loops are + removed from the final completion-based implementation. +- Moved-bookmark test observed red: chooser received old parent rather than the + resolver's current parent. Authorizer now resolves the selected file before + constructing or comparing any proposed folder grant. +- The separate diagnostic app's folder grant was blocked by automatic approval + review as outside scope. It was cancelled; qualification instead used PicFetch + with only generated fixture images and the authorized production flow. + +- Final completion-based sandbox native rerun: exact parent with spaces accepted; + window title is first.jpg `(1/2)`. Cancelling a separate fresh fixture closes + the dialog, yields a single-image title and Right cannot reach second.jpg. +- Focused filepicker race suite and actual appleappstore-tagged suite pass. + GoLand Project inspections of all three changed Go files, weak warnings + included, report no findings. `make verify-build` passes. + +Native Save also accepted a new destination and wrote a valid 981-byte PNG. +Computer Use capture refreshes then timed out. A native thread sample shows +normal GLFW event polling on UI and the following Open request waiting only on +its tracked worker, so no post-Save native-modal deadlock is present. Remembered +native reuse remains unverified until capture service recovers. +Latest-head hosted gates and the fresh deliverable package remain pending. diff --git a/todos.md b/todos.md index 2da48ee5..7a67ce59 100644 --- a/todos.md +++ b/todos.md @@ -13,6 +13,11 @@ #### Bugfix +- [ ] Restore Store folder consent: native panels complete without blocking the + Fyne dispatcher; moved file bookmarks suggest the current parent. Sandbox + confirmation/cancel and local regressions pass; hosted review is pending. + See [regression evidence](plans/2026-10-01-folder-access-regression.md). + - [x] Bound Save Changes and JPEG Export secondary source reads by the encoded input limit, including growth after stat. Save leaves oversized sources intact; Export preserves pixel-only fallback. Local race, Store-tagged and static From d82483f50b2ae8dc61af6fef24b7a137121d8939 Mon Sep 17 00:00:00 2001 From: frathe Date: Thu, 1 Oct 2026 10:02:25 +0200 Subject: [PATCH 29/63] Normalize Windows paths in moved folder consent regression --- internal/filepicker/folders_test.go | 2 +- plans/2026-10-01-folder-access-regression.md | 5 +++++ 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/internal/filepicker/folders_test.go b/internal/filepicker/folders_test.go index 6dc6be38..856cb84e 100644 --- a/internal/filepicker/folders_test.go +++ b/internal/filepicker/folders_test.go @@ -184,7 +184,7 @@ func TestSiblingApprovalUsesResolvedImageParent(t *testing.T) { return resolved, func() { active-- }, nil } authorizer.choose = func(_ context.Context, directory string) (fyne.URI, error) { - if directory != current { + if filepath.Clean(directory) != filepath.Clean(current) { t.Fatalf("permission panel starts at stale folder %q; want %q", directory, current) } if active != 1 { diff --git a/plans/2026-10-01-folder-access-regression.md b/plans/2026-10-01-folder-access-regression.md index 83980181..b1ee18f8 100644 --- a/plans/2026-10-01-folder-access-regression.md +++ b/plans/2026-10-01-folder-access-regression.md @@ -67,3 +67,8 @@ normal GLFW event polling on UI and the following Open request waiting only on its tracked worker, so no post-Save native-modal deadlock is present. Remembered native reuse remains unverified until capture service recovers. Latest-head hosted gates and the fresh deliverable package remain pending. + +Windows CI at d402f38 exposed a test assertion comparing URI forward slashes to +native TempDir backslashes. The raw native-guards artifact confirms both paths +name the correct resolved `002` folder. The assertion now compares filepath.Clean +values; no production behavior changed. Fresh Windows CI must verify the correction. From 56b6166462d7c187b573dc02456b85cdb7f24cd2 Mon Sep 17 00:00:00 2001 From: frathe Date: Thu, 1 Oct 2026 10:17:00 +0200 Subject: [PATCH 30/63] Preserve screenshot pixels across scale changes and terminal interruption --- README.md | 6 ++- internal/consolehelp/resolution.go | 8 +-- internal/screenshots/app.go | 81 ++++++++++++++++++++++------- internal/screenshots/app_test.go | 72 +++++++++++++++++++++++++ plans/2026-10-01-fixed-size-mode.md | 32 ++++++++++++ 5 files changed, 175 insertions(+), 24 deletions(-) diff --git a/README.md b/README.md index 6f7fc728..5d4c526a 100644 --- a/README.md +++ b/README.md @@ -258,7 +258,7 @@ picfetch -fixed-size-mode # Optional preset for scripts: picfetch -fixed-size-mode=1280x800 ~/photos # Packaged macOS application: -/path/to/PicFetch.app/Contents/MacOS/picfetch -fixed-size-mode +/path/to/PicFetch.app/Contents/MacOS/PicFetch -fixed-size-mode ``` Dimensions are physical pixels of the whole window, including its title bar. @@ -268,7 +268,9 @@ Saved window geometry is preserved. Native OS file dialogs retain their OS sizin On macOS use a window capture without a shadow: Cmd+Shift+4, Space, then hold Option while clicking the window. Shadows add pixels outside the selected size. The exact sizes have been measured on Retina macOS; native Windows/X11 captures -still need qualification, and unsupported native geometry is logged. +still need qualification. If native decorations cannot be measured (including +Wayland), the content size remains scaled correctly, but the title-bar/frame +allowance is unavailable and logged; exact whole-window pixels are unverified. ## Building diff --git a/internal/consolehelp/resolution.go b/internal/consolehelp/resolution.go index f9f84bf9..fde0e476 100644 --- a/internal/consolehelp/resolution.go +++ b/internal/consolehelp/resolution.go @@ -20,6 +20,11 @@ func SelectResolution(in *os.File, out io.Writer) (size launch.Resolution, err e if !ok || !term.IsTerminal(int(in.Fd())) || !term.IsTerminal(int(output.Fd())) || os.Getenv("TERM") == "dumb" { return size, errors.New("fixed-size-mode needs an interactive terminal; use -fixed-size-mode=1280x800 for a scripted launch") } + // Cover setup and cleanup too: default signal handling must never terminate + // the process while its terminal is still raw. + notices := make(chan os.Signal, 2) + signal.Notify(notices, os.Interrupt, syscall.SIGTERM) + defer signal.Stop(notices) restoreANSI, err := enableANSI(output) if err != nil { return size, err @@ -30,9 +35,6 @@ func SelectResolution(in *os.File, out io.Writer) (size launch.Resolution, err e return size, err } defer func() { err = errors.Join(err, term.Restore(int(in.Fd()), state)) }() - notices := make(chan os.Signal, 2) - signal.Notify(notices, os.Interrupt, syscall.SIGTERM) - defer signal.Stop(notices) defer func() { _, restoreErr := io.WriteString(out, "\x1b[?25h\x1b[?1049l") err = errors.Join(err, restoreErr) diff --git a/internal/screenshots/app.go b/internal/screenshots/app.go index c026a62c..3fe64a72 100644 --- a/internal/screenshots/app.go +++ b/internal/screenshots/app.go @@ -25,31 +25,55 @@ func App(app fyne.App, width, height int) fyne.App { } func (a *application) NewWindow(title string) fyne.Window { - w := &window{Window: a.App.NewWindow(title), app: a.App, width: a.width, height: a.height} - w.Window.SetContent(container.New(fixedLayout{})) - w.Window.SetPadded(false) + w := &window{Window: a.App.NewWindow(title), app: a.App, width: a.width, height: a.height, measure: winpos.ScreenshotContentSize} + w.Window.SetContent(container.New(fixedLayout{window: w})) w.Window.Resize(fyne.NewSize(float32(a.width), float32(a.height))) w.Window.SetFixedSize(true) + w.Window.SetOnClosed(func() { + w.closed, w.shown = true, false + if w.onClosed != nil { + w.onClosed() + } + }) return w } type window struct { fyne.Window - app fyne.App - width, height int - content fyne.CanvasObject + app fyne.App + width, height int + content fyne.CanvasObject + shown, closed bool + sizing, queued bool + warned bool + applied fyne.Size + onClosed func() + measure func(fyne.Window, int, int) (fyne.Size, bool) } // fixedLayout keeps a panel's natural minimum from expanding the native window. // Its children still get normal layout inside the available screenshot surface. -type fixedLayout struct{} +type fixedLayout struct{ window *window } func (fixedLayout) MinSize(_ []fyne.CanvasObject) fyne.Size { return fyne.Size{} } -func (fixedLayout) Layout(objects []fyne.CanvasObject, size fyne.Size) { +func (layout fixedLayout) Layout(objects []fyne.CanvasObject, size fyne.Size) { for _, object := range objects { object.Move(fyne.Position{}) object.Resize(size) } + // Scale changes refresh content. Check native metrics too, because macOS + // backing scale is independent of Canvas.Scale. Queue correction after layout. + if w := layout.window; w != nil && w.shown && !w.closed && !w.sizing && !w.queued { + if target := w.targetSize(); target != w.applied { + w.queued = true + fyne.Do(func() { + w.queued = false + if w.shown && !w.closed { + w.applySize() + } + }) + } + } } func (w *window) Resize(_ fyne.Size) {} @@ -57,9 +81,12 @@ func (w *window) SetFullScreen(_ bool) {} func (w *window) SetFixedSize(_ bool) {} func (w *window) RequestFullScreenSecondary() {} func (w *window) Content() fyne.CanvasObject { return w.content } +func (w *window) SetOnClosed(fn func()) { w.onClosed = fn } +func (w *window) Hide() { w.shown = false; w.Window.Hide() } +func (w *window) Close() { w.closed = true; w.Window.Close() } func (w *window) SetContent(content fyne.CanvasObject) { w.content = content - root := container.New(fixedLayout{}) + root := container.New(fixedLayout{window: w}) if content != nil { root.Add(content) } @@ -67,12 +94,18 @@ func (w *window) SetContent(content fyne.CanvasObject) { } func (w *window) Show() { w.Window.Show() - size, ok := winpos.ScreenshotContentSize(w.Window, w.width, w.height) + w.shown = true + w.applySize() +} +func (w *window) targetSize() fyne.Size { + size, ok := w.measure(w.Window, w.width, w.height) + _, native := w.Window.(driver.NativeWindow) if !ok { - // The test driver has no chrome or backing scale. Native unsupported - // backends retain the bounded canvas size and report the missing metric. - if _, native := w.Window.(driver.NativeWindow); native { + // Unknown chrome cannot be measured, but physical content pixels still + // require scale conversion. Report the decoration limitation once. + if native && !w.warned { fyne.LogError("screenshot window geometry unavailable", nil) + w.warned = true } size = fyne.NewSize(float32(w.width), float32(w.height)) } @@ -80,12 +113,22 @@ func (w *window) Show() { if scale <= 0 { scale = 1 } - // Fyne rounds native coordinates upward. Bias by one float step so division - // by a fractional UI scale cannot add a physical pixel at that boundary. - logical := fyne.NewSize(math.Nextafter32(size.Width/scale, 0), math.Nextafter32(size.Height/scale, 0)) - if !ok { - logical = size - } // exact dimensions for the headless test driver + logical := fyne.NewSize(size.Width/scale, size.Height/scale) + // Fyne rounds native coordinates upward. Avoid an extra pixel caused by + // fractional floating-point division at that boundary. + if native { + logical = fyne.NewSize(math.Nextafter32(logical.Width, 0), math.Nextafter32(logical.Height, 0)) + } + return logical +} +func (w *window) applySize() { + if w.sizing || w.closed { + return + } + w.sizing = true + defer func() { w.sizing = false }() + logical := w.targetSize() + w.applied = logical w.Window.SetFixedSize(false) w.Window.Resize(logical) w.Window.SetFixedSize(true) diff --git a/internal/screenshots/app_test.go b/internal/screenshots/app_test.go index 0e33370e..d0018f8e 100644 --- a/internal/screenshots/app_test.go +++ b/internal/screenshots/app_test.go @@ -1,10 +1,13 @@ package screenshots import ( + "image/color" + "math" "testing" "fyne.io/fyne/v2" "fyne.io/fyne/v2/canvas" + "fyne.io/fyne/v2/driver/software" "fyne.io/fyne/v2/test" ) @@ -39,3 +42,72 @@ func TestWindowConstraint(t *testing.T) { w.Close() } } + +func TestWindowConstraintRetainsPadding(t *testing.T) { + application := test.NewApp() + defer application.Quit() + w := App(application, 1280, 800).NewWindow("padded screenshot") + if !w.Canvas().(software.WindowlessCanvas).Padded() { + t.Fatal("screenshot mode removed normal window padding") + } + w.Close() +} + +type fallbackApp struct{ fyne.App } + +func (a fallbackApp) NewWindow(title string) fyne.Window { + return fallbackWindow{Window: a.App.NewWindow(title)} +} + +type fallbackWindow struct{ fyne.Window } + +func (w fallbackWindow) RunNative(fn func(any)) { fn(struct{}{}) } + +func TestWindowConstraintTracksScale(t *testing.T) { + application := test.NewApp() + defer application.Quit() + for _, native := range []bool{false, true} { + underlying := application + if native { + underlying = fallbackApp{App: application} + } + w := App(underlying, 1280, 800).NewWindow("scaled screenshot") + c := w.Canvas().(software.WindowlessCanvas) + w.SetContent(canvas.NewRectangle(color.Black)) + c.SetScale(2) + w.Show() + if got := c.Capture().Bounds().Size(); got.X != 1280 || got.Y != 800 { + t.Fatalf("native=%v at 2x screenshot=%v", native, got) + } + c.SetScale(1) + c.Content().Refresh() + if got := c.Capture().Bounds().Size(); got.X != 1280 || got.Y != 800 { + t.Fatalf("native=%v after 1x move screenshot=%v", native, got) + } + w.Close() + c.SetScale(2) + c.Content().Refresh() + } +} + +func TestWindowConstraintTracksNativeBackingWithoutCanvasScaleChange(t *testing.T) { + application := test.NewApp() + defer application.Quit() + w := App(fallbackApp{App: application}, 1280, 800).NewWindow("backing scale").(*window) + backing := float32(2) + w.measure = func(_ fyne.Window, width, height int) (fyne.Size, bool) { + return fyne.NewSize(float32(width)/backing, float32(height)/backing), true + } + w.SetContent(canvas.NewRectangle(color.Black)) + w.Show() + for _, factor := range []float32{2, 1, 2} { + backing = factor + w.Canvas().Content().Refresh() + physical := fyne.NewSize(float32(math.Ceil(float64(w.Canvas().Size().Width)))*backing, + float32(math.Ceil(float64(w.Canvas().Size().Height)))*backing) + if physical != fyne.NewSize(1280, 800) { + t.Fatalf("backing=%v canvas scale=%v frame=%v", backing, w.Canvas().Scale(), physical) + } + } + w.Close() +} diff --git a/plans/2026-10-01-fixed-size-mode.md b/plans/2026-10-01-fixed-size-mode.md index 73f8123b..dd268f4a 100644 --- a/plans/2026-10-01-fixed-size-mode.md +++ b/plans/2026-10-01-fixed-size-mode.md @@ -98,3 +98,35 @@ metadata removal/export identity. Those confirmed older findings must be address in the requested review loop before the new App Store package is built. Recursive scan limits are explicitly classified by the bot as deferrable availability hardening and remain an open backlog item. + +## Review fixes after 1439746 + +Four confirmed code-review findings are fixed inline. Signal notification now +starts before any raw terminal mutation and stops after cursor, terminal and +ANSI restoration. Fixed windows retain ordinary Fyne padding. Unknown native +chrome still reports a limitation, but fallback sizes divide by canvas scale. +Content layout refresh checks native dimensions as well as Fyne scale, and queues +one correction on UI; hidden/closed windows discard it, with no new workers. + +Padding and 2x fallback tests were observed red (padding removed and 2560 x 1600 +instead of 1280 x 800). Tests now cover native/non-native fallback and 2x -> 1x +changes. A separate simulated native 2x -> 1x -> 2x backing transition keeps +Canvas.Scale at 1 and preserves the physical target. Disabling correction made +that guard fail at 640 x 400; it was restored and the race suite passes. + +Focused screenshot/console/root UI race regressions pass. All four actual Retina +native frames still equal their selected pixels after forbidden resize, +fullscreen/maximize and large-content attempts. The real executable passes the +pseudo-terminal Ctrl+C, SIGTERM and Down/Return restoration checks. Windows +cross-compilation passes. GoLand inspections of all three changed Go files, +including weak warnings, are clear after fixing one inferred-type warning. +`make verify-build` passes. Actual mixed-monitor movement remains unqualified; +its independent backing conversion is covered by the per-instance native-metric +seam test. The README states the unknown-chrome limitation explicitly. + +Hosted evidence for 1439746: full platform/race CI and CodeQL pass; artifact +11146375885 from Qodana run 36828289980 has the exact revision and 10 final +post-suppression SARIF results, all unused-export false positives with production +callers (the prior eight plus ScreenshotContentSize and SelectResolution). +Fresh code review has the four addressed findings above; a new clean code/security +round on the final subsequent head remains required. From a664aad4fbc4a5f7bdf31c5963d5e3946d84ff74 Mon Sep 17 00:00:00 2001 From: frathe Date: Thu, 1 Oct 2026 10:34:26 +0200 Subject: [PATCH 31/63] Bind metadata removal and export to the inspected source --- ARCHITECTURE.md | 1 + internal/fileaccess/reader.go | 10 +++ internal/imaging/loader.go | 7 ++ internal/imaging/mutations.go | 1 + internal/imaging/save.go | 99 +++++++++++++++------ internal/imaging/sourceversion.go | 67 ++++++++++++++ internal/ui/display/capture.go | 9 +- internal/ui/display/contract_test.go | 10 ++- internal/ui/display/feature.go | 3 +- internal/ui/exifwin/exifwin.go | 28 +++--- internal/ui/exifwin/exifwin_test.go | 2 +- internal/ui/exifwin/metadata.go | 5 +- internal/ui/exifwin/stripwork.go | 7 +- internal/ui/exifwin/stripwork_test.go | 91 +++++++++++++++++-- internal/ui/export.go | 5 +- internal/ui/export_test.go | 38 ++++++++ internal/ui/save.go | 2 +- internal/uitest/reader.go | 9 ++ plans/2026-10-01-metadata-source-binding.md | 76 ++++++++++++++++ todos.md | 12 +++ 20 files changed, 425 insertions(+), 57 deletions(-) create mode 100644 internal/imaging/sourceversion.go create mode 100644 plans/2026-10-01-metadata-source-binding.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 61dec3b3..0d5a0580 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -930,6 +930,7 @@ Encode/write-back for a subset of formats lives in `save.go`; `mutations.go` ser | File | Responsibility | |------|----------------| | `bytecache.go` | `ByteCache[V]`: goroutine-safe LRU by estimated bytes. `Add` admits foreground images even over budget; generation-bound `CacheWriter.AddIfRoom` admits display preloads only into remaining space without eviction or promotion. `RefreshIfRoom` lets Favorite warming replace stale keys, dropping only that key if the replacement cannot fit. `AddIfFits` keeps its existing individual-size gate and may evict. `LoadedImage.DecodedBytes` shares retained pixel/vector accounting with the mosaic repeat cache. | +| `sourceversion.go` | Descriptor identity plus encoded-byte SHA-256 snapshots for confirmed JPEG removal; display/cache digests bind export metadata to decoded pixels. | | `loader.go` | `LoadedImage`, `NewImgCache`, `ReadAndProbe`, `CaptureDateContext` (cancellable metadata reads), `DecodeLoaded` (pixels), `DecodeRecord` (complete full-cache facts), `LoadImage`, `IsSupportedImage`, `SupportedExtensions`, `MaxEncodedBytes` / `InputTooLargeError`. | | `heic.go` | Canonical native HEIC probe/pixels, independent output validation, `ReadMetadataContext` and worker pixel budget. No image decoder is globally registered. | | `recognized.go` | Static `RecognizedExtensions` for package declarations and portable format rules, distinct from unconditional runtime `SupportedExtensions`. | diff --git a/internal/fileaccess/reader.go b/internal/fileaccess/reader.go index fa3853d6..66f00ad0 100644 --- a/internal/fileaccess/reader.go +++ b/internal/fileaccess/reader.go @@ -2,6 +2,7 @@ package fileaccess import ( "context" + "os" "sync" "fyne.io/fyne/v2" @@ -34,3 +35,12 @@ func (r *scopedReader) Close() error { r.once.Do(func() { r.err = r.URIReadCloser.Close(); r.release() }) return r.err } + +// Stat exposes identity from the same open descriptor, never a second path lookup. +// Non-file storage readers cannot supply a filesystem mutation identity. +func (r *scopedReader) Stat() (os.FileInfo, error) { + if f, ok := r.URIReadCloser.(interface{ Stat() (os.FileInfo, error) }); ok { + return f.Stat() + } + return nil, os.ErrInvalid +} diff --git a/internal/imaging/loader.go b/internal/imaging/loader.go index d1423825..3a816e1b 100644 --- a/internal/imaging/loader.go +++ b/internal/imaging/loader.go @@ -11,6 +11,7 @@ package imaging import ( "context" + "crypto/sha256" "fmt" "image" _ "image/jpeg" // registers JPEG with image.Decode @@ -96,6 +97,7 @@ func IsSupportedImage(u fyne.URI) bool { type LoadedImage struct { Frames []image.Image Delays []time.Duration // parallel to Frames; unused when len(Frames) == 1 + Digest SourceDigest // encoded source of these pixels, retained through cache hits FileSize int64 // raw byte count read by ReadAndProbe, for the info overlay // Vector is the parsed source of an SVG, retained so the app can @@ -357,6 +359,10 @@ func ReadAndProbe(ctx context.Context, u fyne.URI) (data []byte, bounds image.Re if err != nil { return nil, image.Rectangle{}, err } + return probeSource(ctx, data) +} + +func probeSource(ctx context.Context, data []byte) ([]byte, image.Rectangle, error) { if heic.IsData(data) { result, readErr := readHEIC(ctx, data, false) if readErr != nil { @@ -471,6 +477,7 @@ func DecodeRecord(ctx context.Context, data []byte, maxAnimBytes int64) (*Loaded return nil, err } loaded.FileSize = int64(len(data)) + loaded.Digest = sha256.Sum256(data) if !heic.IsData(data) { loaded.HasEXIF = !ReadMetadata(data).Empty() } diff --git a/internal/imaging/mutations.go b/internal/imaging/mutations.go index d7beb577..c527c345 100644 --- a/internal/imaging/mutations.go +++ b/internal/imaging/mutations.go @@ -17,6 +17,7 @@ import ( // WriteResult distinguishes an accomplished replacement from cancelled work. // Cancelling presentation after Committed became true cannot undo the file. type WriteResult struct { + Digest SourceDigest // exact bytes encoded by this process, never a path reread Path string Committed bool } diff --git a/internal/imaging/save.go b/internal/imaging/save.go index e2159de3..21fc022b 100644 --- a/internal/imaging/save.go +++ b/internal/imaging/save.go @@ -3,6 +3,7 @@ package imaging import ( "bytes" "context" + "crypto/sha256" "errors" "image" "image/gif" @@ -98,23 +99,32 @@ func SaveRotated(u fyne.URI, img image.Image) error { // SaveRotatedContext holds the selected file's transaction through replacement. func SaveRotatedContext(ctx context.Context, u fyne.URI, img image.Image) (WriteResult, error) { - return writeWithAccess(ctx, u, true, func(path string) (bool, error) { - err := saveRotated(ctx, path, img) - return err == nil, err + var digest SourceDigest + result, err := writeWithAccess(ctx, u, true, func(path string) (bool, error) { + var writeErr error + digest, writeErr = saveRotated(ctx, path, img) + return writeErr == nil, writeErr }) + if err != nil { + return result, err + } + if result.Committed { + result.Digest = digest + } + return result, nil } -func saveRotated(ctx context.Context, path string, img image.Image) error { +func saveRotated(ctx context.Context, path string, img image.Image) (SourceDigest, error) { ext := filepath.Ext(path) encode, ok := encoders[strings.ToLower(ext)] if !ok { - return &UnsupportedSaveFormatError{ext: ext} + return SourceDigest{}, &UnsupportedSaveFormatError{ext: ext} } if isJPEGExt(ext) { orig, err := readFileContext(ctx, path) if err != nil { - return err + return SourceDigest{}, err } encode = func(w io.Writer, img image.Image) error { var corrected image.Point @@ -127,10 +137,14 @@ func saveRotated(ctx context.Context, path string, img image.Image) error { info, err := os.Stat(path) if err != nil { - return err + return SourceDigest{}, err } - return writeEncodedContext(ctx, path, info.Mode().Perm(), encode, img) + digest := sha256.New() + err = writeFileContext(ctx, path, info.Mode().Perm(), func(w io.Writer) error { return encode(io.MultiWriter(w, digest), img) }) + var sum SourceDigest + copy(sum[:], digest.Sum(nil)) + return sum, err } // defaultExportPerm is what a file Export creates from scratch gets, the @@ -145,6 +159,8 @@ const defaultExportPerm = 0o644 // wallpaper and mosaic paths, which write generated pixels with no source // file at all) passes ExportOptions{} and gets exactly that. type ExportOptions struct { + SourceDigest SourceDigest + // FallbackExt selects an encoder when the exact destination has no // supported extension. Native save callers set this from the chosen format; // the destination path is never changed after confirmation. Empty retains @@ -161,6 +177,9 @@ type ExportOptions struct { // StripJPEGMetadata's irreversible in-place metadata removal. The colour // profile still survives; see encodeJPEGKeepingICC. OmitMetadata bool + // VerifySource binds reread metadata to the pixels' decoded source. A missing + // or mismatched digest yields a pixel-only export, including color profiles. + VerifySource bool } // Export writes img to the exact dest path, using its supported extension or @@ -217,7 +236,7 @@ func exportImage(ctx context.Context, path, ext string, img image.Image, src fyn } if isJPEGExt(ext) && src != nil && src.Path() != "" { - if orig, err := jpegSourceBytesContext(ctx, src); err == nil && orig != nil { + if orig, err := jpegSourceBytesContext(ctx, src); err == nil && orig != nil && (!opts.VerifySource || opts.SourceDigest == sha256.Sum256(orig)) { // Answered here, once, while both frames are still in scope // under their own names: out is what will be written, img is // what arrived. Inside the closure below only one of them has a @@ -332,6 +351,10 @@ func writeEncodedContext(ctx context.Context, path string, perm os.FileMode, enc // file-only grant cannot create siblings. Other builds stage beside the target. // Both routes preserve atomic same-volume replacement and cleanup on failure. func writeFileContext(ctx context.Context, path string, perm os.FileMode, write func(io.Writer) error) error { + return writeFileContextChecked(ctx, path, perm, write, nil) +} + +func writeFileContextChecked(ctx context.Context, path string, perm os.FileMode, write func(io.Writer) error, check func() error) error { if err := ctx.Err(); err != nil { return err } @@ -369,6 +392,11 @@ func writeFileContext(ctx context.Context, path string, perm os.FileMode, write return err } + if check != nil { + if err := check(); err != nil { + return err + } + } return os.Rename(tmpPath, path) } @@ -386,12 +414,21 @@ func StripJPEGMetadata(u fyne.URI) error { // StripJPEGMetadataContext serializes the source read as well as its rewrite. func StripJPEGMetadataContext(ctx context.Context, u fyne.URI) (WriteResult, error) { return writeWithAccess(ctx, u, false, func(path string) (bool, error) { - return stripJPEGMetadata(ctx, path) + return stripJPEGMetadata(ctx, path, nil) }) } -func stripJPEGMetadata(ctx context.Context, path string) (bool, error) { - data, err := readJPEGRemovalSource(ctx, path) +// StripJPEGMetadataVerified refuses a reread whose identity or bytes differ +// from the inspected snapshot. Scoped resolution and reads stay inside its turn. +func StripJPEGMetadataVerified(ctx context.Context, u fyne.URI, expected SourceVersion) (WriteResult, error) { + if !expected.Valid() { + return WriteResult{}, ErrSourceChanged + } + return writeWithAccess(ctx, u, false, func(path string) (bool, error) { return stripJPEGMetadata(ctx, path, &expected) }) +} + +func stripJPEGMetadata(ctx context.Context, path string, expected *SourceVersion) (bool, error) { + data, info, err := readJPEGRemovalSourceVersion(ctx, path) if err != nil { return false, err } @@ -403,47 +440,55 @@ func stripJPEGMetadata(ctx context.Context, path string) (bool, error) { return false, nil } - info, err := os.Stat(path) - if err != nil { - return false, err + if expected != nil && !expected.matches(data, info) { + return false, ErrSourceChanged } - - err = writeFileContext(ctx, path, info.Mode().Perm(), func(w io.Writer) error { + check := func() error { + current, err := os.Stat(path) + if err != nil { + return err + } + if !os.SameFile(info, current) || current.Size() != info.Size() || !current.ModTime().Equal(info.ModTime()) { + return ErrSourceChanged + } + return nil + } + err = writeFileContextChecked(ctx, path, info.Mode().Perm(), func(w io.Writer) error { _, err := w.Write(p.output) return err - }) + }, check) return err == nil, err } -func readJPEGRemovalSource(ctx context.Context, path string) ([]byte, error) { +func readJPEGRemovalSourceVersion(ctx context.Context, path string) ([]byte, os.FileInfo, error) { f, err := os.Open(path) if err != nil { - return nil, err + return nil, nil, err } defer func() { _ = f.Close() }() configuredLimit := MaxEncodedBytes() limit := min(configuredLimit, jpegRemovalSourceBytes) info, err := f.Stat() if err != nil { - return nil, err + return nil, nil, err } if info.Size() > configuredLimit { - return nil, &InputTooLargeError{limit: configuredLimit} + return nil, nil, &InputTooLargeError{limit: configuredLimit} } if info.Size() > jpegRemovalSourceBytes { - return nil, ErrJPEGMetadataMemory + return nil, nil, ErrJPEGMetadataMemory } data, err := io.ReadAll(contextRead{ctx: ctx, in: io.LimitReader(f, limit+1)}) if err != nil { - return nil, err + return nil, nil, err } if int64(len(data)) > limit { if limit < configuredLimit { - return nil, ErrJPEGMetadataMemory + return nil, nil, ErrJPEGMetadataMemory } - return nil, &InputTooLargeError{limit: limit} + return nil, nil, &InputTooLargeError{limit: limit} } - return data, nil + return data, info, nil } // CanStripJPEGMetadata is the compatibility predicate for the shared inspection. diff --git a/internal/imaging/sourceversion.go b/internal/imaging/sourceversion.go new file mode 100644 index 00000000..3bf23cff --- /dev/null +++ b/internal/imaging/sourceversion.go @@ -0,0 +1,67 @@ +package imaging + +import ( + "context" + "crypto/sha256" + "errors" + "image" + "os" + + "fyne.io/fyne/v2" + + "github.com/frathe/picfetch/internal/fileaccess" +) + +// SourceDigest binds metadata to the exact encoded bytes that supplied pixels. +type SourceDigest [sha256.Size]byte + +// SourceVersion additionally binds destructive removal to the inspected inode. +// Its fields are private so callers cannot manufacture a filesystem identity. +type SourceVersion struct { + digest SourceDigest + info os.FileInfo +} + +// ErrSourceChanged refuses a mutation whose inspected source has been replaced. +var ErrSourceChanged = errors.New("source changed after inspection") + +// Valid reports whether inspection obtained an opened filesystem identity. +func (v SourceVersion) Valid() bool { return v.info != nil && v.info.Mode().IsRegular() } + +func (v SourceVersion) matches(data []byte, info os.FileInfo) bool { + return v.Valid() && info != nil && os.SameFile(v.info, info) && v.digest == sha256.Sum256(data) +} + +// ReadAndProbeSnapshot captures identity and bytes from one scoped reader. +// Unavailable descriptor identity leaves ordinary metadata readable but disables +// destructive removal. Probe and input limits are shared with image loading. +func ReadAndProbeSnapshot(ctx context.Context, u fyne.URI) ([]byte, image.Rectangle, SourceVersion, error) { + if err := ctx.Err(); err != nil { + return nil, image.Rectangle{}, SourceVersion{}, err + } + rc, err := fileaccess.Reader(ctx, u) + if err != nil { + return nil, image.Rectangle{}, SourceVersion{}, err + } + defer func() { _ = rc.Close() }() + var info os.FileInfo + if f, ok := rc.(interface{ Stat() (os.FileInfo, error) }); ok { + info, _ = f.Stat() + } + data, err := readMutationBytes(ctx, rc, MaxEncodedBytes()) + if err != nil { + return nil, image.Rectangle{}, SourceVersion{}, err + } + if err := ctx.Err(); err != nil { + return nil, image.Rectangle{}, SourceVersion{}, err + } + version := SourceVersion{digest: sha256.Sum256(data), info: info} + if f, ok := rc.(interface{ Stat() (os.FileInfo, error) }); ok && info != nil { + after, statErr := f.Stat() + if statErr != nil || !os.SameFile(info, after) || info.Size() != after.Size() || !info.ModTime().Equal(after.ModTime()) { + return nil, image.Rectangle{}, SourceVersion{}, ErrSourceChanged + } + } + data, bounds, err := probeSource(ctx, data) + return data, bounds, version, err +} diff --git a/internal/ui/display/capture.go b/internal/ui/display/capture.go index 4c63e46e..361d1cd9 100644 --- a/internal/ui/display/capture.go +++ b/internal/ui/display/capture.go @@ -10,6 +10,7 @@ import ( // Capture retains read-only pixels and the presentation they came from. type Capture struct { + Digest imaging.SourceDigest Pixels image.Image Identity Identity Rotation int @@ -25,17 +26,23 @@ func (f *Feature) Capture() (Capture, bool) { return Capture{}, false } return Capture{Pixels: f.surface.Image, Identity: f.snapshot.Displayed, - Rotation: f.Rotation(), owner: f, requestRevision: f.revision, + Digest: f.snapshot.Digest, Rotation: f.Rotation(), owner: f, requestRevision: f.revision, Vector: f.vector.svg, LogicalSize: image.Pt(int(f.logical.Width+0.5), int(f.logical.Height+0.5)), Rasterize: f.vector.options.Rasterize}, true } // ReconcileSaved adopts a committed capture as the saved baseline, preserving // any subsequent turns and the published image itself. func (f *Feature) ReconcileSaved(capture Capture) bool { + return f.ReconcileSavedDigest(capture, imaging.SourceDigest{}) +} + +// ReconcileSavedDigest retains the digest of our own committed encoded output. +func (f *Feature) ReconcileSavedDigest(capture Capture, digest imaging.SourceDigest) bool { if capture.owner != f || f.stopped || f.snapshot.Loading || capture.requestRevision != f.revision || capture.Identity.Revision != f.snapshot.Displayed.Revision || capture.Pixels == nil || f.Count() != 1 { return false } + f.snapshot.Digest = digest f.frames = []image.Image{capture.Pixels} f.rotation = normalizedRotation(f.rotation - capture.Rotation) b := capture.Pixels.Bounds() diff --git a/internal/ui/display/contract_test.go b/internal/ui/display/contract_test.go index 021c7e5d..53f9b014 100644 --- a/internal/ui/display/contract_test.go +++ b/internal/ui/display/contract_test.go @@ -27,7 +27,7 @@ func TestPresentationContract(t *testing.T) { t.Cleanup(f.Stop) source := storage.NewFileURI("/saved.png") pixels := image.NewNRGBA(image.Rect(0, 0, 2, 3)) - loaded := &imaging.LoadedImage{Frames: []image.Image{pixels}} + loaded := &imaging.LoadedImage{Frames: []image.Image{pixels}, Digest: imaging.SourceDigest{1}} f.Present(source, loaded, false) f.RotateBy(1) capture, ok := f.Capture() @@ -40,9 +40,12 @@ func TestPresentationContract(t *testing.T) { f.RotateBy(1) } visible := f.Surface().Image - if !f.ReconcileSaved(capture) { + if !f.ReconcileSavedDigest(capture, imaging.SourceDigest{2}) { t.Fatal("current committed capture was rejected") } + if f.Snapshot().Digest != (imaging.SourceDigest{2}) { + t.Fatal("save reconciliation did not adopt committed output digest") + } wantRotation := 1 if reset { wantRotation = 3 @@ -50,6 +53,9 @@ func TestPresentationContract(t *testing.T) { if f.Snapshot().Rotation != wantRotation || f.Surface().Image != visible { t.Fatal("save reconciliation discarded a later view adjustment or republished pixels") } + if capture.Digest != loaded.Digest { + t.Fatal("capture lost decoded source digest") + } if capture.Rotation != 1 || capture.Pixels.Bounds().Size() != image.Pt(3, 2) { t.Fatal("capture changed after the view changed") } diff --git a/internal/ui/display/feature.go b/internal/ui/display/feature.go index 0a2c35a3..ac6d49e6 100644 --- a/internal/ui/display/feature.go +++ b/internal/ui/display/feature.go @@ -31,6 +31,7 @@ type Snapshot struct { Rotation int Vector, Animated bool Duration time.Duration + Digest imaging.SourceDigest FileSize int64 HasEXIF, Preview bool } @@ -163,7 +164,7 @@ func (f *Feature) present(source fyne.URI, loaded *imaging.LoadedImage, transiti f.clearVector() id := Identity{Source: source, Revision: f.revision} f.snapshot = Snapshot{Requested: id, Displayed: id, Vector: loaded.Vector != nil, - Animated: len(loaded.Frames) > 1, FileSize: loaded.FileSize, HasEXIF: loaded.HasEXIF, Preview: loaded.Preview} + Animated: len(loaded.Frames) > 1, FileSize: loaded.FileSize, HasEXIF: loaded.HasEXIF, Preview: loaded.Preview, Digest: loaded.Digest} if f.snapshot.Animated { for _, delay := range loaded.Delays { f.snapshot.Duration += delay diff --git a/internal/ui/exifwin/exifwin.go b/internal/ui/exifwin/exifwin.go index 4beafdcd..fa679dd3 100644 --- a/internal/ui/exifwin/exifwin.go +++ b/internal/ui/exifwin/exifwin.go @@ -118,17 +118,18 @@ type Window struct { // see tiles.go for why the widget's own fetching can't be left to it. // warming and warmGen track the prefetch that fills the first view, // warm is the completion.Signal tests wait on - see internal/completion. - tiles *tileFetcher - warming bool - warmGen int - warm completion.Signal - warmWorkers sync.WaitGroup - ui UIQueue - stopped bool - stripFile func(context.Context, fyne.URI) (imaging.WriteResult, error) - stripWork stripMutation - metadata metadataRead - heic *heic.Capability + tiles *tileFetcher + warming bool + warmGen int + warm completion.Signal + warmWorkers sync.WaitGroup + ui UIQueue + stopped bool + stripFile func(context.Context, fyne.URI, imaging.SourceVersion) (imaging.WriteResult, error) + stripWork stripMutation + metadata metadataRead + sourceVersion imaging.SourceVersion + heic *heic.Capability onClosed func() } @@ -144,7 +145,7 @@ func New(application fyne.App, host Host) *Window { host: host, tiles: newTileFetcher(osmTiles, nil), ui: fyneQueue{}, - stripFile: imaging.StripJPEGMetadataContext, + stripFile: imaging.StripJPEGMetadataVerified, } // The panel is read against the photo it describes, so it floats above @@ -353,12 +354,13 @@ func (w *Window) requestStrip() { return } + version := w.sourceVersion if w.showConfirm(confirmation{ title: lang.L("Remove Metadata?"), message: fmt.Sprintf(lang.L("Remove identifying metadata, previews, additional pictures, and audio/video from %q? The original file will be replaced. Image data and essential orientation/color instructions are preserved without recompression. This cannot be undone."), u.Name()), action: lang.L("Remove Metadata"), importance: widget.DangerImportance, - onConfirm: func() { w.performStrip(u) }, + onConfirm: func() { w.performStripVersion(u, version) }, onCancel: func() { w.pending = nil }, onClosed: func() { w.pending = nil }, }) == nil { diff --git a/internal/ui/exifwin/exifwin_test.go b/internal/ui/exifwin/exifwin_test.go index 742fd229..9a1d5ba3 100644 --- a/internal/ui/exifwin/exifwin_test.go +++ b/internal/ui/exifwin/exifwin_test.go @@ -936,7 +936,7 @@ func TestJPEGMetadataRemovalUI(t *testing.T) { defer w.Window().Close() // Exercise error delivery through the existing per-window mutation // boundary without making the UI test allocate a large image. - w.stripFile = func(_ context.Context, _ fyne.URI) (imaging.WriteResult, error) { + w.stripFile = func(_ context.Context, _ fyne.URI, _ imaging.SourceVersion) (imaging.WriteResult, error) { return imaging.WriteResult{}, imaging.ErrJPEGMetadataMemory } if _, found := absolutePos(w.Window().Content(), w.StripButton()); !found { diff --git a/internal/ui/exifwin/metadata.go b/internal/ui/exifwin/metadata.go index 471e91ca..f7053555 100644 --- a/internal/ui/exifwin/metadata.go +++ b/internal/ui/exifwin/metadata.go @@ -65,7 +65,7 @@ func (w *Window) Refresh() { generation := w.metadata.generation done := w.metadata.done.Begin() w.metadata.workers.Go(func() { - data, _, err := imaging.ReadAndProbe(ctx, u) + data, _, version, err := imaging.ReadAndProbeSnapshot(ctx, u) var metadata imaging.Metadata var inspection imaging.JPEGMetadataInspection if err == nil && ctx.Err() == nil { @@ -92,7 +92,8 @@ func (w *Window) Refresh() { } w.text.SetText(formatExifMetadata(metadata)) w.showLocation(metadata) - w.canStrip = inspection.State == imaging.JPEGMetadataRemovable + w.sourceVersion = version + w.canStrip = version.Valid() && inspection.State == imaging.JPEGMetadataRemovable if inspection.State == imaging.JPEGMetadataClean { w.setRemovalStatus(lang.L("Metadata removal: nothing to remove.")) } else if inspection.Err != nil && !errors.Is(inspection.Err, imaging.ErrJPEGMetadataNotJPEG) { diff --git a/internal/ui/exifwin/stripwork.go b/internal/ui/exifwin/stripwork.go index 07f4aa83..6082e8d5 100644 --- a/internal/ui/exifwin/stripwork.go +++ b/internal/ui/exifwin/stripwork.go @@ -9,6 +9,7 @@ import ( "fyne.io/fyne/v2/lang" "github.com/frathe/picfetch/internal/completion" + "github.com/frathe/picfetch/internal/imaging" ) type stripMutation struct { @@ -24,6 +25,10 @@ type stripMutation struct { // A successful commit still notifies the host after navigation/close; only its // current panel request owns the busy state and toast. func (w *Window) performStrip(u fyne.URI) { + w.performStripVersion(u, w.sourceVersion) +} + +func (w *Window) performStripVersion(u fyne.URI, version imaging.SourceVersion) { w.pending = nil if w.stopped || w.text == nil || w.stripWork.pending { return @@ -38,7 +43,7 @@ func (w *Window) performStrip(u fyne.URI) { w.syncStripVisible() strip := w.stripFile w.stripWork.workers.Go(func() { - result, err := strip(ctx, u) + result, err := strip(ctx, u, version) if ctx.Err() != nil && !result.Committed { cancel() done() diff --git a/internal/ui/exifwin/stripwork_test.go b/internal/ui/exifwin/stripwork_test.go index 1f74338a..b32fef71 100644 --- a/internal/ui/exifwin/stripwork_test.go +++ b/internal/ui/exifwin/stripwork_test.go @@ -7,6 +7,7 @@ import ( "fmt" "image/color" "os" + "path/filepath" "sync" "sync/atomic" "testing" @@ -14,8 +15,10 @@ import ( "time" "fyne.io/fyne/v2" + "fyne.io/fyne/v2/storage" "github.com/frathe/picfetch/internal/imaging" + "github.com/frathe/picfetch/internal/ui/widgets" "github.com/frathe/picfetch/internal/uitest" ) @@ -27,10 +30,10 @@ func TestMetadataRemovalLeavesQueuedWindowInputResponsive(t *testing.T) { w.Show() w.Settle() entered, release := make(chan struct{}), make(chan struct{}) - w.stripFile = func(ctx context.Context, u fyne.URI) (imaging.WriteResult, error) { + w.stripFile = func(ctx context.Context, u fyne.URI, version imaging.SourceVersion) (imaging.WriteResult, error) { close(entered) <-release - return imaging.StripJPEGMetadataContext(ctx, u) + return imaging.StripJPEGMetadataVerified(ctx, u, version) } actions := make(chan func(), 2) uiDone := make(chan struct{}) @@ -78,16 +81,16 @@ func TestMetadataRemovalCancellationDistinguishesCommittedDiskChanges(t *testing w.Show() w.Settle() entered, release := make(chan struct{}), make(chan struct{}) - w.stripFile = func(ctx context.Context, u fyne.URI) (imaging.WriteResult, error) { + w.stripFile = func(ctx context.Context, u fyne.URI, version imaging.SourceVersion) (imaging.WriteResult, error) { var result imaging.WriteResult var err error if committed { - result, err = imaging.StripJPEGMetadataContext(ctx, u) + result, err = imaging.StripJPEGMetadataVerified(ctx, u, version) } close(entered) <-release if !committed { - result, err = imaging.StripJPEGMetadataContext(ctx, u) + result, err = imaging.StripJPEGMetadataVerified(ctx, u, version) } return result, err } @@ -169,7 +172,7 @@ func TestMetadataRemovalBusyErrorDeliveryAndRetry(t *testing.T) { entered, release := make(chan struct{}), make(chan struct{}) var once sync.Once var calls atomic.Int32 - w.stripFile = func(_ context.Context, _ fyne.URI) (imaging.WriteResult, error) { + w.stripFile = func(_ context.Context, _ fyne.URI, _ imaging.SourceVersion) (imaging.WriteResult, error) { calls.Add(1) once.Do(func() { close(entered) }) <-release @@ -205,7 +208,7 @@ func TestMetadataRemovalBusyErrorDeliveryAndRetry(t *testing.T) { if w.StripButton().Disabled() { t.Error("failed removal cannot be retried") } - w.stripFile = imaging.StripJPEGMetadataContext + w.stripFile = imaging.StripJPEGMetadataVerified w.performStrip(source) w.Settle() if len(host.toasts) != 2 || host.after != 1 { @@ -215,3 +218,77 @@ func TestMetadataRemovalBusyErrorDeliveryAndRetry(t *testing.T) { t.Error("successful retry left removal in the panel") } } + +func TestMetadataRemovalBindsConfirmedSource(t *testing.T) { + for _, change := range []string{"unchanged link", "retargeted link", "identical replacement", "changed bytes", "refreshed while confirming"} { + t.Run(change, func(t *testing.T) { + app, host := gpsApp(t) + source, _ := host.DisplayedFile() + original := readWindowFile(t, source) + private := filepath.Join(t.TempDir(), "private.jpg") + privateBytes := uitest.GPSJPEG(t, 8, 4, -33.856, 151.215) + if change == "identical replacement" { + privateBytes = bytes.Clone(original) + } + if err := os.WriteFile(private, privateBytes, 0o600); err != nil { + t.Fatal(err) + } + if change == "unchanged link" || change == "retargeted link" { + link := filepath.Join(filepath.Dir(source.Path()), "selected.jpg") + if err := os.Symlink(source.Path(), link); err != nil { + t.Skipf("symlink unavailable: %v", err) + } + source = storage.NewFileURI(link) + host.current = func() (fyne.URI, bool) { return source, true } + } + w := newTestWindow(t, app, host) + w.Show() + w.Settle() + t.Cleanup(func() { w.Window().Close(); w.Settle() }) + w.StripButton().OnTapped() + panel := w.Window().Canvas().Focused().(*widgets.ChoicePanel) + switch change { + case "retargeted link": + if err := os.Remove(source.Path()); err != nil { + t.Fatal(err) + } + if err := os.Symlink(private, source.Path()); err != nil { + t.Fatal(err) + } + case "identical replacement": + if err := os.Rename(private, source.Path()); err != nil { + t.Fatal(err) + } + private = source.Path() + case "changed bytes", "refreshed while confirming": + if err := os.WriteFile(source.Path(), privateBytes, 0o600); err != nil { + t.Fatal(err) + } + private = source.Path() + } + if change == "refreshed while confirming" { + w.Refresh() + w.Settle() + } + panel.TypedKey(&fyne.KeyEvent{Name: fyne.KeyRight}) + panel.TypedKey(&fyne.KeyEvent{Name: fyne.KeyReturn}) + w.Settle() + if change == "unchanged link" { + if host.after != 1 || !imaging.ReadMetadata(readWindowFile(t, source)).Empty() { + t.Fatal("unchanged symlink source cannot be stripped") + } + } else { + got, err := os.ReadFile(private) + if err != nil { + t.Fatal(err) + } + if !bytes.Equal(got, privateBytes) || host.after != 0 { + t.Fatal("confirmation mutated a replacement source") + } + if len(host.toasts) != 1 { + t.Fatalf("missing refusal: %v", host.toasts) + } + } + }) + } +} diff --git a/internal/ui/export.go b/internal/ui/export.go index 99baa927..1a817db5 100644 --- a/internal/ui/export.go +++ b/internal/ui/export.go @@ -131,7 +131,10 @@ func (v *viewer) exportAs(ext string) { if !ok { return } - req := exportRequest{ext: ext, opts: v.exportOptions.Options(), source: capture.Identity.Source, pixels: capture.Pixels, choose: filepicker.ChooseSave, write: v.fileWork.export} + opts := v.exportOptions.Options() + opts.VerifySource = true + opts.SourceDigest = capture.Digest + req := exportRequest{ext: ext, opts: opts, source: capture.Identity.Source, pixels: capture.Pixels, choose: filepicker.ChooseSave, write: v.fileWork.export} // chooser is shared with openFileDialog's own goroutine rather than // given a twin of its own: it means "the native file dialog diff --git a/internal/ui/export_test.go b/internal/ui/export_test.go index e7e150e1..9e392421 100644 --- a/internal/ui/export_test.go +++ b/internal/ui/export_test.go @@ -218,6 +218,44 @@ func TestExportAs_JPEGSourceKeepsGPSExif(t *testing.T) { settleToast(t, v) }) + t.Run("export after our own save keeps source metadata", func(t *testing.T) { + v.rotateBy(1) + v.saveRotation() + drainFileWork(t, v) + dest := filepath.Join(t.TempDir(), "saved-copy.jpg") + uitest.StubSaveChooser(t, func(_ string) (fyne.URI, error) { return storage.NewFileURI(dest), nil }) + v.exportAs(".jpg") + settleChooser(t, v) + data, err := os.ReadFile(dest) + if err != nil { + t.Fatal(err) + } + if !imaging.ReadMetadata(data).HasGPS { + t.Fatal("our own Save discarded metadata binding for subsequent Export") + } + settleToast(t, v) + }) + + t.Run("source changes while chooser is open", func(t *testing.T) { + dest := filepath.Join(t.TempDir(), "copy.jpg") + uitest.StubSaveChooser(t, func(_ string) (fyne.URI, error) { + if err := os.WriteFile(path, uitest.GPSJPEG(t, 8, 4, -33.856, 151.215), 0o600); err != nil { + return nil, err + } + return storage.NewFileURI(dest), nil + }) + v.exportAs(".jpg") + settleChooser(t, v) + got, err := os.ReadFile(dest) + if err != nil { + t.Fatal(err) + } + if imaging.ReadMetadata(got).HasGPS { + t.Fatal("export published metadata from a source changed after display") + } + settleToast(t, v) + }) + } func TestExportAs_SuggestsTheSourceNameWithTheNewExtensionInItsOwnFolder(t *testing.T) { diff --git a/internal/ui/save.go b/internal/ui/save.go index ba423b76..7ef38dbc 100644 --- a/internal/ui/save.go +++ b/internal/ui/save.go @@ -84,7 +84,7 @@ func (v *viewer) saveRotation() { return } if result.Committed { - v.display.ReconcileSaved(capture) + v.display.ReconcileSavedDigest(capture, result.Digest) } v.ShowToast(lang.L("Saved")) }) diff --git a/internal/uitest/reader.go b/internal/uitest/reader.go index de7114eb..c80c7084 100644 --- a/internal/uitest/reader.go +++ b/internal/uitest/reader.go @@ -2,6 +2,7 @@ package uitest import ( "io" + "os" "strings" "fyne.io/fyne/v2" @@ -74,6 +75,14 @@ type uriReader struct { func (r uriReader) URI() fyne.URI { return r.u } +// Stat preserves identity when the controlled factory returns a real descriptor. +func (r uriReader) Stat() (os.FileInfo, error) { + if f, ok := r.ReadCloser.(interface{ Stat() (os.FileInfo, error) }); ok { + return f.Stat() + } + return nil, os.ErrInvalid +} + // ReadCloser supplies controlled read/close effects without replacing a global // storage function. Tests own synchronization and wait for completion before // inspecting state captured by these callbacks. diff --git a/plans/2026-10-01-metadata-source-binding.md b/plans/2026-10-01-metadata-source-binding.md new file mode 100644 index 00000000..2a9328e5 --- /dev/null +++ b/plans/2026-10-01-metadata-source-binding.md @@ -0,0 +1,76 @@ +# Metadata source binding — 2026-10-01 + +Route: Deep (display/cache, EXIF confirmation, filesystem writes, Windows). +Owner: Pico / T0 inline; zero delegates. Hot context remains with the lead. + +## Problem and decisions + +PR #75 threads 4148307899 and 4148307904 identify source replacement while a +confirmation or save chooser is open. Removal can follow a different JPEG; +export can publish its metadata alongside the previously displayed pixels. + +- Bind removal to the opened file identity and encoded-byte digest captured by + inspection, and capture that value in the confirmation continuation. +- Preserve unchanged symlink sources; a retargeted link or replaced inode is a + refusal even when its bytes happen to match. Changed bytes are also a refusal. +- Bind viewer exports to the digest decoded for the displayed pixels, including + cache hits. A changed/unavailable source supplies no metadata; export succeeds. +- Capture the digest of our own committed Save output without rereading a path, + so a subsequent export retains ordinary metadata behavior. +- Retain scoped URIs, transaction admission, cancellation, atomic staging and + Store file-only replacement directories. No dependencies or new UI strings. +- Recursive scan budgeting (4148307910) remains availability backlog. + +## Acceptance and tasks + +1. Removal rejects entry retarget/replacement and in-place content changes after + confirmation, leaving both originals untouched; unchanged symlink works. + `go test -tags no_emoji,nodynamic ./internal/ui/exifwin -run 'TestMetadataRemovalBindsConfirmedSource|TestRequestStrip|TestMetadataRemoval'` +2. Export after a chooser-time source replacement writes the captured pixels + without the replacement's metadata; unchanged source keeps GPS. + `go test -tags no_emoji,nodynamic ./internal/ui -run TestExportAs_JPEGSourceKeepsGPSExif` +3. Cache/display captures retain their byte digest; committed-save reconciliation + adopts only the digest of our own encoded output. + `go test -tags no_emoji,nodynamic ./internal/ui/display ./internal/imaging -run 'TestPresentationContract|TestFileMutation|TestExport|TestSaveRotated'` +4. Focused race checks, Store-tag compilation/tests, Windows compile/vet, + `make verify-build`, GoLand inspections of every changed code file, and latest + hosted Codex/security/CodeQL/Qodana/CI evidence close the gate. + +Task graph: regression tests -> identity/digest contracts -> EXIF/display wiring +-> focused checks/negative guard checks -> commit/push -> hosted review -> build. +All tasks T0 inline. Files: existing imaging loader/save/mutations tests, +fileaccess reader, EXIF metadata/confirmation/mutation/tests, display +presentation/capture/tests, root export/save/tests and this evidence record. +New files, if needed, receive exact Qodana test exclusions. Root UI regression is +an existing top-level test's subtest, so the shard assignment stays unchanged. +Budget: zero spawns; focused local suites; broad suite only native amd64 CI; +review rounds repeat until current-head clean. No claims of an OS atomic +compare-and-swap replacement: guards bind inspected and reread source identity; +existing filesystem transaction/staging semantics remain in force. + +## Evidence + +- End-to-end tests reproduced three destructive swaps (retargeted symlink, + identical-byte replacement inode, and changed bytes) and chooser-time GPS + publication before the guards. Untouched symlink behavior was green. +- Full affected EXIF/display/imaging/fileaccess/uitest race suites and focused + root Export/Save races passed. Store-tagged focused suites passed; Windows + EXIF compilation and affected-package vet passed (runtime qualification CI). +- Removing the identity guard again caused all three mutation regressions to + fail; removing digest verification caused export GPS disclosure again. Both + implementations were restored. Fresh post-restoration focused checks follow. +- `make verify-build` passed after repository import formatting. Existing root + test was extended with subtests; no shard or test-exclusion changes required. +- GoLand inspected every changed file including weak warnings. Redundant casts, + struct padding, obsolete helper and result-flow warning fixed and reinspected. + Six existing EXIF test duplication warnings match its exact Qodana exclusion. + The loader has an IDE configuration error (required `nodynamic` tag absent), + despite real tagged build/vet/tests passing. An ignored module-tag edit did not + refresh the active IDE; it was restored. That IDE resolution gate remains + unverified, requiring current configured hosted Qodana evidence. No blanket + source suppression was introduced. IDE build tool reported limited diagnostics. +- No dependency changes; existing notices remain the shipped closure. Parent + directory replacement races outside these source-read/confirmation guards + remain outside the claim of filesystem atomic compare-and-swap protection. +- Ledger: zero spawns; one lead local review plus targeted inspection fixes; + one local race round; broad native amd64 suite delegated to hosted CI. diff --git a/todos.md b/todos.md index 7a67ce59..2cc235c2 100644 --- a/todos.md +++ b/todos.md @@ -13,6 +13,11 @@ #### Bugfix +- [ ] Bind confirmed metadata removal to inspected file identity and contents; + bind exported metadata to displayed source bytes, including cache and Save. + Local regressions/race/Store checks pass; latest hosted review is pending. + See [source-binding evidence](plans/2026-10-01-metadata-source-binding.md). + - [ ] Restore Store folder consent: native panels complete without blocking the Fyne dispatcher; moved file bookmarks suggest the current parent. Sandbox confirmation/cancel and local regressions pass; hosted review is pending. @@ -98,6 +103,13 @@ Preserve the full feature set. Active plan: [Mac App Store preparation](plans/20 ## Deferred +### Recursive scan work budget + +- [ ] Count examined files/directories and batch directory enumeration, including + rejected files and empty trees. Codex priority advice (PR #75 thread 4148307910) + treats this as deferrable availability hardening of user-selected trees; + image admission limits, cancellation and cycle protection already remain. +