Skip to content

Enforce HTTPS on curl downloads/redirects in calibration and test scripts (SonarCloud S6506) - #47

Open
devin-ai-integration[bot] wants to merge 1 commit into
developfrom
devin/1794421843-enforce-https-curl
Open

devin-ai-integration[bot] wants to merge 1 commit into
developfrom
devin/1794421843-enforce-https-curl

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes 17 SonarCloud shell:S6506 alerts. Each flagged curl -L call now passes --proto '=https' --proto-redir '=https', so neither the first request nor any redirect it follows can drop to plain HTTP or switch to another protocol. Every URL already uses https://github.com/NASA-AMMOS/VICAR/releases/download/..., and GitHub redirects those to https://release-assets.githubusercontent.com, so downloads work the same as before. No suppression comments, and no Dockerfiles were touched.

- curl -fsIL -o /dev/null "${BASE_URL}/${asset}"
+ curl --proto '=https' --proto-redir '=https' -fsIL -o /dev/null "${BASE_URL}/${asset}"

The two hint lines that test-release-regression.sh prints for manual downloads (L271/L278) now include the same flags. These lines were not flagged.

Alert File:line Fix
#36 fetch-calibration.sh:150 HTTPS-only flags on asset HEAD probe
#37 fetch-calibration.sh:155 HTTPS-only flags on multipart-suffix probe
#38 fetch-calibration.sh:164 HTTPS-only flags on Content-Length HEAD
#39 fetch-calibration.sh:281 HTTPS-only flags on resumable download
#40 fetch-calibration.sh:290 HTTPS-only flags on retry download
#10 test-calibration-demo.sh:106 HTTPS-only flags on calibration download
#11 test-calibration-demo.sh:119 HTTPS-only flags on sample-data download
#61 test-fullfeatured-image.sh:194 HTTPS-only flags on sample-data stream
#62 test-visor-image.sh:156 HTTPS-only flags on sample-data stream
#63 test-release-regression.sh:260 HTTPS-only flags on sample-data download
#64 test-release-regression.sh:263 HTTPS-only flags on calibration download
#55 visor/install-visor-calibration.sh:47 HTTPS-only flags on asset HEAD probe
#56 visor/install-visor-calibration.sh:52 HTTPS-only flags on multipart-suffix probe
#57 visor/install-visor-calibration.sh:72 HTTPS-only flags on part download
#58 visor/update-calibration-checksums.sh:37 HTTPS-only flags on asset HEAD probe
#59 visor/update-calibration-checksums.sh:41 HTTPS-only flags on multipart-suffix probe
#60 visor/update-calibration-checksums.sh:53 HTTPS-only flags on checksum stream

No alerts were left as false positives.

Verification

  • bash -n passes on all 7 changed scripts.
  • With the new flags, ./fetch-calibration.sh --list still finds and sizes every mission against the live 5.0 release. This covers the HEAD probe, the multipart-suffix probe and the Content-Length lookup (including the multipart m20 asset).
  • A HEAD request with the new flags follows the github.com redirect to https://release-assets.githubusercontent.com/... and gets 200. A body GET streams data as expected. The same flags make curl fail on an http:// URL.
  • --proto/--proto-redir need curl 7.20.2 or newer. Oracle Linux 8 ships 7.61.

Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/da07e7c14494495c8ff5a9a09d48ff69
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/da07e7c14494495c8ff5a9a09d48ff69?variant=devin
Requested by: @HuiJun


🟢 Devin Review found nothing to fix · View findings

Devin Review

…ipts

Add --proto '=https' --proto-redir '=https' to curl calls that follow
redirects (-L), so a redirect can never downgrade to plain HTTP or
another protocol. Fixes SonarCloud shell:S6506 alerts.

Co-Authored-By: jason.han <hanhuijun@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@github-actions

Copy link
Copy Markdown

Project Requirements: PASS

PR #47 adds --proto '=https' --proto-redir '=https' to every curl call in fetch-calibration.sh, the VISOR calibration install/checksum scripts, and four image/regression test scripts. All base URLs are hard-coded HTTPS GitHub release URLs (github.com/NASA-AMMOS/VICAR), and none of them can be overridden by users. That means the nominal behavior, arguments, outputs, and image contents stay the same. The only thing users can see change is the manual-fetch hint printed by test-release-regression.sh. Documentation parity: pass. Broad deployability: pass. Test coverage: concern, because no test checks that non-HTTPS URLs or redirects are refused, and fetch-calibration.sh is not exercised by any CI workflow.

Requirement Verdict Notes
1. Documentation parity Pass Not user-facing in any meaningful sense. The scripts keep the same arguments, outputs, product names, and image contents. Every download URL was already a fixed HTTPS GitHub URL that users cannot override, so the new flags only refuse transfers or redirects that are not HTTPS, which the published assets never use. The one visible text change is the remediation hint that test-release-regression.sh prints on failure, which now includes the --proto flags. The curl -L commands in docs/demos/downloading-visor-data.md, QUICKSTART.md, and the skills still work. The terrain-intelligence-generator/README.md wording ('hard failure carrying the exact curl command') is also still accurate. No doc update is needed.
2. Broad deployability Pass Generic hardening. The flags have been in curl since 7.20, so they work with the curl in Oracle Linux 8 (7.61), current Ubuntu, and macOS. No new dependencies, runtimes, hosts, or credentials are added. Proxies that users configure still work, because --proto restricts the transfer URL scheme, not the proxy scheme. URLs were already fixed public HTTPS GitHub release URLs, so no configurable HTTP or file:// mirror path gets broken.
3. Test coverage Concern This is security hardening with no change to nominal behavior. The changed test scripts and install-visor-calibration.sh are still run by CI: test-visor-image.sh and the visor Dockerfile build through build-publish-visor-variants.yml (paths visor/**), test-fullfeatured-image.sh through build-publish-fullfeatured.yml, test-calibration-demo.sh through calibration-demo.yml (weekly/manual), and test-release-regression.sh through release-visual-regression.yml (release only). Those runs confirm the HTTPS download path still works. However, nothing checks the new off-nominal behavior (a non-HTTPS URL or a redirect to HTTP being refused with a clear error), and fetch-calibration.sh itself is not run by any workflow. Not blocking: the change is small and mechanical, and the nominal paths are covered.

3. Test coverage

  • fetch-calibration.sh:150: No automated test checks that the new protocol restriction rejects a non-HTTPS URL or a redirect from HTTPS to HTTP. No CI workflow runs fetch-calibration.sh at all (calibration-demo.yml says nothing else in CI exercises the calibration path, and its test script calls curl directly). A regression that drops the flags, or an asset that redirects to HTTP, would go unnoticed. Fix: Add a small shell test, run from test.yml or a lightweight workflow, that points the curl helpers at an http:// URL (for example, by making BASE_URL overridable only for the test, or by checking a local HTTP server that redirects) and asserts a non-zero exit with a clear error message. At minimum, add a grep-style lint check that every curl call in these scripts includes --proto '=https' --proto-redir '=https'.
  • terrain-intelligence-generator/visor/install-visor-calibration.sh:47: The visor image build covers the nominal HTTPS path, but no off-nominal case checks that a redirect to a non-HTTPS location aborts the build with a clear message. Fix: Extend the visor image test, or add the shared curl-protocol test described above, so it also covers install-visor-calibration.sh and update-calibration-checksums.sh.

Evaluated at eb2cd06 · Devin session · workflow run

Criteria: .github/project-requirements/criteria.md. A fail blocks the merge; a maintainer can waive it by adding the requirements-waived label.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant