Skip to content

Pass residential proxy secret during deploy #60

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
protostatis merged 1 commit into from
Jun 1, 2026
Merged

Pass residential proxy secret during deploy #60

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 13 additions & 8 deletions .github/workflows/deploy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,26 +93,25 @@ jobs:

- name: Deploy to EC2
uses: appleboy/ssh-action@v1.0.3
env:

@protostatis protostatis Jun 1, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using env: + envs: together is the correct pattern for appleboy/ssh-action — env makes the variable available to the action process, and envs (comma-separated) tells the action which variables to forward to the remote SSH session. This looks correct.

RESIDENTIAL_PROXY: ${{ secrets.RESIDENTIAL_PROXY }}
with:
host: ${{ secrets.EC2_HOST }}
username: ${{ secrets.EC2_USERNAME }}
key: ${{ secrets.EC2_SSH_KEY }}
command_timeout: 20m
envs: RESIDENTIAL_PROXY
script: |
set -e

echo "========== DEPLOYMENT START: ${{ github.ref_name }} =========="

cd /home/ec2-user/crypto_sentiment_crawler

# If the host still has the legacy Reddit VPN enabled, disable it
# before the first GitHub/Docker network operation. GitHub fetches
# can fail through the old VPN path before the normal deploy setup
# gets a chance to stop wg-quick@wg0.
if [ -f /opt/crypto-sentiment/.env ] && grep -Eq '^(PROXY_URL|RESIDENTIAL_PROXY)=[^[:space:]]+' /opt/crypto-sentiment/.env; then
echo "Residential proxy configured; disabling legacy WireGuard before network operations."
sudo systemctl disable --now wg-quick@wg0 2>/dev/null || true
fi
# Disable the legacy Reddit VPN before the first GitHub/Docker
# network operation. This deploy path now uses an app-level proxy.
echo "Disabling legacy WireGuard before network operations."

@protostatis protostatis Jun 1, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously, WireGuard was only disabled if RESIDENTIAL_PROXY or PROXY_URL was set in /opt/crypto-sentiment/.env. Now it's unconditionally disabled on every deploy. This is a net improvement for reliability — the old check could silently fail if the env file was removed or drifted. Still worth noting in case a future operator expects the conditional behavior (e.g., if WireGuard is ever needed for something else on this host).

sudo systemctl disable --now wg-quick@wg0 2>/dev/null || true

# Pull latest code. Preserve any EC2-local edits so deployment can
# move to the release tag without discarding operational changes.
Expand All @@ -129,6 +128,11 @@ jobs:
sudo chown -R ec2-user:ec2-user /opt/crypto-sentiment

CRAWLER_ENV_ARGS=""
CRAWLER_SECRET_ENV_ARGS=""
if [ -n "$RESIDENTIAL_PROXY" ]; then

@protostatis protostatis Jun 1, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The echo confirms the secret's presence but doesn't leak the value — good. Consider omitting it entirely in deploy scripts to reduce noise and prevent accidental exposure in CI logs if someone later changes the echo.

echo "Residential proxy secret available for crawler."

@protostatis protostatis Jun 1, 2026

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Secrets passed via docker run -e VAR=value are visible in docker inspect, docker logs (occasionally), and the host's /proc/*/environ. On a shared EC2 instance, any user with SSH access can read this value. Consider writing it to a file with restrictive permissions (e.g., chmod 600) and using --env-file instead, or sourcing it via an existing env file that already has limited permissions. This is lower risk on a single-tenant server, but worth flagging.

CRAWLER_SECRET_ENV_ARGS="-e RESIDENTIAL_PROXY=$RESIDENTIAL_PROXY"
fi
if [ -f /opt/crypto-sentiment/.env ]; then
CRAWLER_ENV_ARGS="--env-file /opt/crypto-sentiment/.env"
if grep -Eq '^(PROXY_URL|RESIDENTIAL_PROXY)=[^[:space:]]+' /opt/crypto-sentiment/.env; then
Expand Down Expand Up @@ -236,6 +240,7 @@ jobs:
docker run -d --name crypto-crawler --restart unless-stopped \
--network crypto-sentiment_crypto-net \
$CRAWLER_ENV_ARGS \
$CRAWLER_SECRET_ENV_ARGS \
-v /opt/crypto-sentiment/data:/app/data \
-v /opt/crypto-sentiment/logs:/app/logs \
-e DATABASE_PATH=/app/data/sentiment.db \
Expand Down
Loading