Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
95 commits
Select commit Hold shift + click to select a range
2c94bc1
mark external repo-policy-compliance configurations as deprecated
yanksyoon Oct 28, 2025
7ba3881
add allow-forked-repository charm configuration option
yanksyoon Oct 28, 2025
f19bbe7
add deprecated configuration option warning
yanksyoon Oct 28, 2025
8134a51
use type casting for bool charm configs
yanksyoon Oct 28, 2025
ef9b73c
change config option name to allow-external-contributor
yanksyoon Oct 28, 2025
d753dc7
parse allow-external-contributor charm config option
yanksyoon Oct 28, 2025
89f2bad
test: configuration parsing unit tests
yanksyoon Oct 28, 2025
4ffd599
fix: add use_aproxy default value
yanksyoon Oct 28, 2025
4ecc739
add allow_external_contributor configuration to application config
yanksyoon Oct 28, 2025
2498169
add deprecation warning for repo-policy-compliance-client
yanksyoon Oct 28, 2025
dc3f022
add allow_external_contributor option to application
yanksyoon Oct 28, 2025
1e10fff
test: initial forked repo testing
yanksyoon Oct 30, 2025
d1d4c52
debug
yanksyoon Oct 30, 2025
953fcb5
debug: ping me when test done
yanksyoon Oct 30, 2025
dc7c75d
Merge remote-tracking branch 'origin' into feat/pre-job-fork-check
yanksyoon Nov 18, 2025
f2c7378
feat: enhance integration tests for trusted and untrusted forked repo…
yanksyoon Nov 19, 2025
e2350d0
feat: update GitHub client initialization to use Token authentication
yanksyoon Nov 19, 2025
32f9818
chore: revert charm integration tests
yanksyoon Nov 20, 2025
1eea0f7
chore: revert charm integration tests
yanksyoon Nov 20, 2025
56a182e
chore: merge pre_job_contents_dict generation
yanksyoon Nov 20, 2025
596a559
feat: add GitHub and OpenStack configuration options for integration …
yanksyoon Nov 20, 2025
1e796ff
refactor: simplify configuration file reading in RunningApplication c…
yanksyoon Nov 20, 2025
9200a9b
chore: remove deprecated OpenStack ARM64 options from pytest configur…
yanksyoon Nov 20, 2025
81fa6ec
Merge remote-tracking branch 'origin' into feat/pre-job-fork-check
yanksyoon Nov 20, 2025
d4a3453
feat: add integration tests configuration to GitHub Actions and tox
yanksyoon Nov 20, 2025
7134402
chore: lint fixes
yanksyoon Nov 20, 2025
a850b00
refactor: streamline OpenStack options and remove deprecated token_al…
yanksyoon Nov 21, 2025
3ef26d0
chore: update integration test environment variables for OpenStack
yanksyoon Nov 21, 2025
5c3934b
Merge branch 'chore/cleanup-tests' into feat/pre-job-fork-check
yanksyoon Nov 21, 2025
f295341
chore: undo charm CI testing changes
yanksyoon Nov 21, 2025
97e063f
chore: refactor testing args parsing to upper level
yanksyoon Nov 21, 2025
f528f45
refactor: remove environment variable dependency for alternate GitHub…
yanksyoon Nov 21, 2025
2ffb626
chore: apply pflake8 rules to all tests
yanksyoon Nov 21, 2025
ada2718
Merge branch 'chore/cleanup-tests' into feat/pre-job-fork-check
yanksyoon Nov 21, 2025
0e7133d
chore: remove reference to unused env var
yanksyoon Nov 21, 2025
4b56fcd
Merge branch 'chore/cleanup-tests' into feat/pre-job-fork-check
yanksyoon Nov 21, 2025
eb7a72a
Merge branch 'main' into feat/pre-job-fork-check
yanksyoon Nov 26, 2025
03102bb
test: adapt tests to latest GH repository secrets
yanksyoon Nov 27, 2025
ebcffe6
chore: remove integration test requirements.txt
yanksyoon Nov 27, 2025
9e44423
chore: tidy integration test & fail if required params not supplied
yanksyoon Nov 27, 2025
3784ddb
test: update docstring
yanksyoon Nov 27, 2025
4a7030c
ci: install application package before testing
yanksyoon Nov 27, 2025
2f8b0cd
ci: test
yanksyoon Nov 27, 2025
ee2edee
ci: fix secrets mapping
yanksyoon Nov 27, 2025
8ed2ac2
ci: skip heavy testing for debug
yanksyoon Nov 27, 2025
54caba5
ci: debug
yanksyoon Nov 27, 2025
2cf25d9
fix: proxy configuration check
yanksyoon Nov 27, 2025
6c938ad
test: use proxies
yanksyoon Nov 27, 2025
23d0713
test: use proxies in fixtures
yanksyoon Nov 27, 2025
f40df95
test: use test configured flavor/image
yanksyoon Nov 27, 2025
ee73b59
test: application accept 204
yanksyoon Nov 27, 2025
aa7e39d
test: use configurable metrics log path
yanksyoon Nov 27, 2025
4114abe
test: prefer reactive configuration by default for tests
yanksyoon Nov 27, 2025
68f8de6
test: prefer non reactive configuration by default to save deploy com…
yanksyoon Nov 27, 2025
cf571e2
test: remove duplicate health checks (create app health check enabled)
yanksyoon Nov 27, 2025
f5cb83a
test: fix create PR from fork
yanksyoon Nov 27, 2025
ff22f45
test: fix PR base
yanksyoon Nov 27, 2025
d89fd73
test: stepped PR deployment
yanksyoon Nov 27, 2025
3f24503
refactor: remove unused dispatch workflow function and update tests t…
yanksyoon Nov 27, 2025
185dc80
ci: trigger
yanksyoon Nov 27, 2025
9220288
fix: PR base
yanksyoon Nov 27, 2025
376cf5f
test: origina/upstream/fork repository differentiation
yanksyoon Nov 27, 2025
16a5a0c
ci: trigger
yanksyoon Nov 27, 2025
3f3d3e5
test: fix use aproxy configuration by default
yanksyoon Nov 27, 2025
4a47297
test: get all logs from a run
yanksyoon Nov 27, 2025
39c1a3f
test: log cli output
yanksyoon Nov 27, 2025
3f76829
test: fix log assertion
yanksyoon Nov 27, 2025
2f8bbad
ci: remove debug
yanksyoon Nov 27, 2025
f2c46a4
ci: renable charm tests
yanksyoon Nov 27, 2025
2db2abe
fix: lint fixes
yanksyoon Nov 27, 2025
ecd2bda
Merge branch 'main' into feat/pre-job-fork-check
yanksyoon Nov 28, 2025
459b1e8
docs: update allow-external-contributor configuration name
yanksyoon Nov 28, 2025
00817b2
ci: revert debug test
yanksyoon Nov 28, 2025
6a3a4fc
Merge branch 'feat/pre-job-fork-check' of https://github.com/canonica…
yanksyoon Nov 28, 2025
be8ab20
test: revert subdir tests per-file-ignores settings
yanksyoon Nov 28, 2025
b5b39a9
test: fix syntax err
yanksyoon Nov 28, 2025
9ea9786
test: add openstack user/project domain name
yanksyoon Nov 28, 2025
85574bb
test: add openstack user/project domain name
yanksyoon Nov 28, 2025
fbeb9d6
test: add openstack user/project domain name
yanksyoon Nov 28, 2025
6d97fae
test: add openstack cleanup fixture
yanksyoon Nov 28, 2025
1e764f5
test: remove runtime hook
yanksyoon Nov 28, 2025
35fb1d2
Merge branch 'main' into feat/pre-job-fork-check
yanksyoon Nov 28, 2025
ee17ea1
Merge remote-tracking branch 'origin' into feat/pre-job-fork-check
yanksyoon Nov 30, 2025
461744d
feat: flush runners on external contributor config change
yanksyoon Nov 30, 2025
62ea95a
feat: add allow_external_contributor to stored state
yanksyoon Nov 30, 2025
33d49e6
chore: increment app version & docs: update changelog
yanksyoon Nov 30, 2025
c323b96
fix: use stored state
yanksyoon Dec 1, 2025
30dc913
fix: only check user association for pull_request events
yanksyoon Dec 1, 2025
1a1a037
chore: whitespace for concatenated string
yanksyoon Dec 1, 2025
dadbf41
revert removed whitespace from test fixture docstring
yanksyoon Dec 1, 2025
9886120
Merge branch 'main' into feat/pre-job-fork-check
yanksyoon Dec 1, 2025
f51a1f4
feat: check all events that can be triggered by external contributors
yanksyoon Dec 1, 2025
5697288
Merge branch 'main' into feat/pre-job-fork-check
yanksyoon Dec 3, 2025
b62b44f
Merge branch 'main' into feat/pre-job-fork-check
yanksyoon Dec 3, 2025
f3ecd9f
test: cover different event types
yanksyoon Dec 3, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 38 additions & 1 deletion .github/workflows/test_github_runner_manager.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,47 @@ on:
pull_request:

jobs:
unit-tests-github-runner-manager:
application-unit-tests:
name: Unit tests for github-runner-manager
uses: canonical/operator-workflows/.github/workflows/test.yaml@main
secrets: inherit
with:
self-hosted-runner: true
self-hosted-runner-label: edge
working-directory: ./github-runner-manager/
application-integration-tests:
name: Integration tests for github-runner-manager
runs-on: [self-hosted, pfe-ci]
steps:
- name: Checkout code
uses: actions/checkout@v5.0.1
- name: Setup Astral UV
uses: astral-sh/setup-uv@v7
- name: Install tox with UV
run: uv tool install tox --with tox-uv
- name: Run integration tests
working-directory: ./github-runner-manager/
env:
# GitHub configuration
INTEGRATION_TOKEN: ${{ secrets.INTEGRATION_TEST_SECRET_ENV_VALUE_1 }}
INTEGRATION_TOKEN_ALT: ${{ secrets.INTEGRATION_TEST_SECRET_ENV_VALUE_2 }}
# OpenStack configuration
OS_AUTH_URL: ${{ vars.INTEGRATION_TEST_OPENSTACK_AUTH_URL }}
OS_PROJECT_DOMAIN_NAME: ${{ vars.INTEGRATION_TEST_OPENSTACK_PROJECT_DOMAIN_NAME }}
OS_PROJECT_NAME: ${{ vars.INTEGRATION_TEST_OPENSTACK_PROJECT_NAME }}
OS_USER_DOMAIN_NAME: ${{ vars.INTEGRATION_TEST_OPENSTACK_USER_DOMAIN_NAME }}
OS_USERNAME: ${{ vars.INTEGRATION_TEST_OPENSTACK_USERNAME }}
OS_PASSWORD: ${{ secrets.INTEGRATION_TEST_SECRET_ENV_VALUE_3 }}
OS_NETWORK: ${{ vars.INTEGRATION_TEST_OPENSTACK_NETWORK_NAME }}
OS_REGION_NAME: ${{ vars.INTEGRATION_TEST_OPENSTACK_REGION_NAME }}
Comment thread
yanksyoon marked this conversation as resolved.
run: |
tox -e integration -- -v --tb=native -s \
--github-repository=${{ vars.INTEGRATION_TEST_PATH_2 }} \
--https-proxy="${{ vars.INTEGRATION_TEST_HTTPS_PROXY }}" \
--http-proxy="${{ vars.INTEGRATION_TEST_HTTP_PROXY }}" \
--no-proxy="${{ vars.INTEGRATION_TEST_NO_PROXY }}" \
--openstack-https-proxy="${{ vars.INTEGRATION_TEST_OPENSTACK_HTTPS_PROXY }}" \
--openstack-http-proxy="${{ vars.INTEGRATION_TEST_OPENSTACK_HTTP_PROXY }}" \
--openstack-no-proxy="${{ vars.INTEGRATION_TEST_OPENSTACK_NO_PROXY }}" \
--openstack-flavor="${{ vars.INTEGRATION_TEST_OPENSTACK_FLAVOR_NAME }}" \
--openstack-image-id="${{ vars.INTEGRATION_TEST_IMAGE_ID }}"
11 changes: 11 additions & 0 deletions config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -88,15 +88,26 @@ options:
repo-policy-compliance-token:
type: string
description: >-
DEPRECATED, please use allow-external-contributor configuration option instead.
The token to authenticate with the repository-policy-compliance service in order to
generate one-time-tokens. This option requires the repo-policy-compliance-url to be set.
If not set, the repository-policy-compliance service will not be used.
repo-policy-compliance-url:
type: string
description: >-
DEPRECATED, please use allow-external-contributor configuration option instead
The URL to the repository-policy-compliance service. This option requires the
repo-policy-compliance-token to be set. If not set, the repository-policy-compliance service
will not be used. This option is only supported when using OpenStack Cloud.
allow-external-contributor:
type: boolean
default: True
Comment thread
yanksyoon marked this conversation as resolved.
description: >-
EXPERIMENTAL
Whether to allow runs from forked repository from an external contributor. Enabling this
option will enable all runs from forked repositories. By default, runs from contribution
authors being in COLLABORATOR, MEMBER or OWNER status is allowed. See
https://docs.github.com/en/graphql/reference/enums#commentauthorassociation.
test-mode:
type: string
description: >-
Expand Down
5 changes: 5 additions & 0 deletions docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,11 @@ This changelog documents user-relevant changes to the GitHub runner charm.

- Added timeouts to API calls (Openstack,GitHub,Repo policy compliance) to fix a hanging GitHub runner manager application.

## 2025-11-30

- Added a new configuration option `allow-external-contributor` as an non-external replacement for
`repo-policy-compliance` service.

## 2025-11-21

- Updated broken links in `docs/how-to/index.md`.
Expand Down
2 changes: 1 addition & 1 deletion github-runner-manager/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

[project]
name = "github-runner-manager"
version = "0.7.0"
version = "0.8.0"
authors = [
{ name = "Canonical IS DevOps", email = "is-devops-team@canonical.com" },
]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,11 @@ class ApplicationConfiguration(BaseModel):
"""Main entry point for the Application Configuration.

Attributes:
allow_external_contributor: Whether to allow runs from forked repository from an external
contributor. Enabling this option will enable all runs from forked repositories. By
default, runs from contribution authors being in COLLABORATOR, MEMBER or OWNER status
is allowed. See \
https://docs.github.com/en/graphql/reference/enums#commentauthorassociation.
name: Name to identify the manager. Used for metrics.
extra_labels: Extra labels to add to the runner.
github_config: GitHub configuration.
Expand All @@ -48,6 +53,7 @@ class ApplicationConfiguration(BaseModel):
reconcile_interval: Seconds to wait between reconciliation.
"""

allow_external_contributor: bool = False
name: str
extra_labels: list[str]
github_config: github.GitHubConfiguration | None
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,7 @@ def build( # pylint: disable-msg=too-many-locals
base_quantity = combination.base_virtual_machines

openstack_runner_manager_config = OpenStackRunnerManagerConfig(
allow_external_contributor=application_configuration.allow_external_contributor,
prefix=application_configuration.openstack_configuration.vm_prefix,
credentials=application_configuration.openstack_configuration.credentials,
server_config=server_config,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

"""Models and functions for the metric events."""
import logging
import os
from pathlib import Path
from typing import Any, Optional

Expand All @@ -11,7 +12,7 @@
from github_runner_manager.errors import IssueMetricEventError
from github_runner_manager.manager.vm_manager import CodeInformation

METRICS_LOG_PATH = Path("/var/log/github-runner-metrics.log")
METRICS_LOG_PATH = Path(os.getenv("METRICS_LOG_PATH", "/var/log/github-runner-metrics.log"))


logger = logging.getLogger(__name__)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,15 @@ class OpenStackRunnerManagerConfig:
"""Configuration for OpenStack runner manager.

Attributes:
allow_external_contributor: Whether to allow runs from forked repository from an external
contributor.
prefix: The prefix of the runner names.
credentials: The OpenStack authorization information.
server_config: The configuration for OpenStack server.
service_config: The configuration for supporting services.
"""

allow_external_contributor: bool
prefix: str
credentials: OpenStackCredentials
server_config: OpenStackServerConfig | None
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,7 @@ def _generate_cloud_init(self, runner_context: RunnerContext) -> str:
"metrics_exchange_path": str(METRICS_EXCHANGE_PATH),
"do_repo_policy_check": False,
"custom_pre_job_script": service_config.custom_pre_job_script,
"allow_external_contributor": self._config.allow_external_contributor,
}
repo_policy = self._get_repo_policy_compliance_client()
if repo_policy is not None:
Expand All @@ -196,7 +197,10 @@ def _generate_cloud_init(self, runner_context: RunnerContext) -> str:
pre_job_contents = jinja.get_template("pre-job.j2").render(pre_job_contents_dict)

use_aproxy = service_config.use_aproxy
if not service_config.runner_proxy_config.proxy_address:
if (
not service_config.runner_proxy_config
or not service_config.runner_proxy_config.proxy_address
):
use_aproxy = False
aproxy_redirect_ports = service_config.aproxy_redirect_ports
if not aproxy_redirect_ports:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ def __init__(self, url: str, charm_token: str) -> None:
url: Base URL to the repo policy compliance service.
charm_token: Charm token configured for the repo policy compliance service.
"""
logger.warning(
"Repo policy compliance service is marked for deprecation. "
"Consider using `allow_external_contributor` option instead."
)
self._session = self._create_session()
self.base_url = url
self.token = charm_token
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ jq -n \
{% endif %}

{% if do_repo_policy_check %}
logger -s "Repo policy check is marked for deprecation. Consider using `allow_external_contributor` instead"

# Log common env variables.
logger -s "GITHUB_EVENT_NAME: ${GITHUB_EVENT_NAME}, \
Expand Down Expand Up @@ -132,6 +133,25 @@ jq -n \
logger -s "The repository setup check has passed, proceeding to execute jobs"
{% endif %}

{% if not allow_external_contributor %}
# Check author association for events that can be triggered by external contributors
# https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows
# This does not cover all possible events, only the most common ones that can be triggered by external contributors.
if [[ "${GITHUB_EVENT_NAME}" == "pull_request" || "${GITHUB_EVENT_NAME}" == "pull_request_target" || "${GITHUB_EVENT_NAME}" == "pull_request_review" || "${GITHUB_EVENT_NAME}" == "pull_request_review_comment" || "${GITHUB_EVENT_NAME}" == "issue_comment" ]]; then
AUTHOR_ASSOCIATION=$(jq -r '.pull_request.author_association // .issue.author_association // .review.author_association // .comment.author_association // empty' "$GITHUB_EVENT_PATH")
logger -s "Author association: $AUTHOR_ASSOCIATION"

if [[ "$AUTHOR_ASSOCIATION" != "OWNER" && "$AUTHOR_ASSOCIATION" != "MEMBER" && "$AUTHOR_ASSOCIATION" != "COLLABORATOR" ]]; then
logger -p user.error -s "Insufficient user authorization (author_association). Only OWNER, MEMBER, or COLLABORATOR may run workflows."
exit 1
fi

logger -s "The contributor check has passed, proceeding to execute jobs"
else
logger -s "Skipping contributor check for event: ${GITHUB_EVENT_NAME}"
fi
{% endif %}

if [[ -n "$DOCKERHUB_MIRROR" ]]; then
logger -s "A private docker registry is setup as a dockerhub mirror for this self-hosted runner."
logger -s "The docker daemon on this self-hosted runner is configured to use the dockerhub mirror."
Expand Down
116 changes: 115 additions & 1 deletion github-runner-manager/tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,125 @@

"""Fixtures for charm tests."""

import os


def pytest_addoption(parser):
"""Parse additional pytest options.

Args:
parser: Pytest parser.
"""
parser.addoption("--charm-file", action="store")
parser.addoption(
"--github-token",
action="store",
help="GitHub personal access token for integration tests.",
default=os.getenv("INTEGRATION_TOKEN"),
)
parser.addoption(
"--github-repository",
action="store",
help="The GitHub repository in <owner>/<repo> format for integration tests.",
)
parser.addoption(
"--github-token-alt",
action="store",
help="Alternate GitHub token from a different user for fork testing.",
default=os.getenv("INTEGRATION_TOKEN_ALT"),
)
parser.addoption(
"--openstack-auth-url",
action="store",
help="OpenStack authentication URL for integration tests.",
default=os.getenv("OS_AUTH_URL"),
)
parser.addoption(
"--openstack-project-domain-name",
action="store",
help="OpenStack project domain name for integration tests.",
default=os.getenv("OS_PROJECT_DOMAIN_NAME"),
)
parser.addoption(
"--openstack-project",
action="store",
help="OpenStack project name for integration tests.",
default=os.getenv("OS_PROJECT_NAME"),
)
parser.addoption(
"--openstack-user-domain-name",
action="store",
help="OpenStack user domain name for integration tests.",
default=os.getenv("OS_USER_DOMAIN_NAME"),
)
parser.addoption(
"--openstack-username",
action="store",
help="OpenStack username for integration tests.",
default=os.getenv("OS_USERNAME"),
)
parser.addoption(
"--openstack-password",
action="store",
help="OpenStack password for integration tests.",
default=os.getenv("OS_PASSWORD"),
)
parser.addoption(
"--openstack-network",
action="store",
help="OpenStack network name for integration tests.",
default=os.getenv("OS_NETWORK"),
)
parser.addoption(
"--openstack-region",
action="store",
help="OpenStack region name for integration tests.",
default=os.getenv("OS_REGION_NAME"),
)
parser.addoption(
"--https-proxy",
action="store",
help="HTTPS proxy for runner configuration.",
default=os.getenv("HTTPS_PROXY"),
)
parser.addoption(
"--http-proxy",
action="store",
help="HTTP proxy for runner configuration.",
default=os.getenv("HTTP_PROXY"),
)
parser.addoption(
"--no-proxy",
action="store",
help="No proxy configuration for runner.",
default=os.getenv("NO_PROXY"),
)
parser.addoption(
"--openstack-https-proxy",
action="store",
help="HTTPS proxy for OpenStack runner operations.",
default=os.getenv("OPENSTACK_HTTPS_PROXY"),
)
parser.addoption(
"--openstack-http-proxy",
action="store",
help="HTTP proxy for OpenStack runner operations.",
default=os.getenv("OPENSTACK_HTTP_PROXY"),
)
parser.addoption(
"--openstack-no-proxy",
action="store",
help="No proxy configuration for OpenStack runner operations.",
default=os.getenv("OPENSTACK_NO_PROXY"),
)
parser.addoption(
"--openstack-flavor",
action="store",
help="OpenStack flavor name for runner instances.",
default=os.getenv("OPENSTACK_FLAVOR"),
)
parser.addoption(
"--openstack-image-id",
action="store",
help="OpenStack image ID for runner instances.",
default=os.getenv("OPENSTACK_IMAGE_ID"),
)
2 changes: 2 additions & 0 deletions github-runner-manager/tests/integration/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Copyright 2025 Canonical Ltd.
# See LICENSE file for licensing details.
Loading
Loading