You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
intro: 'Detect secret types specific to your organization with custom patterns.'
4
+
versions:
5
+
fpt: '*'
6
+
ghes: '*'
7
+
ghec: '*'
8
+
topics:
9
+
- Secret scanning
10
+
- Secret Protection
11
+
contentType: concepts
12
+
---
13
+
14
+
You can define custom patterns to identify secrets that are not detected by the default patterns supported by {% data variables.product.prodname_secret_scanning %}. For example, you might have a secret pattern that is internal to your organization. For a list of supported secrets and service providers, see [AUTOTITLE](/code-security/secret-scanning/introduction/supported-secret-scanning-patterns).
15
+
16
+
Custom patterns for {% data variables.product.prodname_secret_scanning %} are defined as regular expressions, and can be created at the enterprise, organization, or repository level. You can also enable push protection for custom patterns, stopping those secrets from ever reaching your repository.
17
+
18
+
## Next steps
19
+
20
+
To start using custom patterns, see [AUTOTITLE](/code-security/how-tos/secure-your-secrets/customize-leak-detection/defining-custom-patterns-for-secret-scanning).
Copy file name to clipboardExpand all lines: content/code-security/how-tos/secure-your-secrets/customize-leak-detection/defining-custom-patterns-for-secret-scanning.md
+4-62Lines changed: 4 additions & 62 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
---
2
2
title: Defining custom patterns for secret scanning
3
3
shortTitle: Define custom patterns
4
-
intro: You can define your own custom patterns to extend the capabilities of {% data variables.product.prodname_secret_scanning %} by generating one or more regular expressions.
4
+
intro: Protect your unique secret types by defining custom patterns with regular expressions.
5
5
product: '{% data reusables.gated-features.secret-scanning-custom-patterns %}'
6
6
permissions: '{% data reusables.permissions.security-enterprise-enable %}'
7
7
redirect_from:
@@ -19,38 +19,11 @@ topics:
19
19
- Secret scanning
20
20
---
21
21
22
-
## About custom patterns for {% data variables.product.prodname_secret_scanning %}
23
-
24
-
You can define custom patterns to identify secrets that are not detected by the default patterns supported by {% data variables.product.prodname_secret_scanning %}. For example, you might have a secret pattern that is internal to your organization. For details of the supported secrets and service providers, see [AUTOTITLE](/code-security/secret-scanning/introduction/supported-secret-scanning-patterns).
25
-
26
-
You can define custom patterns for your enterprise, organization, or repository. {% data variables.product.prodname_secret_scanning_caps %} supports up to 500 custom patterns for each organization or enterprise account, and up to 100 custom patterns per repository.
27
-
28
-
You can also enable push protection for custom patterns. For more information about push protection, see [AUTOTITLE](/code-security/secret-scanning/protecting-pushes-with-secret-scanning).
29
-
30
-
## About using regular expressions for custom patterns
31
-
32
-
You can specify custom patterns for {% data variables.product.prodname_secret_scanning %} as one or more regular expressions.
33
-
34
-
{% data variables.product.prodname_secret_scanning_caps %} uses the [Hyperscan library](https://github.com/intel/hyperscan) and only supports Hyperscan regex constructs, which are a subset of PCRE syntax. Hyperscan option modifiers are not supported. For more information on Hyperscan pattern constructs, see [Pattern support](http://intel.github.io/hyperscan/dev-reference/compilation.html#pattern-support) in the Hyperscan documentation.
35
-
36
-
{% ifversion secret-scanning-custom-pattern-ai-generated %}Regular expressions can be entered manually or generated using {% data variables.secret-scanning.copilot-secret-scanning %}'s {% data variables.secret-scanning.custom-pattern-regular-expression-generator %}.
The **More options {% octicon "chevron-down" aria-hidden="true" aria-label="chevron-down" %}** section in the UI helps you write regular expressions manually.
41
-
42
-
***Secret format:** an expression that describes the format of the secret itself.
43
-
***Before secret:** an expression that describes the characters that come before the secret. By default, this is set to `\A|[^0-9A-Za-z]` which means that the secret must be at the start of a line or be preceded by a non-alphanumeric character.
44
-
***After secret:** an expression that describes the characters that come after the secret. By default, this is set to `\z|[^0-9A-Za-z]` which means that the secret must be followed by a new line or a non-alphanumeric character.
45
-
***Additional match requirements:** one or more optional expressions that the secret itself must or must not match.
46
-
47
-
For simple tokens you will usually only need to specify a secret format. The other fields provide flexibility so that you can specify more complex secrets without creating complex regular expressions. For an example of a custom pattern, see [Example of a custom pattern specified using additional requirements](#example-of-a-custom-pattern-specified-using-additional-requirements) below.
### Using {% data variables.secret-scanning.copilot-secret-scanning %}'s {% data variables.secret-scanning.custom-pattern-regular-expression-generator %}
24
+
##Defining a custom pattern with {% data variables.product.prodname_copilot_short %}
52
25
53
-
{% data reusables.secret-scanning.regular-expression-generator-overview %} For more information, see [AUTOTITLE](/code-security/secret-scanning/copilot-secret-scanning/responsible-ai-regex-generator) and[AUTOTITLE](/code-security/secret-scanning/copilot-secret-scanning/generating-regular-expressions-for-custom-patterns-with-copilot-secret-scanning).
26
+
You can use {% data variables.secret-scanning.copilot-secret-scanning %} to generate regular expressions based on a text description of the type of pattern you would like to detect, including optional example strings that should be detected. See[AUTOTITLE](/code-security/secret-scanning/copilot-secret-scanning/generating-regular-expressions-for-custom-patterns-with-copilot-secret-scanning).
54
27
55
28
{% endif %}
56
29
@@ -68,44 +41,13 @@ Before defining a custom pattern, you must ensure that {% ifversion ghas-product
68
41
1. When you're ready to test your new custom pattern, to identify matches in the repository without creating alerts, click **Save and dry run**.
69
42
{% data reusables.advanced-security.secret-scanning-dry-run-results %}
70
43
{% data reusables.advanced-security.secret-scanning-create-custom-pattern %}
71
-
1. Optionally, to enable push protection for your custom pattern, click **Enable**.
44
+
1. Optionally, to enable push protection for your custom pattern, click **Enable**. For more information, see [AUTOTITLE](/code-security/secret-scanning/protecting-pushes-with-secret-scanning).
72
45
73
46
> [!NOTE]
74
47
> The "Enable" button isn't available until after the dry run succeeds and you publish the pattern.
75
48
76
-
For more information about push protection, see [AUTOTITLE](/code-security/secret-scanning/protecting-pushes-with-secret-scanning).
77
-
78
49
After your pattern is created, {% data reusables.secret-scanning.secret-scanning-process %} For more information on viewing {% data variables.secret-scanning.alerts %}, see [AUTOTITLE](/code-security/secret-scanning/managing-alerts-from-secret-scanning).
79
50
80
-
### Example of a custom pattern specified using additional requirements
81
-
82
-
A company has an internal token with five characteristics. They use the different fields to specify how to identify tokens as follows:
83
-
84
-
|**Characteristic**|**Field and regular expression**|
85
-
|----------------|------------------------------|
86
-
| Length between 5 and 10 characters | Secret format: `[$#%@AA-Za-z0-9]{5,10}`|
87
-
| Does not end in a `.`| After secret: `[^\.]`|
88
-
| Contains numbers and uppercase letters | Additional requirements: secret must match `[A-Z]` and `[0-9]`|
89
-
| Does not include more than one lowercase letter in a row | Additional requirements: secret must not match `[a-z]{2,}`|
90
-
| Contains one of `$%@!`| Additional requirements: secret must match `[$%@!]`|
91
-
92
-
These tokens would match the custom pattern described above:
93
-
94
-
```shell
95
-
a9@AAfT!# Secret string match: a9@AAfT
96
-
ee95GG@ZA942@aa # Secret string match: @ZA942@a
97
-
a9@AA!ee9 # Secret string match: a9@AA
98
-
```
99
-
100
-
These strings would not match the custom pattern described above:
101
-
102
-
```shell
103
-
a9@AA.!
104
-
a@AAAAA
105
-
aa9@AA!ee9
106
-
aAAAe9
107
-
```
108
-
109
51
## Defining a custom pattern for an organization
110
52
111
53
Before defining a custom pattern, you must ensure that you enable {% data variables.product.prodname_secret_scanning %} for the repositories that you want to scan in your organization. {% ifversion security-configurations %} You can use {% data variables.product.prodname_security_configurations %} to enable {% data variables.product.prodname_secret_scanning %} on all repositories in your organization. For more information, see [AUTOTITLE](/code-security/securing-your-organization/introduction-to-securing-your-organization-at-scale/about-enabling-security-features-at-scale).{% else %}
intro: Use specific regular expression syntax to define accurate custom patterns for {% data variables.product.prodname_secret_scanning %}.
5
+
versions:
6
+
fpt: '*'
7
+
ghec: '*'
8
+
ghes: '*'
9
+
topics:
10
+
- Secret scanning
11
+
- Secret Protection
12
+
contentType: reference
13
+
---
14
+
15
+
## Regular expression library for custom patterns
16
+
17
+
{% data variables.product.prodname_secret_scanning_caps %} custom patterns are defined using the [Hyperscan library](https://github.com/intel/hyperscan) and only support Hyperscan regex constructs, which are a subset of PCRE syntax. Hyperscan option modifiers are not supported. For more information on Hyperscan pattern constructs, see [Pattern support](https://intel.github.io/hyperscan/dev-reference/compilation.html#pattern-support) in the Hyperscan documentation.
18
+
19
+
## Syntax for manually defining custom patterns
20
+
21
+
The **More options {% octicon "chevron-down" aria-hidden="true" aria-label="chevron-down" %}** section in the UI helps you write regular expressions manually.
22
+
23
+
***Secret format:** an expression that describes the format of the secret itself.
24
+
***Before secret:** an expression that describes the characters that come before the secret. By default, this is set to `\A|[^0-9A-Za-z]` which means that the secret must be at the start of a line or be preceded by a non-alphanumeric character.
25
+
***After secret:** an expression that describes the characters that come after the secret. By default, this is set to `\z|[^0-9A-Za-z]` which means that the secret must be followed by a new line or a non-alphanumeric character.
26
+
***Additional match requirements:** one or more optional expressions that the secret itself must or must not match.
27
+
28
+
For simple tokens you will usually only need to specify a secret format. The other fields provide flexibility so that you can specify more complex secrets without creating complex regular expressions.
29
+
30
+
### Example custom pattern
31
+
32
+
A company has an internal token with five characteristics. They use the different fields to specify how to identify tokens as follows:
33
+
34
+
|**Characteristic**|**Field and regular expression**|
35
+
|----------------|------------------------------|
36
+
| Length between 5 and 10 characters | Secret format: `[$#%@AA-Za-z0-9]{5,10}`|
37
+
| Does not end in a `.`| After secret: `[^\.]`|
38
+
| Contains numbers and uppercase letters | Additional requirements: secret must match `[A-Z]` and `[0-9]`|
39
+
| Does not include more than one lowercase letter in a row | Additional requirements: secret must not match `[a-z]{2,}`|
40
+
| Contains one of `$%@!`| Additional requirements: secret must match `[$%@!]`|
41
+
42
+
These tokens would match the custom pattern described above:
43
+
44
+
```shell
45
+
a9@AAfT!# Secret string match: a9@AAfT
46
+
ee95GG@ZA942@aa # Secret string match: @ZA942@a
47
+
a9@AA!ee9 # Secret string match: a9@AA
48
+
```
49
+
50
+
These strings would not match the custom pattern described above:
51
+
52
+
```shell
53
+
a9@AA.!
54
+
a@AAAAA
55
+
aa9@AA!ee9
56
+
aAAAe9
57
+
```
58
+
59
+
## Limits
60
+
61
+
{% data variables.product.prodname_secret_scanning_caps %} supports up to 500 custom patterns for each organization or enterprise account, and up to 100 custom patterns per repository.
Copy file name to clipboardExpand all lines: data/reusables/advanced-security/secret-scanning-add-custom-pattern-details.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
1. Enter the details for your new custom pattern. You must at least provide the name for your pattern, and a regular expression for the format of your secret pattern.
2
2
1. In the "Pattern name" field, type a name for your pattern.
3
-
1. In the "Secret format" field, type a regular expression for the format of your secret pattern.{% ifversion secret-scanning-custom-pattern-ai-generated %} Alternatively, you can use the generator to generate a regular expression for you. For more information, see [AUTOTITLE](/code-security/secret-scanning/copilot-secret-scanning/generating-regular-expressions-for-custom-patterns-with-copilot-secret-scanning).{% endif %}
4
-
1. You can click **More options {% octicon "chevron-down" aria-hidden="true" aria-label="chevron-down" %}** to provide other surrounding content or additional match requirements for the secret format.
3
+
1. In the "Secret format" field, type a regular expression for the format of your secret pattern.
4
+
1. You can click **More options {% octicon "chevron-down" aria-hidden="true" aria-label="chevron-down" %}** to provide other surrounding content or additional match requirements for the secret format. See [AUTOTITLE](/code-security/reference/secret-security/custom-patterns#syntax-for-manually-defining-custom-patterns).
5
5
1. Provide a sample test string to make sure your configuration is matching the patterns you expect.
Custom patterns for {% data variables.product.prodname_secret_scanning %} is available for the following repository types:
2
-
3
-
{% ifversion fpt %}
4
-
* Organization-owned repositories on {% data variables.product.prodname_team %} with [{% data variables.product.prodname_GH_secret_protection %}](/get-started/learning-about-github/about-github-advanced-security) enabled{% endif %}
5
-
6
-
{% ifversion ghec %}
7
-
* Organization-owned repositories on {% data variables.product.prodname_team %} or {% data variables.product.prodname_ghe_cloud %} with [{% data variables.product.prodname_GH_secret_protection %}](/get-started/learning-about-github/about-github-advanced-security) enabled{% endif %}
8
-
9
-
{% ifversion ghes %}
10
-
* Organization-owned repositories with [{% data variables.product.prodname_GH_secret_protection %}](/get-started/learning-about-github/about-github-advanced-security) enabled{% endif %}
1
+
Organization-owned repositories on {% data variables.product.prodname_team %} or {% data variables.product.prodname_enterprise %} with [{% data variables.product.prodname_GH_secret_protection %}](/get-started/learning-about-github/about-github-advanced-security) enabled
> * To enable push protection for custom patterns, {% data variables.product.prodname_secret_scanning %} as push protection needs to be enabled at the enterprise level. For more information, see [AUTOTITLE](/code-security/secret-scanning/protecting-pushes-with-secret-scanning#enabling-secret-scanning-as-a-push-protection-for-your-enterprise).
2
+
> * To enable push protection for custom patterns, {% data variables.product.prodname_secret_scanning %} as push protection needs to be enabled at the enterprise level.
3
3
> * Enabling push protection for commonly found custom patterns can be disruptive to contributors.
> * The option to enable push protection is visible for published patterns only.
3
-
> * Push protection for custom patterns will only apply to repositories in your organization that have {% data variables.product.prodname_secret_scanning %} as push protection enabled. For more information, see [AUTOTITLE](/code-security/secret-scanning/protecting-pushes-with-secret-scanning#enabling-secret-scanning-as-a-push-protection-for-an-organization).
3
+
> * Push protection for custom patterns will only apply to repositories in your organization that have {% data variables.product.prodname_secret_scanning %} as push protection enabled.
4
4
> * Enabling push protection for commonly found custom patterns can be disruptive to contributors.
0 commit comments