Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .bumpy/land-pending-fixes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
varlock: patch
env-spec-language: patch
---

Fix enum/url/ip/md5/port coercion, imported @currentEnv (#428), and ServerResponse.end hang (#897)
20 changes: 7 additions & 13 deletions packages/varlock-website/src/content/docs/guides/import.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ Meaning if there was a value for `ITEM` in all 4 files, the final value used wou

### `environment flag "..." must be defined within this schema`

If you use [`@currentEnv`](/reference/root-decorators/#currentenv) to point at a variable (e.g. `# @currentEnv=$DEPLOY_ENV`) and that variable is only brought in via a **partial** `@import()`, varlock validates the env flag during schema initialization, before imported values are merged. The flag must be **defined in the same `.env.schema` file** that declares `@currentEnv`, not only in an imported file.
If you use [`@currentEnv`](/reference/root-decorators/#currentenv) to point at a variable (e.g. `# @currentEnv=$DEPLOY_ENV`), that item must either be defined in the same file or brought in by `@import`. A partial import must include the flag in `pick=[...]` (or not omit it).

This commonly appears in monorepos when a sub-package imports shared keys from a parent schema:

Expand All @@ -177,24 +177,18 @@ This commonly appears in monorepos when a sub-package imports shared keys from a
MY_SERVICE_URL=...
```

Running `varlock load` fails with:
That works: `DEPLOY_ENV` arrives via the import, and varlock uses it to load `.env.<env>` files after imports finish.

If the pick list omits the flag, `varlock load` fails with:

```txt
environment flag "DEPLOY_ENV" must be defined within this schema
environment flag "DEPLOY_ENV" must be defined within this schema or imported via @import
```

**Fixes:**

- Define the env flag locally in the file that uses `@currentEnv`, even if the value comes from elsewhere:

```env-spec title=".env.schema (sub-package)"
# @currentEnv=$DEPLOY_ENV
# @import(../../../, pick=[DEPLOY_ENV, AWS_REGION])
# ---
DEPLOY_ENV=
MY_SERVICE_URL=...
```

- Add the flag to the import filter: `pick=[DEPLOY_ENV, ...]`
- Define the env flag locally in the file that uses `@currentEnv`
- Move `@currentEnv` to the shared schema where the flag is already defined
- Import the full directory (omit the key list) if the sub-package should inherit the parent's `@currentEnv` handling

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,7 @@ To disable runtime log redaction, set the [`@redactLogs`](/reference/root-decora

_Only available in JavaScript/Node.js projects using varlock's runtime integrations._

Varlock scans outgoing HTTP responses at runtime to detect if any sensitive values are being accidentally sent to clients. If a leak is detected, varlock throws an error with a detailed diagnostic message including the config item key and where the leak was detected.
Varlock scans outgoing HTTP responses at runtime to detect if any sensitive values are being accidentally sent to clients. If a leak is detected, varlock throws an error with a detailed diagnostic message including the config item key and where the leak was detected. On `ServerResponse.end` (for example Next.js Pages Router `res.json()`), the response is finished with a 500 (or the socket is destroyed if headers were already sent) so the client is not left hanging.

This works by patching:
- **Node.js `ServerResponse`**: intercepts `write()` and `end()` calls, scanning text and JSON response bodies (including gzip-compressed responses)
Expand Down
19 changes: 14 additions & 5 deletions packages/varlock-website/src/content/docs/reference/data-types.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,7 @@ MY_BOOL=true
**Options:**
- `prependHttps` (boolean): Automatically prepend "https://" if no protocol is specified
- `allowedProtocols` (string[]): List of allowed protocols. Protocol names are case-insensitive and can include the trailing colon. If omitted, any valid URL protocol is allowed
- `allowedDomains` (string[]): List of allowed domains
- `allowedDomains` (string[] or comma-separated string): List of allowed hosts. A quoted comma-string (`allowedDomains="a.com,b.com"`) is treated as a host list, not a substring match
- `noTrailingSlash` (boolean): Disallow a trailing slash on the URL path (except root `/`)
- `matches` (string|RegExp): Regular expression pattern the full URL must match. Use `/pattern/flags` syntax or a quoted string pattern (see [regex-like strings](/reference/functions#regex-like-strings))

Expand Down Expand Up @@ -207,13 +207,18 @@ DB_HOST=10.0.3.12

<div>
### `enum`
Checks a value is contained in a list of possible values - it must match one exactly. Members can also be sourced from other items (see [Dynamic type options](#dynamic-type-options)).
Checks a value is contained in a list of possible values. It must match one exactly. Members can also be sourced from other items (see [Dynamic type options](#dynamic-type-options)).

`process.env` and `overrideValues` are always strings. Numeric and boolean members still match those overrides (`LEVEL=1`, `FLAG=true`) after coercion.

**NOTE** - this is the only type that cannot be used without any additional arguments

```env-spec
# @type=enum(development, staging, production)
ENV=development

# @type=enum(1, 2, 3)
LEVEL=2
```
</div>

Expand All @@ -230,7 +235,7 @@ MY_EMAIL=User@Example.com

<div>
### `port`
Checks for valid port number. Coerces to a number.
Checks for a valid integer port number (0-65535). Coerces to a number. Fractional values like `80.5` are rejected.

**Options:**
- `min` (number): Minimum port number (default: 0)
Expand All @@ -244,7 +249,7 @@ MY_PORT=3000

<div>
### `ip`
Checks for a valid [IP address](https://en.wikipedia.org/wiki/IP_address).
Checks for a valid [IP address](https://en.wikipedia.org/wiki/IP_address). IPv6 accepts IPv4-mapped addresses such as `::ffff:192.168.1.1`.

**Options:**
- `version` (`4|6`): IPv4 or IPv6
Expand All @@ -253,6 +258,9 @@ Checks for a valid [IP address](https://en.wikipedia.org/wiki/IP_address).
```env-spec
# @type=ip(version=4, normalize=true)
MY_IP=192.168.1.1

# @type=ip(version=6)
MAPPED=::ffff:192.168.1.1
```
</div>

Expand Down Expand Up @@ -286,7 +294,8 @@ MY_UUID=123e4567-e89b-12d3-a456-426614174000

<div>
### `md5`
Checks for valid [MD5 hash](https://en.wikipedia.org/wiki/MD5).
Checks for a valid [MD5 hash](https://en.wikipedia.org/wiki/MD5) (32 hex digits). Uppercase hex is accepted and normalized to lowercase.

```env-spec
# @type=md5
MY_HASH=d41d8cd98f00b204e9800998ecf8427e
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ and also may affect other dynamic behaviour in your schema, such as the [`forEnv

- It _must_ be set to a simple reference to a single config item (e.g. `$APP_ENV`).
- This decorator should only be set in your `.env.schema` file.
- The referenced item _must_ be defined within the same file.
- The referenced item must be defined in the same file, or brought in by `@import` (and included by any `pick`/`omit` filter on that import).
- This will override the `--env` CLI flag if it is set.
- We do not recommend using `NODE_ENV` as your environment flag, as it has other implications, and is often set out of your control.

Expand All @@ -43,6 +43,14 @@ See [environments guide](/guides/environments) for more info.
# @type=enum(dev, preview, prod, test)
APP_ENV=dev
```

In a monorepo, the flag can live in a shared schema and be imported:

```env-spec
# @currentEnv=$DEPLOY_ENV
# @import(../../../.env.schema, pick=[DEPLOY_ENV])
# ---
```
</div>

<div>
Expand Down
95 changes: 91 additions & 4 deletions packages/varlock/src/env-graph/lib/data-source.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,10 @@ import path from 'node:path';
import _ from '@env-spec/utils/my-dash';
import { tryCatch } from '@env-spec/utils/try-catch';
import {
ParsedEnvSpecArrayLiteral,
ParsedEnvSpecDecorator, ParsedEnvSpecDecoratorComment, ParsedEnvSpecFile,
ParsedEnvSpecFunctionCall, ParsedEnvSpecStaticValue, parseEnvSpecDotEnvFile,
ParsedEnvSpecFunctionCall, ParsedEnvSpecKeyValuePair, ParsedEnvSpecStaticValue,
parseEnvSpecDotEnvFile,
} from '@env-spec/parser';

import { ConfigItem, type ConfigItemDef } from './config-item';
Expand Down Expand Up @@ -35,6 +37,66 @@ export function keyPassesImportFilter(
return keyMatchesFilter(key, importFilter);
}

/**
* Peek unprocessed `@import(...)` decorators on `source` to see if any would statically
* bring in `key`. Used during finishInit so `@currentEnv=$FLAG` can reference a flag that
* arrives via import (imports are processed later).
*/
export function importStaticallyProvidesKey(source: EnvGraphDataSource, key: string): boolean {
for (const importDec of source.getRootDecFns('import')) {
const args = importDec.parsedDecorator.bareFnArgs;
if (!args) continue;

let enabled: boolean | 'dynamic' = true;
let pickPatterns: Array<string> | undefined;
let omitPatterns: Array<string> | undefined;
const positionalKeys: Array<string> = [];
let sawPath = false;

for (const arg of args.values) {
if (arg instanceof ParsedEnvSpecKeyValuePair) {
if (arg.key === 'enabled') {
if (arg.value instanceof ParsedEnvSpecStaticValue) {
enabled = arg.value.value === true;
} else {
enabled = 'dynamic';
}
} else if (arg.key === 'pick' && arg.value instanceof ParsedEnvSpecArrayLiteral) {
pickPatterns = arg.value.simplifiedValue.filter((v): v is string => typeof v === 'string' && !!v.trim());
} else if (arg.key === 'omit' && arg.value instanceof ParsedEnvSpecArrayLiteral) {
omitPatterns = arg.value.simplifiedValue.filter((v): v is string => typeof v === 'string' && !!v.trim());
}
} else if (arg instanceof ParsedEnvSpecStaticValue) {
// first positional arg is the import path; later ones are deprecated key allowlist
if (!sawPath) {
sawPath = true;
} else if (typeof arg.value === 'string' && arg.value.trim()) {
positionalKeys.push(arg.value.trim());
}
}
}

// static enabled=false: this import will not run
if (enabled === false) continue;

if (pickPatterns?.length) {
if (keyMatchesFilter(key, { mode: 'pick', patterns: pickPatterns })) return true;
continue;
}
if (omitPatterns?.length) {
if (keyMatchesFilter(key, { mode: 'omit', patterns: omitPatterns })) return true;
continue;
}
if (positionalKeys.length) {
if (positionalKeys.includes(key)) return true;
continue;
}
// full import (no pick/omit/positional filter) brings every key
return true;
}
return false;
}

const DATA_SOURCE_TYPES = Object.freeze({
schema: {
fileSuffixes: ['schema'],
Expand Down Expand Up @@ -317,6 +379,14 @@ export abstract class EnvGraphDataSource {
// For files, @currentEnv won't take effect and forEnv will fall back to parent's env setting
if (this.isPartialImport && !this.isKeyImported(envFlagItemKey)) {
skipCurrentEnvProcessing = true;
} else if (
// Flag arrives via @import later. Do not process ref() yet (it would SchemaError
// "invalid dependency" and mark this source invalid, which skips _processImports).
!this.configItemDefs[envFlagItemKey]
&& !isBuiltinVar(envFlagItemKey)
&& importStaticallyProvidesKey(this, envFlagItemKey)
) {
skipCurrentEnvProcessing = true;
}
}
}
Expand All @@ -342,8 +412,14 @@ export abstract class EnvGraphDataSource {
}

if (envFlagItemKey) {
if (!this.configItemDefs[envFlagItemKey] && !isBuiltinVar(envFlagItemKey)) {
this._errors.push(new LoadingError(`environment flag "${envFlagItemKey}" must be defined within this schema`));
const definedLocally = !!this.configItemDefs[envFlagItemKey] || isBuiltinVar(envFlagItemKey);
// Flag may arrive later via @import. Allow that without erroring or early-returning
// (early return used to skip @defaultSensitive processing and cascade into a crash).
const providedByImport = !definedLocally && importStaticallyProvidesKey(this, envFlagItemKey);
if (!definedLocally && !providedByImport) {
this._errors.push(new LoadingError(
`environment flag "${envFlagItemKey}" must be defined within this schema or imported via @import`,
));
return;
}

Expand All @@ -353,7 +429,7 @@ export abstract class EnvGraphDataSource {
}

// Always set the envFlagKey so parent directories can check it
// (even if we're skipping processing for a file partial import)
// (even if we're skipping processing for a file partial import, or waiting on @import)
this.setEnvFlag(envFlagItemKey);
}

Expand Down Expand Up @@ -965,6 +1041,9 @@ export class DirectoryDataSource extends EnvGraphDataSource {
if (!envFlagItem.resolvedValue) await envFlagItem.earlyResolve();
return { env: envFlagItem.resolvedValue?.toString(), fromFallback: false };
}
// Schema declared @currentEnv=$FLAG but FLAG is not in the graph yet (still waiting on
// @import). Do not fall back to parent/CLI env, or we would load the wrong .env.* files.
return { env: undefined, fromFallback: false };
}
// Fall back to parent chain or fallback value
const fromEnvFlagItem = !!this.envFlagConfigItem;
Expand Down Expand Up @@ -1029,6 +1108,14 @@ export class DirectoryDataSource extends EnvGraphDataSource {
for (const source of envSources) {
await source._processImports();
}
} else if (this.schemaDataSource?._envFlagKey) {
const envFlagKey = this.schemaDataSource._envFlagKey;
if (!this.graph.configSchema[envFlagKey]) {
this._errors.push(new LoadingError(
`environment flag "${envFlagKey}" was expected from @import but was not provided. `
+ 'Include it in pick=[...] (or omit filters), or define it in this schema.',
));
}
}
}
}
Expand Down
Loading
Loading