fix(config): say what type a key takes when its value will not parse (MAPCO-11617) - #34
Merged
Merged
Conversation
…1617) ErrType formatted its value with %t, the boolean verb, so every wrong-typed config value read "type %!t(int64=1) could not be converted". It now names the value and what it should have been: `1 (int64) is not a boolean (true or false)`. A string that does not parse as a bool, int, uint or float now returns the same error instead of strconv's text, which named a Go function rather than the config. The two ErrType sites in dict.go know their key, so they return dict.ErrKeyType like every other branch in that file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(MAPCO-11617) A value the TOML lexer cannot read, like `serve_layer_collections = xx`, is rejected before any type reaches our code, so the error could only say it "expected value". toml.ParseError records the key, though, and the key names a field of Config. Parse now walks Config through its toml tags to that field and appends what it takes: Near line 3 (last key parsed 'maps.serve_layer_collections'): expected value but found "xx" instead (maps.serve_layer_collections takes a boolean, true or false) The hint is read off the field's kind, so it cannot go stale as keys are added. It is given only for kinds that are unambiguous: env.URL is a struct written as a string and env.Dict is a table of anything, so structs, maps and arrays of tables stay silent, as does any type that decodes itself outside env. The result wraps the ParseError. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Name the singular/plural hint pair instead of indexing a [2]string, share the pointer-stripping loop, and describe every int, uint and float width rather than only the ones Config uses today. Scope the env-var test's Setenv to its own subtest, and drop a comment that claimed dict.go shares the want constants. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Coverage Report for CI Build 265Coverage increased (+0.6%) to 56.751%Details
Uncovered Changes
Coverage Regressions2 previously-covered lines in 1 file lost coverage.
Coverage Stats
💛 - Coveralls |
…APCO-11617)
v0.4.1 called UnmarshalTOML without the key it was decoding and returned
its error bare, so a bad quoted or wrong-typed value, or a missing ${VAR},
failed with a message that named no key. v1.6.0 wraps that error in a
ParseError carrying the key and the line:
toml: line 5 (last key "webserver.headers"): environment variable
"I_AM_MISSING" not found
Only config/ imports the package. Every .toml in the tree, and the
workspace config, decodes to identical JSON under both versions; configs
that failed still fail, now with a key and a line. v1 decodes in struct
order rather than map order, so a config missing several env vars may
report a different one first.
ParseError's own format changes from "Near line N (last key parsed 'k')"
to "toml: line N (last key \"k\")"; the two tests that pin it follow.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (MAPCO-11617)
With toml v1.6.0, a value that is valid TOML but wrong for its field
fails with the key, the line, and the env error naming the wanted type:
toml: line 1 (last key "tile_buffer"): "x" is not an integer
Appending "(tile_buffer takes an integer)" to that repeats it, and on a
missing ${VAR} it is beside the point. The hint now goes only on syntax
errors, the one path where nothing else says what the value should be.
Parse reads the config whole so it can tell them apart: a syntax error is
one plain TOML rejects too, checked only once decoding has failed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Jira: MAPCO-11617. Base is
development, the integration branch that shigola feature work targets.What and why
A mistyped config value used to fail in one of three ways. None of them said what the value should have been, and two of them did not name the key:
serve_layer_collections = xxNear line 3 (last key parsed 'maps.serve_layer_collections'): expected value but found "xx" insteadtoml: line 3 (last key "maps.serve_layer_collections"): expected value but found "xx" instead (maps.serve_layer_collections takes a boolean, true or false)= "xx"strconv.ParseBool: parsing "xx": invalid syntaxtoml: line 3 (last key "maps.serve_layer_collections"): "xx" is not a boolean (true or false)= 1type %!t(int64=1) could not be convertedtoml: line 3 (last key "maps.serve_layer_collections"): 1 (int64) is not a boolean (true or false)= "${NOPE}"environment variable "NOPE" not foundtoml: line 3 (last key "maps.serve_layer_collections"): environment variable "NOPE" not foundEvery failure now names the key and the line. A failure on a bad value also says what the value should have been.
Three separate problems were behind this:
github.com/BurntSushi/tomlv0.4.1 calledUnmarshalTOMLwithout the key and returned its error bare. So a quoted or wrong-typed value, or a missing${VAR}, failed with no key and no line. v1.6.0 wraps that error in aParseErrorcarrying both. That is the reason for the dependency bump, and the bump alone fixes the key.env.env.ErrTypeformatted its value with%t, the boolean verb. It now names the value and what it should have been. Strings that fail to parse as a bool, int, uint or float used to surfacestrconv's text, which names a Go function rather than anything in the config. They now get the same message.config.Parsenow takes the key fromParseError.LastKeyand walksConfigthrough itstomltags to that field, descending through arrays of tables such asmaps.layers.*. It then describes the field from itsreflect.Kind. The hint comes from the struct, so it cannot go stale as keys are added.The hint is added only to syntax errors. A decode-stage error already names the wanted type, so a hint there would just repeat it, and next to a missing
${VAR}it would be irrelevant.Parsetells the two kinds apart after decoding has failed, by checking whether plain TOML rejects the text too. The result wraps theParseError, soerrors.Asstill finds it.In
dict.go, the twoErrTypesites know their key. They now returndict.ErrKeyType, as every other branch in that file already does.The dependency bump: v0.4.1 → v1.6.0
Only
config/imports the package, so the change has a small footprint. It was checked in three ways:.tomlin the tree (the CITE config and theconfig/testdatafixtures) and the workspaceconfig.tomlwere parsed under both versions and compared as JSON. The configs that parse produce identical results. The configs that fail still fail, but now with a key and a line.ParseErrorformat. The message changes fromNear line N (last key parsed 'k')totoml: line N (last key "k"). The two tests that pin the format were updated. Nothing in production code reads these errors.govulncheck: 0 vulnerabilities affecting our code, and none in the toml module.It covers every scalar type, not just booleans
The ticket's examples are all booleans, but the change applies to every key. These cases are pinned by
config/config_type_errors_test.goandinternal/env/parse_errors_test.go.Syntax error: the hint added by
config.Parse[[maps.layers]]min_zoom = abc(maps.layers.min_zoom takes a non-negative integer)tile_buffer = xx(tile_buffer takes an integer)[webserver]port = xx(webserver.port takes a string)bounds = [1.0, x](maps.bounds takes an array of floating-point numbers)center = [1.0, x](maps.center takes an array of floating-point numbers)[tracing]enabled = yes(tracing.enabled takes a boolean, true or false)Valid TOML of the wrong type: the message after
toml: line N (last key "…"):tile_buffer = "x"strconv.Atoi: parsing "x": invalid syntaxtoml: line 1 (last key "tile_buffer"): "x" is not an integermin_zoom = "x"strconv.ParseUint: parsing "x": invalid syntaxtoml: line 4 (last key "maps.layers.min_zoom"): "x" is not a non-negative integer1for a stringtype %!t(int64=1) could not be converted… 1 (int64) is not a string1.5for an integertype %!t(float64=1.5) could not be converted… 1.5 (float64) is not an integer-1for an unsigned keytype %!t(int64=-1) could not be converted… -1 (int64) is not a non-negative integer1for a floattype %!t(int64=1) could not be converted… 1 (int64) is not a floating-point number (e.g. 1.0)"xx"for a floatstrconv.ParseFloat: parsing "xx": invalid syntax… "xx" is not a floating-point number (e.g. 1.0)1forhostnametype %!t(int64=1) could not be converted… 1 (int64) is not a URL stringA value that comes from
${VAR}is reported after substitution. For example,"${FOO}"withFOO=xxon a boolean key gives"xx" is not a boolean.Deliberately no hint. These cases still name the key and line, but get no type description:
hostname = xxenv.URLis a struct but is written as a string. Describing it by its kind would call it a table.[webserver.headers],[cache],[observer]or[[providers]]env.Dict: a table whose keys are not struct fields, and whose values can be anything.[[maps]]foo barLastKeyis the table itself, and there is nothing useful to say about an array of tables.nope = xx"5s")int64) would describe the wrong thing. Onlyenv's scalar types are trusted, because each accepts exactly its own kind or a${VAR}string. Pinned byconfig/type_hint_internal_test.go.Reading the diff
Read the commits in order:
fix(env):ErrTypeand theParse*functions ininternal/env.fix(config):config/type_hint.go(the walker) and its call site inconfig.Parse.refactor(config): fixes from the review. The only behaviour change is hints for the narrower int and float widths, which no current field uses.build(deps): the toml bump and thevendor/refresh. Most of the diff is here, and it is vendored code. Outsidevendor/, only the two tests that pin theParseErrorformat change.fix(config): the hint is limited to syntax errors, now that the decoder names the key.ParseBool,ParseInt,ParseUintandParseFloatnow return anilpointer instead of a pointer to the zero value when they fail. Every caller checkserrfirst, and all callers are ininternal/env.Parsenow reads the whole config withio.ReadAllbefore decoding. The file is small, and the v0.4.1 decoder already read it all internally.Verification
RUN_POSTGIS_TESTS=yes, run with bothgo test -race ./...andCGO_ENABLED=0 go test ./...: all green. The Redis tests were not run.gofmt -s -lon everything outsidevendor/: clean.govulncheck: see above./code-reviewwas run on the standards and spec axes, and its structural findings are addressed in commit 3.Left out on purpose
Parse*Slicehelpers still returnstrconverrors. Their only callers, indict.go, replace those errors withdict.ErrKeyType, so thestrconvtext never reaches an operator.ParseFloatrejects TOML integers.sample_ratio = 1fails and= 1.0works. This is existing behaviour. The new message says so (e.g. 1.0) but does not change it.shigola-docschange. No docs page quotes these messages.🤖 Generated with Claude Code