Skip to content

Implement IgnoreExtraFieldsFromTextual - #303

Merged
rockwotj merged 4 commits into
linkedin:masterfrom
passuied:feature/ignore-additional-fields
Jan 14, 2026
Merged

Implement IgnoreExtraFieldsFromTextual#303
rockwotj merged 4 commits into
linkedin:masterfrom
passuied:feature/ignore-additional-fields

Conversation

@passuied

@passuied passuied commented Jan 4, 2026

Copy link
Copy Markdown
Contributor

Description

  • Add ability to ignore extra fields from textual representation

  • Approach:

    1. A way to skip arbitrary JSON values without a codec
    2. An option to enable lenient behavior
    3. Passing this option through the codec builder chain
  • Usage example:

opt := &goavro.CodecOption{
    IgnoreExtraFieldsFromTextual: true,  // Ignore unknown fields during JSON decoding
}
codec, err := goavro.NewCodecWithOptions(schema, opt)

// Now this will work even with extra fields in JSON
native, _, err := codec.NativeFromTextual([]byte(`{"knownField": "value", "unknownField": "ignored"}`))

Issue Reference

#294

@passuied

passuied commented Jan 4, 2026

Copy link
Copy Markdown
Contributor Author

@rockwotj Re-implemented the solution. Please review

@passuied

Copy link
Copy Markdown
Contributor Author

@rockwotj @mihaitodor any chance you could look at this PR shortly?

@rockwotj rockwotj left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally LGTM, some nitpicks during review

Comment thread record_test.go
t.Fatalf("expected empty remaining buffer, got: %v", remaining)
}

m, ok := native.(map[string]interface{})

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prefer any in new code

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of the support for GO 1.12, we still need to keep interface{}. Or am I missing something?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should probably bump that, it's EOL

Comment thread json_utils.go Outdated
Comment on lines +166 to +168
if i >= len(buf) {
return nil, io.ErrShortBuffer
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this check isn't required right? the loop conditional should catch it?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point!

Comment thread json_utils.go
}

// skipJSONString skips a JSON string starting with '"' and returns the buffer after the closing '"'.
func skipJSONString(buf []byte) ([]byte, error) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's OK, but this doesn't validate the string

@rockwotj

Copy link
Copy Markdown
Collaborator

Merge conflicts @passuied

@passuied
passuied requested a review from rockwotj January 14, 2026 04:53
@passuied

Copy link
Copy Markdown
Contributor Author

Addressed your feedback @rockwotj please re-review

@rockwotj
rockwotj merged commit 36e223d into linkedin:master Jan 14, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants