CONSIDER - YAML library replacement

[lostsnow]

Currently the yaml library go-yaml/yaml no longer appears to be under active development, which means that some issues may not be solved upstream:

• Is this (v3) repo still maintained? go-yaml/yaml#788
• Is this library still maintained? go-yaml/yaml#776

So there are a couple of options right now

• Keep using yaml.v2.
• Use yaml.v3 and follow the v3 format (this option might as well keep the yaml.v2 option in use now).
• Switch to another yaml library (may have to follow their format too), such as:
  • goccy/go-yaml
  • sigs.k8s.io/yaml

[Volte6]

goccy looks promising.

I think we need to evaluate what effects this will have and what would need to be changed/updated.

[Jasrags]

I’d suggest we use yaml.v3 if it’s compatible with the functionality we need. I've been using it in a few projects without issue. While upstream maintenance is a concern, v3 currently seems stable enough for typical use cases.

One way to future-proof ourselves would be to define an interface around our YAML functionality. This would allow us to isolate any future changes or library swaps behind a single implementation, making it easy to adapt if we decide to switch libraries later on.

Here’s a simple example of what that interface might look like:

type YAML interface {
    LoadFromFile(path string, out interface{}) error
    SaveToFile(path string, in interface{}) error
}

[lostsnow]

There is an additional problem, the formatting is not consistent from version to version or library to library, so the yaml files under _datafiles change a lot.

[Jasrags]

Given we have access to the current and new library it would not be too much effort to write a go program to read in and then rewrite the yaml files.

[lostsnow]

go-yaml/yaml#865

Yes, rewrite the Marshal() function is a way

[Jasrags]

I'm going to start digging into this to see the level of effort to switch and expected changes.

[Volte6]

V3 is no longer maintained either.

https://github.com/go-yaml/yaml?tab=readme-ov-file#this-project-is-unmaintained

[Jasrags]

Understood, will still dig into overall evaluation on what we should do going forward.

[Jasrags]

Here are some benchmarks

Sample Struct (NOTE: sigs_k8s_io_yaml uses the json tag for yaml parsing)

type SampleData struct {
	Name         string            `yaml:"name" json:"name"`
	Count        int               `yaml:"count" json:"count"`
	Tags         []string          `yaml:"tags" json:"tags"`
	Metadata     map[string]string `yaml:"metadata" json:"metadata"`
	Active       bool              `yaml:"active" json:"active"`
	FloatValue   float64           `yaml:"float_value" json:"float_value"`
	Int64Value   int64             `yaml:"int64_value" json:"int64_value"`
	UintValue    uint              `yaml:"uint_value" json:"uint_value"`
	ByteValue    byte              `yaml:"byte_value" json:"byte_value"`
	RuneValue    rune              `yaml:"rune_value" json:"rune_value"`
	EmbeddedData SampleEmbdedData  `yaml:"embedded_data" json:"embedded_data"`
}

type SampleEmbdedData struct {
	ID   string `yaml:"id" json:"id"`
	Name string `yaml:"name" json:"name"`
}

BenchmarkMarshal

goos: darwin
goarch: amd64
pkg: yamleval
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkMarshal/yaml_v2-12         	   61995	     18220 ns/op	   24693 B/op	      98 allocs/op
BenchmarkMarshal/yaml_v3-12         	   57420	     20769 ns/op	   37436 B/op	      99 allocs/op
BenchmarkMarshal/sigs_k8s_io_yaml-12         	   20882	     57766 ns/op	   81010 B/op	     330 allocs/op
BenchmarkMarshal/goccy_go-yaml-12            	   26550	     45611 ns/op	   29100 B/op	     767 allocs/op

BenchmarkUnmarshal

goos: darwin
goarch: amd64
pkg: yamleval
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkUnmarshal/yaml_v2-12         	   49386	     22921 ns/op	   11806 B/op	     185 allocs/op
BenchmarkUnmarshal/yaml_v3-12         	   39450	     30325 ns/op	   15480 B/op	     218 allocs/op
BenchmarkUnmarshal/sigs_k8s_io_yaml-12         	   28678	     40945 ns/op	   18688 B/op	     320 allocs/op
BenchmarkUnmarshal/goccy_go-yaml-12            	   19525	     61845 ns/op	   39045 B/op	     773 allocs/op

Output files

test_goccy_go-yaml.yaml

name: Example
count: 42
tags:
- alpha
- beta
metadata:
  env: production
  version: "1.0"
active: true
float_value: 3.14
int64_value: 1234567890123456789
uint_value: 42
byte_value: 255
rune_value: 65
embedded_data:
  id: "123"
  name: Embedded Example

test_sigs_k8s_io_yaml.yaml

active: true
byte_value: 255
count: 42
embedded_data:
  id: "123"
  name: Embedded Example
float_value: 3.14
int64_value: 1234567890123456789
metadata:
  env: production
  version: "1.0"
name: Example
rune_value: 65
tags:
- alpha
- beta
uint_value: 42

test_yaml_v2.yaml

name: Example
count: 42
tags:
- alpha
- beta
metadata:
  env: production
  version: "1.0"
active: true
float_value: 3.14
int64_value: 1234567890123456789
uint_value: 42
byte_value: 255
rune_value: 65
embedded_data:
  id: "123"
  name: Embedded Example

test_yaml_v3.yaml

name: Example
count: 42
tags:
    - alpha
    - beta
metadata:
    env: production
    version: "1.0"
active: true
float_value: 3.14
int64_value: 1234567890123456789
uint_value: 42
byte_value: 255
rune_value: 65
embedded_data:
    id: "123"
    name: Embedded Example

test_yaml_output.zip

[Jasrags]

Overall, I think a good next step would be to add the following interface and implementation of the v2 engine and move usage to that.

// YAMLEngine defines a common interface for YAML marshal/unmarshal engines.
type YAMLEngine interface {
	Name() string
	Marshal(v any) ([]byte, error)
	Unmarshal(data []byte, v any) error
}

// YAMLV2Engine adapts gopkg.in/yaml.v2 to YAMLEngine.
type YAMLV2Engine struct{}

func (e YAMLV2Engine) Name() string {
	return "yaml_v2"
}

func (e YAMLV2Engine) Marshal(v any) ([]byte, error) {
	return yamlv2.Marshal(v)
}

func (e YAMLV2Engine) Unmarshal(data []byte, v any) error {
	return yamlv2.Unmarshal(data, v)
}

[Volte6]

We need to understand how this will affect current yaml usage, especially around struct tags:

Supported/Used tags:

• 'omitempty'
• '-'
• 'flow'

Custom Tags:

• 'env'

Custom Marshal/Unmarshal

Also, how it will affect the requirements around how we layout the yaml - will the current yaml remain valid? Will it need to be mass-updated in some way? For example, are tabbing changes required? Do rules for 'empty' field detection change?

[Jasrags]

I'll add those examples into the tests and add the results along with diffs, overall it seems like minor spacing changes on list items.

[Jasrags]

diff --color data/test_yaml_v2.yaml data/test_goccy_go-yaml.yaml

no diff

diff --color data/test_yaml_v2.yaml data/test_sigs_k8s_io_yaml.yaml

1,8d0
< name: Example
< count: 42
< tags:
< - alpha
< - beta
< metadata:
<   env: production
<   version: "1.0"
10,12d1
< float_value: 3.14
< int64_value: 1234567890123456789
< uint_value: 42
14c3
< rune_value: 65
---
> count: 42
17a7
> float_value: 3.14
18a9,18
> int64_value: 1234567890123456789
> metadata:
>   env: production
>   version: "1.0"
> name: Example
> rune_value: 65
> tags:
> - alpha
> - beta
> uint_value: 42

diff --color data/test_yaml_v2.yaml data/test_yaml_v3.yaml

4,5c4,5
< - alpha
< - beta
---
>     - alpha
>     - beta
7,8c7,8
<   env: production
<   version: "1.0"
---
>     env: production
>     version: "1.0"
16,17c16,17
<   id: "123"
<   name: Embedded Example
---
>     id: "123"
>     name: Embedded Example