Skip to content

Commit c39a892

Browse files
kpaulissekpaulisse
authored
Merge pull request #18 from github/kpaulisse-octocatalog-diff-0-5-4
octocatalog-diff 0.5.4
2 parents 8e14d69 + 48f5040 commit c39a892

27 files changed

Lines changed: 703 additions & 162 deletions

File tree

‎.version‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
0.5.3
1+
0.5.4

‎doc/CHANGELOG.md‎

Lines changed: 38 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,40 @@
11
# octocatalog-diff change log
22

3-
| Version | Date | Description / Changes |
4-
| ------- | ---- | ----------- |
5-
| 0.5.3 | 2016-10-31 | https://github.com/github/octocatalog-diff/pull/10: facts terminus optimization |
6-
| 0.5.2 | - | Unreleased internal version |
7-
| 0.5.1 | 2016-10-20 | Initial release |
3+
<table><thead>
4+
<tr>
5+
<th>Version</th>
6+
<th>Date</th>
7+
<th>Description / Changes</th>
8+
</tr>
9+
</thead><tbody>
10+
<tr valign=top>
11+
<td>0.5.4</td>
12+
<td>2016-11-07</td>
13+
<td>
14+
<ul>
15+
<li><a href="https://github.com/github/octocatalog-diff/pull/16">https://github.com/github/octocatalog-diff/pull/16</a>: environment running <code>puppet --version</code></li>
16+
<li><a href="https://github.com/github/octocatalog-diff/pull/5">https://github.com/github/octocatalog-diff/pull/5</a>: bootstrap debugging</li>
17+
<li><a href="https://github.com/github/octocatalog-diff/pull/17">https://github.com/github/octocatalog-diff/pull/17</a>: hiera simplification and <code>--hiera-path</code> option</li>
18+
</ul>
19+
</td>
20+
</tr>
21+
<tr valign=top>
22+
<td>0.5.3</td>
23+
<td>2016-10-31</td>
24+
<td>
25+
<ul>
26+
<li><a href="https://github.com/github/octocatalog-diff/pull/10">https://github.com/github/octocatalog-diff/pull/10</a>: facts terminus optimization</li>
27+
</ul>
28+
</td>
29+
</tr>
30+
<tr valign=top>
31+
<td>0.5.2</td>
32+
<td>-</td>
33+
<td>Unreleased internal version</td>
34+
</tr>
35+
<tr valign=top>
36+
<td>0.5.1</td>
37+
<td>2016-10-20</td>
38+
<td>Initial release</td>
39+
</tr>
40+
</tbody></table>

‎doc/advanced-hiera-path-stripping.md‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Configuring octocatalog-diff to use Hiera path stripping
2+
3+
This is a different, and potentially more complex, alternative to `--hiera-path` / `settings[:hiera_path]` described in the [Configuring octocatalog-diff to use Hiera](/doc/configuration-hiera.md) document. Unless you have a very good reason, you should prefer to use the instructions in that document instead of using the more complicated option that is described herein.
4+
5+
The command line option `--hiera-path-strip PATH` allows you to manipulate directory paths for the JSON or YAML hiera backends. This setting only has an effect on the copy of hiera.yaml that is copied into the temporary compilation directory. This does not make any changes to the actual source hiera.yaml file on your system or in your checkout.
6+
7+
For example, perhaps your production hiera.yaml file has entries such as the following:
8+
9+
```
10+
---
11+
:backends:
12+
- yaml
13+
:hierarchy:
14+
- "nodes/%{::trusted.certname}"
15+
- common
16+
17+
:yaml:
18+
:datadir: /etc/puppetlabs/code/environments/%{environment}/hieradata
19+
```
20+
21+
However, when you run octocatalog-diff on a machine that is not a Puppet master, the hiera data will not actually be found in `/etc/puppetlabs/code/environments/production/hieradata`, but rather in a directory called `hieradata` relative to the checkout of your Puppet code.
22+
23+
Specifying `--hiera-path-strip PATH` causes octocatalog-diff to munge the datadir for the YAML and JSON configuration. The correct command in this case is now:
24+
25+
```
26+
bin/octocatalog-diff --hiera-config hiera.yaml --hiera-path-strip /etc/puppetlabs/code
27+
```
28+
29+
```
30+
---
31+
:backends:
32+
- yaml
33+
:hierarchy:
34+
- "nodes/%{::trusted.certname}"
35+
- common
36+
37+
:yaml:
38+
:datadir: /var/tmp/puppet-compile-dir-92347829847/environments/%{environment}/hieradata
39+
```

‎doc/configuration-hiera.md‎

Lines changed: 37 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Before you start, please understand how octocatalog-diff compiles a catalog:
1010
- It compiles the catalog, based on the temporary directory, for environment=production
1111
- It removes the temporary directory
1212

13-
## Configuring the path to hiera.yaml
13+
## Configuring the location of hiera.yaml
1414

1515
The command line option `--hiera-config PATH` allows you to set the path to hiera.yaml.
1616

@@ -21,22 +21,26 @@ You may specify this as either an absolute or a relative path.
2121
octocatalog-diff knows to use a relative path when the supplied path for `--hiera-config` does not start with a `/`.
2222

2323
```
24-
bin/octocatalog-diff --hiera-config config/hiera.yaml ...
24+
bin/octocatalog-diff --hiera-config hiera.yaml ...
2525
```
2626
27-
The path is relative to a checkout of your Puppet repository. As per the example in the introduction, say that octocatalog-diff is using a temporary directory of `/var/tmp/puppet-compile-dir-92347829847` when compiling a Puppet catalog. With the setting above, it will copy `config/hiera.yaml` (relative to your Puppet checkout) into the temporary directory.
27+
The path is relative to a checkout of your Puppet repository. With the setting above, it will use the file named `hiera.yaml` that is at the top level
28+
of your Puppet checkout.
2829
29-
If you use Puppet to manage your hiera.yaml file on Puppet masters, perhaps it is found in one of the modules in your code. In that case, you may use syntax like:
30+
Perhaps your hiera.yaml file is in a subdirectory of your Puppet checkout. In that case, just use the relative directory path. Be sure not to add a leading `/` though,
31+
because you don't want octocatalog-diff to treat it as an absolute path. In the following example, suppose you have a top level directory called `config` and your
32+
`hiera.yaml` file is contained within it. You could then use:
3033
3134
```
32-
bin/octocatalog-diff --hiera-config modules/puppet/files/hiera.yaml ...
35+
bin/octocatalog-diff --hiera-config config/hiera.yaml ...
3336
```
3437
3538
If you are specifying the hiera.yaml path in the [configuration file](/doc/configuration.md), you will instead set the variable like this:
3639
3740
```
41+
settings[:hiera_config] = 'hiera.yaml'
42+
(or)
3843
settings[:hiera_config] = 'config/hiera.yaml'
39-
settings[:hiera_config] = 'modules/puppet/files/hiera.yaml'
4044
```
4145
4246
octocatalog-diff will fail if you specify a hiera configuration location that cannot be opened.
@@ -57,47 +61,35 @@ You may specify this as either an absolute or a relative path.
5761
settings[:hiera_config] = '/etc/puppetlabs/puppet/hiera.yaml'
5862
```
5963
60-
Please note that octocatalog-diff will copy the file from the specified location into the compile directory. Since this hiera.yaml file is not copied from your Puppet repo, there is no way to compile the "to" and "from" branches using different hiera.yaml files. Furthermore, you are responsible for getting this file into place on any machine that will run octocatalog-diff.
64+
Please note that octocatalog-diff will copy the file from the specified location into the compile directory. Since this hiera.yaml file is not copied from your Puppet repo, there is no way to compile the "to" and "from" branches using different hiera.yaml files. Furthermore, you are responsible for getting this file into place on any machine that will run octocatalog-diff. An absolute path may make octocatalog-diff work correctly on your Puppet master servers, but the structure may differ on other machines where you wish to run the utility.
6165
6266
We strongly recommend that you version-control your hiera.yaml file within your Puppet repository, and use the relative path option described above.
6367
68+
## Configuring the directory in your repository in which hiera data files are found
69+
70+
The command line option `--hiera-path PATH` allows you to set the directory path, relative to the checkout of your Puppet repository, of your Hiera YAML/JSON data files.
71+
72+
If you are using the out-of-the-box Puppet Enterprise configuration, or the [Puppet Control Repo template](https://github.com/puppetlabs/control-repo), then the correct setting here is simply 'hieradata'.
73+
74+
You must specify this as a relative path. octocatalog-diff knows to use a relative path when the supplied path for `--hiera-path` does not start with a `/`.
75+
76+
```
77+
bin/octocatalog-diff --hiera-path hieradata ...
78+
```
79+
80+
The path is relative to a checkout of your Puppet repository. With the setting above, it will look for Hiera data in a directory called `hieradata` that is at the top level
81+
of your Puppet checkout.
82+
83+
If you are specifying the Hiera data path in the [configuration file](/doc/configuration.md), you will instead set the variable like this:
84+
85+
```
86+
settings[:hiera_path] = 'hieradata'
87+
```
88+
89+
octocatalog-diff will fail if you specify a path that is not a directory.
90+
6491
## Configuring the prefix path to strip
6592
66-
The command line option `--hiera-path-strip PATH` allows you to manipulate directory paths for the JSON or YAML hiera backends. This setting only has an effect on the copy of hiera.yaml that is copied into the temporary compilation directory. This does not make any changes to the actual source hiera.yaml file on your system or in your checkout.
67-
68-
For example, perhaps your production hiera.yaml file has entries such as the following:
69-
70-
```
71-
:backends:
72-
- yaml
73-
:yaml:
74-
:datadir: /var/lib/puppet/environments/%{::environment}/hieradata
75-
:hierarchy:
76-
- servers/%{::fqdn}
77-
- platform/%{::virtual}
78-
- datacenter/%{::datacenter}
79-
- os/%{::operatingsystem}
80-
- common
81-
```
82-
83-
However, when you run octocatalog-diff, the hiera data will not actually be found in `/var/lib/puppet/environments/production/hieradata`, but rather in a directory called `environments/production/hieradata` relative to the checkout of your Puppet code.
84-
85-
Specifying `--hiera-path-strip PATH` causes octocatalog-diff will rewrite the datadir for the YAML and JSON configuration. In the example above, the correct setting is `--hiera-path-strip /var/lib/puppet`, which will result in the following configuration in the hiera.yaml file:
86-
87-
```
88-
bin/octocatalog-diff --hiera-config environments/production/config/hiera.yaml --hiera-path-strip /var/lib/puppet
89-
```
90-
91-
```
92-
# This is the temporary hiera.yaml file used for octocatalog-diff catalog compilation
93-
:backends:
94-
- yaml
95-
:yaml:
96-
:datadir: /var/tmp/puppet-compile-dir-92347829847/environments/%{::environment}/hieradata
97-
:hierarchy:
98-
- servers/%{::fqdn}
99-
- platform/%{::virtual}
100-
- datacenter/%{::datacenter}
101-
- os/%{::operatingsystem}
102-
- common
103-
```
93+
This is a different, and potentially more complex, alternative to `--hiera-path` / `settings[:hiera_path]` described in the prior section. Unless you have a very good reason, you should prefer to use the instructions above.
94+
95+
If you need to use the more complex path strip alternative, see: [Configuring octocatalog-diff to use Hiera path stripping](/doc/advanced-hiera-path-stripping.md).

‎examples/octocatalog-diff.cfg.rb‎

Lines changed: 26 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -34,21 +34,37 @@ def self.config
3434
##############################################################################################
3535

3636
# settings[:hiera_config] = '/etc/puppetlabs/puppet/hiera.yaml' # Absolute path
37-
# settings[:hiera_config] = 'environments/production/config/hiera.yaml' # Relative path
37+
settings[:hiera_config] = 'hiera.yaml' # Relative path, assumes hiera.yaml at top of repo
3838

3939
##############################################################################################
40+
# hiera_path
4041
# hiera_path_strip
41-
# Portion of the `:datadir:` to strip (used for JSON and YAML data sources). For
42-
# example, perhaps your hiera.yaml file contains this code:
43-
# :yaml:
44-
# :datadir: /var/lib/puppet/environments/%{::environment}/hieradata
45-
# In this case, you desire to strip `/var/lib/puppet` from the beginning of the path,
46-
# in order that octocatalog-diff can find your hiera datafiles in the compilation
47-
# location, which is {temporary directory}/environments/production/hieradata.
48-
# More: https://github.com/github/octocatalog-diff/blob/master/doc/configuration-hiera.md
42+
# These control the setup of the 'datadir' when you are using the JSON or YAML data source.
43+
# There are two ways to configure this setting - do one or the other but not both.
44+
#
45+
# 1. (EASIEST METHOD)
46+
# You can specify the path to the hieradata relative to the checkout of your Puppet repo.
47+
# This may be the most straightforward to configure. For example, if your Hiera data YAML
48+
# and JSON files are found under a `hieradata` directory in the top level of your Puppet
49+
# repo, simply set `settings[:hiera_path] = 'hieradata'` and you're done!
50+
#
51+
# 2. (MORE COMPLEX METHOD)
52+
# You can specify a string that will be stripped off the existing defined data directory
53+
# in the hiera.yaml file. For example, perhaps your hiera.yaml file contains this code:
54+
# :yaml:
55+
# :datadir: /etc/puppetlabs/code/environments/%{environment}/hieradata
56+
# In this case, you desire to strip `/etc/puppetlabs/code` from the beginning of the path,
57+
# in order that octocatalog-diff can find your hiera datafiles in the compilation
58+
# location, which is {temporary directory}/environments/production/hieradata.
59+
#
60+
# More: https://github.com/github/octocatalog-diff/blob/master/doc/configuration-hiera.md
4961
##############################################################################################
5062

51-
# settings[:hiera_path_strip] = '/var/lib/puppet'
63+
# This should work out-of-the-box with a default Puppet Enterprise or Puppet Control Repo setup.
64+
settings[:hiera_path] = 'hieradata'
65+
66+
# If you want to use the 'strip' method described above, this may work.
67+
# settings[:hiera_path_strip] = '/etc/puppetlabs/code'
5268

5369
##############################################################################################
5470
# puppetdb_url

‎lib/octocatalog-diff/catalog-diff/cli.rb‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,8 @@ class Cli
3939
compare_file_text: true,
4040
display_datatype_changes: true,
4141
parallel: true,
42-
suppress_absent_file_details: true
42+
suppress_absent_file_details: true,
43+
hiera_path: 'hieradata'
4344
}.freeze
4445

4546
# This method is the one to call externally. It is possible to specify alternate
@@ -75,6 +76,8 @@ def self.cli(argv = ARGV, logger = Logger.new(STDERR), opts = {})
7576
# Note: do NOT use 'options[k] ||= v' here because if the value of options[k] is boolean(false)
7677
# it will then be overridden. Whereas the intent is to define values only for those keys that don't exist.
7778
DEFAULT_OPTIONS.each { |k, v| options[k] = v unless options.key?(k) }
79+
veto_with_none_options = %w(hiera_path hiera_path_strip)
80+
veto_with_none_options.each { |x| options.delete(x.to_sym) if options[x.to_sym] == :none }
7881

7982
# Fact overrides come in here - 'options' is modified
8083
setup_fact_overrides(options)

‎lib/octocatalog-diff/catalog-diff/cli/options/bootstrap_current.rb‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Option to bootstrap the current directory (by default, the bootstrap script is NOT
2+
# run when the catalog builds in the current directory).
3+
# @param parser [OptionParser object] The OptionParser argument
4+
# @param options [Hash] Options hash being constructed; this is modified in this method.
5+
OctocatalogDiff::CatalogDiff::Cli::Options::Option.newoption(:bootstrap_current) do
6+
has_weight 48
7+
8+
def parse(parser, options)
9+
parser.on('--bootstrap-current', 'Run bootstrap script for the current directory too') do
10+
options[:bootstrap_current] = true
11+
end
12+
end
13+
end

‎lib/octocatalog-diff/catalog-diff/cli/options/debug_bootstrap.rb‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# Option to print debugging output for the bootstrap script in addition to the normal
2+
# debugging output. Note that `--debug` must also be enabled for this option to have
3+
# any effect.
4+
# @param parser [OptionParser object] The OptionParser argument
5+
# @param options [Hash] Options hash being constructed; this is modified in this method.
6+
OctocatalogDiff::CatalogDiff::Cli::Options::Option.newoption(:debug_bootstrap) do
7+
has_weight 49
8+
9+
def parse(parser, options)
10+
parser.on('--debug-bootstrap', 'Print debugging output for bootstrap script') do
11+
options[:debug_bootstrap] = true
12+
end
13+
end
14+
end

‎lib/octocatalog-diff/catalog-diff/cli/options/hiera_path.rb‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Specify the path to the Hiera data directory (relative to the top level Puppet checkout). For Puppet Enterprise and the
2+
# Puppet control repo template, the value of this should be 'hieradata', which is the default.
3+
# @param parser [OptionParser object] The OptionParser argument
4+
# @param options [Hash] Options hash being constructed; this is modified in this method.
5+
OctocatalogDiff::CatalogDiff::Cli::Options::Option.newoption(:hiera_path) do
6+
has_weight 181
7+
8+
def parse(parser, options)
9+
parser.on('--hiera-path PATH', 'Path to hiera data directory, relative to top directory of repository') do |path_in|
10+
if options.key?(:hiera_path_strip) && options[:hiera_path_strip] != :none
11+
raise ArgumentError, '--hiera-path and --hiera-path-strip are mutually exclusive'
12+
end
13+
14+
if options[:hiera_path] == :none
15+
raise ArgumentError, '--hiera-path and --no-hiera-path are mutually exclusive'
16+
end
17+
18+
options[:hiera_path] = path_in
19+
20+
if options[:hiera_path].start_with?('/')
21+
raise ArgumentError, '--hiera-path PATH must be a relative path not an absolute path'
22+
end
23+
24+
options[:hiera_path].sub!(%r{/+$}, '')
25+
raise ArgumentError, '--hiera-path must not be empty' if options[:hiera_path].empty?
26+
end
27+
28+
parser.on('--no-hiera-path', 'Do not use any default hiera path settings') do
29+
if options[:hiera_path].is_a?(String)
30+
raise ArgumentError, '--hiera-path and --no-hiera-path are mutually exclusive'
31+
end
32+
33+
options[:hiera_path] = :none
34+
end
35+
end
36+
end

‎lib/octocatalog-diff/catalog-diff/cli/options/hiera_path_strip.rb‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,11 +2,26 @@
22
# @param parser [OptionParser object] The OptionParser argument
33
# @param options [Hash] Options hash being constructed; this is modified in this method.
44
OctocatalogDiff::CatalogDiff::Cli::Options::Option.newoption(:hiera_path_strip) do
5-
has_weight 181
5+
has_weight 182
66

77
def parse(parser, options)
88
parser.on('--hiera-path-strip PATH', 'Path prefix to strip when munging hiera.yaml') do |path_in|
9+
if options.key?(:hiera_path) && options[:hiera_path] != :none
10+
raise ArgumentError, '--hiera-path and --hiera-path-strip are mutually exclusive'
11+
end
12+
13+
if options[:hiera_path_strip] == :none
14+
raise ArgumentError, '--hiera-path-strip and --no-hiera-path-strip are mutually exclusive'
15+
end
16+
917
options[:hiera_path_strip] = path_in
1018
end
19+
20+
parser.on('--no-hiera-path-strip', 'Do not use any default hiera path strip settings') do
21+
if options[:hiera_path_strip].is_a?(String)
22+
raise ArgumentError, '--hiera-path-strip and --no-hiera-path-strip are mutually exclusive'
23+
end
24+
options[:hiera_path_strip] = :none
25+
end
1126
end
1227
end

0 commit comments

Comments
 (0)