Merge pull request #74 from github/kpaulisse-enc-override

Enhancement: Allow override of ENC parameters

Author: kpaulisse

.version

@@ -1 +1 @@
-0.99.rc1
+0.99.rc2

doc/CHANGELOG.md

@@ -7,6 +7,18 @@
 <th>Description / Changes</th>
 </tr>
 </thead><tbody>
+<tr valign=top>
+<td>0.99.0rc2</td>
+<td>2017-01-23</td>
+<td>
+<ul>
+<li><a href="https://github.com/github/octocatalog-diff/pull/70">#70</a> Missing `[]=` method in display code</li>
+<li><a href="https://github.com/github/octocatalog-diff/pull/77">#77</a> `FactOverride` -> `Override`</li>
+<li><a href="https://github.com/github/octocatalog-diff/pull/74">#74</a> Allow override of ENC parameters</li>
+</ul>
+</td>
+</tr>
+
 <tr valign=top>
 <td>0.99.0rc1</td>
 <td>2017-01-16</td>

doc/advanced-override-enc.md

@@ -0,0 +1,54 @@
+# Overriding ENC parameters
+
+One powerful feature of `octocatalog-diff` allows you to override ENC parameters when compiling the catalogs, to predict the effect of an ENC parameter change on the catalog. This is useful to simulate a change in agent node configuration without actually setting up an agent to do so.
+
+## Usage
+
+To override an ENC parameter in both catalogs:
+
+```
+--enc-override parameters::some_class::some_param=value
+```
+
+To override an ENC parameter in the "to" catalog:
+
+```
+--to-enc-override parameters::some_class::some_param=value
+```
+
+To override an ENC parameter in the "from" catalog:
+
+```
+--from-enc-override parameters::some_class::some_param=value
+```
+
+You may use as many of these arguments as you wish to adjust as many ENC parameters as you wish.
+
+## Limitations
+
+As presently implemented, this only works on ENCs that supply their results as YAML.
+
+Is your ENC doing something different? Please [let us know](https://github.com/github/octocatalog-diff/issues/new) so we can enhance octocatalog-diff to handle it!
+
+## Examples
+
+Simulate a change to a top-level parameter named "role" only in the "to" catalog:
+
+```
+octocatalog-diff -n some-node.example.com -f master -t master \
+  --to-enc-override parameters::role=some_new_role
+```
+
+Simulate a change in a class parameter between the catalogs:
+
+```
+octocatalog-diff -n some-node.example.com -f master -t master \
+  --from-enc-override parameters::my_class::my_value=value_in_old \
+  --to-enc-override parameters::my_class::my_value=value_in_new
+```
+
+Note that each of the examples specified the from branch and to branch to be `master`. There is no requirement that you do this, but you can generally obtain the most accurate test results by changing only one variable at a time.
+
+## Advanced usage
+
+The format for declaring overrides with data types is the same as [overriding facts](/doc/advanced-override-facts.md#advanced-usage).

doc/advanced.md

@@ -18,6 +18,7 @@ See also:
 - [Building catalogs instead of diffing catalogs](/doc/advanced-catalog-only.md)
 - [Enabling storeconfigs for exported resources in PuppetDB](/doc/advanced-storeconfigs.md)
 - [Fetching catalogs from Puppet Master / PuppetServer](/doc/advanced-puppet-master.md)
+- [Overriding ENC parameters](/doc/advanced-override-enc.md)
 - [Overriding facts](/doc/advanced-override-facts.md)
 - [Puppet Enterprise node classification service](/doc/advanced-pe-enc.md)
 - [Using `octocatalog-diff` without git](/doc/advanced-using-without-git.md)

doc/dev/api/v1/objects/fact_override.md

@@ -118,6 +118,12 @@ Usage: octocatalog-diff [command line options]
                                      Full path to key file for SSL client auth to Puppet Master for the to branch
         --from-puppet-master-ssl-client-key STRING
                                      Full path to key file for SSL client auth to Puppet Master for the from branch
+        --enc-override STRING1[,STRING2[,...]]
+                                     Override parameter from ENC globally
+        --to-enc-override STRING1[,STRING2[,...]]
+                                     Override parameter from ENC for the to branch
+        --from-enc-override STRING1[,STRING2[,...]]
+                                     Override parameter from ENC for the from branch
         --pe-enc-url URL             Base URL for Puppet Enterprise ENC endpoint
         --pe-enc-token TOKEN         Token to access the Puppet Enterprise ENC API
         --pe-enc-token-file PATH     Path containing token for PE node classifier, relative or absolute

doc/optionsref.md

@@ -195,6 +195,7 @@ def install_directory_symlink(logger, dir, target = 'environments/production')
       def install_enc(logger)
         raise ArgumentError, 'A node must be specified when using an ENC' unless @node.is_a?(String)
         enc_obj = OctocatalogDiff::CatalogUtil::ENC.new(@options.merge(tempdir: @tempdir))
+        enc_obj.execute(logger)
         raise "Failed ENC: #{enc_obj.error_message}" if enc_obj.error_message
 
         enc_path = File.join(@tempdir, 'enc.sh')

lib/octocatalog-diff/catalog-util/builddir.rb

@@ -5,6 +5,7 @@
 require_relative 'enc/script'
 
 require 'stringio'
+require 'yaml'
 
 module OctocatalogDiff
   module CatalogUtil
@@ -32,20 +33,64 @@ def initialize(options = {})
 
       # Retrieve content
       # @return [String] ENC content, or nil if there was an error
-      def content
-        execute
+      def content(logger = nil)
+        execute(logger)
         @content ||= @enc_obj.content
       end
 
       # Retrieve error message
       # @return [String] Error message, or nil if there was no error
-      def error_message
-        execute
+      def error_message(logger = nil)
+        execute(logger)
         @error_message ||= @enc_obj.error_message
       end
 
+      # Execute the 'execute' method of the object, but only once
+      # @param [Logger] Logger (optional) - if not supplied any logger messages will be discarded
+      def execute(logger = nil)
+        return if @executed
+        logger ||= @options[:logger]
+        logger ||= Logger.new(StringIO.new)
+        @enc_obj.execute(logger) if @enc_obj.respond_to?(:execute)
+        @executed = true
+        override_enc_parameters(logger)
+      end
+
       private
 
+      # Override of ENC parameters with parameters specified on the command line.
+      # Modifies structures in @enc_obj.
+      # @param logger [Logger] Logger object
+      def override_enc_parameters(logger)
+        return unless @options[:enc_override].is_a?(Array) && @options[:enc_override].any?
+        content_structure = YAML.load(content)
+        @options[:enc_override].each do |x|
+          merge_enc_param(content_structure, x.key, x.value)
+          logger.debug "ENC override: #{x.key} #{x.value.nil? ? 'DELETED' : '= ' + x.value.inspect}"
+        end
+        @content = content_structure.to_yaml
+      end
+
+      # Merging behavior for ENC overrides
+      # @param pointer [Hash] Portion of the content structure to modify
+      # @param key [String] String representing structure, delimited by '::'
+      # @param value [?] Value to insert at structure point
+      def merge_enc_param(pointer, key, value)
+        if key =~ /::/
+          first_key, the_rest = key.split(/::/, 2)
+          if pointer[first_key].nil?
+            pointer[first_key] = {}
+          elsif !pointer[first_key].is_a?(Hash)
+            raise ArgumentError, "Attempt to override #{pointer[first_key].class} with hash for #{key}"
+          end
+          merge_enc_param(pointer[first_key], the_rest, value)
+        elsif value.nil?
+          pointer.delete(key)
+        else
+          pointer[key] = value
+        end
+      end
+
       # Backend - given options, choose an appropriate backend and construct the corresponding object.
       # @return [?] Backend object
       def backend
@@ -64,16 +109,6 @@ def backend
         # At this point we do not know what backend to use for the ENC
         raise ArgumentError, 'Unable to determine ENC backend to use'
       end
-
-      # Execute the 'execute' method of the object, but only once
-      # @param [Logger] Logger (optional) - if not supplied any logger messages will be discarded
-      def execute(logger = nil)
-        return if @executed
-        logger ||= @options[:logger]
-        logger ||= Logger.new(StringIO.new)
-        @enc_obj.execute(logger) if @enc_obj.respond_to?(:execute)
-        @executed = true
-      end
     end
   end
 end

lib/octocatalog-diff/catalog-util/enc.rb

@@ -81,7 +81,9 @@ def self.cli(argv = ARGV, logger = Logger.new(STDERR), opts = {})
       veto_with_none_options = %w(hiera_path hiera_path_strip)
       veto_with_none_options.each { |x| options.delete(x.to_sym) if options[x.to_sym] == :none }
 
+      # Fact and ENC overrides come in here - 'options' is modified
       setup_fact_overrides(options)
+      setup_enc_overrides(options)
 
       # Configure the logger and logger.debug initial information
       # 'logger' is modified and used
@@ -142,6 +144,12 @@ def self.setup_fact_overrides(options)
       setup_overrides(:to_fact_override, options)
     end
 
+    # ENC parameter overrides come in here
+    def self.setup_enc_overrides(options)
+      setup_overrides(:from_enc_override, options)
+      setup_overrides(:to_enc_override, options)
+    end
+
     # Helper method: Configure and setup logger
     def self.setup_logger(logger, options, argv_save)
       # Configure the logger

lib/octocatalog-diff/cli.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+# Allow override of ENC parameters on the command line. ENC parameter overrides can be supplied for the 'to' or 'from' catalog,
+# or for both. There is some attempt to handle data types here (since all items on the command line are strings)
+# by permitting a data type specification as well. For parameters nested in hashes, use `::` as the delimiter.
+OctocatalogDiff::Cli::Options::Option.newoption(:enc_override) do
+  has_weight 322
+
+  def parse(parser, options)
+    # Set 'enc_override_in' because more processing is needed, once the command line options
+    # have been parsed, to make this into the final form 'enc_override'.
+    OctocatalogDiff::Cli::Options.option_globally_or_per_branch(
+      parser: parser,
+      options: options,
+      cli_name: 'enc-override',
+      option_name: 'enc_override_in',
+      desc: 'Override parameter from ENC',
+      datatype: []
+    )
+  end
+end