Merge remote-tracking branch 'origin/master' into kpaulisse-environment

Author: Kevin Paulisse

doc/advanced-catalog-validation.md

@@ -21,7 +21,7 @@ file { '/usr/local/bin/some-script.sh':
 
 The catalog for this code would build, whether or not the `exec { 'execute /usr/local/bin/some-script.sh': ... }` resource was part of the catalog. However, when the catalog is applied on the Puppet agent, it would fail if this resource is missing.
 
-With the `--validate-references` command line flag (or the `settings[:validate_references]` [configuration setting](/doc/configuration.md), you can instruct `octocatalog-diff` to confirm that any resource targeted by a `before`, `notify`, `require`, and `subscribe` parameter actually exists. If the resource is missing from the catalog, an error will be raised, just as if the catalog failed to compile.
+With the `--validate-references` command line flag (or the `settings[:validate_references]` [configuration setting](/doc/configuration.md)), you can instruct `octocatalog-diff` to confirm that any resource targeted by a `before`, `notify`, `require`, and `subscribe` parameter actually exists. If the resource is missing from the catalog, an error will be raised, just as if the catalog failed to compile.
 
 The command line argument is demonstrated here:
 

doc/dev/releasing.md

@@ -2,8 +2,47 @@
 
 The project maintainers are responsible for bumping the version number, regenerating auto-generated documentation, tagging the release, and uploading to rubygems.
 
-0. Ensure that all changes have been merged to master.
+## Local testing
+
+To test the new version of `octocatalog-diff` in the Puppet repository:
+
+0. In the Puppet checkout, start a new branch based off master.
+0. In the `octocatalog-diff` checkout:
+  - Ensure that the desired branch is checked out.
+  - Choose a unique internal version number which has never been used in CI. A good guideline is that if you're planning to release a version `0.6.0` then for these tests, use `0.6.0a`, `0.6.0b`, ...
+  - Build the gem using your internal version number:
+
+    ```
+    OCTOCATALOG_DIFF_VERSION=0.6.0a rake gem:force-build
+    ```
+  - Run the task to install the gem into your Puppet checkout:
+
+    ```
+    OCTOCATALOG_DIFF_VERSION=0.6.0a rake gem:localinstall
+    ```
+
+0. Back in the Puppet checkout, ensure that the changes are as expected (updates to Gemfile / Gemfile.lock, addition of new gem). Push the change and build appropriate CI job(s) to validate the changes.
+
+## Merging
+
 0. If necessary, complete a Pull Request to update the [version file](/.version).
+0. If necessary, auto-generate the build documentation.
+
+  ```
+  rake doc:build
+  ```
+
+0. Ensure that CI tests are all passing.
+0. Merge and delete the branch.
+
+## Releasing
+
+Generally, a new release will correspond to a merge to master of one or more Pull Requests.
+
+0. Ensure that all changes associated with the release have been merged to master.
+  - Merge all Pull Requests associated with release.
+  - If necessary, complete a Pull Request to update the [change log](/doc/CHANGELOG.md).
+  - If necessary (for significant changes), complete a Pull Request to update the top-level README file.
 0. Ensure the the master branch is checked out on your system.
 0. Run the release procedure:
 
@@ -13,7 +52,6 @@ The project maintainers are responsible for bumping the version number, regenera
 
 This rake task handles the following:
 
-- Auto-generates the [options reference](/doc/optionsref.md) (`rake doc:build`)
 - Build the gem file (`rake gem:build`)
 - Tag the release in the repository (`rake gem:tag`)
 - Upload the gem file to rubygems (`rake gem:push`)

doc/dev/run-from-branch.md

@@ -0,0 +1,62 @@
+# Running octocatalog-diff from a branch
+
+When we are assisting with troubleshooting, or implementing a feature you've requested, we may ask you to run `octocatalog-diff` from a non-master branch to try it out.
+
+This document is intended for people who may not be familiar with git, GitHub, and/or ruby. If you already know how to do this in another way, feel free!
+
+## Installation
+
+1. Determine the branch name. If there's an open Pull Request, you can see the branch name near the top of the page.
+
+  ![Pull Request branch](/doc/images/pull-request-identify-branch.png)
+
+2. Clone the `octocatalog-diff` repository in your home directory. From the command line:
+
+  ```
+  cd $HOME
+  git clone https://github.com/github/octocatalog-diff.git
+  ```
+
+3. Change into the directory created by your checkout:
+
+  ```
+  cd $HOME/octocatalog-diff
+  ```
+
+4. Check out the branch you wish to use, filling in the branch name you determined in the first step:
+
+  ```
+  git checkout BRANCH_NAME_FROM_STEP_1
+  ```
+
+5. Bootstrap the repository to pull in dependencies:
+
+  ```
+  ./script/bootstrap
+  ```
+
+6. Optional but recommended - run the test suite:
+
+  ```
+  rake
+  ```
+
+## Use
+
+Now that you have `octocatalog-diff` checked out and bootstrapped, it's time to use it.
+
+We've created a wrapper script to make this easier for you.
+
+1. Change directories to the location where you ordinarily run `octocatalog-diff` (for example: in your Puppet repository).
+
+    ```
+    cd /etc/puppetlabs/code
+    ```
+
+2. Run the `script/octocatalog-diff-wrapper` script from *this* checkout. For example, if you checked out `octocatalog-diff` to your home directory, you could use:
+
+    ```
+    $HOME/octocatalog-diff/script/octocatalog-diff-wrapper <options>
+    ```
+
+:warning: Note: If you are requesting our help, please use the debug option (`-d`) to display debugging information.

doc/images/pull-request-identify-branch.png

@@ -47,3 +47,9 @@ To install from source, you'll need a git client and internet access.
   Note: If tests fail on your machine with a clean checkout of the master branch, we would definitely appreciate if you would report it. Please [open an issue](https://github.com/github/octocatalog-diff/issues/new) with the output and some information about your system (e.g. OS, ruby version, etc.) to let us know.
 
 Once the code is downloaded and bootstrapped, please proceed to [Configuration](/doc/configuration.md).
+
+## Running from an alternate branch
+
+We have prepared specific instructions for running `octocatalog-diff` from a non-master branch, for testing changes that may be requested by the developers.
+
+- [Running octocatalog-diff from a branch](/doc/dev/run-from-branch.md)

doc/installation.md

@@ -166,7 +166,8 @@ def self.setup_logger(logger, options, argv_save)
         logger.level = Logger::ERROR if options[:quiet]
 
         # Some debugging information up front
-        logger.debug "Running octocatalog-diff #{VERSION} with ruby #{RUBY_VERSION}"
+        version_display = ENV['OCTOCATALOG_DIFF_CUSTOM_VERSION'] || VERSION
+        logger.debug "Running octocatalog-diff #{version_display} with ruby #{RUBY_VERSION}"
         logger.debug "Command line arguments: #{argv_save.inspect}"
         logger.debug "Running on host #{Socket.gethostname} (#{RUBY_PLATFORM})"
       end

lib/octocatalog-diff/catalog-diff/cli.rb

@@ -198,12 +198,11 @@ def validate_references
         @options[:validate_references].each do |r|
           next unless x.key?('parameters')
           next unless x['parameters'].key?(r)
-          missing_resources = remove_existing_resources(x['parameters'][r])
-          missing << missing_resources.map { |missing_target| { source: x, target_type: r, target_value: missing_target } }
+          missing_resources = resources_missing_from_catalog(x['parameters'][r])
+          next unless missing_resources.any?
+          missing.concat missing_resources.map { |missing_target| { source: x, target_type: r, target_value: missing_target } }
         end
       end
-      missing.flatten!
-      missing.compact!
       return if missing.empty?
 
       # At this point there is at least one broken/missing reference. Format an error message and
@@ -229,20 +228,18 @@ def validate_references
 
     private
 
-    # Private method: Determine if a catalog contains resource or resources, which may
-    # have been passed in as an array or a string. Return the references to resources
-    # that are missing from the catalog. (An empty array would indicate all references
-    # are present.)
+    # Private method: Given a list of resources to check, return the references from
+    # that list that are missing from the catalog. (An empty array returned would indicate
+    # all references are present in the catalog.)
     # @param resources_to_check [String / Array] Resources to check
     # @return [Array] References that are missing from catalog
-    def remove_existing_resources(resources_to_check)
-      rtc_array = resources_to_check.is_a?(Array) ? resources_to_check : [resources_to_check]
-      rtc_array.map do |res|
+    def resources_missing_from_catalog(resources_to_check)
+      [resources_to_check].flatten.select do |res|
         unless res =~ /\A([\w:]+)\[(.+)\]\z/
           raise ArgumentError, "Resource #{res} is not in the expected format"
         end
-        resource(type: Regexp.last_match(1), title: Regexp.last_match(2)).nil? ? res : nil
-      end.compact
+        resource(type: Regexp.last_match(1), title: Regexp.last_match(2)).nil?
+      end
     end
 
     # Private method: Choose backend based on passed-in options

lib/octocatalog-diff/catalog.rb

@@ -43,12 +43,15 @@ def self.build(target = GEMFILE)
 
     # Push the gem to rubygems
     def self.push
+      raise 'Cannot push version that does not match .version file' unless version == OctocatalogDiff::Version::VERSION
       raise "The gem file doesn't exist: #{FINAL_GEMFILE}" unless File.file?(FINAL_GEMFILE)
       exec_command("gem push #{Shellwords.escape(FINAL_GEMFILE)}")
     end
 
     # Tag the release on GitHub
     def self.tag
+      raise 'Cannot tag version that does not match .version file' unless version == OctocatalogDiff::Version::VERSION
+
       # Make sure we have not released this version before
       exec_command('git fetch -t origin')
       tags = exec_command('git tag -l').split(/\n/)

rake/gem.rb

@@ -0,0 +1,17 @@
+#!/bin/bash
+
+# This script exists to make it easier to test alternate branches of octocatalog-diff.
+# It is intended as a one-for-one replacement of `octocatalog-diff` as installed by the gem.
+
+CURRENT_PWD="$(pwd)"
+CHECKOUT_BASE="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && cd .. && pwd )"
+
+if [ -z "$OCTOCATALOG_DIFF_CONFIG_FILE" ]; then
+  if [ -f "${CURRENT_PWD}/.octocatalog-diff.cfg.rb" ]; then
+    export OCTOCATALOG_DIFF_CONFIG_FILE="${CURRENT_PWD}/.octocatalog-diff.cfg.rb"
+  fi
+fi
+
+cd "$CHECKOUT_BASE"
+export OCTOCATALOG_DIFF_CUSTOM_VERSION="@$(git rev-parse HEAD)"
+bundle exec bin/octocatalog-diff --basedir "$CURRENT_PWD" $*

script/octocatalog-diff-wrapper

@@ -115,6 +115,37 @@
     end
   end
 
+  describe '#setup_logger' do
+    context 'with custom version specified in environment' do
+      before(:each) do
+        ENV['OCTOCATALOG_DIFF_CUSTOM_VERSION'] = '@d05c30152c897219367d586414ccb1f651ab7221'
+      end
+
+      after(:each) do
+        ENV.delete 'OCTOCATALOG_DIFF_CUSTOM_VERSION'
+      end
+
+      it 'should log custom version' do
+        logger, logger_str = OctocatalogDiff::Spec.setup_logger
+        described_class.setup_logger(logger, { debug: true }, nil)
+        expect(logger_str.string).to match(/Running octocatalog-diff @d05c30152c897219367d586414ccb1f651ab7221 with ruby/)
+      end
+    end
+
+    context 'with default version' do
+      before(:each) do
+        ENV.delete 'OCTOCATALOG_DIFF_CUSTOM_VERSION'
+      end
+
+      it 'should log current version' do
+        logger, logger_str = OctocatalogDiff::Spec.setup_logger
+        described_class.setup_logger(logger, { debug: true }, nil)
+        version = described_class::VERSION
+        expect(logger_str.string).to match(/Running octocatalog-diff #{version} with ruby/)
+      end
+    end
+  end
+
   describe '#setup_fact_overrides' do
     it 'should make no adjustments when there are no fact overrides' do
       options = {}