Best Practices

Each pattern requires a separate molecule test setup.

Test Patterns

• Do NOT re-test encapsulated roles and collections.

• Focus on testing all options and code paths. A component may test many options if they can be individually explicitly verified.

  Use Decisions

  Create decisions when explicitly NOT testing a task.

• Use least privilege. Comments are required when using these features in Molecule.

  • Skipping idempotent step.
  • root or SYS_ capabilities.
  • gather_facts: true.
• Roles should be thoroughly tested with VMs for tasks not covered in containers.
• Molecule test name reflects component tested.

• All tests must pass with molecule test --all.

  Require Enablement

  Live testing (e.g. external API use) requires explicit enablement (See example).

  Live testing should always occur before submitting changes if code paths were affected.

• Make use of Molecule test variables

molecule.yml

• Describe test.
• Define test variables for test setup.

converge.yml

• unsafe variables must be defined here. Reference.
• Edge cases outside of the base setup (e.g. multiple convergence steps).
• Use mock scripts to replace binaries when binaries cannot be used in a container and they do not affect testing (See example).
• Do live testing here (See example).
  • Use {ROLE}_testing_enable.
  • Disable live API testing by default to ensure all tests pass.
  • Disable live API usage (which cannot be mocked easily).

prepare.yml

• Generate static testing data (certs, keys, network) (See example).
• Cache generated files in molecule directory:

  '{{ lookup("env", "MOLECULE_PROJECT_DIRECTORY") ~ "/molecule/cache" }}'

• Cache runtime test files in /tmp on remote host.
• Specify dynamic target files (See example).
• Load generated files and inject in converge.yml (See example).

verify.yml

• Use ansible.builtin.assert to validate test results. Any tests missing assert are not explicitly validating correctness.

Reference:

• https://www.puppeteers.net/blog/ansible-quality-assurance-part-1-ansible-variable-validation-with-assert/

Documenting Tests

Explicitly state test conditions in molecule.yml using the following format:

###############################################################################
# [Molecule Test Name]
###############################################################################
# Description of the molecule test / setup.
#
# Tests:
# * Each condition to verify on completion.

Default

default is effectively an role integration test with default values - only disabling components which cannot be tested on testing platform - to validate no runtime errors.

converge.yml

- name: 'Converge | default role settings'
  ansible.builtin.include_role:
    name: 'r_pufky.deb.os'

Regression

Explicitly test major bugs to ensure bug is fixed.

• Prefix test with regression_.
• Reference bug in verify.yml with detailed information on cause and resolution.

Assertions

Clearly state expected results and failure reasons (See example).

Simple assertion failure

- name: 'Verify | script files | assert scripts'
  ansible.builtin.assert:
    quiet: true
    that:
      - _test_nzbget_skel.files | length == 2
    fail_msg: |
      ╭───────────────────────────────────────────────────────────────────╮
      │                                                                   │
      │     /var/lib/nzbget/scripts{scan,post_processing} not found.      │
      │                                                                   │
      ╰───────────────────────────────────────────────────────────────────╯

Complex error with variables

- name: 'Verify | expressions | assert nzbget_extensions'
  ansible.builtin.assert:
    quiet: true
    that:
      - _test_nzbget_config is search('Extensions=scan,post_processing')
    fail_msg: |
      ╭───────────────────────────────────────────────────────────────────╮
      │                                                                   │
      │                nzbget_extensions format incorrect.                │
      │                                                                   │
      ├───────────────────────────────────────────────────────────────────╯
      │ Expected:
      │   Extensions=scan,post_processing
      │
      │ _test_nzbget_config: {{ _test_nzbget_config }}