mig5/enroll
1
0
Fork 0
Code Issues Pull requests Projects Releases 21 Packages Wiki Activity Actions

Compare commits

base: mig5:main
Branches Tags
mig5:main
mig5:0.6.0
mig5:0.5.0
mig5:0.4.4
mig5:0.4.3
mig5:0.4.2
mig5:0.4.1
mig5:0.4.0
mig5:0.3.0
mig5:0.2.3
mig5:0.2.2
mig5:0.2.0
mig5:0.1.7
mig5:0.1.6
mig5:0.1.5
mig5:0.1.4
mig5:0.1.3
mig5:0.1.2
mig5:0.1.1
mig5:0.1.0
mig5:0.0.4
mig5:0.0.2
..
compare: mig5:0.0.2
Branches Tags
mig5:main
mig5:0.6.0
mig5:0.5.0
mig5:0.4.4
mig5:0.4.3
mig5:0.4.2
mig5:0.4.1
mig5:0.4.0
mig5:0.3.0
mig5:0.2.3
mig5:0.2.2
mig5:0.2.0
mig5:0.1.7
mig5:0.1.6
mig5:0.1.5
mig5:0.1.4
mig5:0.1.3
mig5:0.1.2
mig5:0.1.1
mig5:0.1.0
mig5:0.0.4
mig5:0.0.2

No commits in common. "main" and "0.0.2" have entirely different histories.
main ... 0.0.2

76 changed files with 1123 additions and 19199 deletions
Download patch file Download diff file Expand all files Collapse all files

66
.forgejo/workflows/build-deb.yml
View file

@ -1,66 +0,0 @@
name: CI
on:
push:
jobs:
test:
runs-on: docker
steps:
- name: Install system dependencies
run: |
apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
build-essential \
devscripts \
debhelper \
dh-python \
pybuild-plugin-pyproject \
python3-all \
python3-poetry-core \
python3-yaml \
python3-paramiko \
python3-jsonschema \
rsync \
ca-certificates
- name: Checkout
uses: actions/checkout@v4
with:
submodules: recursive
- name: Build deb
run: |
mkdir /out
rsync -a --delete \
--exclude '.git' \
--exclude '.venv' \
--exclude 'dist' \
--exclude 'build' \
--exclude '__pycache__' \
--exclude '.pytest_cache' \
--exclude '.mypy_cache' \
./ /out/
cd /out/
export DEBEMAIL="mig@mig5.net"
export DEBFULLNAME="Miguel Jacq"
dch --distribution "trixie" --local "~trixie" "CI build for trixie"
dpkg-buildpackage -us -uc -b
# Notify if any previous step in this job failed
- name: Notify on failure
if: ${{ failure() }}
env:
WEBHOOK_URL: ${{ secrets.NODERED_WEBHOOK_URL }}
REPOSITORY: ${{ forgejo.repository }}
RUN_NUMBER: ${{ forgejo.run_number }}
SERVER_URL: ${{ forgejo.server_url }}
run: |
curl -X POST \
-H "Content-Type: application/json" \
-d "{\"repository\":\"$REPOSITORY\",\"run_number\":\"$RUN_NUMBER\",\"status\":\"failure\",\"url\":\"$SERVER_URL/$REPOSITORY/actions/runs/$RUN_NUMBER\"}" \
"$WEBHOOK_URL"

4
.forgejo/workflows/ci.yml
View file

@ -15,7 +15,7 @@ jobs:
run: |
apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
ansible ansible-lint python3-venv pipx systemctl python3-apt jq python3-jsonschema
ansible ansible-lint python3-venv pipx
- name: Install Poetry
run: |
@ -25,7 +25,7 @@ jobs:
- name: Install project deps (including test extras)
run: |
poetry install --with dev
poetry install --with test
- name: Run test script
run: |

40
.forgejo/workflows/trivy.yml Normal file
View file

@ -0,0 +1,40 @@
name: Trivy
on:
schedule:
- cron: '0 1 * * *'
push:
jobs:
test:
runs-on: docker
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Install system dependencies
run: |
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends wget gnupg
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | tee /usr/share/keyrings/trivy.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb generic main" | tee -a /etc/apt/sources.list.d/trivy.list
apt-get update
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends trivy
- name: Run trivy
run: |
trivy fs --no-progress --ignore-unfixed --format table --disable-telemetry .
# Notify if any previous step in this job failed
- name: Notify on failure
if: ${{ failure() }}
env:
WEBHOOK_URL: ${{ secrets.NODERED_WEBHOOK_URL }}
REPOSITORY: ${{ forgejo.repository }}
RUN_NUMBER: ${{ forgejo.run_number }}
SERVER_URL: ${{ forgejo.server_url }}
run: |
curl -X POST \
-H "Content-Type: application/json" \
-d "{\"repository\":\"$REPOSITORY\",\"run_number\":\"$RUN_NUMBER\",\"status\":\"failure\",\"url\":\"$SERVER_URL/$REPOSITORY/actions/runs/$RUN_NUMBER\"}" \
"$WEBHOOK_URL"

25
.pre-commit-config.yaml
View file

@ -1,25 +0,0 @@
repos:
- repo: https://github.com/pycqa/flake8
rev: 7.3.0
hooks:
- id: flake8
args: ["--select=F"]
types: [python]
- repo: https://github.com/psf/black-pre-commit-mirror
rev: 25.11.0
hooks:
- id: black
language_version: python3
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- repo: https://github.com/PyCQA/bandit
rev: 1.9.2
hooks:
- id: bandit
files: ^enroll/

154
CHANGELOG.md
View file

@ -1,154 +0,0 @@
# 0.6.0
* Add support for capturing ipset and iptables configuration files
* Add support for generating ipset and iptables configuration files from runtime, if the former weren't present (`firewall_runtime` role)
* Dependency updates
# 0.5.0
* Add support for templating `sshd_config`, if a compatible version of JinjaTurtle is also present.
* Dependency updates
# 0.4.4
* Update cryptography dependency
* Add capability to handle passphrases on encrypted SSH private keys. Prompting can be forced with `--ask-key-passphrase` or automated (e.g for CI) with `--ssh-key-passphrase env SOMEVAR`
# 0.4.3
* Add support for AddressFamily and ConnectTimeout in the .ssh/config when using `--remote-ssh-config`.
* Update dependencies
# 0.4.2
* Support `--remote-ssh-config [path-to-ssh-config]` as an argument in case extra params are required beyond `--remote-port` or `--remote-user`. Note: `--remote-host` must still be set, but it can be an 'alias' represented by the 'Host' value in the ssh config.
# 0.4.1
* Add interactive output when 'enroll diff --enforce' is invoking Ansible.
# 0.4.0
* Introduce `enroll validate` - a tool to validate a harvest against the state schema, or check for missing or orphaned obsolete artifacts in a harvest.
* Attempt to generate Jinja2 templates of systemd unit files and Postfix main.cf (now that JinjaTurtle supports it)
* Update pynacl dependency to resolve CVE-2025-69277
* Add `--exclude-path` to `enroll diff` command, so that you can ignore certain churn from the diff (stuff you still wanted to harvest as a baseline but don't care if it changes day to day)
* Add `--ignore-package-versions` to `enroll diff` command, to optionally ignore package upgrades (e.g due to patching) from the diff.
* Add tags to the playbook for each role, to allow easier targeting of specific roles during play later.
* Add `--enforce` mode to `enroll diff`. If there is diff detected between the two harvests, and it can enforce restoring the state from the older harvest, it will manifest the state and apply it with ansible. Only the specific roles that had diffed will be applied (via the new tags capability)
# 0.3.0
* Introduce `enroll explain` - a tool to analyze and explain what's in (or not in) a harvest and why.
* Centralise the cron and logrotate stuff into their respective roles, we had a bit of duplication between roles based on harvest discovery.
* Capture other files in the user's home directory such as `.bashrc`, `.bash_aliases`, `.profile`, if these files differ from the `/etc/skel` defaults
* Ignore files that end with a tilde or - (probably backup files generated by editors or shadow file changes)
* Manage certain symlinks e.g for apache2/nginx sites-enabled and so on
# 0.2.3
* Introduce --ask-become-pass or -K to support password-required sudo on remote hosts, just like Ansible. It will also fall back to this prompt if a password is required but the arg wasn't passed in.
# 0.2.2
* Fix stat() of parent directory so that we set directory perms correct on --include paths.
* Set pty for remote calls when sudo is required, to help systems with limits on sudo without pty
# 0.2.1
* Don't accidentally add `extra_paths` role to `usr_local_custom` list, resulting in `extra_paths` appearing twice in manifested playbook
* Ensure directories in the tree of anything included with --include are defined in the state and manifest so we make dirs before we try to create files
# 0.2.0
* Add version CLI arg
* Add ability to enroll RH-style systems (DNF5/DNF/RPM)
* Refactor harvest state to track package versions
# 0.1.7
* Fix an attribution bug for certain files ending up in the wrong package/role.
# 0.1.6
* DRY up some code logic
* More test coverage
# 0.1.5
* Consolidate logrotate and cron files into their main service/package roles if they exist.
* Standardise on `MAX_FILES_CAP` in one place
* Manage apt stuff in its own role, not in `etc_custom`
# 0.1.4
* Attempt to capture more stuff from /etc that might not be attributable to a specific package. This includes common singletons and systemd timers
* Avoid duplicate apt data in package-specific roles.
# 0.1.3
* Allow the user to add extra paths to harvest, or paths to ignore, using `--exclude-path` and `--include-path`
arguments.
* Add support for an enroll.ini config file to store arguments per subcommand, to avoid having to remember
them all for repetitive executions.
# 0.1.2
* Include files from `/usr/local/bin` and `/usr/local/etc` in harvest (assuming they aren't binaries or
symlinks) and store in `usr_local_custom` role, similar to `etc_custom`.
# 0.1.1
* Add `diff` subcommand which can compare two harvests and send email or webhook notifications in different
formats.
# 0.1.0
* Add remote mode for harvesting a remote machine via a local workstation (no need to install enroll remotely)
Optionally use `--no-sudo` if you don't want the remote user to have passwordless sudo when conducting the
harvest, albeit you'll end up with less useful data (same as if running `enroll harvest` on a machine without
sudo)
* Add `--dangerous` flag to capture even sensitive data (use at your own risk!)
* Add `--sops` flag which makes the harvest and the manifest 'out' data encrypted as a single SOPS data file.
This would make `--dangerous` a little bit safer, if your intention is just to store the Ansible manifest
in git or somewhere similar for disaster-recovery purposes (e.g encrypted at rest for safe-keeping).
* Do a better job at capturing other config files in `/etc/<package>/` even if that package doesn't normally
ship or manage those files.
* Don't collect files ending in `.log`
# 0.0.5
* Use JinjaTurtle to generate dynamic template/inventory if it's on the PATH
* Support --fqdn flag for site-specific inventory and an inventory hosts file.
This radically re-architects the roles to loop through abstract inventory
because otherwise different servers can collide with each other through use
of the same role. Use 'single site' mode (no `--fqdn`) if you want more readable,
self-contained roles (in which case, store each manifested output in its own
repo per server)
* Generate an ansible.cfg if not present, to support `host_vars` plugin and other params,
when using `--fqdn` mode
* Be more permissive with files that we previously thought contained secrets (ignore commented lines)
# 0.0.4
* Fix dash package detection issue
* Reorder which roles install first
# 0.0.3
* various bug fixes
* Add debian packaging
# 0.0.2
* Merge pkg_ and roles created based on file/service detection
* Avoid idempotency issue with users (`password_lock`)
* Rename subcommands/args ('export' is now 'enroll', '--bundle' is now '--harvest')
* Don't try and start systemd services that were Inactive at harvest time
* Capture miscellaneous files in /etc under their own `etc_custom` role, but not backup files
* Add tests
* Various other bug fixes
# 0.0.1
* Initial commit

13
CHANGELOG.txt Normal file
View file

@ -0,0 +1,13 @@
# 0.0.2
* Merge pkg_ and roles created based on file/service detection
* Avoid idempotency issue with users (password_lock)
* Rename subcommands/args ('export' is now 'enroll', '--bundle' is now '--harvest')
* Don't try and start systemd services that were Inactive at harvest time
* Capture miscellaneous files in /etc under their own etc_custom role, but not backup files
* Add tests
* Various other bug fixes
# 0.0.1
* Initial commit

5
CONTRIBUTORS.md
View file

@ -1,5 +0,0 @@
## Contributors
mig5 would like to thank the following people for their contributions to Enroll.
* [slhck](https://slhck.info/)

80
Dockerfile.debbuild
View file

@ -1,80 +0,0 @@
# syntax=docker/dockerfile:1
ARG BASE_IMAGE=debian:bookworm
FROM ${BASE_IMAGE}
ENV DEBIAN_FRONTEND=noninteractive
# If Ubuntu, ensure Universe is enabled.
RUN set -eux; \
. /etc/os-release; \
if [ "${ID:-}" = "ubuntu" ]; then \
apt-get update; \
apt-get install -y --no-install-recommends software-properties-common ca-certificates; \
add-apt-repository -y universe; \
fi
# Build deps
RUN set -eux; \
apt-get update; \
apt-get install -y --no-install-recommends \
build-essential \
devscripts \
debhelper \
dh-python \
pybuild-plugin-pyproject \
python3-all \
python3-poetry-core \
python3-yaml \
python3-paramiko \
python3-jsonschema \
rsync \
ca-certificates \
; \
rm -rf /var/lib/apt/lists/*
# Build runner script
RUN set -eux; \
cat > /usr/local/bin/build-deb <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
SRC="${SRC:-/src}"
WORKROOT="${WORKROOT:-/work}"
WORK="${WORKROOT}/src"
OUT="${OUT:-/out}"
mkdir -p "$WORK" "$OUT"
rsync -a --delete \
--exclude '.git' \
--exclude '.venv' \
--exclude 'dist' \
--exclude 'build' \
--exclude '__pycache__' \
--exclude '.pytest_cache' \
--exclude '.mypy_cache' \
"${SRC}/" "${WORK}/"
cd "${WORK}"
if [ -n "${SUITE:-}" ]; then
export DEBEMAIL="mig@mig5.net"
export DEBFULLNAME="Miguel Jacq"
dch --distribution "$SUITE" --local "~${SUITE}" "CI build for $SUITE"
fi
dpkg-buildpackage -us -uc -b
shopt -s nullglob
cp -v "${WORKROOT}"/*.deb \
"${WORKROOT}"/*.changes \
"${WORKROOT}"/*.buildinfo \
"${WORKROOT}"/*.dsc \
"${WORKROOT}"/*.tar.* \
"${OUT}/" || true
echo "Artifacts copied to ${OUT}"
EOF
RUN chmod +x /usr/local/bin/build-deb
WORKDIR /work
ENTRYPOINT ["/usr/local/bin/build-deb"]

88
Dockerfile.rpmbuild
View file

@ -1,88 +0,0 @@
# syntax=docker/dockerfile:1
ARG BASE_IMAGE=fedora:42
FROM ${BASE_IMAGE}
RUN set -eux; \
dnf -y update; \
dnf -y install \
rpm-build \
rpmdevtools \
redhat-rpm-config \
gcc \
make \
findutils \
tar \
gzip \
rsync \
python3 \
python3-devel \
python3-setuptools \
python3-wheel \
pyproject-rpm-macros \
python3-rpm-macros \
python3-yaml \
python3-paramiko \
python3-jsonschema \
openssl-devel \
python3-poetry-core ; \
dnf -y clean all
# Build runner script (copies repo, tars, runs rpmbuild)
RUN set -eux; cat > /usr/local/bin/build-rpm <<'EOF'
#!/usr/bin/env bash
set -euo pipefail
SRC="${SRC:-/src}"
WORKROOT="${WORKROOT:-/work}"
OUT="${OUT:-/out}"
VERSION_ID="$(grep VERSION_ID /etc/os-release | cut -d= -f2)"
echo "Version ID is ${VERSION_ID}"
mkdir -p "${WORKROOT}" "${OUT}"
WORK="${WORKROOT}/src"
rm -rf "${WORK}"
mkdir -p "${WORK}"
rsync -a --delete \
--exclude '.git' \
--exclude '.venv' \
--exclude 'dist' \
--exclude 'build' \
--exclude '__pycache__' \
--exclude '.pytest_cache' \
--exclude '.mypy_cache' \
"${SRC}/" "${WORK}/"
cd "${WORK}"
# Determine version from pyproject.toml unless provided
if [ -n "${VERSION:-}" ]; then
ver="${VERSION}"
else
ver="$(grep -m1 '^version = ' pyproject.toml | sed -E 's/version = "([^"]+)".*/\1/')"
fi
TOPDIR="${WORKROOT}/rpmbuild"
mkdir -p "${TOPDIR}"/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
tarball="${TOPDIR}/SOURCES/enroll-${ver}.tar.gz"
tar -czf "${tarball}" --transform "s#^#enroll/#" .
spec_src="rpm/enroll.spec"
cp -v "${spec_src}" "${TOPDIR}/SPECS/enroll.spec"
rpmbuild -ba "${TOPDIR}/SPECS/enroll.spec" \
--define "_topdir ${TOPDIR}" \
--define "upstream_version ${ver}"
shopt -s nullglob
cp -v "${TOPDIR}"/RPMS/*/*.rpm "${OUT}/" || true
cp -v "${TOPDIR}"/SRPMS/*.src.rpm "${OUT}/" || true
echo "Artifacts copied to ${OUT}"
EOF
RUN chmod +x /usr/local/bin/build-rpm
WORKDIR /work
ENTRYPOINT ["/usr/local/bin/build-rpm"]

615
README.md
View file

@ -4,629 +4,88 @@
<img src="https://git.mig5.net/mig5/enroll/raw/branch/main/enroll.svg" alt="Enroll logo" width="240" />
</div>
**enroll** inspects a Linux machine (Debian-like or RedHat-like) and generates Ansible roles/playbooks (and optionally inventory) for what it finds.
**enroll** inspects a Linux machine (currently Debian-only) and generates Ansible roles for things it finds running on the machine.
- Detects packages that have been installed.
- Detects package ownership of `/etc` files where possible
- Captures config that has **changed from packaged defaults** where possible (e.g dpkg conffile hashes + package md5sums when available).
It aims to be **optimistic and noninteractive**:
- Detects packages that have been installed
- Detects Debian package ownership of `/etc` files using dpkg’s local database.
- Captures config that has **changed from packaged defaults** (dpkg conffile hashes + package md5sums when available).
- Also captures **service-relevant custom/unowned files** under `/etc/<service>/...` (e.g. drop-in config includes).
- Defensively excludes likely secrets (path denylist + content sniff + size caps).
- Captures non-system users and their SSH public keys and any .bashrc or .bash_aliases or .profile files that deviate from the skel defaults.
- Captures miscellaneous `/etc` files it can't attribute to a package and installs them in an `etc_custom` role.
- Captures live ipset and iptables runtime state into a fallback `firewall_runtime` role, when active ipsets/iptables rules are present *and* no corresponding persistent ipset/iptables *files* were found.
- Captures symlinks in common applications that rely on them, e.g apache2/nginx 'sites-enabled'
- Ditto for /usr/local/bin (for non-binary files) and /usr/local/etc
- Avoids trying to start systemd services that were detected as inactive during harvest.
- Captures non-system users that exist on the system, and their SSH public keys
- Captures miscellaneous `/etc` files that it can't attribute to a package, and installs it in an `etc_custom` role
- Avoids trying to start systemd services that were detected as being Inactive during harvest
---
## Install
## Mental model
### AppImage
`enroll` works in two phases:
1) **Harvest**: collect host facts + relevant files into a harvest bundle (`state.json` + harvested artifacts)
2) **Manifest**: turn that harvest into Ansible roles/playbooks (and optionally inventory)
Additionally, some other functionalities exist:
- **Diff**: compare two harvests and report what changed (packages/services/users/files) since the previous snapshot.
- **Single-shot mode**: run both harvest and manifest at once.
---
## Output modes: single-site vs multi-site (`--fqdn`)
`enroll manifest` (and `enroll single-shot`) support two distinct output styles.
### Single-site mode (default: *no* `--fqdn`)
Use when enrolling **one server** (or generating a “golden” role set you intend to reuse).
**Characteristics**
- Roles are more self-contained.
- Raw config files live in the role's `files/`.
- Template variables live in the role's `defaults/main.yml`.
### Multi-site mode (`--fqdn`)
Use when enrolling **several existing servers** quickly, especially if they differ.
**Characteristics**
- Roles are shared, host-specific state lives in inventory.
- Host inventory drives what gets managed (files/packages/services).
- Non-templated raw files live per-host under `inventory/host_vars/<fqdn>/<role>/.files/...`.
**Rule of thumb**
- “Make this one server reproducible/provisionable” → start with **single-site**
- “Get multiple already-running servers under management quickly” → use **multi-site**
---
## Subcommands
### `enroll harvest`
Harvest state about a host and write a harvest bundle.
**What it captures (high level)**
- Detected services + service-relevant packages
- “Manual” packages
- Changed-from-default config (plus related custom/unowned files under service dirs)
- Non-system users + SSH public keys
- Misc `/etc` that can't be attributed to a package (`etc_custom` role)
- Static firewall config files such as nftables, UFW, firewalld, `/etc/iptables/rules.v4`, `/etc/iptables/rules.v6`, and `/etc/ipset*`
- Live kernel ipset/iptables state via `ipset save`, `iptables-save`, and `ip6tables-save` as a fallback, but only when the corresponding persistent config was not found (`firewall_runtime` role at manifest time)
- Optional user-specified extra files/dirs via `--include-path` (emitted as an `extra_paths` role at manifest time)
**Common flags**
- Remote harvesting:
- `--remote-host`, `--remote-user`, `--remote-port`, `--remote-ssh-config`
- `--no-sudo` (if you don't want/need sudo)
- Sensitive-data behaviour:
- default: tries to avoid likely secrets
- `--dangerous`: disables secret-safety checks (see “Sensitive data” below)
- Encrypt bundles at rest:
- `--sops <FINGERPRINT...>`: writes a single encrypted `harvest.tar.gz.sops` instead of a plaintext directory
- Path selection (include/exclude):
- `--include-path <PATTERN>` (repeatable): add extra files/dirs to harvest (even from locations normally ignored, like `/home`). Still subject to secret-safety checks unless `--dangerous`.
- `--exclude-path <PATTERN>` (repeatable): skip files/dirs even if they would normally be harvested.
- Pattern syntax:
- plain path: matches that file; directories match the directory + everything under it
- glob (default): supports `*` and `**` (prefix with `glob:` to force)
- regex: prefix with `re:` or `regex:`
- Precedence: excludes win over includes.
* Using remote mode and auth requires secrets?
* sudo password:
* `--ask-become-pass` (or `-K`) prompts for the sudo password.
* If you forget, and remote sudo requires a password, Enroll will still fall back to prompting in interactive mode (slightly slower due to retry).
* SSH private-key passphrase:
* `--ask-key-passphrase` prompts for the SSH key passphrase.
* `--ssh-key-passphrase-env ENV_VAR` reads the SSH key passphrase from an environment variable (useful for CI/non-interactive runs).
* If neither is provided, and Enroll detects an encrypted key in an interactive session, it will still fall back to prompting on-demand.
* In non-interactive sessions, pass `--ask-key-passphrase` or `--ssh-key-passphrase-env ENV_VAR` when using encrypted private keys.
* Note: `--ask-key-passphrase` and `--ssh-key-passphrase-env` are mutually exclusive.
Examples (encrypted SSH key)
```bash
# Interactive
enroll harvest --remote-host myhost.example.com --remote-user myuser --ask-key-passphrase --out /tmp/enroll-harvest
# Non-interactive / CI
export ENROLL_SSH_KEY_PASSPHRASE='correct horse battery staple'
enroll single-shot --remote-host myhost.example.com --remote-user myuser --ssh-key-passphrase-env ENROLL_SSH_KEY_PASSPHRASE --harvest /tmp/enroll-harvest --out /tmp/enroll-ansible --fqdn myhost.example.com
```
---
### `enroll manifest`
Generate Ansible output from an existing harvest bundle.
**Inputs**
- `--harvest /path/to/harvest` (directory)
or `--harvest /path/to/harvest.tar.gz.sops` (if using `--sops`)
**Output**
- In plaintext mode: an Ansible repo-like directory structure (roles/playbooks, and inventory in multi-site mode).
- In `--sops` mode: a single encrypted file `manifest.tar.gz.sops` containing the generated output.
**Common flags**
- `--fqdn <host>`: enables **multi-site** output style
**Role tags**
Generated playbooks tag each role so you can target just the parts you need:
- Tag format: `role_<role_name>` (e.g. `role_services`, `role_users`)
- Fallback/safe tag: `role_other`
Example:
```bash
ansible-playbook -i "localhost," -c local /tmp/enroll-ansible/playbook.yml --tags role_services,role_users
```
---
### `enroll single-shot`
Convenience wrapper that runs **harvest → manifest** in one command.
Use this when you want “get me something workable ASAP”.
Supports the same general flags as harvest/manifest, including `--fqdn`, remote harvest flags, and `--sops`.
---
### `enroll diff`
Compare two harvest bundles and report what changed.
**What it reports**
- Packages added/removed
- Services enabled added/removed, plus key state changes
- Users added/removed, plus field changes (uid/gid/home/shell/groups, etc.)
- Managed files added/removed/changed (metadata + content hash changes where available)
**Inputs**
- `--old <harvest>` and `--new <harvest>` (directories or `state.json` paths)
- `--sops` when comparing SOPS-encrypted harvest bundles
- `--exclude-path <PATTERN>` (repeatable) to ignore file/dir drift under matching paths (same pattern syntax as harvest)
- `--ignore-package-versions` to ignore package version-only drift (upgrades/downgrades)
- `--enforce` to apply the **old** harvest state locally (requires `ansible-playbook` on `PATH`)
**Noise suppression**
- `--exclude-path` is useful for things that change often but you still want in the harvest baseline (e.g. `/var/anacron`).
- `--ignore-package-versions` keeps routine upgrades from alerting; package add/remove drift is still reported.
**Enforcement (`--enforce`)**
If a diff exists and `ansible-playbook` is available, Enroll will:
1) generate a manifest from the **old** harvest into a temporary directory
2) run `ansible-playbook -i localhost, -c local <tmp>/playbook.yml` (often with `--tags role_<...>` to limit runtime)
3) record in the diff report that the old harvest was enforced
Enforcement is intentionally “safe”:
- reinstalls packages that were removed (`state: present`), but does **not** attempt downgrades/pinning
- restores users, files (contents + permissions/ownership), and service enable/start state
If `ansible-playbook` is not on `PATH`, Enroll returns an error and does not enforce.
**Output formats**
- `--format json` (default for webhooks)
- `--format markdown` / `--format text` (human-oriented)
**Notifications**
- Webhook:
- `--webhook <url>`
- `--webhook-format json|markdown|text`
- `--webhook-header 'Header-Name: value'` (repeatable)
- Email (optional):
- `--email-to <addr>` (plus optional SMTP/sendmail-related flags, depending on your install)
---
### `enroll explain`
Analyze a harvest and provide user-friendly explanations for what's in it and why.
This may also explain why something *wasn't* included (e.g a binary file, a file that was too large, unreadable due to permissions, or looked like a log file/secret.
Provide either the path to the harvest or the path to its state.json. It can also handle SOPS-encrypted harvests.
Output can be provided in plaintext or json.
---
### `enroll validate`
Validates a harvest by checking:
* state.json exists and is valid JSON
* state.json validates against a JSON Schema (by default the vendored one)
* Every `managed_file` entry has a corresponding artifact at: `artifacts/<role_name>/<src_rel>`
* That there are no **unreferenced files** sitting in `artifacts/` that aren't in the state.
#### Schema location + overrides
The master schema lives at: `enroll/schema/state.schema.json`.
You can override with a local file or URL:
```
enroll validate /path/to/harvest --schema ./state.schema.json
enroll validate /path/to/harvest --schema https://enroll.sh/schema/state.schema.json
```
Or skip schema checks (still does artifact consistency checks):
```
enroll validate /path/to/harvest --no-schema
```
#### CLI usage examples
Validate a local harvest:
```
enroll validate ./harvest
```
Validate a harvest tarball or a sops bundle:
```
enroll validate ./harvest.tar.gz
enroll validate ./harvest.sops --sops
```
JSON output + write to file:
```
enroll validate ./harvest --format json --out validate.json
```
Return exit code 1 for any warnings, not just errors (useful for CI):
```
enroll validate ./harvest --fail-on-warnings
```
---
## Sensitive data
By default, `enroll` does **not** assume how you handle secrets in Ansible. It will attempt to avoid harvesting likely sensitive data (private keys, passwords, tokens, etc.). This can mean it skips some config files you may ultimately want to manage.
If you opt in to collecting everything:
### `--dangerous`
**WARNING:** disables “likely secret” safety checks. This can copy private keys, TLS key material, API tokens, database passwords, and other credentials into the harvest output **in plaintext**.
If you intend to keep harvests/manifests long-term (especially in git), strongly consider encrypting them at rest.
### Encrypt bundles at rest with `--sops`
`--sops` encrypts the harvest and/or manifest outputs into a single `.tar.gz.sops` file (GPG). This is for **storage-at-rest**, not for direct “Ansible SOPS inventory” workflows.
⚠️ Important: `manifest --sops` produces one encrypted file. You must decrypt + extract it before running `ansible-playbook`.
---
## JinjaTurtle integration (both modes)
If [JinjaTurtle](https://git.mig5.net/mig5/jinjaturtle) is installed, `enroll` can generate Jinja2 templates for ini/json/xml/toml-style config.
- Templates live in `roles/<role>/templates/...`
- Variables live in:
- single-site: `roles/<role>/defaults/main.yml`
- multi-site: `inventory/host_vars/<fqdn>/<role>.yml`
You can force it on with `--jinjaturtle` or disable with `--no-jinjaturtle`.
---
## How multi-site avoids “shared role breaks a host”
In multi-site mode, roles are **data-driven**. The role tasks are generic (“deploy the files listed for this host”, “install the packages listed for this host”, “apply systemd enable/start state listed for this host”). Host inventory decides what applies per-host, avoiding the classic “host2 adds config, host1 breaks” failure mode.
---
# Install
## Ubuntu/Debian apt repository
```bash
sudo mkdir -p /usr/share/keyrings
curl -fsSL https://mig5.net/static/mig5.asc | sudo gpg --dearmor -o /usr/share/keyrings/mig5.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/mig5.gpg] https://apt.mig5.net $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/mig5.list
sudo apt update
sudo apt install enroll
```
## Fedora
```bash
sudo rpm --import https://mig5.net/static/mig5.asc
sudo tee /etc/yum.repos.d/mig5.repo > /dev/null << 'EOF'
[mig5]
name=mig5 Repository
baseurl=https://rpm.mig5.net/$releasever/rpm/$basearch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://mig5.net/static/mig5.asc
EOF
sudo dnf upgrade --refresh
sudo dnf install enroll
```
## AppImage
Download it from my Releases page, then:
Download the AppImage file from the Releases page (verify with GPG if you wish, my fingerprint is [here](https://mig5.net/static/mig5.asc),
then make it executable and run it:
```bash
chmod +x Enroll.AppImage
./Enroll.AppImage
```
## Pip/PipX
### Pip
```bash
pip install enroll
```
## Poetry (dev)
### Poetry
Clone this repository with git, then:
```bash
poetry install
poetry run enroll --help
```
---
## Usage
## Found a bug / have a suggestion?
On the host (root recommended):
My Forgejo doesn't currently support federation, so I haven't opened registration/login for issues.
Instead, email me (see `pyproject.toml`) or contact me on the Fediverse:
https://goto.mig5.net/@mig5
---
# Examples
## Harvest
### Local harvest
```bash
enroll harvest --out /tmp/enroll-harvest
```
### Remote harvest over SSH
```bash
enroll harvest --remote-host myhost.example.com --remote-user myuser --out /tmp/enroll-harvest
```
### Remote harvest over SSH, where the SSH configuration is in ~/.ssh/config (e.g a different SSH key)
Note: you must still pass `--remote-host`, but in this case, its value can be the 'Host' alias of an entry in your `~/.ssh/config`.
### 1. Harvest state/information about the host
```bash
enroll harvest --remote-host myhostalias --remote-ssh-config ~/.ssh/config --out /tmp/enroll-harvest
sudo poetry run enroll harvest --out /tmp/enroll-harvest
```
### Include paths (`--include-path`)
```bash
# Add a few dotfiles from /home (still secret-safe unless --dangerous)
enroll harvest --out /tmp/enroll-harvest --include-path '/home/*/.bashrc' --include-path '/home/*/.profile'
```
### Exclude paths (`--exclude-path`)
```bash
# Skip specific /usr/local/bin entries (or patterns)
enroll harvest --out /tmp/enroll-harvest --exclude-path '/usr/local/bin/docker-*' --exclude-path '/usr/local/bin/some-tool'
```
### Regex include
```bash
enroll harvest --out /tmp/enroll-harvest --include-path 're:^/home/[^/]+/\.config/myapp/.*$'
```
### `--dangerous`
```bash
enroll harvest --out /tmp/enroll-harvest --dangerous
```
### Remote + dangerous:
```bash
enroll harvest --remote-host myhost.example.com --remote-user myuser --dangerous
```
### `--sops` (encrypt at rest)
```bash
# Encrypted harvest bundle (writes /tmp/enroll-harvest/harvest.tar.gz.sops)
enroll harvest --out /tmp/enroll-harvest --dangerous --sops <FINGERPRINT(s)>
```
---
## Manifest
### Single-site (default: no --fqdn)
```bash
enroll manifest --harvest /tmp/enroll-harvest --out /tmp/enroll-ansible
```
### Multi-site (--fqdn)
```bash
enroll manifest --harvest /tmp/enroll-harvest --out /tmp/enroll-ansible --fqdn "$(hostname -f)"
```
### Manifest with `--sops`
```bash
# Generate encrypted manifest bundle (writes /tmp/enroll-ansible/manifest.tar.gz.sops)
enroll manifest --harvest /tmp/enroll-harvest/harvest.tar.gz.sops --out /tmp/enroll-ansible --sops <FINGERPRINT(s)>
# Decrypt/extract the manifest bundle, then run Ansible from inside ./manifest/
cd /tmp/enroll-ansible
sops -d manifest.tar.gz.sops | tar -xzvf -
cd manifest
```
---
## Single-shot
### 2. Generate Ansible manifests (roles/playbook) from that harvest
```bash
enroll single-shot --harvest /tmp/enroll-harvest --out /tmp/enroll-ansible --fqdn "$(hostname -f)"
sudo poetry run enroll manifest --harvest /tmp/enroll-harvest --out /tmp/enroll-ansible
```
Remote single-shot (run harvest over SSH, then manifest locally):
```bash
enroll single-shot --remote-host myhost.example.com --remote-user myuser --harvest /tmp/enroll-harvest --out /tmp/enroll-ansible --fqdn "myhost.example.com"
```
---
## Diff
### Compare two harvest directories, output in json
```bash
enroll diff --old /path/to/harvestA --new /path/to/harvestB --format json
```
### Diff + webhook notify
```bash
enroll diff --old /path/to/golden/harvest --new /path/to/new/harvest --webhook https://nr.mig5.net/forms/webhooks/xxxx --webhook-format json --webhook-header 'X-Enroll-Secret: xxxx'
```
`diff` mode also supports email sending and text or markdown format, as well as `--exit-code` mode to trigger a return code of 2 (useful for crons or CI)
### Ignore a specific directory or file from the diff
```bash
enroll diff --old /path/to/harvestA --new /path/to/harvestB --exclude-path /var/anacron
```
### Ignore package version drift (routine upgrades) but still alert on add/remove
```bash
enroll diff --old /path/to/harvestA --new /path/to/harvestB --ignore-package-versions
```
### Enforce the old harvest state when drift is detected (requires Ansible)
```bash
enroll diff --old /path/to/harvestA --new /path/to/harvestB --enforce --ignore-package-versions --exclude-path /var/anacron
```
---
## Explain
### Explain a harvest
All of these do the same thing:
### Alternatively, do both steps in one shot:
```bash
enroll explain /path/to/state.json
enroll explain /path/to/bundle_dir
enroll explain /path/to/harvest.tar.gz
sudo poetry run enroll enroll --harvest /tmp/enroll-harvest --out /tmp/enroll-ansible
```
### Explain a SOPS-encrypted harvest
Then run:
```bash
enroll explain /path/to/harvest.tar.gz.sops --sops
```
### Explain with JSON output and more examples
```bash
enroll explain /path/to/state.json --format json --max-examples 25
```
### Example output
```
❯ enroll explain /tmp/syrah.harvest
Enroll explain: /tmp/syrah.harvest
Host: syrah.mig5.net (os: debian, pkg: dpkg)
Enroll: 0.2.3
Inventory
- Packages: 254
- Why packages were included (observed_via):
- user_installed: 248 – Package appears explicitly installed (as opposed to only pulled in as a dependency).
- package_role: 232 – Package was referenced by an enroll packages snapshot/role. (e.g. acl, acpid, adduser)
- systemd_unit: 22 – Package is associated with a systemd unit that was harvested. (e.g. postfix.service, tor.service, apparmor.service)
Roles collected
- users: 1 user(s), 1 file(s), 0 excluded
- services: 19 unit(s), 111 file(s), 6 excluded
- packages: 232 package snapshot(s), 41 file(s), 0 excluded
- apt_config: 26 file(s), 7 dir(s), 10 excluded
- dnf_config: 0 file(s), 0 dir(s), 0 excluded
- firewall_runtime: 2 snapshot(s), 1 ipset(s)
- etc_custom: 70 file(s), 20 dir(s), 0 excluded
- usr_local_custom: 35 file(s), 1 dir(s), 0 excluded
- extra_paths: 0 file(s), 0 dir(s), 0 excluded
Why files were included (managed_files.reason)
- custom_unowned (179): A file not owned by any package (often custom/operator-managed).. Examples: /etc/apparmor.d/local/lsb_release, /etc/apparmor.d/local/nvidia_modprobe, /etc/apparmor.d/local/sbin.dhclient
- usr_local_bin_script (35): Executable scripts under /usr/local/bin (often operator-installed).. Examples: /usr/local/bin/check_firewall, /usr/local/bin/awslogs
- apt_keyring (13): Repository signing key material used by APT.. Examples: /etc/apt/keyrings/openvpn-repo-public.asc, /etc/apt/trusted.gpg, /etc/apt/trusted.gpg.d/deb.torproject.org-keyring.gpg
- modified_conffile (10): A package-managed conffile differs from the packaged/default version.. Examples: /etc/dnsmasq.conf, /etc/ssh/moduli, /etc/tor/torrc
- logrotate_snippet (9): logrotate snippets/configs referenced in system configuration.. Examples: /etc/logrotate.d/rsyslog, /etc/logrotate.d/tor, /etc/logrotate.d/apt
- apt_config (7): APT configuration affecting package installation and repository behavior.. Examples: /etc/apt/apt.conf.d/01autoremove, /etc/apt/apt.conf.d/20listchanges, /etc/apt/apt.conf.d/70debconf
[...]
```
---
## Run Ansible
### Single-site
```bash
ansible-playbook -i "localhost," -c local /tmp/enroll-ansible/playbook.yml
```
### Multi-site (--fqdn)
```bash
ansible-playbook /tmp/enroll-ansible/playbooks/"$(hostname -f)".yml
```
### Run only specific roles (tags)
Generated playbooks tag each role as `role_<name>` (e.g. `role_users`, `role_services`), so you can speed up targeted runs:
```bash
ansible-playbook -i "localhost," -c local /tmp/enroll-ansible/playbook.yml --tags role_users
```
## Notes / Safety
## Configuration file
- enroll **skips** common sensitive locations like `/etc/ssl/private/*`, `/etc/ssh/ssh_host_*`, and files that look like private keys/tokens.
- It also skips symlinks, binary-ish files, and large files by default.
- Review each generated role’s README before committing it anywhere.
- It only stores the raw config files. If you want to turn these into Jinja2 templates with dynamic inventory, see my other tool https://git.mig5.net/mig5/jinjaturtle .
As can be seen above, there are a lot of powerful 'permutations' available to all four subcommands.
Sometimes, it can be easier to store them in a config file so you don't have to remember them!
## Troubleshooting
Enroll supports reading an ini-style file of all the arguments for each subcommand.
- Run as root for the most complete harvest (`sudo ...`).
### Location of the config file
## Found a bug, have a suggestion?
The path the config file can be specified with `-c` or `--config` on the command-line. Otherwise,
Enroll will look for `./enroll.ini`, `./.enroll.ini` (in the current working directory),
`~/.config/enroll/enroll.ini` (or `$XDG_CONFIG_HOME/enroll/enroll.ini`).
You can e-mail me (see the pyproject.toml for details) or contact me on the Fediverse:
You may also pass `--no-config` if you deliberately want to ignore the config file even if it existed.
### Precedence
Highest wins:
* Explicit CLI flags
* INI config ([cmd], [enroll])
* argparse defaults
### Example config file
Here is an example.
Whenever an argument on the command-line has a 'hyphen' in it, just be sure to change it to an underscore in the ini file.
```ini
[enroll]
# (future global flags may live here)
[harvest]
dangerous = false
include_path =
/home/*/.bashrc
/home/*/.profile
exclude_path = /usr/local/bin/docker-*, /usr/local/bin/some-tool
# remote_host = yourserver.example.com
# remote_user = you
# remote_port = 2222
[manifest]
# you can set defaults here too, e.g.
no_jinjaturtle = true
sops = 54A91143AE0AB4F7743B01FE888ED1B423A3BC99
[diff]
# ignore noisy drift
exclude_path = /var/anacron
ignore_package_versions = true
# enforce = true # requires ansible-playbook on PATH
[single-shot]
# if you use single-shot, put its defaults here.
# It does not inherit those of the subsections above, so you
# may wish to repeat them here.
include_path = re:^/home/[^/]+/\.config/myapp/.*$
```
https://goto.mig5.net/@mig5

173
debian/changelog vendored
View file

@ -1,173 +0,0 @@
enroll (0.6.0) unstable; urgency=medium
* Add support for capturing ipset and iptables configuration files
* Add support for generating ipset and iptables configuration files from runtime, if the former weren't present ('firewall_runtime' role)
-- Miguel Jacq <mig@mig5.net> Thu, 14 May 2026 15:00 +1000
enroll (0.5.0) unstable; urgency=medium
* Add ssh config support where JinjaTurtle is used
-- Miguel Jacq <mig@mig5.net> Tue, 12 May 2026 12:00 +1000
enroll (0.4.4) unstable; urgency=medium
* Add capability to handle passphrases on encrypted SSH private keys. Prompting can be forced with `--ask-key-passphrase` or automated (e.g for CI) with `--ssh-key-passphrase env SOMEVAR`
-- Miguel Jacq <mig@mig5.net> Tue, 17 Feb 2026 11:00 +1100
enroll (0.4.3) unstable; urgency=medium
* Add support for AddressFamily and ConnectTimeout in the .ssh/config when using `--remote-ssh-config`.
-- Miguel Jacq <mig@mig5.net> Fri, 16 Jan 2026 11:00 +1100
enroll (0.4.2) unstable; urgency=medium
* Support `--remote-ssh-config [path-to-ssh-config]` as an argument in case extra params are required beyond `--remote-port` or `--remote-user`. Note: `--remote-host` must still be set, but it can be an 'alias' represented by the 'Host' value in the ssh config.
-- Miguel Jacq <mig@mig5.net> Tue, 13 Jan 2026 21:55:00 +1100
enroll (0.4.1) unstable; urgency=medium
* Add interactive output when 'enroll diff --enforce' is invoking Ansible.
-- Miguel Jacq <mig@mig5.net> Sun, 11 Jan 2026 10:00:00 +1100
enroll (0.4.0) unstable; urgency=medium
* Introduce `enroll validate` - a tool to validate a harvest against the state schema, or check for missing or orphaned obsolete artifacts in a harvest.
* Attempt to generate Jinja2 templates of systemd unit files and Postfix main.cf (now that JinjaTurtle supports it)
* Update pynacl dependency to resolve CVE-2025-69277
* Add `--exclude-path` to `enroll diff` command, so that you can ignore certain churn from the diff (stuff you still wanted to harvest as a baseline but don't care if it changes day to day)
* Add `--ignore-package-versions` to `enroll diff` command, to optionally ignore package upgrades (e.g due to patching) from the diff.
* Add tags to the playbook for each role, to allow easier targeting of specific roles during play later.
* Add `--enforce` mode to `enroll diff`. If there is diff detected between the two harvests, and it can enforce restoring the state from the older harvest, it will manifest the state and apply it with ansible.
Only the specific roles that had diffed will be applied (via the new tags capability)
-- Miguel Jacq <mig@mig5.net> Sat, 10 Jan 2026 10:30:00 +1100
enroll (0.3.0) unstable; urgency=medium
* Introduce `enroll explain` - a tool to analyze and explain what's in (or not in) a harvest and why.
* Centralise the cron and logrotate stuff into their respective roles, we had a bit of duplication between roles based on harvest discovery.
* Capture other files in the user's home directory such as `.bashrc`, `.bash_aliases`, `.profile`, if these files differ from the `/etc/skel` defaults
* Ignore files that end with a tilde or - (probably backup files generated by editors or shadow file changes)
* Manage certain symlinks e.g for apache2/nginx sites-enabled and so on
-- Miguel Jacq <mig@mig5.net> Mon, 05 Jan 2026 17:00:00 +1100
enroll (0.2.3) unstable; urgency=medium
* Introduce --ask-become-pass or -K to support password-required sudo on remote hosts, just like Ansible. It will also fall back to this prompt if a password is required but the arg wasn't passed in.
-- Miguel Jacq <mig@mig5.net> Sun, 04 Jan 2026 20:38:00 +1100
enroll (0.2.2) unstable; urgency=medium
* Fix stat() of parent directory so that we set directory perms correct on --include paths.
* Set pty for remote calls when sudo is required, to help systems with limits on sudo without pty
-- Miguel Jacq <mig@mig5.net> Sat, 03 Jan 2026 09:56:00 +1100
enroll (0.2.1) unstable; urgency=medium
* Don't accidentally add extra_paths role to usr_local_custom list, resulting in extra_paths appearing twice in manifested playbook
* Ensure directories in the tree of anything included with --include are defined in the state and manifest so we make dirs before we try to create files
-- Miguel Jacq <mig@mig5.net> Fri, 02 Jan 2026 21:30:00 +1100
enroll (0.2.0) unstable; urgency=medium
* Add version CLI arg
* Add ability to enroll RH-style systems (DNF5/DNF/RPM)
* Refactor harvest state to track package versions
-- Miguel Jacq <mig@mig5.net> Mon, 29 Dec 2025 17:30:00 +1100
enroll (0.1.7) unstable; urgency=medium
* Fix an attribution bug for certain files ending up in the wrong package/role.
-- Miguel Jacq <mig@mig5.net> Sun, 28 Dec 2025 18:30:00 +1100
enroll (0.1.6) unstable; urgency=medium
* DRY up some code logic
* More test coverage
-- Miguel Jacq <mig@mig5.net> Sun, 28 Dec 2025 15:30:00 +1100
enroll (0.1.5) unstable; urgency=medium
* Consolidate logrotate and cron files into their main service/package roles if they exist.
* Standardise on MAX_FILES_CAP in one place
* Manage apt stuff in its own role, not in etc_custom
-- Miguel Jacq <mig@mig5.net> Sun, 28 Dec 2025 10:00:00 +1100
enroll (0.1.4) unstable; urgency=medium
* Attempt to capture more stuff from /etc that might not be attributable to a specific package. This includes common singletons and systemd timers
* Avoid duplicate apt data in package-specific roles.
-- Miguel Jacq <mig@mig5.net> Sat, 27 Dec 2025 19:00:00 +1100
enroll (0.1.3) unstable; urgency=medium
* Allow the user to add extra paths to harvest, or paths to ignore, using `--exclude-path` and `--include-path`
arguments.
* Add support for an enroll.ini config file to store arguments per subcommand, to avoid having to remember
them all for repetitive executions.
-- Miguel Jacq <mig@mig5.net> Sat, 20 Dec 2025 18:24:00 +1100
enroll (0.1.2) unstable; urgency=medium
* Include files from `/usr/local/bin` and `/usr/local/etc` in harvest (assuming they aren't binaries or
symlinks) and store in `usr_local_custom` role, similar to `etc_custom`.
-- Miguel Jacq <mig@mig5.net> Thu, 18 Dec 2025 17:07:00 +1100
enroll (0.1.1) unstable; urgency=medium
* Add `diff` subcommand which can compare two harvests and send email or webhook notifications in different
formats.
-- Miguel Jacq <mig@mig5.net> Thu, 18 Dec 2025 15:00:00 +1100
enroll (0.1.0) unstable; urgency=medium
* Add remote mode for harvesting a remote machine via a local workstation (no need to install enroll remotely)
Optionally use `--no-sudo` if you don't want the remote user to have passwordless sudo when conducting the
harvest, albeit you'll end up with less useful data (same as if running `enroll harvest` on a machine without
sudo)
* Add `--dangerous` flag to capture even sensitive data (use at your own risk!)
* Add `--sops` flag which makes the harvest and the manifest 'out' data encrypted as a single SOPS data file.
This would make `--dangerous` a little bit safer, if your intention is just to store the Ansible manifest
in git or somewhere similar for disaster-recovery purposes (e.g encrypted at rest for safe-keeping).
* Do a better job at capturing other config files in `/etc/<package>/` even if that package doesn't normally
ship or manage those files.
* Don't collect files ending in `.log`
-- Miguel Jacq <mig@mig5.net> Wed, 17 Dec 2025 18:00:00 +1100
enroll (0.0.5) unstable; urgency=medium
* Use JinjaTurtle to generate dynamic template/inventory if it's on the PATH
* Support --fqdn flag for site-specific inventory and an inventory hosts file
* Generate an ansible.cfg if not present, to support host_vars plugin and other params
* Be more permissive with files that we previously thought contained secrets (ignore commented lines)
-- Miguel Jacq <mig@mig5.net> Tue, 16 Dec 2025 12:00:00 +1100
enroll (0.0.4) unstable; urgency=medium
* Fix dash package detection issue
* Reorder which roles install first
-- Miguel Jacq <mig@mig5.net> Mon, 15 Dec 2025 17:00:00 +1100
enroll (0.0.3) unstable; urgency=medium
* Initial package
-- Miguel Jacq <mig@mig5.net> Mon, 15 Dec 2025 12:00:00 +1100

22
debian/control vendored
View file

@ -1,22 +0,0 @@
Source: enroll
Section: admin
Priority: optional
Maintainer: Miguel Jacq <mig@mig5.net>
Rules-Requires-Root: no
Build-Depends:
debhelper-compat (= 13),
dh-python,
pybuild-plugin-pyproject,
python3-all,
python3-yaml,
python3-poetry-core,
python3-paramiko,
python3-jsonschema
Standards-Version: 4.6.2
Homepage: https://git.mig5.net/mig5/enroll
Package: enroll
Architecture: all
Depends: ${misc:Depends}, ${python3:Depends}, python3-yaml, python3-paramiko, python3-jsonschema
Description: Harvest a host into Ansible roles
A tool that inspects a system and emits Ansible roles/playbooks to reproduce it.

6
debian/rules vendored
View file

@ -1,6 +0,0 @@
#!/usr/bin/make -f
export PYBUILD_NAME=enroll
export PYBUILD_SYSTEM=pyproject
%:
dh $@ --with python3 --buildsystem=pybuild

1
debian/source/format vendored
View file

@ -1 +0,0 @@
3.0 (quilt)

6
debian/source/options vendored
View file

@ -1,6 +0,0 @@
tar-ignore = ".git"
tar-ignore = ".venv"
tar-ignore = "__pycache__"
tar-ignore = ".pytest_cache"
tar-ignore = "dist"
tar-ignore = "build"

1
enroll.svg
View file

@ -109,3 +109,4 @@
<tspan class="text-dark">en</tspan><tspan class="text-light">roll</tspan>
</text>
</svg>

Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 4.4 KiB

After

Width:  |  Height:  |  Size: 4.4 KiB

Before After
Before After

79
enroll/cache.py
View file

@ -1,79 +0,0 @@
from __future__ import annotations
import os
import re
import tempfile
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
from typing import Optional
def _safe_component(s: str) -> str:
s = s.strip()
if not s:
return "unknown"
s = re.sub(r"[^A-Za-z0-9_.-]+", "_", s)
s = re.sub(r"_+", "_", s)
return s[:64]
def enroll_cache_dir() -> Path:
"""Return the base cache directory for enroll.
We default to ~/.local/cache to match common Linux conventions in personal
homedirs, but honour XDG_CACHE_HOME if set.
"""
base = os.environ.get("XDG_CACHE_HOME")
if base:
root = Path(base).expanduser()
else:
root = Path.home() / ".local" / "cache"
return root / "enroll"
@dataclass(frozen=True)
class HarvestCache:
"""A locally-persistent directory that holds a harvested bundle."""
dir: Path
@property
def state_json(self) -> Path:
return self.dir / "state.json"
def _ensure_dir_secure(path: Path) -> None:
"""Create a directory with restrictive permissions; refuse symlinks."""
# Refuse a symlink at the leaf.
if path.exists() and path.is_symlink():
raise RuntimeError(f"Refusing to use symlink path: {path}")
path.mkdir(parents=True, exist_ok=True, mode=0o700)
try:
os.chmod(path, 0o700)
except OSError:
# Best-effort; on some FS types chmod may fail.
pass
def new_harvest_cache_dir(*, hint: Optional[str] = None) -> HarvestCache:
"""Create a new, unpredictable harvest directory under the user's cache.
This mitigates pre-guessing attacks (e.g. an attacker creating a directory
in advance in a shared temp location) by creating the bundle directory under
the user's home and using mkdtemp() randomness.
"""
base = enroll_cache_dir() / "harvest"
_ensure_dir_secure(base)
ts = datetime.now().strftime("%Y%m%d-%H%M%S")
safe = _safe_component(hint or "harvest")
prefix = f"{ts}-{safe}-"
# mkdtemp creates a new directory with a random suffix.
d = Path(tempfile.mkdtemp(prefix=prefix, dir=str(base)))
try:
os.chmod(d, 0o700)
except OSError:
pass
return HarvestCache(dir=d)

1134
enroll/cli.py
View file

File diff suppressed because it is too large Load diff

103
enroll/debian.py
View file

@ -1,12 +1,11 @@
from __future__ import annotations
import glob
import hashlib
import os
import subprocess # nosec
from typing import Dict, List, Optional, Set, Tuple
_DIVERSION_PREFIX = "diversion by "
def _run(cmd: list[str]) -> str:
p = subprocess.run(cmd, check=False, text=True, capture_output=True) # nosec
@ -19,32 +18,9 @@ def dpkg_owner(path: str) -> Optional[str]:
p = subprocess.run(["dpkg", "-S", path], text=True, capture_output=True) # nosec
if p.returncode != 0:
return None
for raw in (p.stdout or "").splitlines():
line = raw.strip()
if not line:
continue
# dpkg diversion chatter; not an ownership line
if line.startswith(_DIVERSION_PREFIX):
continue
# Expected: "<pkg>[, <pkg2>...][:<arch>]: <path>"
if ":" not in line:
continue
left, _ = line.split(":", 1)
# If multiple pkgs listed, pick the first (common case is just one)
left = left.split(",", 1)[0].strip()
# Strip any ":arch" suffix from left side
left = p.stdout.split(":", 1)[0].strip()
pkg = left.split(":", 1)[0].strip()
if pkg and not pkg.startswith(_DIVERSION_PREFIX):
return pkg
return None
return pkg or None
def list_manual_packages() -> List[str]:
@ -63,50 +39,6 @@ def list_manual_packages() -> List[str]:
return sorted(set(pkgs))
def list_installed_packages() -> Dict[str, List[Dict[str, str]]]:
"""Return mapping of installed package name -> installed instances.
Uses dpkg-query and is expected to work on Debian/Ubuntu-like systems.
Output format:
{"pkg": [{"version": "...", "arch": "..."}, ...], ...}
"""
try:
p = subprocess.run(
[
"dpkg-query",
"-W",
"-f=${Package}\t${Version}\t${Architecture}\n",
],
text=True,
capture_output=True,
check=False,
) # nosec
except Exception:
return {}
out: Dict[str, List[Dict[str, str]]] = {}
for raw in (p.stdout or "").splitlines():
line = raw.strip("\n")
if not line:
continue
parts = line.split("\t")
if len(parts) < 3:
continue
name, ver, arch = parts[0].strip(), parts[1].strip(), parts[2].strip()
if not name:
continue
out.setdefault(name, []).append({"version": ver, "arch": arch})
# Stable ordering for deterministic JSON dumps.
for k in list(out.keys()):
out[k] = sorted(
out[k], key=lambda x: (x.get("arch") or "", x.get("version") or "")
)
return out
def build_dpkg_etc_index(
info_dir: str = "/var/lib/dpkg/info",
) -> Tuple[Set[str], Dict[str, str], Dict[str, Set[str]], Dict[str, List[str]]]:
@ -197,9 +129,7 @@ def parse_status_conffiles(
if ":" in line:
k, v = line.split(":", 1)
key = k
# Preserve leading spaces in continuation lines, but strip
# the trailing newline from the initial key line value.
cur[key] = v.lstrip().rstrip("\n")
cur[key] = v.lstrip()
if cur:
flush()
@ -223,3 +153,28 @@ def read_pkg_md5sums(pkg: str) -> Dict[str, str]:
md5, rel = line.split(None, 1)
m[rel.strip()] = md5.strip()
return m
def file_md5(path: str) -> str:
h = hashlib.md5() # nosec
with open(path, "rb") as f:
for chunk in iter(lambda: f.read(1024 * 1024), b""):
h.update(chunk)
return h.hexdigest()
def stat_triplet(path: str) -> Tuple[str, str, str]:
st = os.stat(path, follow_symlinks=True)
mode = oct(st.st_mode & 0o777)[2:].zfill(4)
import pwd, grp
try:
owner = pwd.getpwuid(st.st_uid).pw_name
except KeyError:
owner = str(st.st_uid)
try:
group = grp.getgrgid(st.st_gid).gr_name
except KeyError:
group = str(st.st_gid)
return owner, group, mode

1353
enroll/diff.py
View file

File diff suppressed because it is too large Load diff

598
enroll/explain.py
View file

@ -1,598 +0,0 @@
from __future__ import annotations
import json
from collections import Counter, defaultdict
from dataclasses import dataclass
from typing import Any, Dict, Iterable, List, Tuple
from .diff import _bundle_from_input, _load_state # reuse existing bundle handling
@dataclass(frozen=True)
class ReasonInfo:
title: str
why: str
_MANAGED_FILE_REASONS: Dict[str, ReasonInfo] = {
# Package manager / repo config
"apt_config": ReasonInfo(
"APT configuration",
"APT configuration affecting package installation and repository behavior.",
),
"apt_source": ReasonInfo(
"APT repository source",
"APT source list entries (e.g. sources.list or sources.list.d).",
),
"apt_keyring": ReasonInfo(
"APT keyring",
"Repository signing key material used by APT.",
),
"apt_signed_by_keyring": ReasonInfo(
"APT Signed-By keyring",
"Keyring referenced via a Signed-By directive in an APT source.",
),
"yum_conf": ReasonInfo(
"YUM/DNF main config",
"Primary YUM configuration (often /etc/yum.conf).",
),
"yum_config": ReasonInfo(
"YUM/DNF config",
"YUM/DNF configuration files (including conf.d).",
),
"yum_repo": ReasonInfo(
"YUM/DNF repository",
"YUM/DNF repository definitions (e.g. yum.repos.d).",
),
"dnf_config": ReasonInfo(
"DNF configuration",
"DNF configuration affecting package installation and repositories.",
),
"rpm_gpg_key": ReasonInfo(
"RPM GPG key",
"Repository signing keys used by RPM/YUM/DNF.",
),
# SSH
"authorized_keys": ReasonInfo(
"SSH authorized keys",
"User authorized_keys files (controls who can log in with SSH keys).",
),
"ssh_public_key": ReasonInfo(
"SSH public key",
"SSH host/user public keys relevant to authentication.",
),
# System config / security
"system_security": ReasonInfo(
"Security configuration",
"Security-sensitive configuration (SSH, sudoers, PAM, auth, etc.).",
),
"system_network": ReasonInfo(
"Network configuration",
"Network configuration (interfaces, resolv.conf, network managers, etc.).",
),
"system_firewall": ReasonInfo(
"Firewall configuration",
"Firewall rules/configuration (ufw, nftables, iptables, ipset, etc.).",
),
"system_sysctl": ReasonInfo(
"sysctl configuration",
"Kernel sysctl tuning (sysctl.conf / sysctl.d).",
),
"system_modprobe": ReasonInfo(
"modprobe configuration",
"Kernel module configuration (modprobe.d).",
),
"system_mounts": ReasonInfo(
"Mount configuration",
"Mount configuration (e.g. /etc/fstab and related).",
),
"system_rc": ReasonInfo(
"Startup/rc configuration",
"Startup scripts / rc configuration that can affect boot behavior.",
),
# systemd + timers
"systemd_dropin": ReasonInfo(
"systemd drop-in",
"systemd override/drop-in files that modify a unit's behavior.",
),
"systemd_envfile": ReasonInfo(
"systemd EnvironmentFile",
"Files referenced by systemd units via EnvironmentFile.",
),
"related_timer": ReasonInfo(
"Related systemd timer",
"A systemd timer captured because it is related to a unit/service.",
),
# cron / logrotate
"system_cron": ReasonInfo(
"System cron",
"System cron configuration (crontab, cron.d, etc.).",
),
"cron_snippet": ReasonInfo(
"Cron snippet",
"Cron snippets referenced/used by harvested services or configs.",
),
"system_logrotate": ReasonInfo(
"System logrotate",
"System logrotate configuration.",
),
"logrotate_snippet": ReasonInfo(
"logrotate snippet",
"logrotate snippets/configs referenced in system configuration.",
),
# Custom paths / drift signals
"modified_conffile": ReasonInfo(
"Modified package conffile",
"A package-managed conffile differs from the packaged/default version.",
),
"modified_packaged_file": ReasonInfo(
"Modified packaged file",
"A file owned by a package differs from the packaged version.",
),
"custom_unowned": ReasonInfo(
"Unowned custom file",
"A file not owned by any package (often custom/operator-managed).",
),
"custom_specific_path": ReasonInfo(
"Custom specific path",
"A specific path included by a custom rule or snapshot.",
),
"usr_local_bin_script": ReasonInfo(
"/usr/local/bin script",
"Executable scripts under /usr/local/bin (often operator-installed).",
),
"usr_local_etc_custom": ReasonInfo(
"/usr/local/etc custom",
"Custom configuration under /usr/local/etc.",
),
# User includes
"user_include": ReasonInfo(
"User-included path",
"Included because you specified it via --include-path / include patterns.",
),
}
_MANAGED_DIR_REASONS: Dict[str, ReasonInfo] = {
"parent_of_managed_file": ReasonInfo(
"Parent directory",
"Included so permissions/ownership can be recreated for managed files.",
),
"user_include_dir": ReasonInfo(
"User-included directory",
"Included because you specified it via --include-path / include patterns.",
),
}
_EXCLUDED_REASONS: Dict[str, ReasonInfo] = {
"user_excluded": ReasonInfo(
"User excluded",
"Excluded because you explicitly excluded it (e.g. --exclude-path / patterns).",
),
"unreadable": ReasonInfo(
"Unreadable",
"Enroll could not read this path with the permissions it had.",
),
"log_file": ReasonInfo(
"Log file",
"Excluded because it appears to be a log file (usually noisy/large).",
),
"denied_path": ReasonInfo(
"Denied path",
"Excluded because the path is in a denylist for safety.",
),
"too_large": ReasonInfo(
"Too large",
"Excluded because it exceeded the size limit for harvested files.",
),
"not_regular_file": ReasonInfo(
"Not a regular file",
"Excluded because it was not a regular file (device, socket, etc.).",
),
"binary_like": ReasonInfo(
"Binary-like",
"Excluded because it looked like binary content (not useful for config management).",
),
"sensitive_content": ReasonInfo(
"Sensitive content",
"Excluded because it likely contains secrets (e.g. shadow, private keys).",
),
}
_OBSERVED_VIA: Dict[str, ReasonInfo] = {
"user_installed": ReasonInfo(
"User-installed",
"Package appears explicitly installed (as opposed to only pulled in as a dependency).",
),
"systemd_unit": ReasonInfo(
"Referenced by systemd unit",
"Package is associated with a systemd unit that was harvested.",
),
"package_role": ReasonInfo(
"Referenced by package role",
"Package was referenced by an enroll packages snapshot/role.",
),
"firewall_runtime": ReasonInfo(
"Referenced by firewall runtime role",
"Package was referenced by captured live ipset/iptables runtime state.",
),
}
def _ri(mapping: Dict[str, ReasonInfo], key: str) -> ReasonInfo:
return mapping.get(key) or ReasonInfo(key, f"Captured with reason '{key}'")
def _role_common_counts(role_obj: Dict[str, Any]) -> Tuple[int, int, int, int]:
"""Return (managed_files, managed_dirs, excluded, notes) counts for a RoleCommon object."""
mf = len(role_obj.get("managed_files") or [])
md = len(role_obj.get("managed_dirs") or [])
ex = len(role_obj.get("excluded") or [])
nt = len(role_obj.get("notes") or [])
return mf, md, ex, nt
def _summarize_reasons(
items: Iterable[Dict[str, Any]],
reason_key: str,
*,
mapping: Dict[str, ReasonInfo],
max_examples: int,
) -> List[Dict[str, Any]]:
by_reason: Dict[str, List[str]] = defaultdict(list)
counts: Counter[str] = Counter()
for it in items:
if not isinstance(it, dict):
continue
r = it.get(reason_key)
if not r:
continue
r = str(r)
counts[r] += 1
p = it.get("path")
if (
max_examples > 0
and isinstance(p, str)
and p
and len(by_reason[r]) < max_examples
):
by_reason[r].append(p)
out: List[Dict[str, Any]] = []
for reason, count in counts.most_common():
info = _ri(mapping, reason)
out.append(
{
"reason": reason,
"count": count,
"title": info.title,
"why": info.why,
"examples": by_reason.get(reason, []),
}
)
return out
def explain_state(
harvest: str,
*,
sops_mode: bool = False,
fmt: str = "text",
max_examples: int = 3,
) -> str:
"""Explain a harvest bundle's state.json.
`harvest` may be:
- a bundle directory
- a path to state.json
- a tarball (.tar.gz/.tgz)
- a SOPS-encrypted bundle (.sops)
"""
bundle = _bundle_from_input(harvest, sops_mode=sops_mode)
state = _load_state(bundle.dir)
host = state.get("host") or {}
enroll = state.get("enroll") or {}
roles = state.get("roles") or {}
inv = state.get("inventory") or {}
inv_pkgs = (inv.get("packages") or {}) if isinstance(inv, dict) else {}
role_summaries: List[Dict[str, Any]] = []
# Users
users_obj = roles.get("users") or {}
user_entries = users_obj.get("users") or []
mf, md, ex, _nt = (
_role_common_counts(users_obj) if isinstance(users_obj, dict) else (0, 0, 0, 0)
)
role_summaries.append(
{
"role": "users",
"summary": f"{len(user_entries)} user(s), {mf} file(s), {ex} excluded",
"notes": users_obj.get("notes") or [],
}
)
# Services
services_list = roles.get("services") or []
if isinstance(services_list, list):
total_mf = sum(
len((s.get("managed_files") or []))
for s in services_list
if isinstance(s, dict)
)
total_ex = sum(
len((s.get("excluded") or [])) for s in services_list if isinstance(s, dict)
)
role_summaries.append(
{
"role": "services",
"summary": f"{len(services_list)} unit(s), {total_mf} file(s), {total_ex} excluded",
"units": [
{
"unit": s.get("unit"),
"active_state": s.get("active_state"),
"sub_state": s.get("sub_state"),
"unit_file_state": s.get("unit_file_state"),
"condition_result": s.get("condition_result"),
}
for s in services_list
if isinstance(s, dict)
],
}
)
# Package snapshots
pkgs_list = roles.get("packages") or []
if isinstance(pkgs_list, list):
total_mf = sum(
len((p.get("managed_files") or []))
for p in pkgs_list
if isinstance(p, dict)
)
total_ex = sum(
len((p.get("excluded") or [])) for p in pkgs_list if isinstance(p, dict)
)
role_summaries.append(
{
"role": "packages",
"summary": f"{len(pkgs_list)} package snapshot(s), {total_mf} file(s), {total_ex} excluded",
"packages": [
p.get("package") for p in pkgs_list if isinstance(p, dict)
],
}
)
# Runtime firewall snapshot
firewall_obj = roles.get("firewall_runtime") or {}
if isinstance(firewall_obj, dict) and firewall_obj:
captures = [
key
for key in ("ipset_save", "iptables_v4_save", "iptables_v6_save")
if firewall_obj.get(key)
]
role_summaries.append(
{
"role": "firewall_runtime",
"summary": f"{len(captures)} snapshot(s), {len(firewall_obj.get('ipset_sets') or [])} ipset(s)",
"notes": firewall_obj.get("notes") or [],
}
)
# Single snapshots
for rname in [
"apt_config",
"dnf_config",
"etc_custom",
"usr_local_custom",
"extra_paths",
]:
robj = roles.get(rname) or {}
if not isinstance(robj, dict):
continue
mf, md, ex, _nt = _role_common_counts(robj)
extra: Dict[str, Any] = {}
if rname == "extra_paths":
extra = {
"include_patterns": robj.get("include_patterns") or [],
"exclude_patterns": robj.get("exclude_patterns") or [],
}
role_summaries.append(
{
"role": rname,
"summary": f"{mf} file(s), {md} dir(s), {ex} excluded",
"notes": robj.get("notes") or [],
**extra,
}
)
# Flatten managed/excluded across roles
all_managed_files: List[Dict[str, Any]] = []
all_managed_dirs: List[Dict[str, Any]] = []
all_excluded: List[Dict[str, Any]] = []
def _consume_role(role_obj: Dict[str, Any]) -> None:
for f in role_obj.get("managed_files") or []:
if isinstance(f, dict):
all_managed_files.append(f)
for d in role_obj.get("managed_dirs") or []:
if isinstance(d, dict):
all_managed_dirs.append(d)
for e in role_obj.get("excluded") or []:
if isinstance(e, dict):
all_excluded.append(e)
if isinstance(users_obj, dict):
_consume_role(users_obj)
if isinstance(services_list, list):
for s in services_list:
if isinstance(s, dict):
_consume_role(s)
if isinstance(pkgs_list, list):
for p in pkgs_list:
if isinstance(p, dict):
_consume_role(p)
for rname in [
"apt_config",
"dnf_config",
"etc_custom",
"usr_local_custom",
"extra_paths",
]:
robj = roles.get(rname)
if isinstance(robj, dict):
_consume_role(robj)
managed_file_reasons = _summarize_reasons(
all_managed_files,
"reason",
mapping=_MANAGED_FILE_REASONS,
max_examples=max_examples,
)
managed_dir_reasons = _summarize_reasons(
all_managed_dirs,
"reason",
mapping=_MANAGED_DIR_REASONS,
max_examples=max_examples,
)
excluded_reasons = _summarize_reasons(
all_excluded,
"reason",
mapping=_EXCLUDED_REASONS,
max_examples=max_examples,
)
# Inventory observed_via breakdown (count packages that contain at least one entry for that kind)
observed_kinds: Counter[str] = Counter()
observed_refs: Dict[str, Counter[str]] = defaultdict(Counter)
for _pkg, entry in inv_pkgs.items():
if not isinstance(entry, dict):
continue
seen_kinds = set()
for ov in entry.get("observed_via") or []:
if not isinstance(ov, dict):
continue
kind = ov.get("kind")
if not kind:
continue
kind = str(kind)
seen_kinds.add(kind)
ref = ov.get("ref")
if isinstance(ref, str) and ref:
observed_refs[kind][ref] += 1
for k in seen_kinds:
observed_kinds[k] += 1
observed_via_summary: List[Dict[str, Any]] = []
for kind, cnt in observed_kinds.most_common():
info = _ri(_OBSERVED_VIA, kind)
top_refs = [
r for r, _ in observed_refs.get(kind, Counter()).most_common(max_examples)
]
observed_via_summary.append(
{
"kind": kind,
"count": cnt,
"title": info.title,
"why": info.why,
"top_refs": top_refs,
}
)
report: Dict[str, Any] = {
"bundle_dir": str(bundle.dir),
"host": host,
"enroll": enroll,
"inventory": {
"package_count": len(inv_pkgs),
"observed_via": observed_via_summary,
},
"roles": role_summaries,
"reasons": {
"managed_files": managed_file_reasons,
"managed_dirs": managed_dir_reasons,
"excluded": excluded_reasons,
},
}
if fmt == "json":
return json.dumps(report, indent=2, sort_keys=True)
# Text rendering
out: List[str] = []
out.append(f"Enroll explained: {harvest}")
hn = host.get("hostname") or "(unknown host)"
os_family = host.get("os") or "unknown"
pkg_backend = host.get("pkg_backend") or "?"
ver = enroll.get("version") or "?"
out.append(f"Host: {hn} (os: {os_family}, pkg: {pkg_backend})")
out.append(f"Enroll: {ver}")
out.append("")
out.append("Inventory")
out.append(f"- Packages: {len(inv_pkgs)}")
if observed_via_summary:
out.append("- Why packages were included (observed_via):")
for ov in observed_via_summary:
extra = ""
if ov.get("top_refs"):
extra = f" (e.g. {', '.join(ov['top_refs'])})"
out.append(f" - {ov['kind']}: {ov['count']} – {ov['why']}{extra}")
out.append("")
out.append("Roles collected")
for rs in role_summaries:
out.append(f"- {rs['role']}: {rs['summary']}")
if rs["role"] == "extra_paths":
inc = rs.get("include_patterns") or []
exc = rs.get("exclude_patterns") or []
if inc:
suffix = "…" if len(inc) > max_examples else ""
out.append(
f" include_patterns: {', '.join(map(str, inc[:max_examples]))}{suffix}"
)
if exc:
suffix = "…" if len(exc) > max_examples else ""
out.append(
f" exclude_patterns: {', '.join(map(str, exc[:max_examples]))}{suffix}"
)
notes = rs.get("notes") or []
if notes:
for n in notes[:max_examples]:
out.append(f" note: {n}")
if len(notes) > max_examples:
out.append(
f" note: (+{len(notes) - max_examples} more. Use --format json to see them all)"
)
out.append("")
out.append("Why files were included (managed_files.reason)")
if managed_file_reasons:
for r in managed_file_reasons[:15]:
exs = r.get("examples") or []
ex_txt = f" Examples: {', '.join(exs)}" if exs else ""
out.append(f"- {r['reason']} ({r['count']}): {r['why']}.{ex_txt}")
if len(managed_file_reasons) > 15:
out.append(
f"- (+{len(managed_file_reasons) - 15} more reasons. Use --format json to see them all)"
)
else:
out.append("- (no managed files)")
if managed_dir_reasons:
out.append("")
out.append("Why directories were included (managed_dirs.reason)")
for r in managed_dir_reasons:
out.append(f"- {r['reason']} ({r['count']}): {r['why']}")
out.append("")
out.append("Why paths were excluded")
if excluded_reasons:
for r in excluded_reasons:
exs = r.get("examples") or []
ex_txt = f" Examples: {', '.join(exs)}" if exs else ""
out.append(f"- {r['reason']} ({r['count']}): {r['why']}.{ex_txt}")
else:
out.append("- (no excluded paths)")
return "\n".join(out) + "\n"

40
enroll/fsutil.py
View file

@ -1,40 +0,0 @@
from __future__ import annotations
import hashlib
import os
from typing import Tuple
def file_md5(path: str) -> str:
"""Return hex MD5 of a file.
Used for Debian dpkg baseline comparisons.
"""
h = hashlib.md5() # nosec
with open(path, "rb") as f:
for chunk in iter(lambda: f.read(1024 * 1024), b""):
h.update(chunk)
return h.hexdigest()
def stat_triplet(path: str) -> Tuple[str, str, str]:
"""Return (owner, group, mode) for a path.
owner/group are usernames/group names when resolvable, otherwise numeric ids.
mode is a zero-padded octal string (e.g. "0644").
"""
st = os.stat(path, follow_symlinks=True)
mode = oct(st.st_mode & 0o7777)[2:].zfill(4)
import grp
import pwd
try:
owner = pwd.getpwuid(st.st_uid).pw_name
except KeyError:
owner = str(st.st_uid)
try:
group = grp.getgrgid(st.st_gid).gr_name
except KeyError:
group = str(st.st_gid)
return owner, group, mode

2300
enroll/harvest.py
View file

File diff suppressed because it is too large Load diff

217
enroll/ignore.py
View file

@ -1,217 +0,0 @@
from __future__ import annotations
import fnmatch
import os
import re
from dataclasses import dataclass
from typing import Optional
DEFAULT_DENY_GLOBS = [
# Common backup copies created by passwd tools (can contain sensitive data)
"/etc/passwd-",
"/etc/group-",
"/etc/shadow-",
"/etc/gshadow-",
"/etc/subuid-",
"/etc/subgid-",
"/etc/*shadow-",
"/etc/*gshadow-",
"/etc/ssl/private/*",
"/etc/ssh/ssh_host_*",
"/etc/shadow",
"/etc/gshadow",
"/etc/*shadow",
"/etc/letsencrypt/*",
"/usr/local/etc/ssl/private/*",
"/usr/local/etc/ssh/ssh_host_*",
"/usr/local/etc/*shadow",
"/usr/local/etc/*gshadow",
"/usr/local/etc/letsencrypt/*",
]
# Allow a small set of binary config artifacts that are commonly required to
# reproduce system configuration (notably APT keyrings). These are still subject
# to size and readability limits, but are exempt from the "binary_like" denial.
DEFAULT_ALLOW_BINARY_GLOBS = [
"/etc/apt/trusted.gpg",
"/etc/apt/trusted.gpg.d/*.gpg",
"/etc/apt/keyrings/*.gpg",
"/etc/apt/keyrings/*.pgp",
"/etc/apt/keyrings/*.asc",
"/usr/share/keyrings/*.gpg",
"/usr/share/keyrings/*.pgp",
"/usr/share/keyrings/*.asc",
"/etc/pki/rpm-gpg/*",
]
SENSITIVE_CONTENT_PATTERNS = [
re.compile(rb"-----BEGIN (RSA |EC |OPENSSH |)PRIVATE KEY-----"),
re.compile(rb"(?i)\bpassword\s*="),
re.compile(rb"(?i)\b(pass|passwd|token|secret|api[_-]?key)\b"),
]
COMMENT_PREFIXES = (b"#", b";", b"//")
BLOCK_START = b"/*"
BLOCK_END = b"*/"
@dataclass
class IgnorePolicy:
deny_globs: Optional[list[str]] = None
allow_binary_globs: Optional[list[str]] = None
max_file_bytes: int = 256_000
sample_bytes: int = 64_000
# If True, be much less conservative about collecting potentially
# sensitive files. This disables deny globs (e.g. /etc/shadow,
# /etc/ssl/private/*) and skips heuristic content scanning.
dangerous: bool = False
def __post_init__(self) -> None:
if self.deny_globs is None:
self.deny_globs = list(DEFAULT_DENY_GLOBS)
if self.allow_binary_globs is None:
self.allow_binary_globs = list(DEFAULT_ALLOW_BINARY_GLOBS)
def iter_effective_lines(self, content: bytes):
in_block = False
for raw in content.splitlines():
line = raw.lstrip()
if in_block:
if BLOCK_END in line:
in_block = False
continue
if not line:
continue
if line.startswith(BLOCK_START):
in_block = True
continue
if line.startswith(COMMENT_PREFIXES) or line.startswith(b"*"):
continue
yield raw
def deny_reason(self, path: str) -> Optional[str]:
# Always ignore plain *.log files (rarely useful as config, often noisy).
if path.endswith(".log"):
return "log_file"
# Ignore editor/backup files that end with a trailing tilde.
if path.endswith("~"):
return "backup_file"
# Ignore backup shadow files
if path.startswith("/etc/") and path.endswith("-"):
return "backup_file"
if not self.dangerous:
for g in self.deny_globs or []:
if fnmatch.fnmatch(path, g):
return "denied_path"
try:
st = os.stat(path, follow_symlinks=True)
except OSError:
return "unreadable"
if st.st_size > self.max_file_bytes:
return "too_large"
if not os.path.isfile(path) or os.path.islink(path):
return "not_regular_file"
try:
with open(path, "rb") as f:
data = f.read(min(self.sample_bytes, st.st_size))
except OSError:
return "unreadable"
if b"\x00" in data:
for g in self.allow_binary_globs or []:
if fnmatch.fnmatch(path, g):
# Binary is acceptable for explicitly-allowed paths.
return None
return "binary_like"
if not self.dangerous:
for line in self.iter_effective_lines(data):
for pat in SENSITIVE_CONTENT_PATTERNS:
if pat.search(line):
return "sensitive_content"
return None
def deny_reason_dir(self, path: str) -> Optional[str]:
"""Directory-specific deny logic.
deny_reason() is file-oriented (it rejects directories as "not_regular_file").
For directory metadata capture (so roles can recreate directory trees), we need
a lighter-weight check:
- apply deny_globs (unless dangerous)
- require the path to be a real directory (no symlink)
- ensure it's stat'able/readable
No size checks or content scanning are performed for directories.
"""
if not self.dangerous:
for g in self.deny_globs or []:
if fnmatch.fnmatch(path, g):
return "denied_path"
try:
os.stat(path, follow_symlinks=True)
except OSError:
return "unreadable"
if os.path.islink(path):
return "symlink"
if not os.path.isdir(path):
return "not_directory"
return None
def deny_reason_link(self, path: str) -> Optional[str]:
"""Symlink-specific deny logic.
Symlinks are meaningful configuration state (e.g. Debian-style
*-enabled directories). deny_reason() is file-oriented and rejects
symlinks as "not_regular_file".
For symlinks we:
- apply the usual deny_globs (unless dangerous)
- ensure the path is a symlink and we can readlink() it
No size checks or content scanning are performed for symlinks.
"""
# Keep the same fast-path filename ignores as deny_reason().
if path.endswith(".log"):
return "log_file"
if path.endswith("~"):
return "backup_file"
if path.startswith("/etc/") and path.endswith("-"):
return "backup_file"
if not self.dangerous:
for g in self.deny_globs or []:
if fnmatch.fnmatch(path, g):
return "denied_path"
try:
os.lstat(path)
except OSError:
return "unreadable"
if not os.path.islink(path):
return "not_symlink"
try:
os.readlink(path)
except OSError:
return "unreadable"
return None

132
enroll/jinjaturtle.py
View file

@ -1,132 +0,0 @@
from __future__ import annotations
import shutil
import subprocess # nosec
import tempfile
from dataclasses import dataclass
from pathlib import Path
from typing import Optional
SYSTEMD_SUFFIXES = {
".service",
".socket",
".target",
".timer",
".path",
".mount",
".automount",
".slice",
".swap",
".scope",
".link",
".netdev",
".network",
}
SUPPORTED_SUFFIXES = {
".ini",
".cfg",
".json",
".toml",
".yaml",
".yml",
".xml",
".repo",
} | SYSTEMD_SUFFIXES
def infer_other_formats(dest_path: str) -> Optional[str]:
p = Path(dest_path)
name = p.name.lower()
suffix = p.suffix.lower()
# postfix
if name == "main.cf":
return "postfix"
# systemd units
if suffix in SYSTEMD_SUFFIXES:
return "systemd"
# OpenSSH system config files and snippets
parts = {part.lower() for part in p.parts}
if name in {"sshd_config", "ssh_config"}:
return "ssh"
if suffix == ".conf" and {"sshd_config.d", "ssh_config.d"} & parts:
return "ssh"
return None
@dataclass(frozen=True)
class JinjifyResult:
template_text: str
vars_text: str # YAML mapping text (no leading --- expected)
def find_jinjaturtle_cmd() -> Optional[str]:
"""Return the executable path for jinjaturtle if found on PATH."""
return shutil.which("jinjaturtle")
def can_jinjify_path(dest_path: str) -> bool:
p = Path(dest_path)
suffix = p.suffix.lower()
if infer_other_formats(dest_path):
return True
# allow unambiguous structured formats
if suffix in SUPPORTED_SUFFIXES:
return True
return False
def run_jinjaturtle(
jt_exe: str,
src_path: str,
*,
role_name: str,
force_format: Optional[str] = None,
) -> JinjifyResult:
"""
Run jinjaturtle against src_path and return (template, defaults-yaml).
Uses tempfiles and captures outputs.
jinjaturtle CLI:
jinjaturtle <config> -r <role> [-f <format>] [-d <defaults-output>] [-t <template-output>]
"""
src = Path(src_path)
if not src.is_file():
raise FileNotFoundError(src_path)
with tempfile.TemporaryDirectory(prefix="enroll-jt-") as td:
td_path = Path(td)
defaults_out = td_path / "defaults.yml"
template_out = td_path / "template.j2"
cmd = [
jt_exe,
str(src),
"-r",
role_name,
"-d",
str(defaults_out),
"-t",
str(template_out),
]
if force_format:
cmd.extend(["-f", force_format])
p = subprocess.run(cmd, text=True, capture_output=True) # nosec
if p.returncode != 0:
raise RuntimeError(
"jinjaturtle failed for %s (role=%s)\ncmd=%r\nstdout=%s\nstderr=%s"
% (src_path, role_name, cmd, p.stdout, p.stderr)
)
vars_text = defaults_out.read_text(encoding="utf-8").strip()
template_text = template_out.read_text(encoding="utf-8")
# jinjaturtle outputs a YAML mapping; strip leading document marker if present
if vars_text.startswith("---"):
vars_text = "\n".join(vars_text.splitlines()[1:]).lstrip()
return JinjifyResult(
template_text=template_text, vars_text=vars_text.rstrip() + "\n"
)

2207
enroll/manifest.py
View file

File diff suppressed because it is too large Load diff

293
enroll/pathfilter.py
View file

@ -1,293 +0,0 @@
from __future__ import annotations
import glob
import os
import re
from dataclasses import dataclass
from pathlib import PurePosixPath
from typing import List, Optional, Sequence, Set, Tuple
_REGEX_PREFIXES = ("re:", "regex:")
def _has_glob_chars(s: str) -> bool:
return any(ch in s for ch in "*?[")
def _norm_abs(p: str) -> str:
"""Normalise a path-ish string to an absolute POSIX path.
We treat inputs that don't start with '/' as being relative to '/'.
"""
p = p.strip()
if not p:
return "/"
if not p.startswith("/"):
p = "/" + p
# `normpath` keeps a leading '/' for absolute paths.
return os.path.normpath(p)
def _posix_match(path: str, pattern: str) -> bool:
"""Path matching with glob semantics.
Uses PurePosixPath.match which:
- treats '/' as a segment separator
- supports '**' for recursive matching
Both `path` and `pattern` are treated as absolute paths.
"""
# PurePosixPath.match is anchored and works best on relative strings.
p = path.lstrip("/")
pat = pattern.lstrip("/")
try:
return PurePosixPath(p).match(pat)
except Exception:
# If the pattern is somehow invalid, fail closed.
return False
def _regex_literal_prefix(regex: str) -> str:
"""Best-effort literal prefix extraction for a regex.
This lets us pick a starting directory to walk when expanding regex-based
include patterns.
"""
s = regex
if s.startswith("^"):
s = s[1:]
out: List[str] = []
escaped = False
meta = set(".^$*+?{}[]\\|()")
for ch in s:
if escaped:
out.append(ch)
escaped = False
continue
if ch == "\\":
escaped = True
continue
if ch in meta:
break
out.append(ch)
return "".join(out)
@dataclass(frozen=True)
class CompiledPathPattern:
raw: str
kind: str # 'prefix' | 'glob' | 'regex'
value: str
regex: Optional[re.Pattern[str]] = None
def matches(self, path: str) -> bool:
p = _norm_abs(path)
if self.kind == "regex":
if not self.regex:
return False
# Search (not match) so users can write unanchored patterns.
return self.regex.search(p) is not None
if self.kind == "glob":
return _posix_match(p, self.value)
# prefix
pref = self.value.rstrip("/")
return p == pref or p.startswith(pref + "/")
def compile_path_pattern(raw: str) -> CompiledPathPattern:
s = raw.strip()
for pre in _REGEX_PREFIXES:
if s.startswith(pre):
rex = s[len(pre) :].strip()
try:
return CompiledPathPattern(
raw=raw, kind="regex", value=rex, regex=re.compile(rex)
)
except re.error:
# Treat invalid regexes as non-matching.
return CompiledPathPattern(raw=raw, kind="regex", value=rex, regex=None)
# If the user explicitly says glob:, honour it.
if s.startswith("glob:"):
pat = s[len("glob:") :].strip()
return CompiledPathPattern(raw=raw, kind="glob", value=_norm_abs(pat))
# Heuristic: if it contains glob metacharacters, treat as a glob.
if _has_glob_chars(s) or "**" in s:
return CompiledPathPattern(raw=raw, kind="glob", value=_norm_abs(s))
# Otherwise treat as an exact path-or-prefix (dir subtree).
return CompiledPathPattern(raw=raw, kind="prefix", value=_norm_abs(s))
@dataclass
class PathFilter:
"""User-provided path filters.
Semantics:
- exclude patterns always win
- include patterns are used only to expand *additional* files to harvest
(they do not restrict the default harvest set)
Patterns:
- By default: glob-like (supports '**')
- Regex: prefix with 're:' or 'regex:'
- Force glob: prefix with 'glob:'
- A plain path without wildcards matches that path and everything under it
(directory-prefix behaviour).
Examples:
--exclude-path /usr/local/bin/docker-*
--include-path /home/*/.bashrc
--include-path 're:^/home/[^/]+/.config/myapp/.*$'
"""
include: Sequence[str] = ()
exclude: Sequence[str] = ()
def __post_init__(self) -> None:
self._include = [
compile_path_pattern(p) for p in self.include if str(p).strip()
]
self._exclude = [
compile_path_pattern(p) for p in self.exclude if str(p).strip()
]
def is_excluded(self, path: str) -> bool:
for pat in self._exclude:
if pat.matches(path):
return True
return False
def iter_include_patterns(self) -> List[CompiledPathPattern]:
return list(self._include)
def expand_includes(
patterns: Sequence[CompiledPathPattern],
*,
exclude: Optional[PathFilter] = None,
max_files: int,
) -> Tuple[List[str], List[str]]:
"""Expand include patterns into concrete file paths.
Returns (paths, notes). The returned paths are absolute paths.
This function is intentionally conservative:
- symlinks are ignored (both dirs and files)
- the number of collected files is capped
Regex patterns are expanded by walking a best-effort inferred root.
"""
out: List[str] = []
notes: List[str] = []
seen: Set[str] = set()
def _maybe_add_file(p: str) -> None:
if len(out) >= max_files:
return
p = _norm_abs(p)
if exclude and exclude.is_excluded(p):
return
if p in seen:
return
if not os.path.isfile(p) or os.path.islink(p):
return
seen.add(p)
out.append(p)
def _walk_dir(root: str, match: Optional[CompiledPathPattern] = None) -> None:
root = _norm_abs(root)
if not os.path.isdir(root) or os.path.islink(root):
return
for dirpath, dirnames, filenames in os.walk(root, followlinks=False):
# Prune excluded directories early.
if exclude:
dirnames[:] = [
d
for d in dirnames
if not exclude.is_excluded(os.path.join(dirpath, d))
and not os.path.islink(os.path.join(dirpath, d))
]
for fn in filenames:
if len(out) >= max_files:
return
p = os.path.join(dirpath, fn)
if os.path.islink(p) or not os.path.isfile(p):
continue
if exclude and exclude.is_excluded(p):
continue
if match is not None and not match.matches(p):
continue
if p in seen:
continue
seen.add(p)
out.append(_norm_abs(p))
for pat in patterns:
if len(out) >= max_files:
notes.append(
f"Include cap reached ({max_files}); some includes were not expanded."
)
break
matched_any = False
if pat.kind == "prefix":
p = pat.value
if os.path.isfile(p) and not os.path.islink(p):
_maybe_add_file(p)
matched_any = True
elif os.path.isdir(p) and not os.path.islink(p):
before = len(out)
_walk_dir(p)
matched_any = len(out) > before
else:
# Still allow prefix patterns that don't exist now (e.g. remote different)
# by matching nothing rather than erroring.
matched_any = False
elif pat.kind == "glob":
# Use glob for expansion; also walk directories that match.
gpat = pat.value
hits = glob.glob(gpat, recursive=True)
for h in hits:
if len(out) >= max_files:
break
h = _norm_abs(h)
if exclude and exclude.is_excluded(h):
continue
if os.path.isdir(h) and not os.path.islink(h):
before = len(out)
_walk_dir(h)
if len(out) > before:
matched_any = True
elif os.path.isfile(h) and not os.path.islink(h):
_maybe_add_file(h)
matched_any = True
else: # regex
rex = pat.value
prefix = _regex_literal_prefix(rex)
# Determine a walk root. If we can infer an absolute prefix, use its
# directory; otherwise fall back to '/'.
if prefix.startswith("/"):
root = os.path.dirname(prefix) or "/"
else:
root = "/"
before = len(out)
_walk_dir(root, match=pat)
matched_any = len(out) > before
if not matched_any:
notes.append(f"Include pattern matched no files: {pat.raw!r}")
return out, notes

282
enroll/platform.py
View file

@ -1,282 +0,0 @@
from __future__ import annotations
import shutil
from dataclasses import dataclass
from typing import Dict, List, Optional, Set, Tuple
from .fsutil import file_md5
def _read_os_release(path: str = "/etc/os-release") -> Dict[str, str]:
out: Dict[str, str] = {}
try:
with open(path, "r", encoding="utf-8", errors="replace") as f:
for raw in f:
line = raw.strip()
if not line or line.startswith("#") or "=" not in line:
continue
k, v = line.split("=", 1)
k = k.strip()
v = v.strip().strip('"')
out[k] = v
except OSError:
return {}
return out
@dataclass
class PlatformInfo:
os_family: str # debian|redhat|unknown
pkg_backend: str # dpkg|rpm|unknown
os_release: Dict[str, str]
def detect_platform() -> PlatformInfo:
"""Detect platform family and package backend.
Uses /etc/os-release when available, with a conservative fallback to
checking for dpkg/rpm binaries.
"""
osr = _read_os_release()
os_id = (osr.get("ID") or "").strip().lower()
likes = (osr.get("ID_LIKE") or "").strip().lower().split()
deb_ids = {"debian", "ubuntu", "linuxmint", "raspbian", "kali"}
rhel_ids = {
"fedora",
"rhel",
"centos",
"rocky",
"almalinux",
"ol",
"oracle",
"scientific",
}
if os_id in deb_ids or "debian" in likes:
return PlatformInfo(os_family="debian", pkg_backend="dpkg", os_release=osr)
if os_id in rhel_ids or any(
x in likes for x in ("rhel", "fedora", "centos", "redhat")
):
return PlatformInfo(os_family="redhat", pkg_backend="rpm", os_release=osr)
# Fallback heuristics.
if shutil.which("dpkg"):
return PlatformInfo(os_family="debian", pkg_backend="dpkg", os_release=osr)
if shutil.which("rpm"):
return PlatformInfo(os_family="redhat", pkg_backend="rpm", os_release=osr)
return PlatformInfo(os_family="unknown", pkg_backend="unknown", os_release=osr)
class PackageBackend:
"""Backend abstraction for package ownership, config detection, and manual package lists."""
name: str
pkg_config_prefixes: Tuple[str, ...]
def owner_of_path(self, path: str) -> Optional[str]: # pragma: no cover
raise NotImplementedError
def list_manual_packages(self) -> List[str]: # pragma: no cover
raise NotImplementedError
def installed_packages(self) -> Dict[str, List[Dict[str, str]]]: # pragma: no cover
"""Return mapping of package name -> installed instances.
Each instance is a dict with at least:
- version: package version string
- arch: architecture string
Backends should be best-effort and return an empty mapping on failure.
"""
raise NotImplementedError
def build_etc_index(
self,
) -> Tuple[
Set[str], Dict[str, str], Dict[str, Set[str]], Dict[str, List[str]]
]: # pragma: no cover
raise NotImplementedError
def specific_paths_for_hints(self, hints: Set[str]) -> List[str]:
return []
def is_pkg_config_path(self, path: str) -> bool:
for pfx in self.pkg_config_prefixes:
if path == pfx or path.startswith(pfx):
return True
return False
def modified_paths(self, pkg: str, etc_paths: List[str]) -> Dict[str, str]:
"""Return a mapping of modified file paths -> reason label."""
return {}
class DpkgBackend(PackageBackend):
name = "dpkg"
pkg_config_prefixes = ("/etc/apt/",)
def __init__(self) -> None:
from .debian import parse_status_conffiles
self._conffiles_by_pkg = parse_status_conffiles()
def owner_of_path(self, path: str) -> Optional[str]:
from .debian import dpkg_owner
return dpkg_owner(path)
def list_manual_packages(self) -> List[str]:
from .debian import list_manual_packages
return list_manual_packages()
def installed_packages(self) -> Dict[str, List[Dict[str, str]]]:
from .debian import list_installed_packages
return list_installed_packages()
def build_etc_index(self):
from .debian import build_dpkg_etc_index
return build_dpkg_etc_index()
def specific_paths_for_hints(self, hints: Set[str]) -> List[str]:
paths: List[str] = []
for h in hints:
paths.extend(
[
f"/etc/default/{h}",
f"/etc/init.d/{h}",
f"/etc/sysctl.d/{h}.conf",
]
)
return paths
def modified_paths(self, pkg: str, etc_paths: List[str]) -> Dict[str, str]:
from .debian import read_pkg_md5sums
out: Dict[str, str] = {}
conff = self._conffiles_by_pkg.get(pkg, {})
md5sums = read_pkg_md5sums(pkg)
for path in etc_paths:
if not path.startswith("/etc/"):
continue
if self.is_pkg_config_path(path):
continue
if path in conff:
try:
current = file_md5(path)
except OSError:
continue
if current != conff[path]:
out[path] = "modified_conffile"
continue
rel = path.lstrip("/")
baseline = md5sums.get(rel)
if baseline:
try:
current = file_md5(path)
except OSError:
continue
if current != baseline:
out[path] = "modified_packaged_file"
return out
class RpmBackend(PackageBackend):
name = "rpm"
pkg_config_prefixes = (
"/etc/dnf/",
"/etc/yum/",
"/etc/yum.repos.d/",
"/etc/yum.conf",
)
def __init__(self) -> None:
self._modified_cache: Dict[str, Set[str]] = {}
self._config_cache: Dict[str, Set[str]] = {}
def owner_of_path(self, path: str) -> Optional[str]:
from .rpm import rpm_owner
return rpm_owner(path)
def list_manual_packages(self) -> List[str]:
from .rpm import list_manual_packages
return list_manual_packages()
def installed_packages(self) -> Dict[str, List[Dict[str, str]]]:
from .rpm import list_installed_packages
return list_installed_packages()
def build_etc_index(self):
from .rpm import build_rpm_etc_index
return build_rpm_etc_index()
def specific_paths_for_hints(self, hints: Set[str]) -> List[str]:
paths: List[str] = []
for h in hints:
paths.extend(
[
f"/etc/sysconfig/{h}",
f"/etc/sysconfig/{h}.conf",
f"/etc/sysctl.d/{h}.conf",
]
)
return paths
def _config_files(self, pkg: str) -> Set[str]:
if pkg in self._config_cache:
return self._config_cache[pkg]
from .rpm import rpm_config_files
s = rpm_config_files(pkg)
self._config_cache[pkg] = s
return s
def _modified_files(self, pkg: str) -> Set[str]:
if pkg in self._modified_cache:
return self._modified_cache[pkg]
from .rpm import rpm_modified_files
s = rpm_modified_files(pkg)
self._modified_cache[pkg] = s
return s
def modified_paths(self, pkg: str, etc_paths: List[str]) -> Dict[str, str]:
out: Dict[str, str] = {}
modified = self._modified_files(pkg)
if not modified:
return out
config = self._config_files(pkg)
for path in etc_paths:
if not path.startswith("/etc/"):
continue
if self.is_pkg_config_path(path):
continue
if path not in modified:
continue
out[path] = (
"modified_conffile" if path in config else "modified_packaged_file"
)
return out
def get_backend(info: Optional[PlatformInfo] = None) -> PackageBackend:
info = info or detect_platform()
if info.pkg_backend == "dpkg":
return DpkgBackend()
if info.pkg_backend == "rpm":
return RpmBackend()
# Unknown: be conservative and use an rpm backend if rpm exists, otherwise dpkg.
if shutil.which("rpm"):
return RpmBackend()
return DpkgBackend()

673
enroll/remote.py
View file

@ -1,673 +0,0 @@
from __future__ import annotations
import getpass
import os
import shlex
import shutil
import sys
import time
import tarfile
import tempfile
import zipapp
from pathlib import Path
from pathlib import PurePosixPath
from typing import Optional, Callable, TextIO
class RemoteSudoPasswordRequired(RuntimeError):
"""Raised when sudo requires a password but none was provided."""
class RemoteSSHKeyPassphraseRequired(RuntimeError):
"""Raised when SSH private key decryption needs a passphrase."""
def _sudo_password_required(out: str, err: str) -> bool:
"""Return True if sudo output indicates it needs a password/TTY."""
blob = (out + "\n" + err).lower()
patterns = (
"a password is required",
"password is required",
"a terminal is required to read the password",
"no tty present and no askpass program specified",
"must have a tty to run sudo",
"sudo: sorry, you must have a tty",
"askpass",
)
return any(p in blob for p in patterns)
def _sudo_not_permitted(out: str, err: str) -> bool:
"""Return True if sudo output indicates the user cannot sudo at all."""
blob = (out + "\n" + err).lower()
patterns = (
"is not in the sudoers file",
"not allowed to execute",
"may not run sudo",
"sorry, user",
)
return any(p in blob for p in patterns)
def _sudo_tty_required(out: str, err: str) -> bool:
"""Return True if sudo output indicates it requires a TTY (sudoers requiretty)."""
blob = (out + "\n" + err).lower()
patterns = (
"must have a tty",
"sorry, you must have a tty",
"sudo: sorry, you must have a tty",
"must have a tty to run sudo",
)
return any(p in blob for p in patterns)
def _resolve_become_password(
ask_become_pass: bool,
*,
prompt: str = "sudo password: ",
getpass_fn: Callable[[str], str] = getpass.getpass,
) -> Optional[str]:
if ask_become_pass:
return getpass_fn(prompt)
return None
def _resolve_ssh_key_passphrase(
ask_key_passphrase: bool,
*,
env_var: Optional[str] = None,
prompt: str = "SSH key passphrase: ",
getpass_fn: Callable[[str], str] = getpass.getpass,
) -> Optional[str]:
"""Resolve SSH private-key passphrase from env and/or prompt.
Precedence:
1) --ssh-key-passphrase-env style input (env_var)
2) --ask-key-passphrase style interactive prompt
3) None
"""
if env_var:
val = os.environ.get(str(env_var))
if val is None:
raise RuntimeError(
"SSH key passphrase environment variable is not set: " f"{env_var}"
)
return val
if ask_key_passphrase:
return getpass_fn(prompt)
return None
def remote_harvest(
*,
ask_become_pass: bool = False,
ask_key_passphrase: bool = False,
ssh_key_passphrase_env: Optional[str] = None,
no_sudo: bool = False,
prompt: str = "sudo password: ",
key_prompt: str = "SSH key passphrase: ",
getpass_fn: Optional[Callable[[str], str]] = None,
stdin: Optional[TextIO] = None,
**kwargs,
):
"""Call _remote_harvest, with a safe sudo password fallback.
Behavior:
- Run without a password unless --ask-become-pass is set.
- If the remote sudo policy requires a password and none was provided,
prompt and retry when running interactively.
"""
# Resolve defaults at call time (easier to test/monkeypatch, and avoids capturing
# sys.stdin / getpass.getpass at import time).
if getpass_fn is None:
getpass_fn = getpass.getpass
if stdin is None:
stdin = sys.stdin
sudo_password = _resolve_become_password(
ask_become_pass and not no_sudo,
prompt=prompt,
getpass_fn=getpass_fn,
)
ssh_key_passphrase = _resolve_ssh_key_passphrase(
ask_key_passphrase,
env_var=ssh_key_passphrase_env,
prompt=key_prompt,
getpass_fn=getpass_fn,
)
while True:
try:
return _remote_harvest(
sudo_password=sudo_password,
no_sudo=no_sudo,
ssh_key_passphrase=ssh_key_passphrase,
**kwargs,
)
except RemoteSSHKeyPassphraseRequired:
# Already tried a passphrase and still failed.
if ssh_key_passphrase is not None:
raise RemoteSSHKeyPassphraseRequired(
"SSH private key could not be decrypted with the supplied "
"passphrase."
) from None
# Fallback prompt if interactive.
if stdin is not None and getattr(stdin, "isatty", lambda: False)():
ssh_key_passphrase = getpass_fn(key_prompt)
continue
raise RemoteSSHKeyPassphraseRequired(
"SSH private key is encrypted and needs a passphrase. "
"Re-run with --ask-key-passphrase or "
"--ssh-key-passphrase-env VAR."
)
except RemoteSudoPasswordRequired:
if sudo_password is not None:
raise
# Fallback prompt if interactive.
if stdin is not None and getattr(stdin, "isatty", lambda: False)():
sudo_password = getpass_fn(prompt)
continue
raise RemoteSudoPasswordRequired(
"Remote sudo requires a password. Re-run with --ask-become-pass."
)
def _safe_extract_tar(tar: tarfile.TarFile, dest: Path) -> None:
"""Safely extract a tar archive into dest.
Protects against path traversal (e.g. entries containing ../).
"""
# Note: tar member names use POSIX separators regardless of platform.
dest = dest.resolve()
for m in tar.getmembers():
name = m.name
# Some tar implementations include a top-level '.' entry when created
# with `tar -C <dir> .`. That's harmless and should be allowed.
if name in {".", "./"}:
continue
# Reject absolute paths and any '..' components up front.
p = PurePosixPath(name)
if p.is_absolute() or ".." in p.parts:
raise RuntimeError(f"Unsafe tar member path: {name}")
# Refuse to extract links or device nodes from an untrusted archive.
# (A symlink can be used to redirect subsequent writes outside dest.)
if m.issym() or m.islnk() or m.isdev():
raise RuntimeError(f"Refusing to extract special tar member: {name}")
member_path = (dest / Path(*p.parts)).resolve()
if member_path != dest and not str(member_path).startswith(str(dest) + os.sep):
raise RuntimeError(f"Unsafe tar member path: {name}")
# Extract members one-by-one after validation.
for m in tar.getmembers():
if m.name in {".", "./"}:
continue
tar.extract(m, path=dest)
def _build_enroll_pyz(tmpdir: Path) -> Path:
"""Build a self-contained enroll zipapp (pyz) on the local machine.
The resulting file is stdlib-only and can be executed on the remote host
as long as it has Python 3 available.
"""
import enroll as pkg
pkg_dir = Path(pkg.__file__).resolve().parent
stage = tmpdir / "stage"
(stage / "enroll").mkdir(parents=True, exist_ok=True)
def _ignore(d: str, names: list[str]) -> set[str]:
return {
n
for n in names
if n in {"__pycache__", ".pytest_cache"} or n.endswith(".pyc")
}
shutil.copytree(pkg_dir, stage / "enroll", dirs_exist_ok=True, ignore=_ignore)
pyz_path = tmpdir / "enroll.pyz"
zipapp.create_archive(
stage,
target=pyz_path,
main="enroll.cli:main",
compressed=True,
)
return pyz_path
def _ssh_run(
ssh,
cmd: str,
*,
get_pty: bool = False,
stdin_text: Optional[str] = None,
close_stdin: bool = False,
) -> tuple[int, str, str]:
"""Run a command over a Paramiko SSHClient.
Paramiko's exec_command runs commands without a TTY by default.
Some hosts have sudoers "requiretty" enabled, which causes sudo to
fail even when passwordless sudo is configured. For those commands,
request a PTY.
We do not request a PTY for commands that stream binary data
(e.g. tar/gzip output), as a PTY can corrupt the byte stream.
"""
stdin, stdout, stderr = ssh.exec_command(cmd, get_pty=get_pty)
# All three file-like objects share the same underlying Channel.
chan = stdout.channel
if stdin_text is not None and stdin is not None:
try:
stdin.write(stdin_text)
stdin.flush()
except Exception:
# If the remote side closed stdin early, ignore.
pass # nosec
finally:
if close_stdin:
# For sudo -S, a wrong password causes sudo to re-prompt and wait
# forever for more input. We try hard to deliver EOF so sudo can
# fail fast.
try:
chan.shutdown_write() # sends EOF to the remote process
except Exception:
pass # nosec
try:
stdin.close()
except Exception:
pass # nosec
# Read incrementally to avoid blocking forever on stdout.read()/stderr.read()
# if the remote process is waiting for more input (e.g. sudo password retry).
out_chunks: list[bytes] = []
err_chunks: list[bytes] = []
# Keep a small tail of stderr to detect sudo retry messages without
# repeatedly joining potentially large buffers.
err_tail = b""
while True:
progressed = False
if chan.recv_ready():
out_chunks.append(chan.recv(1024 * 64))
progressed = True
if chan.recv_stderr_ready():
chunk = chan.recv_stderr(1024 * 64)
err_chunks.append(chunk)
err_tail = (err_tail + chunk)[-4096:]
progressed = True
# If we just attempted sudo -S with a single password line and sudo is
# asking again, detect it and stop waiting.
if close_stdin and stdin_text is not None:
blob = err_tail.lower()
if b"sorry, try again" in blob or b"incorrect password" in blob:
try:
chan.close()
except Exception:
pass # nosec
break
# Exit once the process has exited and we have drained the buffers.
if (
chan.exit_status_ready()
and not chan.recv_ready()
and not chan.recv_stderr_ready()
):
break
if not progressed:
time.sleep(0.05)
out = b"".join(out_chunks).decode("utf-8", errors="replace")
err = b"".join(err_chunks).decode("utf-8", errors="replace")
rc = chan.recv_exit_status() if chan.exit_status_ready() else 1
return rc, out, err
def _ssh_run_sudo(
ssh,
cmd: str,
*,
sudo_password: Optional[str] = None,
get_pty: bool = True,
) -> tuple[int, str, str]:
"""Run cmd via sudo with a safe non-interactive-first strategy.
Strategy:
1) Try `sudo -n`.
2) If sudo reports a password is required and we have one, retry with
`sudo -S` and feed it via stdin.
3) If sudo reports a password is required and we *don't* have one, raise
RemoteSudoPasswordRequired.
We avoid requesting a PTY unless the remote sudo policy requires it.
This makes sudo -S behavior more reliable (wrong passwords fail fast
instead of blocking on a PTY).
"""
cmd_n = f"sudo -n -p '' -- {cmd}"
# First try: never prompt, and prefer no PTY.
rc, out, err = _ssh_run(ssh, cmd_n, get_pty=False)
need_pty = False
# Some sudoers configurations require a TTY even for passwordless sudo.
if get_pty and rc != 0 and _sudo_tty_required(out, err):
need_pty = True
rc, out, err = _ssh_run(ssh, cmd_n, get_pty=True)
if rc == 0:
return rc, out, err
if _sudo_not_permitted(out, err):
return rc, out, err
if _sudo_password_required(out, err):
if sudo_password is None:
raise RemoteSudoPasswordRequired(
"Remote sudo requires a password, but none was provided."
)
cmd_s = f"sudo -S -p '' -- {cmd}"
return _ssh_run(
ssh,
cmd_s,
get_pty=need_pty,
stdin_text=str(sudo_password) + "\n",
close_stdin=True,
)
return rc, out, err
def _remote_harvest(
*,
local_out_dir: Path,
remote_host: str,
remote_port: Optional[int] = None,
remote_user: Optional[str] = None,
remote_ssh_config: Optional[str] = None,
remote_python: str = "python3",
dangerous: bool = False,
no_sudo: bool = False,
sudo_password: Optional[str] = None,
ssh_key_passphrase: Optional[str] = None,
include_paths: Optional[list[str]] = None,
exclude_paths: Optional[list[str]] = None,
) -> Path:
"""Run enroll harvest on a remote host via SSH and pull the bundle locally.
Returns the local path to state.json inside local_out_dir.
"""
try:
import paramiko # type: ignore
except Exception as e:
raise RuntimeError(
"Remote harvesting requires the 'paramiko' package. "
"Install it with: pip install paramiko"
) from e
local_out_dir = Path(local_out_dir)
local_out_dir.mkdir(parents=True, exist_ok=True)
try:
os.chmod(local_out_dir, 0o700)
except OSError:
pass
# Build a zipapp locally and upload it to the remote.
with tempfile.TemporaryDirectory(prefix="enroll-remote-") as td:
td_path = Path(td)
pyz = _build_enroll_pyz(td_path)
local_tgz = td_path / "bundle.tgz"
ssh = paramiko.SSHClient()
ssh.load_system_host_keys()
# Default: refuse unknown host keys.
# Users should add the key to known_hosts.
ssh.set_missing_host_key_policy(paramiko.RejectPolicy())
# Resolve SSH connection parameters.
connect_host = remote_host
connect_port = int(remote_port) if remote_port is not None else 22
connect_user = remote_user
key_filename = None
sock = None
hostkey_name = connect_host
# Timeouts derived from ssh_config if set (ConnectTimeout).
# Used both for socket connect (when we create one) and Paramiko handshake/auth.
connect_timeout: Optional[float] = None
if remote_ssh_config:
from paramiko.config import SSHConfig # type: ignore
from paramiko.proxy import ProxyCommand # type: ignore
import socket as _socket
cfg_path = Path(str(remote_ssh_config)).expanduser()
if not cfg_path.exists():
raise RuntimeError(f"SSH config file not found: {cfg_path}")
cfg = SSHConfig()
with cfg_path.open("r", encoding="utf-8") as _fp:
cfg.parse(_fp)
hcfg = cfg.lookup(remote_host)
connect_host = str(hcfg.get("hostname") or remote_host)
hostkey_name = str(hcfg.get("hostkeyalias") or connect_host)
if remote_port is None and hcfg.get("port"):
try:
connect_port = int(str(hcfg.get("port")))
except ValueError:
pass
if connect_user is None and hcfg.get("user"):
connect_user = str(hcfg.get("user"))
ident = hcfg.get("identityfile")
if ident:
if isinstance(ident, (list, tuple)):
key_filename = [str(Path(p).expanduser()) for p in ident]
else:
key_filename = str(Path(str(ident)).expanduser())
# Honour OpenSSH ConnectTimeout (seconds) if present.
if hcfg.get("connecttimeout"):
try:
connect_timeout = float(str(hcfg.get("connecttimeout")))
except (TypeError, ValueError):
connect_timeout = None
proxycmd = hcfg.get("proxycommand")
# AddressFamily support: inet (IPv4 only), inet6 (IPv6 only), any (default).
addrfam = str(hcfg.get("addressfamily") or "any").strip().lower()
family: Optional[int] = None
if addrfam == "inet":
family = _socket.AF_INET
elif addrfam == "inet6":
family = _socket.AF_INET6
if proxycmd:
# ProxyCommand provides the transport; AddressFamily doesn't apply here.
sock = ProxyCommand(str(proxycmd))
elif family is not None:
# Enforce the requested address family by pre-connecting the socket and
# passing it into Paramiko via sock=.
last_err: Optional[OSError] = None
infos = _socket.getaddrinfo(
connect_host, connect_port, family, _socket.SOCK_STREAM
)
for af, socktype, proto, _, sa in infos:
s = _socket.socket(af, socktype, proto)
if connect_timeout is not None:
s.settimeout(connect_timeout)
try:
s.connect(sa)
sock = s
break
except OSError as e:
last_err = e
try:
s.close()
except Exception:
pass # nosec
if sock is None and last_err is not None:
raise last_err
elif hostkey_name != connect_host:
# If HostKeyAlias is used, connect to HostName via a socket but
# use HostKeyAlias for known_hosts lookups.
sock = _socket.create_connection(
(connect_host, connect_port), timeout=connect_timeout
)
# If we created a socket (sock!=None), pass hostkey_name as hostname so
# known_hosts lookup uses HostKeyAlias (or whatever hostkey_name resolved to).
try:
ssh.connect(
hostname=hostkey_name if sock is not None else connect_host,
port=connect_port,
username=connect_user,
key_filename=key_filename,
sock=sock,
allow_agent=True,
look_for_keys=True,
timeout=connect_timeout,
banner_timeout=connect_timeout,
auth_timeout=connect_timeout,
passphrase=ssh_key_passphrase,
)
except paramiko.PasswordRequiredException as e: # type: ignore[attr-defined]
raise RemoteSSHKeyPassphraseRequired(
"SSH private key is encrypted and no passphrase was provided."
) from e
# If no username was explicitly provided, SSH may have selected a default.
# We need a concrete username for the (sudo) chown step below.
resolved_user = remote_user
if not resolved_user:
rc, out, err = _ssh_run(ssh, "id -un")
if rc == 0 and out.strip():
resolved_user = out.strip()
sftp = ssh.open_sftp()
rtmp: Optional[str] = None
try:
rc, out, err = _ssh_run(ssh, "mktemp -d")
if rc != 0:
raise RuntimeError(f"Remote mktemp failed: {err.strip()}")
rtmp = out.strip()
# Be explicit: restrict the remote staging area to the current user.
rc, out, err = _ssh_run(ssh, f"chmod 700 {rtmp}")
if rc != 0:
raise RuntimeError(f"Remote chmod failed: {err.strip()}")
rapp = f"{rtmp}/enroll.pyz"
rbundle = f"{rtmp}/bundle"
sftp.put(str(pyz), rapp)
# Run remote harvest.
argv: list[str] = [
remote_python,
rapp,
"harvest",
"--out",
rbundle,
]
if dangerous:
argv.append("--dangerous")
for p in include_paths or []:
argv.extend(["--include-path", str(p)])
for p in exclude_paths or []:
argv.extend(["--exclude-path", str(p)])
_cmd = " ".join(map(shlex.quote, argv))
if not no_sudo:
# Prefer non-interactive sudo first; retry with -S only when needed.
rc, out, err = _ssh_run_sudo(
ssh, _cmd, sudo_password=sudo_password, get_pty=True
)
cmd = f"sudo {_cmd}"
else:
cmd = _cmd
rc, out, err = _ssh_run(ssh, cmd, get_pty=False)
if rc != 0:
raise RuntimeError(
"Remote harvest failed.\n"
f"Command: {cmd}\n"
f"Exit code: {rc}\n"
f"Stdout: {out.strip()}\n"
f"Stderr: {err.strip()}"
)
if not no_sudo:
# Ensure user can read the files, before we tar it.
if not resolved_user:
raise RuntimeError(
"Unable to determine remote username for chown. "
"Pass --remote-user explicitly or use --no-sudo."
)
chown_cmd = f"chown -R {resolved_user} {rbundle}"
rc, out, err = _ssh_run_sudo(
ssh,
chown_cmd,
sudo_password=sudo_password,
get_pty=True,
)
if rc != 0:
raise RuntimeError(
"chown of harvest failed.\n"
f"Command: sudo {chown_cmd}\n"
f"Exit code: {rc}\n"
f"Stdout: {out.strip()}\n"
f"Stderr: {err.strip()}"
)
# Stream a tarball back to the local machine (avoid creating a tar file on the remote).
cmd = f"tar -cz -C {rbundle} ."
_stdin, stdout, stderr = ssh.exec_command(cmd) # nosec
with open(local_tgz, "wb") as f:
while True:
chunk = stdout.read(1024 * 128)
if not chunk:
break
f.write(chunk)
rc = stdout.channel.recv_exit_status()
err_text = stderr.read().decode("utf-8", errors="replace")
if rc != 0:
raise RuntimeError(
"Remote tar stream failed.\n"
f"Command: {cmd}\n"
f"Exit code: {rc}\n"
f"Stderr: {err_text.strip()}"
)
# Extract into the destination.
with tarfile.open(local_tgz, mode="r:gz") as tf:
_safe_extract_tar(tf, local_out_dir)
finally:
# Cleanup remote tmpdir even on failure.
if rtmp:
_ssh_run(ssh, f"rm -rf {rtmp}")
try:
sftp.close()
ssh.close()
except Exception:
ssh.close()
raise RuntimeError("Something went wrong generating the harvest")
return local_out_dir / "state.json"

323
enroll/rpm.py
View file

@ -1,323 +0,0 @@
from __future__ import annotations
import os
import re
import shutil
import subprocess # nosec
from typing import Dict, List, Optional, Set, Tuple
def _run(
cmd: list[str], *, allow_fail: bool = False, merge_err: bool = False
) -> tuple[int, str]:
"""Run a command and return (rc, stdout).
If merge_err is True, stderr is merged into stdout to preserve ordering.
"""
p = subprocess.run(
cmd,
check=False,
text=True,
stdout=subprocess.PIPE,
stderr=(subprocess.STDOUT if merge_err else subprocess.PIPE),
) # nosec
out = p.stdout or ""
if (not allow_fail) and p.returncode != 0:
err = "" if merge_err else (p.stderr or "")
raise RuntimeError(f"Command failed: {cmd}\n{err}{out}")
return p.returncode, out
def rpm_owner(path: str) -> Optional[str]:
"""Return owning package name for a path, or None if unowned."""
if not path:
return None
rc, out = _run(
["rpm", "-qf", "--qf", "%{NAME}\n", path], allow_fail=True, merge_err=True
)
if rc != 0:
return None
for line in out.splitlines():
line = line.strip()
if not line:
continue
if "is not owned" in line:
return None
# With --qf we expect just the package name.
if re.match(r"^[A-Za-z0-9_.+:-]+$", line):
# Strip any accidental epoch/name-version-release output.
return line.split(":", 1)[-1].strip() if line else None
return None
_ARCH_SUFFIXES = {
"noarch",
"x86_64",
"i686",
"aarch64",
"armv7hl",
"ppc64le",
"s390x",
"riscv64",
}
def _strip_arch(token: str) -> str:
"""Strip a trailing .ARCH from a yum/dnf package token."""
t = token.strip()
if "." not in t:
return t
head, tail = t.rsplit(".", 1)
if tail in _ARCH_SUFFIXES:
return head
return t
def list_manual_packages() -> List[str]:
"""Return packages considered "user-installed" on RPM-based systems.
Best-effort:
1) dnf repoquery --userinstalled
2) dnf history userinstalled
3) yum history userinstalled
If none are available, returns an empty list.
"""
def _dedupe(pkgs: List[str]) -> List[str]:
return sorted({p for p in (pkgs or []) if p})
if shutil.which("dnf"):
# Prefer a machine-friendly output.
for cmd in (
["dnf", "-q", "repoquery", "--userinstalled", "--qf", "%{name}\n"],
["dnf", "-q", "repoquery", "--userinstalled"],
):
rc, out = _run(cmd, allow_fail=True, merge_err=True)
if rc == 0 and out.strip():
pkgs = []
for line in out.splitlines():
line = line.strip()
if not line or line.startswith("Loaded plugins"):
continue
pkgs.append(_strip_arch(line.split()[0]))
if pkgs:
return _dedupe(pkgs)
# Fallback
rc, out = _run(
["dnf", "-q", "history", "userinstalled"], allow_fail=True, merge_err=True
)
if rc == 0 and out.strip():
pkgs = []
for line in out.splitlines():
line = line.strip()
if not line or line.startswith("Installed") or line.startswith("Last"):
continue
# Often: "vim-enhanced.x86_64"
tok = line.split()[0]
pkgs.append(_strip_arch(tok))
if pkgs:
return _dedupe(pkgs)
if shutil.which("yum"):
rc, out = _run(
["yum", "-q", "history", "userinstalled"], allow_fail=True, merge_err=True
)
if rc == 0 and out.strip():
pkgs = []
for line in out.splitlines():
line = line.strip()
if (
not line
or line.startswith("Installed")
or line.startswith("Loaded")
):
continue
tok = line.split()[0]
pkgs.append(_strip_arch(tok))
if pkgs:
return _dedupe(pkgs)
return []
def list_installed_packages() -> Dict[str, List[Dict[str, str]]]:
"""Return mapping of installed package name -> installed instances.
Uses `rpm -qa` and is expected to work on RHEL/Fedora-like systems.
Output format:
{"pkg": [{"version": "...", "arch": "..."}, ...], ...}
The version string is formatted as:
- "<version>-<release>" for typical packages
- "<epoch>:<version>-<release>" if a non-zero epoch is present
"""
try:
_, out = _run(
[
"rpm",
"-qa",
"--qf",
"%{NAME}\t%{EPOCHNUM}\t%{VERSION}\t%{RELEASE}\t%{ARCH}\n",
],
allow_fail=False,
merge_err=True,
)
except Exception:
return {}
pkgs: Dict[str, List[Dict[str, str]]] = {}
for raw in (out or "").splitlines():
line = raw.strip("\n")
if not line:
continue
parts = line.split("\t")
if len(parts) < 5:
continue
name, epoch, ver, rel, arch = [p.strip() for p in parts[:5]]
if not name or not ver:
continue
# Normalise epoch.
epoch = epoch.strip()
if epoch.lower() in ("(none)", "none", ""):
epoch = "0"
v = f"{ver}-{rel}" if rel else ver
if epoch and epoch.isdigit() and epoch != "0":
v = f"{epoch}:{v}"
pkgs.setdefault(name, []).append({"version": v, "arch": arch})
for k in list(pkgs.keys()):
pkgs[k] = sorted(
pkgs[k], key=lambda x: (x.get("arch") or "", x.get("version") or "")
)
return pkgs
def _walk_etc_files() -> List[str]:
out: List[str] = []
for dirpath, _, filenames in os.walk("/etc"):
for fn in filenames:
p = os.path.join(dirpath, fn)
if os.path.islink(p) or not os.path.isfile(p):
continue
out.append(p)
return out
def build_rpm_etc_index() -> (
Tuple[Set[str], Dict[str, str], Dict[str, Set[str]], Dict[str, List[str]]]
):
"""Best-effort equivalent of build_dpkg_etc_index for RPM systems.
This builds indexes by walking the live /etc tree and querying RPM ownership
for each file.
Returns:
owned_etc_paths: set of /etc paths owned by rpm
etc_owner_map: /etc/path -> pkg
topdir_to_pkgs: "nginx" -> {"nginx", ...} based on /etc/<topdir>/...
pkg_to_etc_paths: pkg -> list of owned /etc paths
"""
owned: Set[str] = set()
owner: Dict[str, str] = {}
topdir_to_pkgs: Dict[str, Set[str]] = {}
pkg_to_etc: Dict[str, List[str]] = {}
paths = _walk_etc_files()
# Query in chunks to avoid excessive process spawns.
chunk_size = 250
not_owned_re = re.compile(
r"^file\s+(?P<path>.+?)\s+is\s+not\s+owned\s+by\s+any\s+package", re.IGNORECASE
)
for i in range(0, len(paths), chunk_size):
chunk = paths[i : i + chunk_size]
rc, out = _run(
["rpm", "-qf", "--qf", "%{NAME}\n", *chunk],
allow_fail=True,
merge_err=True,
)
lines = [ln.strip() for ln in out.splitlines() if ln.strip()]
# Heuristic: rpm prints one output line per input path. If that isn't
# true (warnings/errors), fall back to per-file queries for this chunk.
if len(lines) != len(chunk):
for p in chunk:
pkg = rpm_owner(p)
if not pkg:
continue
owned.add(p)
owner.setdefault(p, pkg)
pkg_to_etc.setdefault(pkg, []).append(p)
parts = p.split("/", 3)
if len(parts) >= 3 and parts[2]:
topdir_to_pkgs.setdefault(parts[2], set()).add(pkg)
continue
for pth, line in zip(chunk, lines):
if not line:
continue
if not_owned_re.match(line) or "is not owned" in line:
continue
pkg = line.split()[0].strip()
if not pkg:
continue
owned.add(pth)
owner.setdefault(pth, pkg)
pkg_to_etc.setdefault(pkg, []).append(pth)
parts = pth.split("/", 3)
if len(parts) >= 3 and parts[2]:
topdir_to_pkgs.setdefault(parts[2], set()).add(pkg)
for k, v in list(pkg_to_etc.items()):
pkg_to_etc[k] = sorted(set(v))
return owned, owner, topdir_to_pkgs, pkg_to_etc
def rpm_config_files(pkg: str) -> Set[str]:
"""Return config files for a package (rpm -qc)."""
rc, out = _run(["rpm", "-qc", pkg], allow_fail=True, merge_err=True)
if rc != 0:
return set()
files: Set[str] = set()
for line in out.splitlines():
line = line.strip()
if line.startswith("/"):
files.add(line)
return files
def rpm_modified_files(pkg: str) -> Set[str]:
"""Return files reported as modified by rpm verification (rpm -V).
rpm -V only prints lines for differences/missing files.
"""
rc, out = _run(["rpm", "-V", pkg], allow_fail=True, merge_err=True)
# rc is non-zero when there are differences; we still want the output.
files: Set[str] = set()
for raw in out.splitlines():
line = raw.strip()
if not line:
continue
# Typical forms:
# S.5....T. c /etc/foo.conf
# missing /etc/bar
m = re.search(r"\s(/\S+)$", line)
if m:
files.add(m.group(1))
continue
if line.startswith("missing"):
parts = line.split()
if parts and parts[-1].startswith("/"):
files.add(parts[-1])
return files

4
enroll/schema/__init__.py
View file

@ -1,4 +0,0 @@
"""Vendored JSON schemas.
These are used by `enroll validate` so validation can run offline.
"""

788
enroll/schema/state.schema.json
View file

@ -1,788 +0,0 @@
{
"$defs": {
"AptConfigSnapshot": {
"allOf": [
{
"$ref": "#/$defs/RoleCommon"
},
{
"properties": {
"role_name": {
"const": "apt_config"
}
},
"type": "object"
}
],
"unevaluatedProperties": false
},
"DnfConfigSnapshot": {
"allOf": [
{
"$ref": "#/$defs/RoleCommon"
},
{
"properties": {
"role_name": {
"const": "dnf_config"
}
},
"type": "object"
}
],
"unevaluatedProperties": false
},
"EtcCustomSnapshot": {
"allOf": [
{
"$ref": "#/$defs/RoleCommon"
},
{
"properties": {
"role_name": {
"const": "etc_custom"
}
},
"type": "object"
}
],
"unevaluatedProperties": false
},
"ExcludedFile": {
"additionalProperties": false,
"properties": {
"path": {
"minLength": 1,
"pattern": "^/.*",
"type": "string"
},
"reason": {
"enum": [
"user_excluded",
"unreadable",
"backup_file",
"log_file",
"denied_path",
"too_large",
"not_regular_file",
"not_symlink",
"binary_like",
"sensitive_content"
],
"type": "string"
}
},
"required": [
"path",
"reason"
],
"type": "object"
},
"ExtraPathsSnapshot": {
"allOf": [
{
"$ref": "#/$defs/RoleCommon"
},
{
"properties": {
"exclude_patterns": {
"items": {
"type": "string"
},
"type": "array"
},
"include_patterns": {
"items": {
"type": "string"
},
"type": "array"
},
"role_name": {
"const": "extra_paths"
}
},
"required": [
"include_patterns",
"exclude_patterns"
],
"type": "object"
}
],
"unevaluatedProperties": false
},
"InstalledPackageInstance": {
"additionalProperties": false,
"properties": {
"arch": {
"minLength": 1,
"type": "string"
},
"version": {
"minLength": 1,
"type": "string"
}
},
"required": [
"version",
"arch"
],
"type": "object"
},
"ManagedDir": {
"additionalProperties": false,
"properties": {
"group": {
"minLength": 1,
"type": "string"
},
"mode": {
"pattern": "^[0-7]{4}$",
"type": "string"
},
"owner": {
"minLength": 1,
"type": "string"
},
"path": {
"minLength": 1,
"pattern": "^/.*",
"type": "string"
},
"reason": {
"enum": [
"parent_of_managed_file",
"user_include_dir"
],
"type": "string"
}
},
"required": [
"path",
"owner",
"group",
"mode",
"reason"
],
"type": "object"
},
"ManagedFile": {
"additionalProperties": false,
"properties": {
"group": {
"minLength": 1,
"type": "string"
},
"mode": {
"pattern": "^[0-7]{4}$",
"type": "string"
},
"owner": {
"minLength": 1,
"type": "string"
},
"path": {
"minLength": 1,
"pattern": "^/.*",
"type": "string"
},
"reason": {
"enum": [
"apt_config",
"apt_keyring",
"apt_signed_by_keyring",
"apt_source",
"authorized_keys",
"cron_snippet",
"custom_specific_path",
"custom_unowned",
"dnf_config",
"logrotate_snippet",
"modified_conffile",
"modified_packaged_file",
"related_timer",
"rpm_gpg_key",
"ssh_public_key",
"system_cron",
"system_firewall",
"system_logrotate",
"system_modprobe",
"system_mounts",
"system_network",
"system_rc",
"system_security",
"system_sysctl",
"systemd_dropin",
"systemd_envfile",
"user_include",
"user_profile",
"user_shell_aliases",
"user_shell_logout",
"user_shell_rc",
"usr_local_bin_script",
"usr_local_etc_custom",
"yum_conf",
"yum_config",
"yum_repo"
],
"type": "string"
},
"src_rel": {
"minLength": 1,
"pattern": "^[^/].*",
"type": "string"
}
},
"required": [
"path",
"src_rel",
"owner",
"group",
"mode",
"reason"
],
"type": "object"
},
"ManagedLink": {
"additionalProperties": false,
"type": "object",
"properties": {
"path": {
"type": "string",
"minLength": 1,
"pattern": "^/.*"
},
"target": {
"type": "string",
"minLength": 1
},
"reason": {
"type": "string",
"enum": [
"enabled_symlink"
]
}
},
"required": [
"path",
"target",
"reason"
]
},
"ObservedVia": {
"oneOf": [
{
"additionalProperties": false,
"properties": {
"kind": {
"const": "user_installed"
}
},
"required": [
"kind"
],
"type": "object"
},
{
"additionalProperties": false,
"properties": {
"kind": {
"const": "systemd_unit"
},
"ref": {
"minLength": 1,
"type": "string"
}
},
"required": [
"kind",
"ref"
],
"type": "object"
},
{
"additionalProperties": false,
"properties": {
"kind": {
"const": "package_role"
},
"ref": {
"minLength": 1,
"type": "string"
}
},
"required": [
"kind",
"ref"
],
"type": "object"
},
{
"additionalProperties": false,
"properties": {
"kind": {
"const": "firewall_runtime"
},
"ref": {
"minLength": 1,
"type": "string"
}
},
"required": [
"kind",
"ref"
],
"type": "object"
}
]
},
"PackageInventoryEntry": {
"additionalProperties": false,
"properties": {
"arches": {
"items": {
"minLength": 1,
"type": "string"
},
"type": "array"
},
"installations": {
"items": {
"$ref": "#/$defs/InstalledPackageInstance"
},
"type": "array"
},
"observed_via": {
"items": {
"$ref": "#/$defs/ObservedVia"
},
"type": "array"
},
"roles": {
"items": {
"minLength": 1,
"type": "string"
},
"type": "array"
},
"version": {
"type": [
"string",
"null"
]
}
},
"required": [
"version",
"arches",
"installations",
"observed_via",
"roles"
],
"type": "object"
},
"PackageSnapshot": {
"allOf": [
{
"$ref": "#/$defs/RoleCommon"
},
{
"properties": {
"package": {
"minLength": 1,
"type": "string"
}
},
"required": [
"package"
],
"type": "object"
}
],
"unevaluatedProperties": false
},
"RoleCommon": {
"properties": {
"excluded": {
"items": {
"$ref": "#/$defs/ExcludedFile"
},
"type": "array"
},
"managed_dirs": {
"items": {
"$ref": "#/$defs/ManagedDir"
},
"type": "array"
},
"managed_files": {
"items": {
"$ref": "#/$defs/ManagedFile"
},
"type": "array"
},
"managed_links": {
"items": {
"$ref": "#/$defs/ManagedLink"
},
"type": "array"
},
"notes": {
"items": {
"type": "string"
},
"type": "array"
},
"role_name": {
"minLength": 1,
"pattern": "^[A-Za-z0-9_]+$",
"type": "string"
}
},
"required": [
"role_name",
"managed_dirs",
"managed_files",
"excluded",
"notes"
],
"type": "object"
},
"ServiceSnapshot": {
"allOf": [
{
"$ref": "#/$defs/RoleCommon"
},
{
"properties": {
"active_state": {
"type": [
"string",
"null"
]
},
"condition_result": {
"type": [
"string",
"null"
]
},
"packages": {
"items": {
"minLength": 1,
"type": "string"
},
"type": "array"
},
"role_name": {
"minLength": 1,
"pattern": "^[a-z_][a-z0-9_]*$",
"type": "string"
},
"sub_state": {
"type": [
"string",
"null"
]
},
"unit": {
"minLength": 1,
"type": "string"
},
"unit_file_state": {
"type": [
"string",
"null"
]
}
},
"required": [
"unit",
"packages",
"active_state",
"sub_state",
"unit_file_state",
"condition_result"
],
"type": "object"
}
],
"unevaluatedProperties": false
},
"UserEntry": {
"additionalProperties": false,
"properties": {
"gecos": {
"type": "string"
},
"gid": {
"minimum": 0,
"type": "integer"
},
"home": {
"type": "string"
},
"name": {
"minLength": 1,
"type": "string"
},
"primary_group": {
"minLength": 1,
"type": "string"
},
"shell": {
"type": "string"
},
"supplementary_groups": {
"items": {
"minLength": 1,
"type": "string"
},
"type": "array"
},
"uid": {
"minimum": 0,
"type": "integer"
}
},
"required": [
"name",
"uid",
"gid",
"gecos",
"home",
"shell",
"primary_group",
"supplementary_groups"
],
"type": "object"
},
"UsersSnapshot": {
"allOf": [
{
"$ref": "#/$defs/RoleCommon"
},
{
"properties": {
"role_name": {
"const": "users"
},
"users": {
"items": {
"$ref": "#/$defs/UserEntry"
},
"type": "array"
}
},
"required": [
"users"
],
"type": "object"
}
],
"unevaluatedProperties": false
},
"UsrLocalCustomSnapshot": {
"allOf": [
{
"$ref": "#/$defs/RoleCommon"
},
{
"properties": {
"role_name": {
"const": "usr_local_custom"
}
},
"type": "object"
}
],
"unevaluatedProperties": false
},
"FirewallRuntimeSnapshot": {
"additionalProperties": false,
"properties": {
"role_name": {
"const": "firewall_runtime"
},
"packages": {
"items": {
"minLength": 1,
"type": "string"
},
"type": "array"
},
"ipset_save": {
"type": [
"string",
"null"
]
},
"ipset_sets": {
"items": {
"minLength": 1,
"type": "string"
},
"type": "array"
},
"iptables_v4_save": {
"type": [
"string",
"null"
]
},
"iptables_v6_save": {
"type": [
"string",
"null"
]
},
"notes": {
"items": {
"type": "string"
},
"type": "array"
}
},
"required": [
"role_name",
"packages",
"ipset_save",
"ipset_sets",
"iptables_v4_save",
"iptables_v6_save",
"notes"
],
"type": "object"
}
},
"$id": "https://enroll.sh/schema/state.schema.json",
"$schema": "https://json-schema.org/draft/2020-12/schema",
"additionalProperties": false,
"properties": {
"enroll": {
"additionalProperties": false,
"properties": {
"harvest_time": {
"minimum": 0,
"type": "integer"
},
"version": {
"type": "string"
}
},
"required": [
"version",
"harvest_time"
],
"type": "object"
},
"host": {
"additionalProperties": false,
"properties": {
"hostname": {
"minLength": 1,
"type": "string"
},
"os": {
"enum": [
"debian",
"redhat",
"unknown"
],
"type": "string"
},
"os_release": {
"additionalProperties": {
"type": "string"
},
"type": "object"
},
"pkg_backend": {
"enum": [
"dpkg",
"rpm"
],
"type": "string"
}
},
"required": [
"hostname",
"os",
"pkg_backend",
"os_release"
],
"type": "object"
},
"inventory": {
"additionalProperties": false,
"properties": {
"packages": {
"additionalProperties": {
"$ref": "#/$defs/PackageInventoryEntry"
},
"type": "object"
}
},
"required": [
"packages"
],
"type": "object"
},
"roles": {
"additionalProperties": false,
"properties": {
"apt_config": {
"$ref": "#/$defs/AptConfigSnapshot"
},
"dnf_config": {
"$ref": "#/$defs/DnfConfigSnapshot"
},
"etc_custom": {
"$ref": "#/$defs/EtcCustomSnapshot"
},
"extra_paths": {
"$ref": "#/$defs/ExtraPathsSnapshot"
},
"packages": {
"items": {
"$ref": "#/$defs/PackageSnapshot"
},
"type": "array"
},
"services": {
"items": {
"$ref": "#/$defs/ServiceSnapshot"
},
"type": "array"
},
"users": {
"$ref": "#/$defs/UsersSnapshot"
},
"usr_local_custom": {
"$ref": "#/$defs/UsrLocalCustomSnapshot"
},
"firewall_runtime": {
"$ref": "#/$defs/FirewallRuntimeSnapshot"
}
},
"required": [
"users",
"services",
"packages",
"apt_config",
"dnf_config",
"etc_custom",
"usr_local_custom",
"extra_paths"
],
"type": "object"
}
},
"required": [
"enroll",
"host",
"inventory",
"roles"
],
"title": "Enroll harvest state.json schema (latest)",
"type": "object"
}

74
enroll/secrets.py Normal file
View file

@ -0,0 +1,74 @@
from __future__ import annotations
import fnmatch
import os
import re
from dataclasses import dataclass
from typing import Optional
DEFAULT_DENY_GLOBS = [
# Common backup copies created by passwd tools (can contain sensitive data)
"/etc/passwd-",
"/etc/group-",
"/etc/shadow-",
"/etc/gshadow-",
"/etc/subuid-",
"/etc/subgid-",
"/etc/*shadow-",
"/etc/*gshadow-",
"/etc/ssl/private/*",
"/etc/ssh/ssh_host_*",
"/etc/shadow",
"/etc/gshadow",
"/etc/*shadow",
"/etc/letsencrypt/*",
]
SENSITIVE_CONTENT_PATTERNS = [
re.compile(rb"-----BEGIN (RSA |EC |OPENSSH |)PRIVATE KEY-----"),
re.compile(rb"(?i)\bpassword\s*="),
re.compile(rb"(?i)\b(pass|passwd|token|secret|api[_-]?key)\b"),
]
@dataclass
class SecretPolicy:
deny_globs: list[str] = None
max_file_bytes: int = 256_000
sample_bytes: int = 64_000
def __post_init__(self) -> None:
if self.deny_globs is None:
self.deny_globs = list(DEFAULT_DENY_GLOBS)
def deny_reason(self, path: str) -> Optional[str]:
for g in self.deny_globs:
if fnmatch.fnmatch(path, g):
return "denied_path"
try:
st = os.stat(path, follow_symlinks=True)
except OSError:
return "unreadable"
if st.st_size > self.max_file_bytes:
return "too_large"
if not os.path.isfile(path) or os.path.islink(path):
return "not_regular_file"
try:
with open(path, "rb") as f:
data = f.read(min(self.sample_bytes, st.st_size))
except OSError:
return "unreadable"
if b"\x00" in data:
return "binary_like"
for pat in SENSITIVE_CONTENT_PATTERNS:
if pat.search(data):
return "sensitive_content"
return None

137
enroll/sopsutil.py
View file

@ -1,137 +0,0 @@
from __future__ import annotations
import os
import shutil
import subprocess # nosec
import tempfile
from pathlib import Path
from typing import Iterable, List, Optional
class SopsError(RuntimeError):
pass
def find_sops_cmd() -> Optional[str]:
"""Return the `sops` executable path if present on PATH."""
return shutil.which("sops")
def require_sops_cmd() -> str:
exe = find_sops_cmd()
if not exe:
raise SopsError(
"--sops was requested but `sops` was not found on PATH. "
"Install sops and ensure it is available as `sops`."
)
return exe
def _pgp_arg(fingerprints: Iterable[str]) -> str:
fps = [f.strip() for f in fingerprints if f and f.strip()]
if not fps:
raise SopsError("No GPG fingerprints provided for --sops")
# sops accepts a comma-separated list for --pgp.
return ",".join(fps)
def encrypt_file_binary(
src_path: Path,
dst_path: Path,
*,
pgp_fingerprints: List[str],
mode: int = 0o600,
) -> None:
"""Encrypt src_path with sops (binary) and write to dst_path atomically."""
sops = require_sops_cmd()
src_path = Path(src_path)
dst_path = Path(dst_path)
dst_path.parent.mkdir(parents=True, exist_ok=True)
res = subprocess.run(
[
sops,
"--encrypt",
"--input-type",
"binary",
"--output-type",
"binary",
"--pgp",
_pgp_arg(pgp_fingerprints),
str(src_path),
],
capture_output=True,
check=False,
) # nosec
if res.returncode != 0:
raise SopsError(
"sops encryption failed:\n"
f" cmd: {sops} --encrypt ... {src_path}\n"
f" rc: {res.returncode}\n"
f" stderr: {res.stderr.decode('utf-8', errors='replace').strip()}"
)
# Write atomically in the destination directory.
fd, tmp = tempfile.mkstemp(prefix=".enroll-sops-", dir=str(dst_path.parent))
try:
with os.fdopen(fd, "wb") as f:
f.write(res.stdout)
try:
os.chmod(tmp, mode)
except OSError:
pass
os.replace(tmp, dst_path)
finally:
try:
os.unlink(tmp)
except FileNotFoundError:
pass
def decrypt_file_binary_to(
src_path: Path,
dst_path: Path,
*,
mode: int = 0o600,
) -> None:
"""Decrypt a sops-encrypted file (binary) into dst_path."""
sops = require_sops_cmd()
src_path = Path(src_path)
dst_path = Path(dst_path)
dst_path.parent.mkdir(parents=True, exist_ok=True)
res = subprocess.run(
[
sops,
"--decrypt",
"--input-type",
"binary",
"--output-type",
"binary",
str(src_path),
],
capture_output=True,
check=False,
) # nosec
if res.returncode != 0:
raise SopsError(
"sops decryption failed:\n"
f" cmd: {sops} --decrypt ... {src_path}\n"
f" rc: {res.returncode}\n"
f" stderr: {res.stderr.decode('utf-8', errors='replace').strip()}"
)
fd, tmp = tempfile.mkstemp(prefix=".enroll-sops-", dir=str(dst_path.parent))
try:
with os.fdopen(fd, "wb") as f:
f.write(res.stdout)
try:
os.chmod(tmp, mode)
except OSError:
pass
os.replace(tmp, dst_path)
finally:
try:
os.unlink(tmp)
except FileNotFoundError:
pass

97
enroll/systemd.py
View file

@ -33,19 +33,6 @@ def _run(cmd: list[str]) -> str:
return p.stdout
@dataclass
class TimerInfo:
name: str
fragment_path: Optional[str]
dropin_paths: List[str]
env_files: List[str]
trigger_unit: Optional[str]
active_state: Optional[str]
sub_state: Optional[str]
unit_file_state: Optional[str]
condition_result: Optional[str]
def list_enabled_services() -> List[str]:
out = _run(
[
@ -71,31 +58,6 @@ def list_enabled_services() -> List[str]:
return sorted(set(units))
def list_enabled_timers() -> List[str]:
out = _run(
[
"systemctl",
"list-unit-files",
"--type=timer",
"--state=enabled",
"--no-legend",
]
)
units: List[str] = []
for line in out.splitlines():
parts = line.split()
if not parts:
continue
unit = parts[0].strip()
if not unit.endswith(".timer"):
continue
# Skip template units like "foo@.timer"
if unit.endswith("@.timer"):
continue
units.append(unit)
return sorted(set(units))
def get_unit_info(unit: str) -> UnitInfo:
p = subprocess.run(
[
@ -155,62 +117,3 @@ def get_unit_info(unit: str) -> UnitInfo:
unit_file_state=kv.get("UnitFileState") or None,
condition_result=kv.get("ConditionResult") or None,
)
def get_timer_info(unit: str) -> TimerInfo:
p = subprocess.run(
[
"systemctl",
"show",
unit,
"-p",
"FragmentPath",
"-p",
"DropInPaths",
"-p",
"EnvironmentFiles",
"-p",
"Unit",
"-p",
"ActiveState",
"-p",
"SubState",
"-p",
"UnitFileState",
"-p",
"ConditionResult",
],
text=True,
capture_output=True,
) # nosec
if p.returncode != 0:
raise RuntimeError(f"systemctl show failed for {unit}: {p.stderr}")
kv: dict[str, str] = {}
for line in (p.stdout or "").splitlines():
if "=" in line:
k, v = line.split("=", 1)
kv[k] = v.strip()
fragment = kv.get("FragmentPath") or None
dropins = [pp for pp in (kv.get("DropInPaths", "") or "").split() if pp]
env_files: List[str] = []
for token in (kv.get("EnvironmentFiles", "") or "").split():
token = token.lstrip("-")
if token:
env_files.append(token)
trigger = kv.get("Unit") or None
return TimerInfo(
name=unit,
fragment_path=fragment,
dropin_paths=dropins,
env_files=env_files,
trigger_unit=trigger,
active_state=kv.get("ActiveState") or None,
sub_state=kv.get("SubState") or None,
unit_file_state=kv.get("UnitFileState") or None,
condition_result=kv.get("ConditionResult") or None,
)

254
enroll/validate.py
View file

@ -1,254 +0,0 @@
from __future__ import annotations
import json
import urllib.request
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Dict, List, Optional, Set, Tuple
import jsonschema
from .diff import BundleRef, _bundle_from_input
@dataclass
class ValidationResult:
errors: List[str]
warnings: List[str]
@property
def ok(self) -> bool:
return not self.errors
def to_dict(self) -> Dict[str, Any]:
return {
"ok": self.ok,
"errors": list(self.errors),
"warnings": list(self.warnings),
}
def to_text(self) -> str:
lines: List[str] = []
if not self.errors and not self.warnings:
lines.append("OK: harvest bundle validated")
elif not self.errors and self.warnings:
lines.append(f"WARN: {len(self.warnings)} warning(s)")
else:
lines.append(f"ERROR: {len(self.errors)} validation error(s)")
if self.errors:
lines.append("")
lines.append("Errors:")
for e in self.errors:
lines.append(f"- {e}")
if self.warnings:
lines.append("")
lines.append("Warnings:")
for w in self.warnings:
lines.append(f"- {w}")
return "\n".join(lines) + "\n"
def _default_schema_path() -> Path:
# Keep the schema vendored with the codebase so enroll can validate offline.
return Path(__file__).resolve().parent / "schema" / "state.schema.json"
def _load_schema(schema: Optional[str]) -> Dict[str, Any]:
"""Load a JSON schema.
If schema is None, load the vendored schema.
If schema begins with http(s)://, fetch it.
Otherwise, treat it as a local file path.
"""
if not schema:
p = _default_schema_path()
with open(p, "r", encoding="utf-8") as f:
return json.load(f)
if schema.startswith("http://") or schema.startswith("https://"):
with urllib.request.urlopen(schema, timeout=10) as resp: # nosec
data = resp.read()
return json.loads(data.decode("utf-8"))
p = Path(schema).expanduser()
with open(p, "r", encoding="utf-8") as f:
return json.load(f)
def _json_pointer(err: jsonschema.ValidationError) -> str:
# Build a JSON pointer-ish path that is easy to read.
if err.absolute_path:
parts = [str(p) for p in err.absolute_path]
return "/" + "/".join(parts)
return "/"
def _iter_managed_files(state: Dict[str, Any]) -> List[Tuple[str, Dict[str, Any]]]:
"""Return (role_name, managed_file_dict) tuples across all roles."""
roles = state.get("roles") or {}
out: List[Tuple[str, Dict[str, Any]]] = []
# Singleton roles
for rn in [
"users",
"apt_config",
"dnf_config",
"etc_custom",
"usr_local_custom",
"extra_paths",
]:
snap = roles.get(rn) or {}
for mf in snap.get("managed_files") or []:
if isinstance(mf, dict):
out.append((rn, mf))
# Array roles
for s in roles.get("services") or []:
if not isinstance(s, dict):
continue
role_name = str(s.get("role_name") or "unknown")
for mf in s.get("managed_files") or []:
if isinstance(mf, dict):
out.append((role_name, mf))
for p in roles.get("packages") or []:
if not isinstance(p, dict):
continue
role_name = str(p.get("role_name") or "unknown")
for mf in p.get("managed_files") or []:
if isinstance(mf, dict):
out.append((role_name, mf))
return out
def validate_harvest(
harvest_input: str,
*,
sops_mode: bool = False,
schema: Optional[str] = None,
no_schema: bool = False,
) -> ValidationResult:
"""Validate an enroll harvest bundle.
Checks:
- state.json parses
- state.json validates against the schema (unless no_schema)
- every managed_file src_rel exists in artifacts/<role>/<src_rel>
"""
errors: List[str] = []
warnings: List[str] = []
bundle: BundleRef = _bundle_from_input(harvest_input, sops_mode=sops_mode)
try:
state_path = bundle.state_path
if not state_path.exists():
return ValidationResult(
errors=[f"missing state.json at {state_path}"], warnings=[]
)
try:
state = json.loads(state_path.read_text(encoding="utf-8"))
except Exception as e: # noqa: BLE001
return ValidationResult(
errors=[f"failed to parse state.json: {e!r}"], warnings=[]
)
if not no_schema:
try:
sch = _load_schema(schema)
validator = jsonschema.Draft202012Validator(sch)
for err in sorted(validator.iter_errors(state), key=str):
ptr = _json_pointer(err)
msg = err.message
errors.append(f"schema {ptr}: {msg}")
except Exception as e: # noqa: BLE001
errors.append(f"failed to load/validate schema: {e!r}")
# Artifact existence checks
artifacts_dir = bundle.dir / "artifacts"
referenced: Set[Tuple[str, str]] = set()
for role_name, mf in _iter_managed_files(state):
src_rel = str(mf.get("src_rel") or "")
if not src_rel:
errors.append(
f"managed_file missing src_rel for role {role_name} (path={mf.get('path')!r})"
)
continue
if src_rel.startswith("/") or ".." in src_rel.split("/"):
errors.append(
f"managed_file has suspicious src_rel for role {role_name}: {src_rel!r}"
)
continue
referenced.add((role_name, src_rel))
p = artifacts_dir / role_name / src_rel
if not p.exists():
errors.append(
f"missing artifact for role {role_name}: artifacts/{role_name}/{src_rel}"
)
continue
if not p.is_file():
errors.append(
f"artifact is not a file for role {role_name}: artifacts/{role_name}/{src_rel}"
)
# Runtime firewall snapshots are generated artifacts rather than managed files.
fw = (state.get("roles") or {}).get("firewall_runtime") or {}
if isinstance(fw, dict):
for key in ("ipset_save", "iptables_v4_save", "iptables_v6_save"):
src_rel = str(fw.get(key) or "")
if not src_rel:
continue
if src_rel.startswith("/") or ".." in src_rel.split("/"):
errors.append(
f"firewall_runtime {key} has suspicious src_rel: {src_rel!r}"
)
continue
referenced.add(
(str(fw.get("role_name") or "firewall_runtime"), src_rel)
)
p = (
artifacts_dir
/ str(fw.get("role_name") or "firewall_runtime")
/ src_rel
)
if not p.exists():
errors.append(
"missing firewall runtime artifact: "
f"artifacts/{fw.get('role_name') or 'firewall_runtime'}/{src_rel}"
)
elif not p.is_file():
errors.append(
"firewall runtime artifact is not a file: "
f"artifacts/{fw.get('role_name') or 'firewall_runtime'}/{src_rel}"
)
# Warn if there are extra files in artifacts not referenced.
if artifacts_dir.exists() and artifacts_dir.is_dir():
for fp in artifacts_dir.rglob("*"):
if not fp.is_file():
continue
try:
rel = fp.relative_to(artifacts_dir)
except ValueError:
continue
parts = rel.parts
if len(parts) < 2:
continue
role_name = parts[0]
src_rel = "/".join(parts[1:])
if (role_name, src_rel) not in referenced:
warnings.append(
f"unreferenced artifact present: artifacts/{role_name}/{src_rel}"
)
return ValidationResult(errors=errors, warnings=warnings)
finally:
# Ensure any temp extraction dirs are cleaned up.
if bundle.tempdir is not None:
bundle.tempdir.cleanup()

32
enroll/version.py
View file

@ -1,32 +0,0 @@
from __future__ import annotations
def get_enroll_version() -> str:
"""
Best-effort version lookup that works when installed via:
- poetry/pip/wheel
- deb/rpm system packages
Falls back to "0+unknown" when running from an unpacked source tree.
"""
try:
from importlib.metadata import (
packages_distributions,
version,
)
except Exception:
# Very old Python or unusual environment
return "unknown"
# Map import package -> dist(s)
dist_names = []
try:
dist_names = (packages_distributions() or {}).get("enroll", []) or []
except Exception:
dist_names = []
# Try mapped dists first, then a reasonable default
for dist in [*dist_names, "enroll"]:
try:
return version(dist)
except Exception:
return "unknown"

1199
poetry.lock generated
View file

File diff suppressed because it is too large Load diff

21
pyproject.toml
View file

@ -1,25 +1,27 @@
[tool.poetry]
name = "enroll"
version = "0.6.0"
version = "0.0.2"
description = "Enroll a server's running state retrospectively into Ansible"
authors = ["Miguel Jacq <mig@mig5.net>"]
license = "GPL-3.0-or-later"
readme = "README.md"
packages = [{ include = "enroll" }]
repository = "https://git.mig5.net/mig5/enroll"
include = [
{ path = "enroll/schema/state.schema.json", format = ["sdist", "wheel"] }
]
[tool.poetry.dependencies]
python = "^3.10"
pyyaml = "^6"
paramiko = ">=3.5"
jsonschema = "^4.23.0"
[tool.poetry.scripts]
enroll = "enroll.cli:main"
[tool.poetry.group.test.dependencies]
pytest = "^9.0.2"
pytest-cov = "^7.0.0"
[tool.poetry.group.dev.dependencies]
pyproject-appimage = "^4.2"
[build-system]
requires = ["poetry-core>=1.8.0"]
build-backend = "poetry.core.masonry.api"
@ -27,8 +29,3 @@ build-backend = "poetry.core.masonry.api"
[tool.pyproject-appimage]
script = "enroll"
output = "Enroll.AppImage"
[tool.poetry.dev-dependencies]
pytest = "^8"
pytest-cov = "^5"
pyproject-appimage = "^4.2"

76
release.sh
View file

@ -15,79 +15,3 @@ mv Enroll.AppImage dist/
# Sign packages
for file in `ls -1 dist/`; do qubes-gpg-client --batch --armor --detach-sign dist/$file > dist/$file.asc; done
# Deb stuff
DISTS=(
debian:bookworm
debian:trixie
ubuntu:jammy
ubuntu:noble
)
for dist in ${DISTS[@]}; do
release=$(echo ${dist} | cut -d: -f2)
mkdir -p dist/${release}
docker build -f Dockerfile.debbuild -t enroll-deb:${release} \
--no-cache \
--progress=plain \
--build-arg BASE_IMAGE=${dist} .
docker run --rm \
-e SUITE="${release}" \
-v "$PWD":/src \
-v "$PWD/dist/${release}":/out \
enroll-deb:${release}
debfile=$(ls -1 dist/${release}/*.deb)
reprepro -b /home/user/git/repo includedeb "${release}" "${debfile}"
done
# RPM
sudo apt-get -y install createrepo-c rpm
BUILD_OUTPUT="${HOME}/git/enroll/dist"
KEYID="54A91143AE0AB4F7743B01FE888ED1B423A3BC99"
REPO_ROOT="${HOME}/git/repo_rpm"
REMOTE="letessier.mig5.net:/opt/repo_rpm"
DISTS=(
fedora:43
fedora:42
)
for dist in ${DISTS[@]}; do
release=$(echo ${dist} | cut -d: -f2)
REPO_RELEASE_ROOT="${REPO_ROOT}/${release}"
RPM_REPO="${REPO_RELEASE_ROOT}/rpm/x86_64"
mkdir -p "$RPM_REPO"
docker build \
--no-cache \
-f Dockerfile.rpmbuild \
-t enroll-rpm:${release} \
--progress=plain \
--build-arg BASE_IMAGE=${dist} \
.
rm -rf "$PWD/dist/rpm"/*
mkdir -p "$PWD/dist/rpm"
docker run --rm -v "$PWD":/src -v "$PWD/dist/rpm":/out enroll-rpm:${release}
sudo chown -R "${USER}" "$PWD/dist"
for file in `ls -1 "${BUILD_OUTPUT}/rpm"`; do
rpmsign --addsign "${BUILD_OUTPUT}/rpm/$file"
done
cp "${BUILD_OUTPUT}/rpm/"*.rpm "$RPM_REPO/"
createrepo_c "$RPM_REPO"
echo "==> Signing repomd.xml..."
qubes-gpg-client --local-user "$KEYID" --detach-sign --armor "$RPM_REPO/repodata/repomd.xml" > "$RPM_REPO/repodata/repomd.xml.asc"
done
echo "==> Syncing repo to server..."
rsync -aHPvz --exclude=.git --delete "$REPO_ROOT/" "$REMOTE/"
echo "Done!"

100
rpm/enroll.spec
View file

@ -1,100 +0,0 @@
%global upstream_version 0.6.0
Name: enroll
Version: %{upstream_version}
Release: 1%{?dist}.enroll1
Summary: Enroll a server's running state retrospectively into Ansible.
License: GPL-3.0-or-later
URL: https://git.mig5.net/mig5/enroll
Source0: %{name}-%{version}.tar.gz
BuildArch: noarch
BuildRequires: pyproject-rpm-macros
BuildRequires: python3-devel
BuildRequires: python3-poetry-core
Requires: python3-yaml
Requires: python3-paramiko
Requires: python3-jsonschema
Recommends: jinjaturtle
%description
Enroll a server's running state retrospectively into Ansible.
%prep
%autosetup -n enroll
%generate_buildrequires
%pyproject_buildrequires
%build
%pyproject_wheel
%install
%pyproject_install
%pyproject_save_files enroll
%files -f %{pyproject_files}
%license LICENSE
%doc README.md CHANGELOG.md
%{_bindir}/enroll
%changelog
* Thu May 14 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Add support for capturing ipset and iptables configuration files
- Add support for generating ipset and iptables configuration files from runtime, if the former weren't present ('firewall_runtime' role)
* Tue May 12 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Add ssh config support where JinjaTurtle is used
* Tue Feb 16 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Add capability to handle passphrases on encrypted SSH private keys. Prompting can be forced with `--ask-key-passphrase` or automated (e.g for CI) with `--ssh-key-passphrase env SOMEVAR`
* Fri Jan 16 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Add support for AddressFamily and ConnectTimeout in the .ssh/config when using `--remote-ssh-config`.
* Tue Jan 13 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Support `--remote-ssh-config [path-to-ssh-config]` as an argument in case extra params are required beyond `--remote-port` or `--remote-user`. Note: `--remote-host` must still be s
et, but it can be an 'alias' represented by the 'Host' value in the ssh config.
* Sun Jan 11 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Add interactive output when 'enroll diff --enforce' is invoking Ansible.
* Sat Jan 10 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Introduce `enroll validate` - a tool to validate a harvest against the state schema, or check for missing or orphaned obsolete artifacts in a harvest.
- Attempt to generate Jinja2 templates of systemd unit files and Postfix main.cf (now that JinjaTurtle supports it)
- Update pynacl dependency to resolve CVE-2025-69277
- Add `--exclude-path` to `enroll diff` command, so that you can ignore certain churn from the diff (stuff you still wanted to harvest as a baseline but don't care if it changes day to day)
- Add `--ignore-package-versions` to `enroll diff` command, to optionally ignore package upgrades (e.g due to patching) from the diff.
- Add tags to the playbook for each role, to allow easier targeting of specific roles during play later.
- Add `--enforce` mode to `enroll diff`. If there is diff detected between the two harvests, and it can enforce restoring the state from the older harvest, it will manifest the state and apply it with ansible.
Only the specific roles that had diffed will be applied (via the new tags capability)
* Mon Jan 05 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Introduce `enroll explain` - a tool to analyze and explain what's in (or not in) a harvest and why.
- Centralise the cron and logrotate stuff into their respective roles, we had a bit of duplication between roles based on harvest discovery.
- Capture other files in the user's home directory such as `.bashrc`, `.bash_aliases`, `.profile`, if these files differ from the `/etc/skel` defaults
- Ignore files that end with a tilde or - (probably backup files generated by editors or shadow file changes)
- Manage certain symlinks e.g for apache2/nginx sites-enabled and so on
* Sun Jan 04 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Introduce --ask-become-pass or -K to support password-required sudo on remote hosts, just like Ansible. It will also fall back to this prompt if a password is required but the arg wasn't passed in.
* Sat Jan 03 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Fix stat() of parent directory so that we set directory perms correct on --include paths.
- Set pty for remote calls when sudo is required, to help systems with limits on sudo without pty
* Fri Jan 02 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Don't accidentally add extra_paths role to usr_local_custom list, resulting in extra_paths appearing twice in manifested playbook
- Ensure directories in the tree of anything included with --include are defined in the state and manifest so we make dirs before we try to create files
* Mon Dec 29 2025 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Add version CLI arg
- Add ability to enroll RH-style systems (DNF5/DNF/RPM)
- Refactor harvest state to track package versions
* Sun Dec 28 2025 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Fix an attribution bug for certain files ending up in the wrong package/role.
* Sun Dec 28 2025 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- DRY up some code logic
- More test coverage
* Sun Dec 28 2025 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Consolidate logrotate and cron files into their main service/package roles if they exist.
- Standardise on MAX_FILES_CAP in one place
- Manage apt stuff in its own role, not in etc_custom
* Sat Dec 27 2025 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Attempt to capture more stuff from /etc that might not be attributable to a specific package. This includes common singletons and systemd timers
- Avoid duplicate apt data in package-specific roles.
* Sat Dec 27 2025 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Initial RPM packaging for Fedora 42

35
tests.sh
View file

@ -9,45 +9,16 @@ BUNDLE_DIR="/tmp/bundle"
ANSIBLE_DIR="/tmp/ansible"
rm -rf "${BUNDLE_DIR}" "${ANSIBLE_DIR}"
# Install something that has symlinks like apache2,
# to extend the manifests that will be linted later
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends apache2
# Generate data
poetry run \
enroll single-shot \
enroll enroll \
--harvest "${BUNDLE_DIR}" \
--out "${ANSIBLE_DIR}"
# Analyse
poetry run \
enroll explain "${BUNDLE_DIR}"
poetry run \
enroll explain "${BUNDLE_DIR}" --format json | jq
# Validate
poetry run \
enroll validate --fail-on-warnings "${BUNDLE_DIR}"
# Install/remove something, harvest again and diff the harvests
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends cowsay
poetry run \
enroll harvest --out "${BUNDLE_DIR}2"
# Validate
poetry run \
enroll validate --fail-on-warnings "${BUNDLE_DIR}2"
# Diff
poetry run \
enroll diff \
--old "${BUNDLE_DIR}" \
--new "${BUNDLE_DIR}2" \
--format json | jq
DEBIAN_FRONTEND=noninteractive apt-get remove -y --purge cowsay
# Ansible test
builtin cd "${ANSIBLE_DIR}"
# Lint
ansible-lint "${ANSIBLE_DIR}"
# Run
ansible-playbook playbook.yml -i "localhost," -c local --check --diff
sudo ansible-playbook playbook.yml -i "localhost," -c local --check --diff

18
tests/test___main__.py
View file

@ -1,18 +0,0 @@
from __future__ import annotations
import runpy
def test_module_main_invokes_cli_main(monkeypatch):
import enroll.cli
called = {"ok": False}
def fake_main() -> None:
called["ok"] = True
monkeypatch.setattr(enroll.cli, "main", fake_main)
# Execute enroll.__main__ as if `python -m enroll`.
runpy.run_module("enroll.__main__", run_name="__main__")
assert called["ok"] is True

143
tests/test_accounts.py
View file

@ -1,143 +0,0 @@
from __future__ import annotations
import os
from pathlib import Path
def test_parse_login_defs_parses_known_keys(tmp_path: Path):
from enroll.accounts import parse_login_defs
p = tmp_path / "login.defs"
p.write_text(
"""
# comment
UID_MIN 1000
UID_MAX 60000
SYS_UID_MIN 100
SYS_UID_MAX 999
UID_MIN not_an_int
OTHER 123
""",
encoding="utf-8",
)
vals = parse_login_defs(str(p))
assert vals["UID_MIN"] == 1000
assert vals["UID_MAX"] == 60000
assert vals["SYS_UID_MIN"] == 100
assert vals["SYS_UID_MAX"] == 999
assert "OTHER" not in vals
def test_parse_passwd_and_group_and_ssh_files(tmp_path: Path):
from enroll.accounts import find_user_ssh_files, parse_group, parse_passwd
passwd = tmp_path / "passwd"
passwd.write_text(
"\n".join(
[
"root:x:0:0:root:/root:/bin/bash",
"# comment",
"alice:x:1000:1000:Alice:/home/alice:/bin/bash",
"bob:x:1001:1000:Bob:/home/bob:/usr/sbin/nologin",
"badline",
"cathy:x:notint:1000:Cathy:/home/cathy:/bin/bash",
"",
]
),
encoding="utf-8",
)
group = tmp_path / "group"
group.write_text(
"\n".join(
[
"root:x:0:",
"users:x:1000:alice,bob",
"admins:x:1002:alice",
"badgroup:x:notint:alice",
"",
]
),
encoding="utf-8",
)
rows = parse_passwd(str(passwd))
assert ("alice", 1000, 1000, "Alice", "/home/alice", "/bin/bash") in rows
assert all(r[0] != "cathy" for r in rows) # skipped invalid UID
gid_to_name, name_to_gid, members = parse_group(str(group))
assert gid_to_name[1000] == "users"
assert name_to_gid["admins"] == 1002
assert "alice" in members["admins"]
# ssh discovery: only authorized_keys, no symlinks
home = tmp_path / "home" / "alice"
sshdir = home / ".ssh"
sshdir.mkdir(parents=True)
ak = sshdir / "authorized_keys"
ak.write_text("ssh-ed25519 AAA...", encoding="utf-8")
# a symlink should be ignored
(sshdir / "authorized_keys2").write_text("x", encoding="utf-8")
os.symlink(str(sshdir / "authorized_keys2"), str(sshdir / "authorized_keys_link"))
assert find_user_ssh_files(str(home)) == [str(ak)]
def test_collect_non_system_users(monkeypatch, tmp_path: Path):
import enroll.accounts as a
orig_parse_login_defs = a.parse_login_defs
orig_parse_passwd = a.parse_passwd
orig_parse_group = a.parse_group
# Provide controlled passwd/group/login.defs inputs via monkeypatch.
passwd = tmp_path / "passwd"
passwd.write_text(
"\n".join(
[
"root:x:0:0:root:/root:/bin/bash",
"nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin",
"alice:x:1000:1000:Alice:/home/alice:/bin/bash",
"sysuser:x:200:200:Sys:/home/sys:/bin/bash",
"bob:x:1001:1000:Bob:/home/bob:/bin/false",
"",
]
),
encoding="utf-8",
)
group = tmp_path / "group"
group.write_text(
"\n".join(
[
"users:x:1000:alice,bob",
"admins:x:1002:alice",
"",
]
),
encoding="utf-8",
)
defs = tmp_path / "login.defs"
defs.write_text("UID_MIN 1000\n", encoding="utf-8")
monkeypatch.setattr(
a, "parse_login_defs", lambda path=str(defs): orig_parse_login_defs(path)
)
monkeypatch.setattr(
a, "parse_passwd", lambda path=str(passwd): orig_parse_passwd(path)
)
monkeypatch.setattr(
a, "parse_group", lambda path=str(group): orig_parse_group(path)
)
# Use a stable fake ssh discovery.
monkeypatch.setattr(
a, "find_user_ssh_files", lambda home: [f"{home}/.ssh/authorized_keys"]
)
users = a.collect_non_system_users()
assert [u.name for u in users] == ["alice"]
u = users[0]
assert u.primary_group == "users"
assert u.supplementary_groups == ["admins"]
assert u.ssh_files == ["/home/alice/.ssh/authorized_keys"]

33
tests/test_cache_security.py
View file

@ -1,33 +0,0 @@
from __future__ import annotations
import os
from pathlib import Path
import pytest
def test_ensure_dir_secure_refuses_symlink(tmp_path: Path):
from enroll.cache import _ensure_dir_secure
target = tmp_path / "target"
target.mkdir()
link = tmp_path / "link"
link.symlink_to(target, target_is_directory=True)
with pytest.raises(RuntimeError):
_ensure_dir_secure(link)
def test_ensure_dir_secure_ignores_chmod_failures(tmp_path: Path, monkeypatch):
from enroll.cache import _ensure_dir_secure
d = tmp_path / "d"
def boom(_path: str, _mode: int):
raise OSError("no")
monkeypatch.setattr(os, "chmod", boom)
# Should not raise.
_ensure_dir_secure(d)
assert d.exists() and d.is_dir()

629
tests/test_cli.py
View file

@ -1,29 +1,13 @@
from __future__ import annotations
import sys
import pytest
import enroll.cli as cli
from pathlib import Path
from enroll.remote import RemoteSudoPasswordRequired
from enroll.sopsutil import SopsError
def test_cli_harvest_subcommand_calls_harvest(monkeypatch, capsys, tmp_path):
called = {}
def fake_harvest(
out: str,
dangerous: bool = False,
include_paths=None,
exclude_paths=None,
**_kwargs,
):
def fake_harvest(out: str):
called["out"] = out
called["dangerous"] = dangerous
called["include_paths"] = include_paths or []
called["exclude_paths"] = exclude_paths or []
return str(tmp_path / "state.json")
monkeypatch.setattr(cli, "harvest", fake_harvest)
@ -31,9 +15,6 @@ def test_cli_harvest_subcommand_calls_harvest(monkeypatch, capsys, tmp_path):
cli.main()
assert called["out"] == str(tmp_path)
assert called["dangerous"] is False
assert called["include_paths"] == []
assert called["exclude_paths"] == []
captured = capsys.readouterr()
assert str(tmp_path / "state.json") in captured.out
@ -41,12 +22,9 @@ def test_cli_harvest_subcommand_calls_harvest(monkeypatch, capsys, tmp_path):
def test_cli_manifest_subcommand_calls_manifest(monkeypatch, tmp_path):
called = {}
def fake_manifest(harvest_dir: str, out_dir: str, **kwargs):
def fake_manifest(harvest_dir: str, out_dir: str):
called["harvest"] = harvest_dir
called["out"] = out_dir
# Common manifest args should be passed through by the CLI.
called["fqdn"] = kwargs.get("fqdn")
called["jinjaturtle"] = kwargs.get("jinjaturtle")
monkeypatch.setattr(cli, "manifest", fake_manifest)
monkeypatch.setattr(
@ -65,35 +43,17 @@ def test_cli_manifest_subcommand_calls_manifest(monkeypatch, tmp_path):
cli.main()
assert called["harvest"] == str(tmp_path / "bundle")
assert called["out"] == str(tmp_path / "ansible")
assert called["fqdn"] is None
assert called["jinjaturtle"] == "auto"
def test_cli_enroll_subcommand_runs_harvest_then_manifest(monkeypatch, tmp_path):
calls = []
def fake_harvest(
bundle_dir: str,
dangerous: bool = False,
include_paths=None,
exclude_paths=None,
**_kwargs,
):
calls.append(
("harvest", bundle_dir, dangerous, include_paths or [], exclude_paths or [])
)
def fake_harvest(bundle_dir: str):
calls.append(("harvest", bundle_dir))
return str(tmp_path / "bundle" / "state.json")
def fake_manifest(bundle_dir: str, out_dir: str, **kwargs):
calls.append(
(
"manifest",
bundle_dir,
out_dir,
kwargs.get("fqdn"),
kwargs.get("jinjaturtle"),
)
)
def fake_manifest(bundle_dir: str, out_dir: str):
calls.append(("manifest", bundle_dir, out_dir))
monkeypatch.setattr(cli, "harvest", fake_harvest)
monkeypatch.setattr(cli, "manifest", fake_manifest)
@ -102,7 +62,7 @@ def test_cli_enroll_subcommand_runs_harvest_then_manifest(monkeypatch, tmp_path)
"argv",
[
"enroll",
"single-shot",
"enroll",
"--harvest",
str(tmp_path / "bundle"),
"--out",
@ -112,577 +72,6 @@ def test_cli_enroll_subcommand_runs_harvest_then_manifest(monkeypatch, tmp_path)
cli.main()
assert calls == [
("harvest", str(tmp_path / "bundle"), False, [], []),
("manifest", str(tmp_path / "bundle"), str(tmp_path / "ansible"), None, "auto"),
("harvest", str(tmp_path / "bundle")),
("manifest", str(tmp_path / "bundle"), str(tmp_path / "ansible")),
]
def test_cli_harvest_dangerous_flag_is_forwarded(monkeypatch, tmp_path):
called = {}
def fake_harvest(
out: str,
dangerous: bool = False,
include_paths=None,
exclude_paths=None,
**_kwargs,
):
called["out"] = out
called["dangerous"] = dangerous
called["include_paths"] = include_paths or []
called["exclude_paths"] = exclude_paths or []
return str(tmp_path / "state.json")
monkeypatch.setattr(cli, "harvest", fake_harvest)
monkeypatch.setattr(
sys, "argv", ["enroll", "harvest", "--out", str(tmp_path), "--dangerous"]
)
cli.main()
assert called["dangerous"] is True
assert called["include_paths"] == []
assert called["exclude_paths"] == []
def test_cli_harvest_remote_calls_remote_harvest_and_uses_cache_dir(
monkeypatch, capsys, tmp_path
):
from enroll.cache import HarvestCache
cache_dir = tmp_path / "cache"
cache_dir.mkdir()
called = {}
def fake_cache_dir(*, hint=None):
called["hint"] = hint
return HarvestCache(dir=cache_dir)
def fake_remote_harvest(
*,
local_out_dir,
remote_host,
remote_port,
remote_user,
dangerous,
no_sudo,
include_paths=None,
exclude_paths=None,
**_kwargs,
):
called.update(
{
"local_out_dir": local_out_dir,
"remote_host": remote_host,
"remote_port": remote_port,
"remote_user": remote_user,
"dangerous": dangerous,
"no_sudo": no_sudo,
"include_paths": include_paths or [],
"exclude_paths": exclude_paths or [],
}
)
return cache_dir / "state.json"
monkeypatch.setattr(cli, "new_harvest_cache_dir", fake_cache_dir)
monkeypatch.setattr(cli, "remote_harvest", fake_remote_harvest)
monkeypatch.setattr(
sys,
"argv",
[
"enroll",
"harvest",
"--remote-host",
"example.test",
"--remote-user",
"alice",
],
)
cli.main()
out = capsys.readouterr().out
assert str(cache_dir / "state.json") in out
assert called["hint"] == "example.test"
assert called["local_out_dir"] == cache_dir
assert called["remote_host"] == "example.test"
assert called["remote_port"] == 22
assert called["remote_user"] == "alice"
assert called["dangerous"] is False
assert called["no_sudo"] is False
assert called["include_paths"] == []
assert called["exclude_paths"] == []
def test_cli_single_shot_remote_without_harvest_prints_state_path(
monkeypatch, capsys, tmp_path
):
from enroll.cache import HarvestCache
cache_dir = tmp_path / "cache"
cache_dir.mkdir()
ansible_dir = tmp_path / "ansible"
calls = []
def fake_cache_dir(*, hint=None):
return HarvestCache(dir=cache_dir)
def fake_remote_harvest(**kwargs):
calls.append(("remote_harvest", kwargs))
return cache_dir / "state.json"
def fake_manifest(harvest_dir: str, out_dir: str, **kwargs):
calls.append(("manifest", harvest_dir, out_dir, kwargs.get("fqdn")))
monkeypatch.setattr(cli, "new_harvest_cache_dir", fake_cache_dir)
monkeypatch.setattr(cli, "remote_harvest", fake_remote_harvest)
monkeypatch.setattr(cli, "manifest", fake_manifest)
monkeypatch.setattr(
sys,
"argv",
[
"enroll",
"single-shot",
"--remote-host",
"example.test",
"--remote-user",
"alice",
"--out",
str(ansible_dir),
"--fqdn",
"example.test",
],
)
cli.main()
out = capsys.readouterr().out
# It should print the derived state.json path for usability when --harvest
# wasn't provided.
assert str(cache_dir / "state.json") in out
# And it should manifest using the cache dir.
assert ("manifest", str(cache_dir), str(ansible_dir), "example.test") in calls
def test_cli_harvest_remote_ask_become_pass_prompts_and_passes_password(
monkeypatch, tmp_path
):
from enroll.cache import HarvestCache
import enroll.remote as r
cache_dir = tmp_path / "cache"
cache_dir.mkdir()
called = {}
def fake_cache_dir(*, hint=None):
return HarvestCache(dir=cache_dir)
def fake__remote_harvest(*, sudo_password=None, **kwargs):
called["sudo_password"] = sudo_password
return cache_dir / "state.json"
monkeypatch.setattr(cli, "new_harvest_cache_dir", fake_cache_dir)
monkeypatch.setattr(r, "_remote_harvest", fake__remote_harvest)
monkeypatch.setattr(r.getpass, "getpass", lambda _prompt="": "pw123")
monkeypatch.setattr(
sys,
"argv",
[
"enroll",
"harvest",
"--remote-host",
"example.test",
"--ask-become-pass",
],
)
cli.main()
assert called["sudo_password"] == "pw123"
def test_cli_harvest_remote_password_required_fallback_prompts_and_retries(
monkeypatch, tmp_path
):
from enroll.cache import HarvestCache
import enroll.remote as r
cache_dir = tmp_path / "cache"
cache_dir.mkdir()
def fake_cache_dir(*, hint=None):
return HarvestCache(dir=cache_dir)
calls = []
def fake__remote_harvest(*, sudo_password=None, **kwargs):
calls.append(sudo_password)
if sudo_password is None:
raise r.RemoteSudoPasswordRequired("pw required")
return cache_dir / "state.json"
class _TTYStdin:
def isatty(self):
return True
monkeypatch.setattr(cli, "new_harvest_cache_dir", fake_cache_dir)
monkeypatch.setattr(r, "_remote_harvest", fake__remote_harvest)
monkeypatch.setattr(r.getpass, "getpass", lambda _prompt="": "pw456")
monkeypatch.setattr(sys, "stdin", _TTYStdin())
monkeypatch.setattr(
sys, "argv", ["enroll", "harvest", "--remote-host", "example.test"]
)
cli.main()
assert calls == [None, "pw456"]
def test_cli_harvest_remote_password_required_noninteractive_errors(
monkeypatch, tmp_path
):
from enroll.cache import HarvestCache
import enroll.remote as r
cache_dir = tmp_path / "cache"
cache_dir.mkdir()
def fake_cache_dir(*, hint=None):
return HarvestCache(dir=cache_dir)
def fake__remote_harvest(*, sudo_password=None, **kwargs):
raise r.RemoteSudoPasswordRequired("pw required")
class _NoTTYStdin:
def isatty(self):
return False
monkeypatch.setattr(cli, "new_harvest_cache_dir", fake_cache_dir)
monkeypatch.setattr(r, "_remote_harvest", fake__remote_harvest)
monkeypatch.setattr(sys, "stdin", _NoTTYStdin())
monkeypatch.setattr(
sys, "argv", ["enroll", "harvest", "--remote-host", "example.test"]
)
with pytest.raises(SystemExit) as e:
cli.main()
assert "--ask-become-pass" in str(e.value)
def test_cli_manifest_common_args(monkeypatch, tmp_path):
"""Ensure --fqdn and jinjaturtle mode flags are forwarded correctly."""
called = {}
def fake_manifest(harvest_dir: str, out_dir: str, **kwargs):
called["harvest"] = harvest_dir
called["out"] = out_dir
called["fqdn"] = kwargs.get("fqdn")
called["jinjaturtle"] = kwargs.get("jinjaturtle")
monkeypatch.setattr(cli, "manifest", fake_manifest)
monkeypatch.setattr(
sys,
"argv",
[
"enroll",
"manifest",
"--harvest",
str(tmp_path / "bundle"),
"--out",
str(tmp_path / "ansible"),
"--fqdn",
"example.test",
"--no-jinjaturtle",
],
)
cli.main()
assert called["fqdn"] == "example.test"
assert called["jinjaturtle"] == "off"
def test_cli_explain_passes_args_and_writes_stdout(monkeypatch, capsys, tmp_path):
called = {}
def fake_explain_state(
harvest: str,
*,
sops_mode: bool = False,
fmt: str = "text",
max_examples: int = 3,
):
called["harvest"] = harvest
called["sops_mode"] = sops_mode
called["fmt"] = fmt
called["max_examples"] = max_examples
return "EXPLAINED\n"
monkeypatch.setattr(cli, "explain_state", fake_explain_state)
monkeypatch.setattr(
sys,
"argv",
[
"enroll",
"explain",
"--sops",
"--format",
"json",
"--max-examples",
"7",
str(tmp_path / "bundle" / "state.json"),
],
)
cli.main()
out = capsys.readouterr().out
assert out == "EXPLAINED\n"
assert called["sops_mode"] is True
assert called["fmt"] == "json"
assert called["max_examples"] == 7
def test_discover_config_path_missing_config_value_returns_none(monkeypatch):
# Covers the "--config" flag present with no value.
monkeypatch.delenv("ENROLL_CONFIG", raising=False)
monkeypatch.delenv("XDG_CONFIG_HOME", raising=False)
assert cli._discover_config_path(["--config"]) is None
def test_discover_config_path_defaults_to_home_config(monkeypatch, tmp_path: Path):
# Covers the Path.home() / ".config" fallback.
monkeypatch.delenv("ENROLL_CONFIG", raising=False)
monkeypatch.delenv("XDG_CONFIG_HOME", raising=False)
monkeypatch.setattr(cli.Path, "home", lambda: tmp_path)
monkeypatch.setattr(cli.Path, "cwd", lambda: tmp_path)
cp = tmp_path / ".config" / "enroll" / "enroll.ini"
cp.parent.mkdir(parents=True)
cp.write_text("[enroll]\n", encoding="utf-8")
assert cli._discover_config_path(["harvest"]) == cp
def test_cli_harvest_local_sops_encrypts_and_prints_path(
monkeypatch, tmp_path: Path, capsys
):
out_dir = tmp_path / "out"
out_dir.mkdir()
calls: dict[str, object] = {}
def fake_harvest(bundle_dir: str, **kwargs):
calls["bundle"] = bundle_dir
# Create a minimal state.json so tooling that expects it won't break.
Path(bundle_dir).mkdir(parents=True, exist_ok=True)
(Path(bundle_dir) / "state.json").write_text("{}", encoding="utf-8")
return str(Path(bundle_dir) / "state.json")
def fake_encrypt(bundle_dir: Path, out_file: Path, fps: list[str]):
calls["encrypt"] = (bundle_dir, out_file, fps)
out_file.write_text("encrypted", encoding="utf-8")
return out_file
monkeypatch.setattr(cli, "harvest", fake_harvest)
monkeypatch.setattr(cli, "_encrypt_harvest_dir_to_sops", fake_encrypt)
monkeypatch.setattr(
sys,
"argv",
[
"enroll",
"harvest",
"--sops",
"ABCDEF",
"--out",
str(out_dir),
],
)
cli.main()
printed = capsys.readouterr().out.strip()
assert printed.endswith("harvest.tar.gz.sops")
assert Path(printed).exists()
assert calls.get("encrypt")
def test_cli_harvest_remote_sops_encrypts_and_prints_path(
monkeypatch, tmp_path: Path, capsys
):
out_dir = tmp_path / "out"
out_dir.mkdir()
calls: dict[str, object] = {}
def fake_remote_harvest(**kwargs):
calls["remote"] = kwargs
# Create a minimal state.json in the temp bundle.
out = Path(kwargs["local_out_dir"]) / "state.json"
out.write_text("{}", encoding="utf-8")
return out
def fake_encrypt(bundle_dir: Path, out_file: Path, fps: list[str]):
calls["encrypt"] = (bundle_dir, out_file, fps)
out_file.write_text("encrypted", encoding="utf-8")
return out_file
monkeypatch.setattr(cli, "remote_harvest", fake_remote_harvest)
monkeypatch.setattr(cli, "_encrypt_harvest_dir_to_sops", fake_encrypt)
monkeypatch.setattr(
sys,
"argv",
[
"enroll",
"harvest",
"--remote-host",
"example.com",
"--remote-user",
"root",
"--sops",
"ABCDEF",
"--out",
str(out_dir),
],
)
cli.main()
printed = capsys.readouterr().out.strip()
assert printed.endswith("harvest.tar.gz.sops")
assert Path(printed).exists()
assert calls.get("remote")
assert calls.get("encrypt")
def test_cli_harvest_remote_password_required_exits_cleanly(monkeypatch):
def boom(**kwargs):
raise RemoteSudoPasswordRequired("pw required")
monkeypatch.setattr(cli, "remote_harvest", boom)
monkeypatch.setattr(
sys,
"argv",
[
"enroll",
"harvest",
"--remote-host",
"example.com",
"--remote-user",
"root",
],
)
with pytest.raises(SystemExit) as e:
cli.main()
assert "--ask-become-pass" in str(e.value)
def test_cli_runtime_error_is_wrapped_as_user_friendly_system_exit(monkeypatch):
def boom(*args, **kwargs):
raise RuntimeError("nope")
monkeypatch.setattr(cli, "harvest", boom)
monkeypatch.setattr(sys, "argv", ["enroll", "harvest", "--out", "/tmp/x"])
with pytest.raises(SystemExit) as e:
cli.main()
assert str(e.value) == "error: nope"
def test_cli_sops_error_is_wrapped_as_user_friendly_system_exit(monkeypatch):
def boom(*args, **kwargs):
raise SopsError("sops broke")
monkeypatch.setattr(cli, "manifest", boom)
monkeypatch.setattr(
sys, "argv", ["enroll", "manifest", "--harvest", "/tmp/x", "--out", "/tmp/y"]
)
with pytest.raises(SystemExit) as e:
cli.main()
assert str(e.value) == "error: sops broke"
def test_cli_diff_notifies_webhook_and_email_and_respects_exit_code(
monkeypatch, capsys
):
calls: dict[str, object] = {}
def fake_compare(old, new, sops_mode=False, **kwargs):
calls["compare"] = (old, new, sops_mode)
return {"dummy": True}, True
def fake_format(report, fmt="text"):
calls.setdefault("format", []).append((report, fmt))
return "REPORT\n"
def fake_post(url, body, headers=None):
calls["webhook"] = (url, body, headers)
return 200, b"ok"
def fake_email(**kwargs):
calls["email"] = kwargs
monkeypatch.setattr(cli, "compare_harvests", fake_compare)
monkeypatch.setattr(cli, "format_report", fake_format)
monkeypatch.setattr(cli, "post_webhook", fake_post)
monkeypatch.setattr(cli, "send_email", fake_email)
monkeypatch.setenv("SMTPPW", "secret")
monkeypatch.setattr(
sys,
"argv",
[
"enroll",
"diff",
"--old",
"/tmp/old",
"--new",
"/tmp/new",
"--webhook",
"https://example.invalid/h",
"--webhook-header",
"X-Test: ok",
"--email-to",
"a@example.com",
"--smtp-password-env",
"SMTPPW",
"--exit-code",
],
)
with pytest.raises(SystemExit) as e:
cli.main()
assert e.value.code == 2
assert calls.get("compare")
assert calls.get("webhook")
assert calls.get("email")
# No report printed when exiting via --exit-code? (we still render and print).
_ = capsys.readouterr()
def test_cli_diff_webhook_http_error_raises_system_exit(monkeypatch):
def fake_compare(old, new, sops_mode=False, **kwargs):
return {"dummy": True}, True
monkeypatch.setattr(cli, "compare_harvests", fake_compare)
monkeypatch.setattr(cli, "format_report", lambda report, fmt="text": "R\n")
monkeypatch.setattr(cli, "post_webhook", lambda url, body, headers=None: (500, b""))
monkeypatch.setattr(
sys,
"argv",
[
"enroll",
"diff",
"--old",
"/tmp/old",
"--new",
"/tmp/new",
"--webhook",
"https://example.invalid/h",
],
)
with pytest.raises(SystemExit) as e:
cli.main()
assert "HTTP 500" in str(e.value)

189
tests/test_cli_config_and_sops.py
View file

@ -1,189 +0,0 @@
from __future__ import annotations
import argparse
import configparser
import tarfile
from pathlib import Path
def test_discover_config_path_precedence(monkeypatch, tmp_path: Path):
from enroll.cli import _discover_config_path
cfg = tmp_path / "cfg.ini"
cfg.write_text("[enroll]\n", encoding="utf-8")
# --no-config always wins
monkeypatch.setenv("ENROLL_CONFIG", str(cfg))
assert _discover_config_path(["--no-config", "harvest"]) is None
# explicit --config wins
assert _discover_config_path(["--config", str(cfg), "harvest"]) == cfg
# env var used when present
assert _discover_config_path(["harvest"]) == cfg
def test_discover_config_path_finds_local_and_xdg(monkeypatch, tmp_path: Path):
from enroll.cli import _discover_config_path
# local file in cwd
cwd = tmp_path / "cwd"
cwd.mkdir()
local = cwd / "enroll.ini"
local.write_text("[enroll]\n", encoding="utf-8")
monkeypatch.chdir(cwd)
monkeypatch.delenv("ENROLL_CONFIG", raising=False)
monkeypatch.delenv("XDG_CONFIG_HOME", raising=False)
assert _discover_config_path(["harvest"]) == local
# xdg config fallback
monkeypatch.chdir(tmp_path)
xdg = tmp_path / "xdg"
(xdg / "enroll").mkdir(parents=True)
xcfg = xdg / "enroll" / "enroll.ini"
xcfg.write_text("[enroll]\n", encoding="utf-8")
monkeypatch.setenv("XDG_CONFIG_HOME", str(xdg))
assert _discover_config_path(["harvest"]) == xcfg
def test_section_to_argv_supports_bool_append_count_and_unknown(monkeypatch, capsys):
from enroll.cli import _section_to_argv
ap = argparse.ArgumentParser(add_help=False)
ap.add_argument("--flag", action="store_true")
ap.add_argument("--no-flag", action="store_false", dest="flag2")
ap.add_argument("--item", action="append", default=[])
ap.add_argument("-v", action="count", default=0)
cfg = configparser.ConfigParser()
cfg.read_dict(
{
"enroll": {
"flag": "true",
"no_flag": "false",
"item": "a,b",
"v": "2",
"unknown_key": "zzz",
}
}
)
argv = _section_to_argv(ap, cfg, "enroll")
# bools set
assert "--flag" in argv
assert "--no-flag" in argv
# append expanded
assert argv.count("--item") == 2
assert "a" in argv and "b" in argv
# count flag expanded
assert argv.count("-v") == 2
# unknown key prints warning
err = capsys.readouterr().err
assert "unknown option" in err
def test_inject_config_argv_inserts_global_and_command_tokens(tmp_path: Path):
from enroll.cli import _inject_config_argv
root = argparse.ArgumentParser(add_help=False)
root.add_argument("--root-flag", action="store_true")
sub = root.add_subparsers(dest="cmd", required=True)
p_h = sub.add_parser("harvest", add_help=False)
p_h.add_argument("--dangerous", action="store_true")
p_h.add_argument("--include-path", action="append", default=[])
cfg_path = tmp_path / "enroll.ini"
cfg_path.write_text(
"""[enroll]
root-flag = true
[harvest]
dangerous = true
include-path = /etc/one,/etc/two
""",
encoding="utf-8",
)
argv = ["harvest", "--include-path", "/etc/cli"]
injected = _inject_config_argv(
argv,
cfg_path=cfg_path,
root_parser=root,
subparsers={"harvest": p_h},
)
# global inserted before cmd, subcommand tokens right after cmd
assert injected[:2] == ["--root-flag", "harvest"]
# include-path from config inserted before CLI include-path (CLI wins later if duplicates)
joined = " ".join(injected)
assert "--include-path /etc/one" in joined
assert "--include-path /etc/cli" in joined
def test_resolve_sops_out_file_and_encrypt_path(monkeypatch, tmp_path: Path):
from enroll import cli
# directory output should yield harvest.tar.gz.sops inside
out_dir = tmp_path / "o"
out_dir.mkdir()
assert (
cli._resolve_sops_out_file(str(out_dir), hint="h").name == "harvest.tar.gz.sops"
)
# file-like output retained
out_file = tmp_path / "x.sops"
assert cli._resolve_sops_out_file(str(out_file), hint="h") == out_file
# None uses cache dir
class HC:
def __init__(self, d: Path):
self.dir = d
monkeypatch.setattr(
cli, "new_harvest_cache_dir", lambda hint: HC(tmp_path / "cache")
)
p = cli._resolve_sops_out_file(None, hint="h")
assert str(p).endswith("harvest.tar.gz.sops")
# Cover _tar_dir_to quickly (writes a tarball)
bundle = tmp_path / "bundle"
bundle.mkdir()
(bundle / "state.json").write_text("{}", encoding="utf-8")
tar_path = tmp_path / "b.tar.gz"
cli._tar_dir_to(bundle, tar_path)
assert tar_path.exists()
with tarfile.open(tar_path, "r:gz") as tf:
names = tf.getnames()
assert "state.json" in names or "./state.json" in names
def test_encrypt_harvest_dir_to_sops_cleans_up_tmp_tgz(monkeypatch, tmp_path: Path):
from enroll.cli import _encrypt_harvest_dir_to_sops
bundle = tmp_path / "bundle"
bundle.mkdir()
(bundle / "state.json").write_text("{}", encoding="utf-8")
out_file = tmp_path / "out.sops"
seen = {}
def fake_encrypt(src: Path, dst: Path, pgp_fingerprints, mode): # noqa: ARG001
# write something so we can see output created
seen["src"] = src
dst.write_bytes(b"enc")
monkeypatch.setattr("enroll.cli.encrypt_file_binary", fake_encrypt)
# Make os.unlink raise FileNotFoundError to hit the except branch in finally.
monkeypatch.setattr(
"enroll.cli.os.unlink", lambda p: (_ for _ in ()).throw(FileNotFoundError())
)
res = _encrypt_harvest_dir_to_sops(bundle, out_file, fps=["ABC"])
assert res == out_file
assert out_file.read_bytes() == b"enc"

177
tests/test_cli_helpers.py
View file

@ -1,177 +0,0 @@
from __future__ import annotations
import argparse
import configparser
import types
import textwrap
from pathlib import Path
def test_discover_config_path_precedence(tmp_path: Path, monkeypatch):
"""_discover_config_path: --config > ENROLL_CONFIG > ./enroll.ini > XDG."""
from enroll.cli import _discover_config_path
cfg1 = tmp_path / "one.ini"
cfg1.write_text("[enroll]\n", encoding="utf-8")
# Explicit --config should win.
assert _discover_config_path(["--config", str(cfg1)]) == cfg1
# --no-config disables config loading.
assert _discover_config_path(["--no-config", "--config", str(cfg1)]) is None
monkeypatch.chdir(tmp_path)
cfg2 = tmp_path / "two.ini"
cfg2.write_text("[enroll]\n", encoding="utf-8")
monkeypatch.setenv("ENROLL_CONFIG", str(cfg2))
assert _discover_config_path([]) == cfg2
# Local ./enroll.ini fallback.
monkeypatch.delenv("ENROLL_CONFIG", raising=False)
local = tmp_path / "enroll.ini"
local.write_text("[enroll]\n", encoding="utf-8")
assert _discover_config_path([]) == local
# XDG fallback.
local.unlink()
xdg = tmp_path / "xdg"
cfg3 = xdg / "enroll" / "enroll.ini"
cfg3.parent.mkdir(parents=True)
cfg3.write_text("[enroll]\n", encoding="utf-8")
monkeypatch.setenv("XDG_CONFIG_HOME", str(xdg))
assert _discover_config_path([]) == cfg3
def test_config_value_parsing_and_list_splitting():
from enroll.cli import _parse_bool, _split_list_value
assert _parse_bool("1") is True
assert _parse_bool("yes") is True
assert _parse_bool("false") is False
assert _parse_bool("maybe") is None
assert _split_list_value("a,b , c") == ["a", "b", "c"]
# When newlines are present, we split on lines (not commas within a line).
assert _split_list_value("a,b\nc") == ["a,b", "c"]
assert _split_list_value("a\n\n b\n") == ["a", "b"]
assert _split_list_value(" ") == []
def test_section_to_argv_handles_types_and_unknown_keys(capsys):
from enroll.cli import _section_to_argv
p = argparse.ArgumentParser(add_help=False)
p.add_argument("--dangerous", action="store_true")
p.add_argument("--no-color", dest="color", action="store_false")
p.add_argument("--include-path", dest="include_path", action="append")
p.add_argument("-v", action="count", default=0)
p.add_argument("--out")
cfg = configparser.ConfigParser()
cfg.read_dict(
{
"harvest": {
"dangerous": "true",
# Keys are matched by argparse dest; store_false actions still use dest.
"color": "false",
"include-path": "a,b,c",
"v": "2",
"out": "/tmp/bundle",
"unknown": "ignored",
}
}
)
argv = _section_to_argv(p, cfg, "harvest")
# Boolean store_true.
assert "--dangerous" in argv
# Boolean store_false: include the flag only when config wants False.
assert "--no-color" in argv
# Append: split lists and add one flag per item.
assert argv.count("--include-path") == 3
assert "a" in argv and "b" in argv and "c" in argv
# Count: repeats.
assert argv.count("-v") == 2
# Scalar.
assert "--out" in argv and "/tmp/bundle" in argv
err = capsys.readouterr().err
assert "unknown option" in err
def test_inject_config_argv_inserts_global_and_subcommand(tmp_path: Path, capsys):
from enroll.cli import _inject_config_argv
cfg = tmp_path / "enroll.ini"
cfg.write_text(
textwrap.dedent(
"""
[enroll]
dangerous = true
[harvest]
include-path = /etc/foo
unknown = 1
"""
).strip()
+ "\n",
encoding="utf-8",
)
root = argparse.ArgumentParser(add_help=False)
root.add_argument("--dangerous", action="store_true")
harvest_p = argparse.ArgumentParser(add_help=False)
harvest_p.add_argument("--include-path", dest="include_path", action="append")
argv = _inject_config_argv(
["harvest", "--out", "x"],
cfg_path=cfg,
root_parser=root,
subparsers={"harvest": harvest_p},
)
# Global tokens should appear before the subcommand.
assert argv[0] == "--dangerous"
assert argv[1] == "harvest"
# Subcommand tokens should appear right after the subcommand.
assert argv[2:4] == ["--include-path", "/etc/foo"]
# Unknown option should have produced a warning.
assert "unknown option" in capsys.readouterr().err
def test_resolve_sops_out_file(tmp_path: Path, monkeypatch):
from enroll import cli
# Make a predictable cache dir for the default case.
fake_cache = types.SimpleNamespace(dir=tmp_path / "cache")
fake_cache.dir.mkdir(parents=True)
monkeypatch.setattr(cli, "new_harvest_cache_dir", lambda hint=None: fake_cache)
# If out is a directory, use it directly.
out_dir = tmp_path / "out"
out_dir.mkdir()
# The output filename is fixed; hint is only used when creating a cache dir.
assert (
cli._resolve_sops_out_file(out=out_dir, hint="bundle.tar.gz")
== out_dir / "harvest.tar.gz.sops"
)
# If out is a file path, keep it.
out_file = tmp_path / "x.sops"
assert cli._resolve_sops_out_file(out=out_file, hint="bundle.tar.gz") == out_file
# None uses the cache dir, and the name is fixed.
assert (
cli._resolve_sops_out_file(out=None, hint="bundle.tar.gz")
== fake_cache.dir / "harvest.tar.gz.sops"
)

98
tests/test_debian.py
View file

@ -1,98 +0,0 @@
from __future__ import annotations
from pathlib import Path
def test_dpkg_owner_parses_output(monkeypatch):
import enroll.debian as d
class P:
def __init__(self, rc: int, out: str):
self.returncode = rc
self.stdout = out
self.stderr = ""
def fake_run(cmd, text, capture_output):
assert cmd[:2] == ["dpkg", "-S"]
return P(
0,
"""
diversion by foo from: /etc/something
nginx-common:amd64: /etc/nginx/nginx.conf
nginx-common, nginx: /etc/nginx/sites-enabled/default
""",
)
monkeypatch.setattr(d.subprocess, "run", fake_run)
assert d.dpkg_owner("/etc/nginx/nginx.conf") == "nginx-common"
def fake_run_none(cmd, text, capture_output):
return P(1, "")
monkeypatch.setattr(d.subprocess, "run", fake_run_none)
assert d.dpkg_owner("/missing") is None
def test_list_manual_packages_parses_and_sorts(monkeypatch):
import enroll.debian as d
class P:
def __init__(self, rc: int, out: str):
self.returncode = rc
self.stdout = out
self.stderr = ""
def fake_run(cmd, text, capture_output):
assert cmd == ["apt-mark", "showmanual"]
return P(0, "\n# comment\nnginx\nvim\nnginx\n")
monkeypatch.setattr(d.subprocess, "run", fake_run)
assert d.list_manual_packages() == ["nginx", "vim"]
def test_build_dpkg_etc_index(tmp_path: Path):
import enroll.debian as d
info = tmp_path / "info"
info.mkdir()
(info / "nginx.list").write_text(
"/etc/nginx/nginx.conf\n/etc/nginx/sites-enabled/default\n/usr/bin/nginx\n",
encoding="utf-8",
)
(info / "vim:amd64.list").write_text(
"/etc/vim/vimrc\n/usr/bin/vim\n",
encoding="utf-8",
)
owned, owner_map, topdir_to_pkgs, pkg_to_etc = d.build_dpkg_etc_index(str(info))
assert "/etc/nginx/nginx.conf" in owned
assert owner_map["/etc/nginx/nginx.conf"] == "nginx"
assert "nginx" in topdir_to_pkgs
assert topdir_to_pkgs["nginx"] == {"nginx"}
assert pkg_to_etc["vim"] == ["/etc/vim/vimrc"]
def test_parse_status_conffiles_handles_continuations(tmp_path: Path):
import enroll.debian as d
status = tmp_path / "status"
status.write_text(
"\n".join(
[
"Package: nginx",
"Version: 1",
"Conffiles:",
" /etc/nginx/nginx.conf abcdef",
" /etc/nginx/mime.types 123456",
"",
"Package: other",
"Version: 2",
"",
]
),
encoding="utf-8",
)
m = d.parse_status_conffiles(str(status))
assert m["nginx"]["/etc/nginx/nginx.conf"] == "abcdef"
assert m["nginx"]["/etc/nginx/mime.types"] == "123456"
assert "other" not in m

89
tests/test_diff_bundle.py
View file

@ -1,89 +0,0 @@
from __future__ import annotations
import os
import tarfile
from pathlib import Path
import pytest
def _make_bundle_dir(tmp_path: Path) -> Path:
b = tmp_path / "bundle"
(b / "artifacts").mkdir(parents=True)
(b / "state.json").write_text("{}\n", encoding="utf-8")
return b
def _tar_gz_of_dir(src: Path, out: Path) -> None:
with tarfile.open(out, mode="w:gz") as tf:
# tar -C src . semantics
for p in src.rglob("*"):
rel = p.relative_to(src)
tf.add(p, arcname=str(rel))
def test_bundle_from_directory_and_statejson_path(tmp_path: Path):
import enroll.diff as d
b = _make_bundle_dir(tmp_path)
br1 = d._bundle_from_input(str(b), sops_mode=False)
assert br1.dir == b
assert br1.state_path.exists()
br2 = d._bundle_from_input(str(b / "state.json"), sops_mode=False)
assert br2.dir == b
def test_bundle_from_tarball_extracts(tmp_path: Path):
import enroll.diff as d
b = _make_bundle_dir(tmp_path)
tgz = tmp_path / "bundle.tgz"
_tar_gz_of_dir(b, tgz)
br = d._bundle_from_input(str(tgz), sops_mode=False)
try:
assert br.dir.is_dir()
assert (br.dir / "state.json").exists()
finally:
if br.tempdir:
br.tempdir.cleanup()
def test_bundle_from_sops_like_file(monkeypatch, tmp_path: Path):
import enroll.diff as d
b = _make_bundle_dir(tmp_path)
tgz = tmp_path / "bundle.tar.gz"
_tar_gz_of_dir(b, tgz)
# Pretend the tarball is an encrypted bundle by giving it a .sops name.
sops_path = tmp_path / "bundle.tar.gz.sops"
sops_path.write_bytes(tgz.read_bytes())
# Stub out sops machinery: "decrypt" just copies through.
monkeypatch.setattr(d, "require_sops_cmd", lambda: "sops")
def fake_decrypt(src: Path, dest: Path, mode: int):
dest.write_bytes(Path(src).read_bytes())
try:
os.chmod(dest, mode)
except OSError:
pass
monkeypatch.setattr(d, "decrypt_file_binary_to", fake_decrypt)
br = d._bundle_from_input(str(sops_path), sops_mode=False)
try:
assert (br.dir / "state.json").exists()
finally:
if br.tempdir:
br.tempdir.cleanup()
def test_bundle_from_input_missing_path(tmp_path: Path):
import enroll.diff as d
with pytest.raises(RuntimeError, match="not found"):
d._bundle_from_input(str(tmp_path / "nope"), sops_mode=False)

400
tests/test_diff_ignore_versions_exclude_enforce.py
View file

@ -1,400 +0,0 @@
from __future__ import annotations
import json
import sys
import types
from pathlib import Path
import pytest
def _write_bundle(
root: Path, state: dict, artifacts: dict[str, bytes] | None = None
) -> None:
root.mkdir(parents=True, exist_ok=True)
(root / "state.json").write_text(json.dumps(state, indent=2), encoding="utf-8")
artifacts = artifacts or {}
for rel, data in artifacts.items():
p = root / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_bytes(data)
def _minimal_roles() -> dict:
"""A small roles structure that's sufficient for enroll.diff file indexing."""
return {
"users": {
"role_name": "users",
"users": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
"services": [],
"packages": [],
"apt_config": {
"role_name": "apt_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"etc_custom": {
"role_name": "etc_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"usr_local_custom": {
"role_name": "usr_local_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"extra_paths": {
"role_name": "extra_paths",
"include_patterns": [],
"exclude_patterns": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
}
def test_diff_ignore_package_versions_suppresses_version_drift(tmp_path: Path):
from enroll.diff import compare_harvests
old = tmp_path / "old"
new = tmp_path / "new"
old_state = {
"schema_version": 3,
"host": {"hostname": "h1"},
"inventory": {
"packages": {
"curl": {
"version": "1.0",
"installations": [{"version": "1.0", "arch": "amd64"}],
}
}
},
"roles": _minimal_roles(),
}
new_state = {
**old_state,
"inventory": {
"packages": {
"curl": {
"version": "1.1",
"installations": [{"version": "1.1", "arch": "amd64"}],
}
}
},
}
_write_bundle(old, old_state)
_write_bundle(new, new_state)
# Without ignore flag, version drift is reported and counts as changes.
report, has_changes = compare_harvests(str(old), str(new))
assert has_changes is True
assert report["packages"]["version_changed"]
# With ignore flag, version drift is suppressed and does not count as changes.
report2, has_changes2 = compare_harvests(
str(old), str(new), ignore_package_versions=True
)
assert has_changes2 is False
assert report2["packages"]["version_changed"] == []
assert report2["packages"]["version_changed_ignored_count"] == 1
assert report2["filters"]["ignore_package_versions"] is True
def test_diff_exclude_path_filters_file_drift_and_affects_has_changes(tmp_path: Path):
from enroll.diff import compare_harvests
old = tmp_path / "old"
new = tmp_path / "new"
# Only file drift is under /var/anacron, which is excluded.
old_state = {
"schema_version": 3,
"host": {"hostname": "h1"},
"inventory": {"packages": {}},
"roles": {
**_minimal_roles(),
"extra_paths": {
**_minimal_roles()["extra_paths"],
"managed_files": [
{
"path": "/var/anacron/daily.stamp",
"src_rel": "var/anacron/daily.stamp",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "extra_path",
}
],
},
},
}
new_state = json.loads(json.dumps(old_state))
_write_bundle(
old,
old_state,
{"artifacts/extra_paths/var/anacron/daily.stamp": b"yesterday\n"},
)
_write_bundle(
new,
new_state,
{"artifacts/extra_paths/var/anacron/daily.stamp": b"today\n"},
)
report, has_changes = compare_harvests(
str(old), str(new), exclude_paths=["/var/anacron"]
)
assert has_changes is False
assert report["files"]["changed"] == []
assert report["filters"]["exclude_paths"] == ["/var/anacron"]
def test_diff_exclude_path_only_filters_files_not_packages(tmp_path: Path):
from enroll.diff import compare_harvests
old = tmp_path / "old"
new = tmp_path / "new"
old_state = {
"schema_version": 3,
"host": {"hostname": "h1"},
"inventory": {"packages": {"curl": {"version": "1.0"}}},
"roles": {
**_minimal_roles(),
"extra_paths": {
**_minimal_roles()["extra_paths"],
"managed_files": [
{
"path": "/var/anacron/daily.stamp",
"src_rel": "var/anacron/daily.stamp",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "extra_path",
}
],
},
},
}
new_state = {
**old_state,
"inventory": {
"packages": {
"curl": {"version": "1.0"},
"htop": {"version": "3.0"},
}
},
}
_write_bundle(
old,
old_state,
{"artifacts/extra_paths/var/anacron/daily.stamp": b"yesterday\n"},
)
_write_bundle(
new,
new_state,
{
"artifacts/extra_paths/var/anacron/daily.stamp": b"today\n",
},
)
report, has_changes = compare_harvests(
str(old), str(new), exclude_paths=["/var/anacron"]
)
assert has_changes is True
# File drift is filtered, but package drift remains.
assert report["files"]["changed"] == []
assert report["packages"]["added"] == ["htop"]
def test_enforce_old_harvest_requires_ansible_playbook(monkeypatch, tmp_path: Path):
import enroll.diff as d
monkeypatch.setattr(d.shutil, "which", lambda name: None)
old = tmp_path / "old"
_write_bundle(old, {"inventory": {"packages": {}}, "roles": _minimal_roles()})
with pytest.raises(RuntimeError, match="ansible-playbook not found"):
d.enforce_old_harvest(str(old))
def test_enforce_old_harvest_runs_ansible_with_tags_from_file_drift(
monkeypatch, tmp_path: Path
):
import enroll.diff as d
import enroll.manifest as mf
# Pretend ansible-playbook is installed.
monkeypatch.setattr(d.shutil, "which", lambda name: "/usr/bin/ansible-playbook")
calls: dict[str, object] = {}
# Stub manifest generation to only create playbook.yml (fast, no real roles needed).
def fake_manifest(_harvest_dir: str, out_dir: str, **_kwargs):
out = Path(out_dir)
(out / "playbook.yml").write_text(
"---\n- hosts: all\n gather_facts: false\n roles: []\n",
encoding="utf-8",
)
monkeypatch.setattr(mf, "manifest", fake_manifest)
def fake_run(
argv, cwd=None, env=None, capture_output=False, text=False, check=False
):
calls["argv"] = list(argv)
calls["cwd"] = cwd
return types.SimpleNamespace(returncode=0, stdout="ok", stderr="")
monkeypatch.setattr(d.subprocess, "run", fake_run)
old = tmp_path / "old"
old_state = {
"schema_version": 3,
"host": {"hostname": "h1"},
"inventory": {"packages": {}},
"roles": {
**_minimal_roles(),
"usr_local_custom": {
**_minimal_roles()["usr_local_custom"],
"managed_files": [
{
"path": "/etc/myapp.conf",
"src_rel": "etc/myapp.conf",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "custom",
}
],
},
},
}
_write_bundle(old, old_state)
# Minimal report containing enforceable drift: a baseline file is "removed".
report = {
"packages": {"added": [], "removed": [], "version_changed": []},
"services": {"enabled_added": [], "enabled_removed": [], "changed": []},
"users": {"added": [], "removed": [], "changed": []},
"files": {
"added": [],
"removed": [{"path": "/etc/myapp.conf", "role": "usr_local_custom"}],
"changed": [],
},
}
info = d.enforce_old_harvest(str(old), report=report)
assert info["status"] == "applied"
assert "--tags" in info["command"]
assert "role_usr_local_custom" in ",".join(info.get("tags") or [])
argv = calls.get("argv")
assert argv and argv[0].endswith("ansible-playbook")
assert "--tags" in argv
# Ensure we pass the computed tag.
i = argv.index("--tags")
assert "role_usr_local_custom" in str(argv[i + 1])
def test_cli_diff_forwards_exclude_and_ignore_flags(monkeypatch, capsys):
import enroll.cli as cli
calls: dict[str, object] = {}
def fake_compare(
old, new, *, sops_mode=False, exclude_paths=None, ignore_package_versions=False
):
calls["compare"] = {
"old": old,
"new": new,
"sops_mode": sops_mode,
"exclude_paths": exclude_paths,
"ignore_package_versions": ignore_package_versions,
}
# No changes -> should not try to enforce.
return {"packages": {}, "services": {}, "users": {}, "files": {}}, False
monkeypatch.setattr(cli, "compare_harvests", fake_compare)
monkeypatch.setattr(cli, "format_report", lambda report, fmt="text": "R\n")
monkeypatch.setattr(
sys,
"argv",
[
"enroll",
"diff",
"--old",
"/tmp/old",
"--new",
"/tmp/new",
"--exclude-path",
"/var/anacron",
"--ignore-package-versions",
],
)
cli.main()
_ = capsys.readouterr()
assert calls["compare"]["exclude_paths"] == ["/var/anacron"]
assert calls["compare"]["ignore_package_versions"] is True
def test_cli_diff_enforce_skips_when_no_enforceable_drift(monkeypatch):
import enroll.cli as cli
# Drift exists, but is not enforceable (only additions / version changes).
report = {
"packages": {"added": ["htop"], "removed": [], "version_changed": []},
"services": {
"enabled_added": ["x.service"],
"enabled_removed": [],
"changed": [],
},
"users": {"added": ["bob"], "removed": [], "changed": []},
"files": {"added": [{"path": "/tmp/new"}], "removed": [], "changed": []},
}
monkeypatch.setattr(cli, "compare_harvests", lambda *a, **k: (report, True))
monkeypatch.setattr(cli, "has_enforceable_drift", lambda r: False)
called = {"enforce": False}
monkeypatch.setattr(
cli, "enforce_old_harvest", lambda *a, **k: called.update({"enforce": True})
)
captured = {}
def fake_format(rep, fmt="text"):
captured["report"] = rep
return "R\n"
monkeypatch.setattr(cli, "format_report", fake_format)
monkeypatch.setattr(
sys,
"argv",
[
"enroll",
"diff",
"--old",
"/tmp/old",
"--new",
"/tmp/new",
"--enforce",
],
)
cli.main()
assert called["enforce"] is False
assert captured["report"]["enforcement"]["status"] == "skipped"

83
tests/test_diff_notifications.py
View file

@ -1,83 +0,0 @@
from __future__ import annotations
import types
import pytest
def test_post_webhook_success(monkeypatch):
from enroll.diff import post_webhook
class FakeResp:
status = 204
def read(self):
return b"OK"
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return False
monkeypatch.setattr(
"enroll.diff.urllib.request.urlopen",
lambda req, timeout=30: FakeResp(),
)
status, body = post_webhook("https://example.com", b"x")
assert status == 204
assert body == "OK"
def test_post_webhook_raises_on_error(monkeypatch):
from enroll.diff import post_webhook
def boom(_req, timeout=30):
raise OSError("nope")
monkeypatch.setattr("enroll.diff.urllib.request.urlopen", boom)
with pytest.raises(RuntimeError):
post_webhook("https://example.com", b"x")
def test_send_email_uses_sendmail_when_present(monkeypatch):
from enroll.diff import send_email
calls = {}
monkeypatch.setattr("enroll.diff.shutil.which", lambda name: "/usr/sbin/sendmail")
def fake_run(argv, input=None, check=False, **_kwargs):
calls["argv"] = argv
calls["input"] = input
return types.SimpleNamespace(returncode=0)
monkeypatch.setattr("enroll.diff.subprocess.run", fake_run)
send_email(
subject="Subj",
body="Body",
from_addr="a@example.com",
to_addrs=["b@example.com"],
)
assert calls["argv"][0].endswith("sendmail")
msg = (calls["input"] or b"").decode("utf-8", errors="replace")
assert "Subject: Subj" in msg
assert "To: b@example.com" in msg
def test_send_email_raises_when_no_delivery_method(monkeypatch):
from enroll.diff import send_email
monkeypatch.setattr("enroll.diff.shutil.which", lambda name: None)
with pytest.raises(RuntimeError):
send_email(
subject="Subj",
body="Body",
from_addr="a@example.com",
to_addrs=["b@example.com"],
)

152
tests/test_diff_usr_local_custom.py
View file

@ -1,152 +0,0 @@
import json
from pathlib import Path
from enroll.diff import compare_harvests
def _write_bundle(root: Path, state: dict, artifacts: dict[str, bytes]) -> None:
root.mkdir(parents=True, exist_ok=True)
(root / "state.json").write_text(json.dumps(state, indent=2), encoding="utf-8")
for rel, data in artifacts.items():
p = root / rel
p.parent.mkdir(parents=True, exist_ok=True)
p.write_bytes(data)
def test_diff_includes_usr_local_custom_files(tmp_path: Path):
old = tmp_path / "old"
new = tmp_path / "new"
old_state = {
"schema_version": 3,
"host": {"hostname": "h1", "os": "debian", "pkg_backend": "dpkg"},
"inventory": {
"packages": {
"curl": {
"version": "1.0",
"arches": [],
"installations": [{"version": "1.0", "arch": "amd64"}],
"observed_via": [{"kind": "user_installed"}],
"roles": [],
}
}
},
"roles": {
"users": {
"role_name": "users",
"users": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
"services": [],
"packages": [],
"apt_config": {
"role_name": "apt_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"etc_custom": {
"role_name": "etc_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"usr_local_custom": {
"role_name": "usr_local_custom",
"managed_files": [
{
"path": "/usr/local/etc/myapp.conf",
"src_rel": "usr/local/etc/myapp.conf",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "usr_local_etc_custom",
}
],
"excluded": [],
"notes": [],
},
"extra_paths": {
"role_name": "extra_paths",
"include_patterns": [],
"exclude_patterns": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
},
}
new_state = {
**old_state,
"inventory": {
"packages": {
**old_state["inventory"]["packages"],
"htop": {
"version": "3.0",
"arches": [],
"installations": [{"version": "3.0", "arch": "amd64"}],
"observed_via": [{"kind": "user_installed"}],
"roles": [],
},
}
},
"roles": {
**old_state["roles"],
"usr_local_custom": {
"role_name": "usr_local_custom",
"managed_files": [
{
"path": "/usr/local/etc/myapp.conf",
"src_rel": "usr/local/etc/myapp.conf",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "usr_local_etc_custom",
},
{
"path": "/usr/local/bin/myscript",
"src_rel": "usr/local/bin/myscript",
"owner": "root",
"group": "root",
"mode": "0755",
"reason": "usr_local_bin_script",
},
],
"excluded": [],
"notes": [],
},
},
}
_write_bundle(
old,
old_state,
{
"artifacts/usr_local_custom/usr/local/etc/myapp.conf": b"myapp=1\n",
},
)
_write_bundle(
new,
new_state,
{
"artifacts/usr_local_custom/usr/local/etc/myapp.conf": b"myapp=2\n",
"artifacts/usr_local_custom/usr/local/bin/myscript": b"#!/bin/sh\necho hi\n",
},
)
report, has_changes = compare_harvests(str(old), str(new))
assert has_changes is True
# Packages: htop was added.
assert report["packages"]["added"] == ["htop"]
# Files: /usr/local/etc/myapp.conf should be detected as changed (content sha differs).
changed_paths = {c["path"] for c in report["files"]["changed"]}
assert "/usr/local/etc/myapp.conf" in changed_paths
# Files: new script was added.
added_paths = {a["path"] for a in report["files"]["added"]}
assert "/usr/local/bin/myscript" in added_paths

222
tests/test_explain.py
View file

@ -1,222 +0,0 @@
from __future__ import annotations
import json
from pathlib import Path
import enroll.explain as ex
def _write_state(bundle: Path, state: dict) -> Path:
bundle.mkdir(parents=True, exist_ok=True)
(bundle / "state.json").write_text(json.dumps(state, indent=2), encoding="utf-8")
return bundle / "state.json"
def test_explain_state_text_renders_roles_inventory_and_reasons(tmp_path: Path):
bundle = tmp_path / "bundle"
state = {
"schema_version": 3,
"host": {"hostname": "h1", "os": "debian", "pkg_backend": "dpkg"},
"enroll": {"version": "0.0.0"},
"inventory": {
"packages": {
"foo": {
"installations": [{"version": "1.0", "arch": "amd64"}],
"observed_via": [
{"kind": "systemd_unit", "ref": "foo.service"},
{"kind": "package_role", "ref": "foo"},
],
"roles": ["foo"],
},
"bar": {
"installations": [{"version": "2.0", "arch": "amd64"}],
"observed_via": [{"kind": "user_installed", "ref": "manual"}],
"roles": ["bar"],
},
}
},
"roles": {
"users": {
"role_name": "users",
"users": [{"name": "alice"}],
"managed_files": [
{
"path": "/home/alice/.ssh/authorized_keys",
"src_rel": "home/alice/.ssh/authorized_keys",
"owner": "alice",
"group": "alice",
"mode": "0600",
"reason": "authorized_keys",
}
],
"managed_dirs": [
{
"path": "/home/alice/.ssh",
"owner": "alice",
"group": "alice",
"mode": "0700",
"reason": "parent_of_managed_file",
}
],
"excluded": [{"path": "/etc/shadow", "reason": "sensitive_content"}],
"notes": ["n1", "n2"],
},
"services": [
{
"unit": "foo.service",
"role_name": "foo",
"packages": ["foo"],
"managed_files": [
{
"path": "/etc/foo.conf",
"src_rel": "etc/foo.conf",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "modified_conffile",
},
# Unknown reason should fall back to generic text.
{
"path": "/etc/odd.conf",
"src_rel": "etc/odd.conf",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "mystery_reason",
},
],
"excluded": [],
"notes": [],
}
],
"packages": [
{
"package": "bar",
"role_name": "bar",
"managed_files": [],
"excluded": [],
"notes": [],
}
],
"extra_paths": {
"role_name": "extra_paths",
"include_patterns": ["/etc/a", "/etc/b"],
"exclude_patterns": ["/etc/x", "/etc/y"],
"managed_files": [],
"excluded": [],
"notes": [],
},
"apt_config": {
"role_name": "apt_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"dnf_config": {
"role_name": "dnf_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"etc_custom": {
"role_name": "etc_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"usr_local_custom": {
"role_name": "usr_local_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
},
}
state_path = _write_state(bundle, state)
out = ex.explain_state(str(state_path), fmt="text", max_examples=1)
assert "Enroll explained:" in out
assert "Host: h1" in out
assert "Inventory" in out
# observed_via summary should include both kinds (order not strictly guaranteed)
assert "observed_via" in out
assert "systemd_unit" in out
assert "user_installed" in out
# extra_paths include/exclude patterns should be rendered with max_examples truncation.
assert "include_patterns:" in out
assert "/etc/a" in out
assert "exclude_patterns:" in out
# Reasons section should mention known and unknown reasons.
assert "modified_conffile" in out
assert "mystery_reason" in out
assert "Captured with reason 'mystery_reason'" in out
# Excluded paths section.
assert "Why paths were excluded" in out
assert "sensitive_content" in out
def test_explain_state_json_contains_structured_report(tmp_path: Path):
bundle = tmp_path / "bundle"
state = {
"schema_version": 3,
"host": {"hostname": "h2", "os": "rhel", "pkg_backend": "rpm"},
"enroll": {"version": "1.2.3"},
"inventory": {"packages": {}},
"roles": {
"users": {
"role_name": "users",
"users": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
"services": [],
"packages": [],
"apt_config": {
"role_name": "apt_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"dnf_config": {
"role_name": "dnf_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"etc_custom": {
"role_name": "etc_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"usr_local_custom": {
"role_name": "usr_local_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"extra_paths": {
"role_name": "extra_paths",
"include_patterns": [],
"exclude_patterns": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
},
}
state_path = _write_state(bundle, state)
raw = ex.explain_state(str(state_path), fmt="json", max_examples=2)
rep = json.loads(raw)
assert rep["host"]["hostname"] == "h2"
assert rep["enroll"]["version"] == "1.2.3"
assert rep["inventory"]["package_count"] == 0
assert isinstance(rep["roles"], list)
assert "reasons" in rep

25
tests/test_fsutil.py
View file

@ -1,25 +0,0 @@
from __future__ import annotations
import hashlib
import os
from pathlib import Path
from enroll.fsutil import file_md5, stat_triplet
def test_file_md5_matches_hashlib(tmp_path: Path):
p = tmp_path / "x"
p.write_bytes(b"hello world")
expected = hashlib.md5(b"hello world").hexdigest() # nosec
assert file_md5(str(p)) == expected
def test_stat_triplet_reports_mode(tmp_path: Path):
p = tmp_path / "x"
p.write_text("x", encoding="utf-8")
os.chmod(p, 0o600)
owner, group, mode = stat_triplet(str(p))
assert mode == "0600"
assert owner # non-empty string
assert group # non-empty string

31
tests/test_fsutil_extra.py
View file

@ -1,31 +0,0 @@
from __future__ import annotations
import os
from pathlib import Path
def test_stat_triplet_falls_back_to_numeric_ids(tmp_path: Path, monkeypatch):
"""If uid/gid cannot be resolved, stat_triplet should return numeric strings."""
from enroll.fsutil import stat_triplet
p = tmp_path / "f"
p.write_text("x", encoding="utf-8")
os.chmod(p, 0o644)
import grp
import pwd
def _no_user(_uid): # pragma: no cover - executed via monkeypatch
raise KeyError
def _no_group(_gid): # pragma: no cover - executed via monkeypatch
raise KeyError
monkeypatch.setattr(pwd, "getpwuid", _no_user)
monkeypatch.setattr(grp, "getgrgid", _no_group)
owner, group, mode = stat_triplet(str(p))
assert owner.isdigit()
assert group.isdigit()
assert mode == "0644"

276
tests/test_harvest.py
View file

@ -2,7 +2,6 @@ import json
from pathlib import Path
import enroll.harvest as h
from enroll.platform import PlatformInfo
from enroll.systemd import UnitInfo
@ -11,74 +10,6 @@ class AllowAllPolicy:
return None
class FakeBackend:
"""Minimal backend stub for harvest tests.
The real backends (dpkg/rpm) enumerate the live system (dpkg status, rpm
databases, etc). These tests instead control all backend behaviour.
"""
def __init__(
self,
*,
name: str,
owned_etc: set[str],
etc_owner_map: dict[str, str],
topdir_to_pkgs: dict[str, set[str]],
pkg_to_etc_paths: dict[str, list[str]],
manual_pkgs: list[str],
owner_fn,
modified_by_pkg: dict[str, dict[str, str]] | None = None,
pkg_config_prefixes: tuple[str, ...] = ("/etc/apt/",),
installed: dict[str, list[dict[str, str]]] | None = None,
):
self.name = name
self.pkg_config_prefixes = pkg_config_prefixes
self._owned_etc = owned_etc
self._etc_owner_map = etc_owner_map
self._topdir_to_pkgs = topdir_to_pkgs
self._pkg_to_etc_paths = pkg_to_etc_paths
self._manual = manual_pkgs
self._owner_fn = owner_fn
self._modified_by_pkg = modified_by_pkg or {}
self._installed = installed or {}
def build_etc_index(self):
return (
self._owned_etc,
self._etc_owner_map,
self._topdir_to_pkgs,
self._pkg_to_etc_paths,
)
def owner_of_path(self, path: str):
return self._owner_fn(path)
def list_manual_packages(self):
return list(self._manual)
def installed_packages(self):
"""Return mapping package -> installations.
The real backends return:
{"pkg": [{"version": "...", "arch": "..."}, ...]}
"""
return dict(self._installed)
def specific_paths_for_hints(self, hints: set[str]):
return []
def is_pkg_config_path(self, path: str) -> bool:
for pfx in self.pkg_config_prefixes:
if path == pfx or path.startswith(pfx):
return True
return False
def modified_paths(self, pkg: str, etc_paths: list[str]):
# Test-controlled; ignore etc_paths.
return dict(self._modified_by_pkg.get(pkg, {}))
def test_harvest_dedup_manual_packages_and_builds_etc_custom(
monkeypatch, tmp_path: Path
):
@ -91,52 +22,31 @@ def test_harvest_dedup_manual_packages_and_builds_etc_custom(
real_exists = os.path.exists
real_islink = os.path.islink
# Fake filesystem: two /etc files exist, only one is package-owned.
# Also include some /usr/local files to populate usr_local_custom.
# Fake filesystem: two /etc files exist, only one is dpkg-owned.
files = {
"/etc/openvpn/server.conf": b"server",
"/etc/default/keyboard": b"kbd",
"/usr/local/etc/myapp.conf": b"myapp=1\n",
"/usr/local/bin/myscript": b"#!/bin/sh\necho hi\n",
# non-executable text under /usr/local/bin should be skipped
"/usr/local/bin/readme.txt": b"hello\n",
}
dirs = {
"/etc",
"/etc/openvpn",
"/etc/default",
"/usr",
"/usr/local",
"/usr/local/etc",
"/usr/local/bin",
}
dirs = {"/etc", "/etc/openvpn", "/etc/default"}
def fake_isfile(p: str) -> bool:
if p.startswith("/etc/") or p == "/etc":
return p in files
if p.startswith("/usr/local/"):
return p in files
return real_isfile(p)
def fake_isdir(p: str) -> bool:
if p.startswith("/etc"):
return p in dirs
if p.startswith("/usr/local") or p in ("/usr", "/usr/local"):
return p in dirs
return real_isdir(p)
def fake_islink(p: str) -> bool:
if p.startswith("/etc"):
return False
if p.startswith("/usr/local"):
return False
return real_islink(p)
def fake_exists(p: str) -> bool:
if p.startswith("/etc"):
return p in files or p in dirs
if p.startswith("/usr/local") or p in ("/usr", "/usr/local"):
return p in files or p in dirs
return real_exists(p)
def fake_walk(root: str):
@ -147,10 +57,6 @@ def test_harvest_dedup_manual_packages_and_builds_etc_custom(
yield ("/etc/openvpn", [], ["server.conf"])
elif root == "/etc/default":
yield ("/etc/default", [], ["keyboard"])
elif root == "/usr/local/etc":
yield ("/usr/local/etc", [], ["myapp.conf"])
elif root == "/usr/local/bin":
yield ("/usr/local/bin", [], ["myscript", "readme.txt"])
else:
yield (root, [], [])
@ -162,7 +68,6 @@ def test_harvest_dedup_manual_packages_and_builds_etc_custom(
# Avoid real system access
monkeypatch.setattr(h, "list_enabled_services", lambda: ["openvpn.service"])
monkeypatch.setattr(h, "list_enabled_timers", lambda: [])
monkeypatch.setattr(
h,
"get_unit_info",
@ -179,39 +84,32 @@ def test_harvest_dedup_manual_packages_and_builds_etc_custom(
),
)
# Package index: openvpn owns /etc/openvpn/server.conf; keyboard is unowned.
# Debian package index: openvpn owns /etc/openvpn/server.conf; keyboard is unowned.
def fake_build_index():
owned_etc = {"/etc/openvpn/server.conf"}
etc_owner_map = {"/etc/openvpn/server.conf": "openvpn"}
topdir_to_pkgs = {"openvpn": {"openvpn"}}
pkg_to_etc_paths = {"openvpn": ["/etc/openvpn/server.conf"], "curl": []}
return owned_etc, etc_owner_map, topdir_to_pkgs, pkg_to_etc_paths
backend = FakeBackend(
name="dpkg",
owned_etc=owned_etc,
etc_owner_map=etc_owner_map,
topdir_to_pkgs=topdir_to_pkgs,
pkg_to_etc_paths=pkg_to_etc_paths,
manual_pkgs=["openvpn", "curl"],
owner_fn=lambda p: "openvpn" if "openvpn" in (p or "") else None,
modified_by_pkg={
"openvpn": {"/etc/openvpn/server.conf": "modified_conffile"},
},
monkeypatch.setattr(h, "build_dpkg_etc_index", fake_build_index)
# openvpn conffile hash mismatch => should be captured under service role
monkeypatch.setattr(
h,
"parse_status_conffiles",
lambda: {"openvpn": {"/etc/openvpn/server.conf": "old"}},
)
monkeypatch.setattr(h, "read_pkg_md5sums", lambda pkg: {})
monkeypatch.setattr(h, "file_md5", lambda path: "new")
monkeypatch.setattr(
h, "detect_platform", lambda: PlatformInfo("debian", "dpkg", {})
h, "dpkg_owner", lambda p: "openvpn" if "openvpn" in p else None
)
monkeypatch.setattr(h, "get_backend", lambda info=None: backend)
monkeypatch.setattr(h, "list_manual_packages", lambda: ["openvpn", "curl"])
monkeypatch.setattr(h, "collect_non_system_users", lambda: [])
def fake_stat_triplet(p: str):
if p == "/usr/local/bin/myscript":
return ("root", "root", "0755")
# /usr/local/bin/readme.txt remains non-executable
return ("root", "root", "0644")
monkeypatch.setattr(h, "stat_triplet", fake_stat_triplet)
monkeypatch.setattr(h, "stat_triplet", lambda p: ("root", "root", "0644"))
# Avoid needing source files on disk by implementing our own bundle copier
def fake_copy(bundle_dir: str, role_name: str, abs_path: str, src_rel: str):
@ -224,146 +122,20 @@ def test_harvest_dedup_manual_packages_and_builds_etc_custom(
state_path = h.harvest(str(bundle), policy=AllowAllPolicy())
st = json.loads(Path(state_path).read_text(encoding="utf-8"))
inv = st["inventory"]["packages"]
assert "openvpn" in inv
assert "curl" in inv
# openvpn is managed by the service role, so it should NOT appear as a package role.
pkg_roles = st["roles"]["packages"]
assert all(pr["package"] != "openvpn" for pr in pkg_roles)
assert any(pr["package"] == "curl" for pr in pkg_roles)
# Inventory provenance: openvpn should be observed via systemd unit.
openvpn_obs = inv["openvpn"]["observed_via"]
assert any(
o.get("kind") == "systemd_unit" and o.get("ref") == "openvpn.service"
for o in openvpn_obs
)
assert "openvpn" in st["manual_packages"]
assert "curl" in st["manual_packages"]
assert "openvpn" in st["manual_packages_skipped"]
assert all(pr["package"] != "openvpn" for pr in st["package_roles"])
assert any(pr["package"] == "curl" for pr in st["package_roles"])
# Service role captured modified conffile
svc = st["roles"]["services"][0]
svc = st["services"][0]
assert svc["unit"] == "openvpn.service"
assert "openvpn" in svc["packages"]
assert any(mf["path"] == "/etc/openvpn/server.conf" for mf in svc["managed_files"])
# Unowned /etc/default/keyboard is attributed to etc_custom only
etc_custom = st["roles"]["etc_custom"]
etc_custom = st["etc_custom"]
assert any(
mf["path"] == "/etc/default/keyboard" for mf in etc_custom["managed_files"]
)
# /usr/local content is attributed to usr_local_custom
ul = st["roles"]["usr_local_custom"]
assert any(mf["path"] == "/usr/local/etc/myapp.conf" for mf in ul["managed_files"])
assert any(mf["path"] == "/usr/local/bin/myscript" for mf in ul["managed_files"])
assert all(mf["path"] != "/usr/local/bin/readme.txt" for mf in ul["managed_files"])
def test_shared_cron_snippet_prefers_matching_role_over_lexicographic(
monkeypatch, tmp_path: Path
):
"""Regression test for shared snippet routing.
When multiple service roles reference the same owning package, we prefer the
role whose name matches the snippet/package (e.g. ntpsec) rather than a
lexicographic tie-break that could incorrectly pick another role.
"""
bundle = tmp_path / "bundle"
files = {"/etc/cron.d/ntpsec": b"# cron\n"}
dirs = {"/etc", "/etc/cron.d"}
monkeypatch.setattr(h.os.path, "isfile", lambda p: p in files)
monkeypatch.setattr(h.os.path, "islink", lambda p: False)
monkeypatch.setattr(h.os.path, "isdir", lambda p: p in dirs)
monkeypatch.setattr(h.os.path, "exists", lambda p: p in files or p in dirs)
monkeypatch.setattr(h.os, "walk", lambda root: [("/etc/cron.d", [], ["ntpsec"])])
# Only include the cron snippet in the system capture set.
monkeypatch.setattr(
h, "_iter_system_capture_paths", lambda: [("/etc/cron.d/ntpsec", "system_cron")]
)
monkeypatch.setattr(
h, "list_enabled_services", lambda: ["apparmor.service", "ntpsec.service"]
)
monkeypatch.setattr(h, "list_enabled_timers", lambda: [])
def fake_unit_info(unit: str) -> UnitInfo:
if unit == "apparmor.service":
return UnitInfo(
name=unit,
fragment_path="/lib/systemd/system/apparmor.service",
dropin_paths=[],
env_files=[],
exec_paths=["/usr/sbin/apparmor"],
active_state="active",
sub_state="running",
unit_file_state="enabled",
condition_result=None,
)
return UnitInfo(
name=unit,
fragment_path="/lib/systemd/system/ntpsec.service",
dropin_paths=[],
env_files=[],
exec_paths=["/usr/sbin/ntpd"],
active_state="active",
sub_state="running",
unit_file_state="enabled",
condition_result=None,
)
monkeypatch.setattr(h, "get_unit_info", fake_unit_info)
# Make apparmor *also* claim the ntpsec package (simulates overly-broad
# package inference). The snippet routing should still prefer role 'ntpsec'.
def fake_owner(p: str):
if p == "/etc/cron.d/ntpsec":
return "ntpsec"
if "apparmor" in (p or ""):
return "ntpsec" # intentionally misleading
if "ntpsec" in (p or "") or "ntpd" in (p or ""):
return "ntpsec"
return None
backend = FakeBackend(
name="dpkg",
owned_etc=set(),
etc_owner_map={},
topdir_to_pkgs={},
pkg_to_etc_paths={},
manual_pkgs=[],
owner_fn=fake_owner,
modified_by_pkg={},
)
monkeypatch.setattr(
h, "detect_platform", lambda: PlatformInfo("debian", "dpkg", {})
)
monkeypatch.setattr(h, "get_backend", lambda info=None: backend)
monkeypatch.setattr(h, "stat_triplet", lambda p: ("root", "root", "0644"))
monkeypatch.setattr(h, "collect_non_system_users", lambda: [])
def fake_copy(bundle_dir: str, role_name: str, abs_path: str, src_rel: str):
dst = Path(bundle_dir) / "artifacts" / role_name / src_rel
dst.parent.mkdir(parents=True, exist_ok=True)
dst.write_bytes(files[abs_path])
monkeypatch.setattr(h, "_copy_into_bundle", fake_copy)
state_path = h.harvest(str(bundle), policy=AllowAllPolicy())
st = json.loads(Path(state_path).read_text(encoding="utf-8"))
# Cron snippet should end up attached to the ntpsec role, not apparmor.
svc_ntpsec = next(s for s in st["roles"]["services"] if s["role_name"] == "ntpsec")
assert any(mf["path"] == "/etc/cron.d/ntpsec" for mf in svc_ntpsec["managed_files"])
svc_apparmor = next(
s for s in st["roles"]["services"] if s["role_name"] == "apparmor"
)
assert all(
mf["path"] != "/etc/cron.d/ntpsec" for mf in svc_apparmor["managed_files"]
)

164
tests/test_harvest_cron_logrotate.py
View file

@ -1,164 +0,0 @@
from __future__ import annotations
import json
from pathlib import Path
import enroll.harvest as h
from enroll.platform import PlatformInfo
from enroll.systemd import UnitInfo
class AllowAllPolicy:
def deny_reason(self, path: str):
return None
class FakeBackend:
def __init__(
self,
*,
name: str,
installed: dict[str, list[dict[str, str]]],
manual: list[str],
):
self.name = name
self._installed = dict(installed)
self._manual = list(manual)
def build_etc_index(self):
# No package ownership information needed for this test.
return set(), {}, {}, {}
def installed_packages(self):
return dict(self._installed)
def list_manual_packages(self):
return list(self._manual)
def owner_of_path(self, path: str):
return None
def specific_paths_for_hints(self, hints: set[str]):
return []
def is_pkg_config_path(self, path: str) -> bool:
return False
def modified_paths(self, pkg: str, etc_paths: list[str]):
return {}
def test_harvest_unifies_cron_and_logrotate_into_dedicated_package_roles(
monkeypatch, tmp_path: Path
):
bundle = tmp_path / "bundle"
# Fake files we want harvested.
files = {
"/etc/crontab": b"* * * * * root echo hi\n",
"/etc/cron.d/php": b"# php cron\n",
"/var/spool/cron/crontabs/alice": b"@daily echo user\n",
"/etc/logrotate.conf": b"weekly\n",
"/etc/logrotate.d/rsyslog": b"/var/log/syslog { rotate 7 }\n",
}
monkeypatch.setattr(h.os.path, "islink", lambda p: False)
monkeypatch.setattr(h.os.path, "isfile", lambda p: p in files)
monkeypatch.setattr(h.os.path, "isdir", lambda p: False)
monkeypatch.setattr(h.os.path, "exists", lambda p: (p in files) or False)
# Expand cron/logrotate globs deterministically.
def fake_iter_matching(spec: str, cap: int = 10000):
mapping = {
"/etc/crontab": ["/etc/crontab"],
"/etc/cron.d/*": ["/etc/cron.d/php"],
"/etc/cron.hourly/*": [],
"/etc/cron.daily/*": [],
"/etc/cron.weekly/*": [],
"/etc/cron.monthly/*": [],
"/etc/cron.allow": [],
"/etc/cron.deny": [],
"/etc/anacrontab": [],
"/etc/anacron/*": [],
"/var/spool/cron/*": [],
"/var/spool/cron/crontabs/*": ["/var/spool/cron/crontabs/alice"],
"/var/spool/crontabs/*": [],
"/var/spool/anacron/*": [],
"/etc/logrotate.conf": ["/etc/logrotate.conf"],
"/etc/logrotate.d/*": ["/etc/logrotate.d/rsyslog"],
}
return list(mapping.get(spec, []))[:cap]
monkeypatch.setattr(h, "_iter_matching_files", fake_iter_matching)
# Avoid real system probing.
monkeypatch.setattr(
h, "detect_platform", lambda: PlatformInfo("debian", "dpkg", {})
)
backend = FakeBackend(
name="dpkg",
installed={
"cron": [{"version": "1", "arch": "amd64"}],
"logrotate": [{"version": "1", "arch": "amd64"}],
},
# Include cron/logrotate in manual packages to ensure they are skipped in the generic loop.
manual=["cron", "logrotate"],
)
monkeypatch.setattr(h, "get_backend", lambda info=None: backend)
# Include a service that would collide with cron role naming.
monkeypatch.setattr(
h, "list_enabled_services", lambda: ["cron.service", "foo.service"]
)
monkeypatch.setattr(h, "list_enabled_timers", lambda: [])
monkeypatch.setattr(
h,
"get_unit_info",
lambda unit: UnitInfo(
name=unit,
fragment_path=None,
dropin_paths=[],
env_files=[],
exec_paths=[],
active_state="active",
sub_state="running",
unit_file_state="enabled",
condition_result=None,
),
)
monkeypatch.setattr(h, "collect_non_system_users", lambda: [])
monkeypatch.setattr(
h,
"stat_triplet",
lambda p: ("alice" if "alice" in p else "root", "root", "0644"),
)
# Avoid needing real source files by implementing our own bundle copier.
def fake_copy(bundle_dir: str, role_name: str, abs_path: str, src_rel: str):
dst = Path(bundle_dir) / "artifacts" / role_name / src_rel
dst.parent.mkdir(parents=True, exist_ok=True)
dst.write_bytes(files.get(abs_path, b""))
monkeypatch.setattr(h, "_copy_into_bundle", fake_copy)
state_path = h.harvest(str(bundle), policy=AllowAllPolicy())
st = json.loads(Path(state_path).read_text(encoding="utf-8"))
# cron.service must be skipped to avoid colliding with the dedicated "cron" package role.
svc_units = [s["unit"] for s in st["roles"]["services"]]
assert "cron.service" not in svc_units
assert "foo.service" in svc_units
pkgs = st["roles"]["packages"]
cron = next(p for p in pkgs if p["role_name"] == "cron")
logrotate = next(p for p in pkgs if p["role_name"] == "logrotate")
cron_paths = {mf["path"] for mf in cron["managed_files"]}
assert "/etc/crontab" in cron_paths
assert "/etc/cron.d/php" in cron_paths
# user crontab captured
assert "/var/spool/cron/crontabs/alice" in cron_paths
lr_paths = {mf["path"] for mf in logrotate["managed_files"]}
assert "/etc/logrotate.conf" in lr_paths
assert "/etc/logrotate.d/rsyslog" in lr_paths

288
tests/test_harvest_helpers.py
View file

@ -1,288 +0,0 @@
from __future__ import annotations
import os
from pathlib import Path
import enroll.harvest as h
def test_iter_matching_files_skips_symlinks_and_walks_dirs(monkeypatch, tmp_path: Path):
# Layout:
# root/real.txt (file)
# root/sub/nested.txt
# root/link -> ... (ignored)
root = tmp_path / "root"
(root / "sub").mkdir(parents=True)
(root / "real.txt").write_text("a", encoding="utf-8")
(root / "sub" / "nested.txt").write_text("b", encoding="utf-8")
paths = {
str(root): "dir",
str(root / "real.txt"): "file",
str(root / "sub"): "dir",
str(root / "sub" / "nested.txt"): "file",
str(root / "link"): "link",
}
monkeypatch.setattr(h.glob, "glob", lambda spec: [str(root), str(root / "link")])
monkeypatch.setattr(h.os.path, "islink", lambda p: paths.get(p) == "link")
monkeypatch.setattr(h.os.path, "isfile", lambda p: paths.get(p) == "file")
monkeypatch.setattr(h.os.path, "isdir", lambda p: paths.get(p) == "dir")
monkeypatch.setattr(
h.os,
"walk",
lambda p: [
(str(root), ["sub"], ["real.txt", "link"]),
(str(root / "sub"), [], ["nested.txt"]),
],
)
out = h._iter_matching_files("/whatever/*", cap=100)
assert str(root / "real.txt") in out
assert str(root / "sub" / "nested.txt") in out
assert str(root / "link") not in out
def test_parse_apt_signed_by_extracts_keyrings(tmp_path: Path):
f1 = tmp_path / "a.list"
f1.write_text(
"deb [signed-by=/usr/share/keyrings/foo.gpg] https://example.invalid stable main\n",
encoding="utf-8",
)
f2 = tmp_path / "b.sources"
f2.write_text(
"Types: deb\nSigned-By: /etc/apt/keyrings/bar.gpg, /usr/share/keyrings/baz.gpg\n",
encoding="utf-8",
)
f3 = tmp_path / "c.sources"
f3.write_text("Signed-By: | /bin/echo nope\n", encoding="utf-8")
out = h._parse_apt_signed_by([str(f1), str(f2), str(f3)])
assert "/usr/share/keyrings/foo.gpg" in out
assert "/etc/apt/keyrings/bar.gpg" in out
assert "/usr/share/keyrings/baz.gpg" in out
def test_iter_apt_capture_paths_includes_signed_by_keyring(monkeypatch):
# Simulate:
# /etc/apt/apt.conf.d/00test
# /etc/apt/sources.list.d/test.list (signed-by outside /etc/apt)
# /usr/share/keyrings/ext.gpg
files = {
"/etc/apt/apt.conf.d/00test": "file",
"/etc/apt/sources.list.d/test.list": "file",
"/usr/share/keyrings/ext.gpg": "file",
}
monkeypatch.setattr(h.os.path, "isdir", lambda p: p in {"/etc/apt"})
monkeypatch.setattr(
h.os,
"walk",
lambda root: [
("/etc/apt", ["apt.conf.d", "sources.list.d"], []),
("/etc/apt/apt.conf.d", [], ["00test"]),
("/etc/apt/sources.list.d", [], ["test.list"]),
],
)
monkeypatch.setattr(h.os.path, "islink", lambda p: False)
monkeypatch.setattr(h.os.path, "isfile", lambda p: files.get(p) == "file")
# Only treat the sources glob as having a hit.
def fake_iter_matching(spec: str, cap: int = 10000):
if spec == "/etc/apt/sources.list.d/*.list":
return ["/etc/apt/sources.list.d/test.list"]
return []
monkeypatch.setattr(h, "_iter_matching_files", fake_iter_matching)
# Provide file contents for the sources file.
real_open = open
def fake_open(path, *a, **k):
if path == "/etc/apt/sources.list.d/test.list":
return real_open(os.devnull, "r", encoding="utf-8") # placeholder
return real_open(path, *a, **k)
# Easier: patch _parse_apt_signed_by directly to avoid filesystem reads.
monkeypatch.setattr(
h, "_parse_apt_signed_by", lambda sfs: {"/usr/share/keyrings/ext.gpg"}
)
out = h._iter_apt_capture_paths()
paths = {p for p, _r in out}
reasons = {p: r for p, r in out}
assert "/etc/apt/apt.conf.d/00test" in paths
assert "/etc/apt/sources.list.d/test.list" in paths
assert "/usr/share/keyrings/ext.gpg" in paths
assert reasons["/usr/share/keyrings/ext.gpg"] == "apt_signed_by_keyring"
def test_iter_dnf_capture_paths(monkeypatch):
files = {
"/etc/dnf/dnf.conf": "file",
"/etc/yum/yum.conf": "file",
"/etc/yum.conf": "file",
"/etc/yum.repos.d/test.repo": "file",
"/etc/pki/rpm-gpg/RPM-GPG-KEY": "file",
}
def isdir(p):
return p in {"/etc/dnf", "/etc/yum", "/etc/yum.repos.d", "/etc/pki/rpm-gpg"}
def walk(root):
if root == "/etc/dnf":
return [("/etc/dnf", [], ["dnf.conf"])]
if root == "/etc/yum":
return [("/etc/yum", [], ["yum.conf"])]
if root == "/etc/pki/rpm-gpg":
return [("/etc/pki/rpm-gpg", [], ["RPM-GPG-KEY"])]
return []
monkeypatch.setattr(h.os.path, "isdir", isdir)
monkeypatch.setattr(h.os, "walk", walk)
monkeypatch.setattr(h.os.path, "islink", lambda p: False)
monkeypatch.setattr(h.os.path, "isfile", lambda p: files.get(p) == "file")
monkeypatch.setattr(
h,
"_iter_matching_files",
lambda spec, cap=10000: (
["/etc/yum.repos.d/test.repo"] if spec.endswith("*.repo") else []
),
)
out = h._iter_dnf_capture_paths()
paths = {p for p, _r in out}
assert "/etc/dnf/dnf.conf" in paths
assert "/etc/yum/yum.conf" in paths
assert "/etc/yum.conf" in paths
assert "/etc/yum.repos.d/test.repo" in paths
assert "/etc/pki/rpm-gpg/RPM-GPG-KEY" in paths
def test_iter_system_capture_paths_dedupes_first_reason(monkeypatch):
monkeypatch.setattr(h, "_SYSTEM_CAPTURE_GLOBS", [("/a", "r1"), ("/b", "r2")])
monkeypatch.setattr(
h,
"_iter_matching_files",
lambda spec, cap=10000: ["/dup"] if spec in {"/a", "/b"} else [],
)
out = h._iter_system_capture_paths()
assert out == [("/dup", "r1")]
def test_ipset_and_iptables_state_helpers(tmp_path: Path):
ipset_save = """create blocklist hash:ip family inet hashsize 1024 maxelem 65536
add blocklist 203.0.113.10
create nets hash:net family inet
"""
assert h._ipset_save_has_state(ipset_save)
assert h._parse_ipset_set_names(ipset_save) == ["blocklist", "nets"]
assert not h._ipset_save_has_state("# empty\n")
empty_iptables = """*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT
"""
assert not h._iptables_save_has_state(empty_iptables)
native_rule = empty_iptables.replace(
"COMMIT", "-A INPUT -p tcp --dport 22 -j ACCEPT\nCOMMIT"
)
assert h._iptables_save_has_state(native_rule)
changed_policy = empty_iptables.replace(":INPUT ACCEPT", ":INPUT DROP")
assert h._iptables_save_has_state(changed_policy)
def test_collect_firewall_runtime_snapshot_writes_generated_artifacts(
monkeypatch, tmp_path: Path
):
outputs = {
"ipset_save": (
"create blocklist hash:ip family inet\nadd blocklist 203.0.113.10\n",
None,
),
"iptables_v4_save": (
"*filter\n:INPUT DROP [0:0]\n-A INPUT -m set --match-set blocklist src -j DROP\nCOMMIT\n",
None,
),
"iptables_v6_save": ("*filter\n:INPUT ACCEPT [0:0]\nCOMMIT\n", None),
}
def fake_run(command_key, *, timeout=10):
return outputs[command_key]
monkeypatch.setattr(h, "_run_capture_command", fake_run)
snap = h._collect_firewall_runtime_snapshot(str(tmp_path))
assert snap.role_name == "firewall_runtime"
assert snap.packages == ["ipset", "iptables"]
assert snap.ipset_save == "firewall/ipset.save"
assert snap.ipset_sets == ["blocklist"]
assert snap.iptables_v4_save == "firewall/iptables.v4"
assert snap.iptables_v6_save is None
assert (
(tmp_path / "artifacts" / "firewall_runtime" / "firewall" / "ipset.save")
.read_text(encoding="utf-8")
.startswith("create blocklist")
)
assert (
(tmp_path / "artifacts" / "firewall_runtime" / "firewall" / "iptables.v4")
.read_text(encoding="utf-8")
.startswith("*filter")
)
def test_collect_firewall_runtime_snapshot_is_per_family_fallback(
monkeypatch, tmp_path: Path
):
calls = []
outputs = {
"ipset_save": (
"create blocklist hash:ip family inet\nadd blocklist 203.0.113.10\n",
None,
),
"iptables_v4_save": (
"*filter\n:INPUT DROP [0:0]\n-A INPUT -p tcp --dport 22 -j ACCEPT\nCOMMIT\n",
None,
),
"iptables_v6_save": (
"*filter\n:INPUT DROP [0:0]\n-A INPUT -p tcp --dport 22 -j ACCEPT\nCOMMIT\n",
None,
),
}
def fake_run(command_key, *, timeout=10):
calls.append(command_key)
return outputs[command_key]
monkeypatch.setattr(h, "_run_capture_command", fake_run)
snap = h._collect_firewall_runtime_snapshot(
str(tmp_path),
persistent_ipset_files=["/etc/ipset.conf"],
persistent_iptables_v4_files=["/etc/iptables/rules.v4"],
persistent_iptables_v6_files=[],
)
assert "ipset_save" not in calls
assert "iptables_v4_save" not in calls
assert "iptables_v6_save" in calls
assert snap.ipset_save is None
assert snap.iptables_v4_save is None
assert snap.iptables_v6_save == "firewall/iptables.v6"
assert snap.packages == ["iptables"]
assert any("persistent ipset configuration" in note for note in snap.notes)
assert any("persistent IPv4 iptables configuration" in note for note in snap.notes)
assert not (
tmp_path / "artifacts" / "firewall_runtime" / "firewall" / "ipset.save"
).exists()
assert not (
tmp_path / "artifacts" / "firewall_runtime" / "firewall" / "iptables.v4"
).exists()
assert (
tmp_path / "artifacts" / "firewall_runtime" / "firewall" / "iptables.v6"
).exists()

263
tests/test_harvest_symlinks.py
View file

@ -1,263 +0,0 @@
import json
from pathlib import Path
import enroll.harvest as h
from enroll.platform import PlatformInfo
from enroll.systemd import UnitInfo
class AllowAllPolicy:
def deny_reason(self, path: str):
return None
def deny_reason_link(self, path: str):
return None
class FakeBackend:
"""Minimal backend stub for harvest tests.
Keep harvest deterministic and avoid enumerating the real system.
"""
name = "dpkg"
def build_etc_index(self):
return (set(), {}, {}, {})
def owner_of_path(self, path: str):
return None
def list_manual_packages(self):
return []
def installed_packages(self):
return {}
def specific_paths_for_hints(self, hints: set[str]):
return []
def is_pkg_config_path(self, path: str) -> bool:
return False
def modified_paths(self, pkg: str, etc_paths: list[str]):
return {}
def _base_monkeypatches(monkeypatch, *, unit: str):
"""Patch harvest to avoid live system access."""
monkeypatch.setattr(
h, "detect_platform", lambda: PlatformInfo("debian", "dpkg", {})
)
monkeypatch.setattr(h, "get_backend", lambda info=None: FakeBackend())
monkeypatch.setattr(h, "list_enabled_timers", lambda: [])
monkeypatch.setattr(
h,
"get_unit_info",
lambda u: UnitInfo(
name=u,
fragment_path=None,
dropin_paths=[],
env_files=[],
exec_paths=[],
active_state="inactive",
sub_state="dead",
unit_file_state="enabled",
condition_result=None,
),
)
# Keep users empty and avoid touching /etc/skel.
monkeypatch.setattr(h, "collect_non_system_users", lambda: [])
# Avoid warning spam from non-root test runs.
if hasattr(h.os, "geteuid"):
monkeypatch.setattr(h.os, "geteuid", lambda: 0)
# Avoid walking the real filesystem.
monkeypatch.setattr(h.os, "walk", lambda root: iter(()))
monkeypatch.setattr(h, "_copy_into_bundle", lambda *a, **k: None)
# Default to a "no files exist" view of the world unless a test overrides.
monkeypatch.setattr(h.os.path, "isfile", lambda p: False)
monkeypatch.setattr(h.os.path, "exists", lambda p: False)
# Minimal enabled services list.
monkeypatch.setattr(h, "list_enabled_services", lambda: [unit] if unit else [])
def test_harvest_captures_nginx_enabled_symlinks(monkeypatch, tmp_path: Path):
bundle = tmp_path / "bundle"
unit = "nginx.service"
_base_monkeypatches(monkeypatch, unit=unit)
# Fake filesystem for nginx enabled dirs.
dirs = {
"/etc",
"/etc/nginx",
"/etc/nginx/sites-enabled",
"/etc/nginx/modules-enabled",
}
links = {
"/etc/nginx/sites-enabled/default": "../sites-available/default",
"/etc/nginx/modules-enabled/mod-http": "../modules-available/mod-http",
}
monkeypatch.setattr(h.os.path, "isdir", lambda p: p in dirs)
monkeypatch.setattr(h.os.path, "islink", lambda p: p in links)
monkeypatch.setattr(h.os, "readlink", lambda p: links[p])
def fake_glob(pat: str):
if pat == "/etc/nginx/sites-enabled/*":
return [
"/etc/nginx/sites-enabled/default",
"/etc/nginx/sites-enabled/README",
]
if pat == "/etc/nginx/modules-enabled/*":
return ["/etc/nginx/modules-enabled/mod-http"]
return []
monkeypatch.setattr(h.glob, "glob", fake_glob)
state_path = h.harvest(str(bundle), policy=AllowAllPolicy())
st = json.loads(Path(state_path).read_text(encoding="utf-8"))
svc = next(s for s in st["roles"]["services"] if s["role_name"] == "nginx")
managed_links = svc.get("managed_links") or []
assert {(ml["path"], ml["target"], ml["reason"]) for ml in managed_links} == {
(
"/etc/nginx/sites-enabled/default",
"../sites-available/default",
"enabled_symlink",
),
(
"/etc/nginx/modules-enabled/mod-http",
"../modules-available/mod-http",
"enabled_symlink",
),
}
def test_harvest_does_not_capture_enabled_symlinks_without_role(
monkeypatch, tmp_path: Path
):
bundle = tmp_path / "bundle"
_base_monkeypatches(monkeypatch, unit="")
# Dirs exist but nginx isn't detected, so nothing should be captured.
monkeypatch.setattr(
h.os.path,
"isdir",
lambda p: p
in {
"/etc",
"/etc/nginx/sites-enabled",
"/etc/nginx/modules-enabled",
},
)
monkeypatch.setattr(
h.glob, "glob", lambda pat: ["/etc/nginx/sites-enabled/default"]
)
monkeypatch.setattr(h.os.path, "islink", lambda p: True)
monkeypatch.setattr(h.os, "readlink", lambda p: "../sites-available/default")
state_path = h.harvest(str(bundle), policy=AllowAllPolicy())
st = json.loads(Path(state_path).read_text(encoding="utf-8"))
# No services => no place to attach nginx links.
assert st["roles"]["services"] == []
# And no package snapshots either.
assert st["roles"]["packages"] == []
def test_harvest_symlink_capture_respects_ignore_policy(monkeypatch, tmp_path: Path):
bundle = tmp_path / "bundle"
_base_monkeypatches(monkeypatch, unit="nginx.service")
dirs = {"/etc", "/etc/nginx/sites-enabled", "/etc/nginx/modules-enabled"}
links = {
"/etc/nginx/sites-enabled/default": "../sites-available/default",
"/etc/nginx/sites-enabled/ok": "../sites-available/ok",
}
monkeypatch.setattr(h.os.path, "isdir", lambda p: p in dirs)
monkeypatch.setattr(h.os.path, "islink", lambda p: p in links)
monkeypatch.setattr(h.os, "readlink", lambda p: links[p])
monkeypatch.setattr(
h.glob,
"glob",
lambda pat: (
sorted(list(links.keys())) if pat == "/etc/nginx/sites-enabled/*" else []
),
)
calls: list[str] = []
class Policy:
def deny_reason(self, path: str):
return None
def deny_reason_link(self, path: str):
calls.append(path)
if path.endswith("/default"):
return "denied_path"
return None
state_path = h.harvest(str(bundle), policy=Policy())
st = json.loads(Path(state_path).read_text(encoding="utf-8"))
svc = next(s for s in st["roles"]["services"] if s["role_name"] == "nginx")
managed_links = svc.get("managed_links") or []
excluded = svc.get("excluded") or []
assert any(p.endswith("/default") for p in calls)
assert any(p.endswith("/ok") for p in calls)
assert {ml["path"] for ml in managed_links} == {"/etc/nginx/sites-enabled/ok"}
assert {ex["path"] for ex in excluded} == {"/etc/nginx/sites-enabled/default"}
assert (
next(ex["reason"] for ex in excluded if ex["path"].endswith("/default"))
== "denied_path"
)
def test_harvest_captures_apache2_enabled_symlinks(monkeypatch, tmp_path: Path):
bundle = tmp_path / "bundle"
_base_monkeypatches(monkeypatch, unit="apache2.service")
dirs = {
"/etc",
"/etc/apache2/conf-enabled",
"/etc/apache2/mods-enabled",
"/etc/apache2/sites-enabled",
}
links = {
"/etc/apache2/sites-enabled/000-default.conf": "../sites-available/000-default.conf",
"/etc/apache2/mods-enabled/rewrite.load": "../mods-available/rewrite.load",
"/etc/apache2/conf-enabled/security.conf": "../conf-available/security.conf",
}
monkeypatch.setattr(h.os.path, "isdir", lambda p: p in dirs)
monkeypatch.setattr(h.os.path, "islink", lambda p: p in links)
monkeypatch.setattr(h.os, "readlink", lambda p: links[p])
def fake_glob(pat: str):
if pat == "/etc/apache2/sites-enabled/*":
return ["/etc/apache2/sites-enabled/000-default.conf"]
if pat == "/etc/apache2/mods-enabled/*":
return ["/etc/apache2/mods-enabled/rewrite.load"]
if pat == "/etc/apache2/conf-enabled/*":
return ["/etc/apache2/conf-enabled/security.conf"]
return []
monkeypatch.setattr(h.glob, "glob", fake_glob)
state_path = h.harvest(str(bundle), policy=AllowAllPolicy())
st = json.loads(Path(state_path).read_text(encoding="utf-8"))
svc = next(s for s in st["roles"]["services"] if s["role_name"] == "apache2")
managed_links = svc.get("managed_links") or []
assert {ml["path"] for ml in managed_links} == set(links.keys())
assert {ml["target"] for ml in managed_links} == set(links.values())
assert all(ml["reason"] == "enabled_symlink" for ml in managed_links)

10
tests/test_ignore.py
View file

@ -1,10 +0,0 @@
from enroll.ignore import IgnorePolicy
def test_ignore_policy_denies_common_backup_files():
pol = IgnorePolicy()
assert pol.deny_reason("/etc/shadow-") == "backup_file"
assert pol.deny_reason("/etc/passwd-") == "backup_file"
assert pol.deny_reason("/etc/group-") == "backup_file"
assert pol.deny_reason("/etc/something~") == "backup_file"
assert pol.deny_reason("/foobar") == "unreadable"

56
tests/test_ignore_dir.py
View file

@ -1,56 +0,0 @@
from __future__ import annotations
from pathlib import Path
def test_iter_effective_lines_skips_comments_and_block_comments():
from enroll.ignore import IgnorePolicy
policy = IgnorePolicy(deny_globs=[])
content = b"""
# comment
; semi
// slash
* c-star
valid=1
/* block
ignored=1
*/
valid=2
"""
lines = [l.strip() for l in policy.iter_effective_lines(content)]
assert lines == [b"valid=1", b"valid=2"]
def test_deny_reason_dir_behaviour(tmp_path: Path):
from enroll.ignore import IgnorePolicy
# Use an absolute pattern matching our temporary path.
deny_glob = str(tmp_path / "deny") + "/*"
pol = IgnorePolicy(deny_globs=[deny_glob], dangerous=False)
d = tmp_path / "dir"
d.mkdir()
f = tmp_path / "file"
f.write_text("x", encoding="utf-8")
link = tmp_path / "link"
link.symlink_to(d)
assert pol.deny_reason_dir(str(d)) is None
assert pol.deny_reason_dir(str(link)) == "symlink"
assert pol.deny_reason_dir(str(f)) == "not_directory"
# Denied by glob.
deny_path = tmp_path / "deny" / "x"
deny_path.mkdir(parents=True)
assert pol.deny_reason_dir(str(deny_path)) == "denied_path"
# Missing/unreadable.
assert pol.deny_reason_dir(str(tmp_path / "missing")) == "unreadable"
# Dangerous disables deny_globs.
pol2 = IgnorePolicy(deny_globs=[deny_glob], dangerous=True)
assert pol2.deny_reason_dir(str(deny_path)) is None

145
tests/test_jinjaturtle.py
View file

@ -1,145 +0,0 @@
import json
from pathlib import Path
import enroll.manifest as manifest_mod
from enroll.jinjaturtle import JinjifyResult
def test_manifest_uses_jinjaturtle_templates_and_does_not_copy_raw(
monkeypatch, tmp_path: Path
):
"""If jinjaturtle can templatisize a file, we should store a template in the role
and avoid keeping the raw file copy in the destination files area.
This test stubs out jinjaturtle execution so it doesn't depend on the external tool.
"""
bundle = tmp_path / "bundle"
out = tmp_path / "ansible"
# A jinjaturtle-compatible config file.
(bundle / "artifacts" / "foo" / "etc").mkdir(parents=True, exist_ok=True)
(bundle / "artifacts" / "foo" / "etc" / "foo.ini").write_text(
"[main]\nkey = 1\n", encoding="utf-8"
)
state = {
"schema_version": 3,
"host": {"hostname": "test", "os": "debian", "pkg_backend": "dpkg"},
"inventory": {
"packages": {
"foo": {
"version": "1.0",
"arches": [],
"installations": [{"version": "1.0", "arch": "amd64"}],
"observed_via": [{"kind": "systemd_unit", "ref": "foo.service"}],
"roles": ["foo"],
}
}
},
"roles": {
"users": {
"role_name": "users",
"users": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
"services": [
{
"unit": "foo.service",
"role_name": "foo",
"packages": ["foo"],
"active_state": "inactive",
"sub_state": "dead",
"unit_file_state": "disabled",
"condition_result": "no",
"managed_files": [
{
"path": "/etc/foo.ini",
"src_rel": "etc/foo.ini",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "modified_conffile",
}
],
"excluded": [],
"notes": [],
}
],
"packages": [],
"apt_config": {
"role_name": "apt_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"etc_custom": {
"role_name": "etc_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"usr_local_custom": {
"role_name": "usr_local_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"extra_paths": {
"role_name": "extra_paths",
"include_patterns": [],
"exclude_patterns": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
},
}
bundle.mkdir(parents=True, exist_ok=True)
(bundle / "state.json").write_text(json.dumps(state, indent=2), encoding="utf-8")
# Pretend jinjaturtle exists.
monkeypatch.setattr(
manifest_mod, "find_jinjaturtle_cmd", lambda: "/usr/bin/jinjaturtle"
)
# Stub jinjaturtle output.
def fake_run_jinjaturtle(
jt_exe: str, src_path: str, *, role_name: str, force_format=None
):
assert role_name == "foo"
return JinjifyResult(
template_text="[main]\nkey = {{ foo_key }}\n",
vars_text="foo_key: 1\n",
)
monkeypatch.setattr(manifest_mod, "run_jinjaturtle", fake_run_jinjaturtle)
manifest_mod.manifest(str(bundle), str(out), jinjaturtle="on")
# Template should exist in the role.
assert (out / "roles" / "foo" / "templates" / "etc" / "foo.ini.j2").exists()
# Raw file should NOT be copied into role files/ because it was templatised.
assert not (out / "roles" / "foo" / "files" / "etc" / "foo.ini").exists()
# Defaults should include jinjaturtle vars.
defaults = (out / "roles" / "foo" / "defaults" / "main.yml").read_text(
encoding="utf-8"
)
assert "foo_key: 1" in defaults
def test_openssh_paths_are_jinjaturtle_supported_and_forced_to_ssh() -> None:
from enroll.jinjaturtle import can_jinjify_path, infer_other_formats
assert infer_other_formats("/etc/ssh/sshd_config") == "ssh"
assert infer_other_formats("/etc/ssh/ssh_config") == "ssh"
assert infer_other_formats("/etc/ssh/sshd_config.d/50-hardening.conf") == "ssh"
assert infer_other_formats("/etc/ssh/ssh_config.d/99-proxy.conf") == "ssh"
assert can_jinjify_path("/etc/ssh/sshd_config")
assert can_jinjify_path("/etc/ssh/ssh_config")

833
tests/test_manifest.py
View file

@ -1,12 +1,7 @@
import json
from pathlib import Path
import os
import stat
import tarfile
import pytest
import enroll.manifest as manifest
from enroll.manifest import manifest
def test_manifest_writes_roles_and_playbook_with_clean_when(tmp_path: Path):
@ -18,27 +13,7 @@ def test_manifest_writes_roles_and_playbook_with_clean_when(tmp_path: Path):
)
state = {
"schema_version": 3,
"host": {"hostname": "test", "os": "debian", "pkg_backend": "dpkg"},
"inventory": {
"packages": {
"foo": {
"version": "1.0",
"arches": [],
"installations": [{"version": "1.0", "arch": "amd64"}],
"observed_via": [{"kind": "systemd_unit", "ref": "foo.service"}],
"roles": ["foo"],
},
"curl": {
"version": "8.0",
"arches": [],
"installations": [{"version": "8.0", "arch": "amd64"}],
"observed_via": [{"kind": "package_role", "ref": "curl"}],
"roles": ["curl"],
},
}
},
"roles": {
"host": {"hostname": "test", "os": "debian"},
"users": {
"role_name": "users",
"users": [
@ -57,6 +32,21 @@ def test_manifest_writes_roles_and_playbook_with_clean_when(tmp_path: Path):
"excluded": [],
"notes": [],
},
"etc_custom": {
"role_name": "etc_custom",
"managed_files": [
{
"path": "/etc/default/keyboard",
"src_rel": "etc/default/keyboard",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "custom_unowned",
}
],
"excluded": [],
"notes": [],
},
"services": [
{
"unit": "foo.service",
@ -80,7 +70,7 @@ def test_manifest_writes_roles_and_playbook_with_clean_when(tmp_path: Path):
"notes": [],
}
],
"packages": [
"package_roles": [
{
"package": "curl",
"role_name": "curl",
@ -89,65 +79,6 @@ def test_manifest_writes_roles_and_playbook_with_clean_when(tmp_path: Path):
"notes": [],
}
],
"apt_config": {
"role_name": "apt_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"dnf_config": {
"role_name": "dnf_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"etc_custom": {
"role_name": "etc_custom",
"managed_files": [
{
"path": "/etc/default/keyboard",
"src_rel": "etc/default/keyboard",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "custom_unowned",
}
],
"excluded": [],
"notes": [],
},
"usr_local_custom": {
"role_name": "usr_local_custom",
"managed_files": [
{
"path": "/usr/local/etc/myapp.conf",
"src_rel": "usr/local/etc/myapp.conf",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "usr_local_etc_custom",
},
{
"path": "/usr/local/bin/myscript",
"src_rel": "usr/local/bin/myscript",
"owner": "root",
"group": "root",
"mode": "0755",
"reason": "usr_local_bin_script",
},
],
"excluded": [],
"notes": [],
},
"extra_paths": {
"role_name": "extra_paths",
"include_patterns": [],
"exclude_patterns": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
},
}
bundle.mkdir(parents=True, exist_ok=True)
@ -161,38 +92,12 @@ def test_manifest_writes_roles_and_playbook_with_clean_when(tmp_path: Path):
"kbd", encoding="utf-8"
)
# Create artifacts for usr_local_custom files so copy works
(bundle / "artifacts" / "usr_local_custom" / "usr" / "local" / "etc").mkdir(
parents=True, exist_ok=True
)
(
bundle
/ "artifacts"
/ "usr_local_custom"
/ "usr"
/ "local"
/ "etc"
/ "myapp.conf"
).write_text("myapp=1\n", encoding="utf-8")
(bundle / "artifacts" / "usr_local_custom" / "usr" / "local" / "bin").mkdir(
parents=True, exist_ok=True
)
(
bundle / "artifacts" / "usr_local_custom" / "usr" / "local" / "bin" / "myscript"
).write_text("#!/bin/sh\necho hi\n", encoding="utf-8")
manifest(str(bundle), str(out))
manifest.manifest(str(bundle), str(out))
# Service role: systemd management should be gated on foo_manage_unit and a probe.
# Service role: conditional start must be a clean Ansible expression
tasks = (out / "roles" / "foo" / "tasks" / "main.yml").read_text(encoding="utf-8")
assert "- name: Probe whether systemd unit exists and is manageable" in tasks
assert "when: foo_manage_unit | default(false)" in tasks
assert (
"when:\n - foo_manage_unit | default(false)\n - _unit_probe is succeeded\n"
in tasks
)
# Ensure we didn't emit deprecated/broken '{{ }}' delimiters in when: lines.
assert "when: foo_start | bool" in tasks
# Ensure we didn't emit deprecated/broken '{{ }}' delimiters in when:
for line in tasks.splitlines():
if line.lstrip().startswith("when:"):
assert "{{" not in line and "}}" not in line
@ -200,695 +105,11 @@ def test_manifest_writes_roles_and_playbook_with_clean_when(tmp_path: Path):
defaults = (out / "roles" / "foo" / "defaults" / "main.yml").read_text(
encoding="utf-8"
)
assert "foo_manage_unit: true" in defaults
assert "foo_systemd_enabled: true" in defaults
assert "foo_systemd_state: stopped" in defaults
assert "foo_start: false" in defaults
# Playbook should include users, etc_custom, packages, and services
pb = (out / "playbook.yml").read_text(encoding="utf-8")
assert "role: users" in pb
assert "role: etc_custom" in pb
assert "role: usr_local_custom" in pb
assert "role: curl" in pb
assert "role: foo" in pb
def test_manifest_site_mode_creates_host_inventory_and_raw_files(tmp_path: Path):
"""In --fqdn mode, host-specific state goes into inventory/host_vars."""
fqdn = "host1.example.test"
bundle = tmp_path / "bundle"
out = tmp_path / "ansible"
# Artifacts for a service-managed file.
(bundle / "artifacts" / "foo" / "etc").mkdir(parents=True, exist_ok=True)
(bundle / "artifacts" / "foo" / "etc" / "foo.conf").write_text(
"x", encoding="utf-8"
)
# Artifacts for etc_custom file so copy works.
(bundle / "artifacts" / "etc_custom" / "etc" / "default").mkdir(
parents=True, exist_ok=True
)
(bundle / "artifacts" / "etc_custom" / "etc" / "default" / "keyboard").write_text(
"kbd", encoding="utf-8"
)
state = {
"schema_version": 3,
"host": {"hostname": "test", "os": "debian", "pkg_backend": "dpkg"},
"inventory": {
"packages": {
"foo": {
"version": "1.0",
"arches": [],
"installations": [{"version": "1.0", "arch": "amd64"}],
"observed_via": [{"kind": "systemd_unit", "ref": "foo.service"}],
"roles": ["foo"],
}
}
},
"roles": {
"users": {
"role_name": "users",
"users": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
"services": [
{
"unit": "foo.service",
"role_name": "foo",
"packages": ["foo"],
"active_state": "active",
"sub_state": "running",
"unit_file_state": "enabled",
"condition_result": "yes",
"managed_files": [
{
"path": "/etc/foo.conf",
"src_rel": "etc/foo.conf",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "modified_conffile",
}
],
"excluded": [],
"notes": [],
}
],
"packages": [],
"apt_config": {
"role_name": "apt_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"dnf_config": {
"role_name": "dnf_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"etc_custom": {
"role_name": "etc_custom",
"managed_files": [
{
"path": "/etc/default/keyboard",
"src_rel": "etc/default/keyboard",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "custom_unowned",
}
],
"excluded": [],
"notes": [],
},
"usr_local_custom": {
"role_name": "usr_local_custom",
"managed_files": [
{
"path": "/usr/local/etc/myapp.conf",
"src_rel": "usr/local/etc/myapp.conf",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "usr_local_etc_custom",
}
],
"excluded": [],
"notes": [],
},
"extra_paths": {
"role_name": "extra_paths",
"include_patterns": [],
"exclude_patterns": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
},
}
bundle.mkdir(parents=True, exist_ok=True)
(bundle / "state.json").write_text(json.dumps(state, indent=2), encoding="utf-8")
# Artifacts for usr_local_custom file so copy works.
(bundle / "artifacts" / "usr_local_custom" / "usr" / "local" / "etc").mkdir(
parents=True, exist_ok=True
)
(
bundle
/ "artifacts"
/ "usr_local_custom"
/ "usr"
/ "local"
/ "etc"
/ "myapp.conf"
).write_text("myapp=1\n", encoding="utf-8")
manifest.manifest(str(bundle), str(out), fqdn=fqdn)
# Host playbook exists.
assert (out / "playbooks" / f"{fqdn}.yml").exists()
# Role defaults are safe/host-agnostic in site mode.
foo_defaults = (out / "roles" / "foo" / "defaults" / "main.yml").read_text(
encoding="utf-8"
)
assert "foo_packages: []" in foo_defaults
assert "foo_managed_files: []" in foo_defaults
assert "foo_manage_unit: false" in foo_defaults
# Host vars contain host-specific state.
foo_hostvars = (out / "inventory" / "host_vars" / fqdn / "foo.yml").read_text(
encoding="utf-8"
)
assert "foo_packages" in foo_hostvars
assert "foo_managed_files" in foo_hostvars
assert "foo_manage_unit: true" in foo_hostvars
assert "foo_systemd_state: started" in foo_hostvars
# Non-templated raw config is stored per-host under .files.
assert (
out / "inventory" / "host_vars" / fqdn / "foo" / ".files" / "etc" / "foo.conf"
).exists()
def test_copy2_replace_overwrites_readonly_destination(tmp_path: Path):
"""Merging into an existing manifest should tolerate read-only files.
Some harvested artifacts (e.g. private keys) may be mode 0400. If a previous
run copied them into the destination tree, a subsequent run must still be
able to update/replace them.
"""
import os
import stat
from enroll.manifest import _copy2_replace
src = tmp_path / "src"
dst = tmp_path / "dst"
src.write_text("new", encoding="utf-8")
dst.write_text("old", encoding="utf-8")
os.chmod(dst, 0o400)
_copy2_replace(str(src), str(dst))
assert dst.read_text(encoding="utf-8") == "new"
mode = stat.S_IMODE(dst.stat().st_mode)
assert mode & stat.S_IWUSR # destination should remain mergeable
def test_manifest_includes_dnf_config_role_when_present(tmp_path: Path):
bundle = tmp_path / "bundle"
out = tmp_path / "ansible"
# Create a dnf_config artifact.
(bundle / "artifacts" / "dnf_config" / "etc" / "dnf").mkdir(
parents=True, exist_ok=True
)
(bundle / "artifacts" / "dnf_config" / "etc" / "dnf" / "dnf.conf").write_text(
"[main]\n", encoding="utf-8"
)
state = {
"schema_version": 3,
"host": {"hostname": "test", "os": "redhat", "pkg_backend": "rpm"},
"inventory": {
"packages": {
"dnf": {
"version": "4.0",
"arches": [],
"installations": [{"version": "4.0", "arch": "x86_64"}],
"observed_via": [{"kind": "dnf_config"}],
"roles": [],
}
}
},
"roles": {
"users": {
"role_name": "users",
"users": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
"services": [],
"packages": [],
"apt_config": {
"role_name": "apt_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"dnf_config": {
"role_name": "dnf_config",
"managed_files": [
{
"path": "/etc/dnf/dnf.conf",
"src_rel": "etc/dnf/dnf.conf",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "dnf_config",
}
],
"excluded": [],
"notes": [],
},
"etc_custom": {
"role_name": "etc_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"usr_local_custom": {
"role_name": "usr_local_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"extra_paths": {
"role_name": "extra_paths",
"include_patterns": [],
"exclude_patterns": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
},
}
bundle.mkdir(parents=True, exist_ok=True)
(bundle / "state.json").write_text(json.dumps(state, indent=2), encoding="utf-8")
manifest.manifest(str(bundle), str(out))
pb = (out / "playbook.yml").read_text(encoding="utf-8")
assert "role: dnf_config" in pb
tasks = (out / "roles" / "dnf_config" / "tasks" / "main.yml").read_text(
encoding="utf-8"
)
# Ensure the role exists and contains some file deployment logic.
assert "Deploy any other managed files" in tasks
def test_render_install_packages_tasks_contains_dnf_branch():
from enroll.manifest import _render_install_packages_tasks
txt = _render_install_packages_tasks("role", "role")
assert "ansible.builtin.apt" in txt
assert "ansible.builtin.dnf" in txt
assert "ansible.builtin.package" in txt
assert "pkg_mgr" in txt
def test_manifest_orders_cron_and_logrotate_at_playbook_tail(tmp_path: Path):
"""Cron/logrotate roles should appear at the end.
The cron role may restore per-user crontabs under /var/spool, so it should
run after users have been created.
"""
bundle = tmp_path / "bundle"
out = tmp_path / "ansible"
state = {
"schema_version": 3,
"host": {"hostname": "test", "os": "debian", "pkg_backend": "dpkg"},
"inventory": {"packages": {}},
"roles": {
"users": {
"role_name": "users",
"users": [{"name": "alice"}],
"managed_files": [],
"excluded": [],
"notes": [],
},
"services": [],
"packages": [
{
"package": "curl",
"role_name": "curl",
"managed_files": [],
"excluded": [],
"notes": [],
},
{
"package": "cron",
"role_name": "cron",
"managed_files": [
{
"path": "/var/spool/cron/crontabs/alice",
"src_rel": "var/spool/cron/crontabs/alice",
"owner": "alice",
"group": "root",
"mode": "0600",
"reason": "system_cron",
}
],
"excluded": [],
"notes": [],
},
{
"package": "logrotate",
"role_name": "logrotate",
"managed_files": [
{
"path": "/etc/logrotate.conf",
"src_rel": "etc/logrotate.conf",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "system_logrotate",
}
],
"excluded": [],
"notes": [],
},
],
"apt_config": {
"role_name": "apt_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"dnf_config": {
"role_name": "dnf_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"etc_custom": {
"role_name": "etc_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"usr_local_custom": {
"role_name": "usr_local_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"extra_paths": {
"role_name": "extra_paths",
"include_patterns": [],
"exclude_patterns": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
},
}
# Minimal artifacts for managed files.
(bundle / "artifacts" / "cron" / "var" / "spool" / "cron" / "crontabs").mkdir(
parents=True, exist_ok=True
)
(
bundle / "artifacts" / "cron" / "var" / "spool" / "cron" / "crontabs" / "alice"
).write_text("@daily echo hi\n", encoding="utf-8")
(bundle / "artifacts" / "logrotate" / "etc").mkdir(parents=True, exist_ok=True)
(bundle / "artifacts" / "logrotate" / "etc" / "logrotate.conf").write_text(
"weekly\n", encoding="utf-8"
)
bundle.mkdir(parents=True, exist_ok=True)
(bundle / "state.json").write_text(json.dumps(state, indent=2), encoding="utf-8")
manifest.manifest(str(bundle), str(out))
pb = (out / "playbook.yml").read_text(encoding="utf-8").splitlines()
# Roles are emitted as indented list items under the `roles:` key.
roles = [
ln.strip().removeprefix("- ").strip() for ln in pb if ln.startswith(" - ")
]
# Ensure tail ordering.
assert roles[-2:] == ["role: cron", "role: logrotate"]
assert "role: users" in roles
assert roles.index("role: users") < roles.index("role: cron")
def test_yaml_helpers_fallback_when_yaml_unavailable(monkeypatch):
monkeypatch.setattr(manifest, "_try_yaml", lambda: None)
assert manifest._yaml_load_mapping("foo: 1\n") == {}
out = manifest._yaml_dump_mapping({"b": 2, "a": 1})
# Best-effort fallback is key: repr(value)
assert out.splitlines()[0].startswith("a: ")
assert out.endswith("\n")
def test_copy2_replace_makes_readonly_sources_user_writable(
monkeypatch, tmp_path: Path
):
src = tmp_path / "src.txt"
dst = tmp_path / "dst.txt"
src.write_text("hello", encoding="utf-8")
# Make source read-only; copy2 preserves mode, so tmp will be read-only too.
os.chmod(src, 0o444)
manifest._copy2_replace(str(src), str(dst))
st = os.stat(dst, follow_symlinks=False)
assert stat.S_IMODE(st.st_mode) & stat.S_IWUSR
def test_prepare_bundle_dir_sops_decrypts_and_extracts(monkeypatch, tmp_path: Path):
enc = tmp_path / "harvest.tar.gz.sops"
enc.write_text("ignored", encoding="utf-8")
def fake_require():
return None
def fake_decrypt(src: str, dst: str, *, mode: int = 0o600):
# Create a minimal tar.gz with a state.json file.
with tarfile.open(dst, "w:gz") as tf:
p = tmp_path / "state.json"
p.write_text("{}", encoding="utf-8")
tf.add(p, arcname="state.json")
monkeypatch.setattr(manifest, "require_sops_cmd", fake_require)
monkeypatch.setattr(manifest, "decrypt_file_binary_to", fake_decrypt)
bundle_dir, td = manifest._prepare_bundle_dir(str(enc), sops_mode=True)
try:
assert (Path(bundle_dir) / "state.json").exists()
finally:
td.cleanup()
def test_prepare_bundle_dir_rejects_non_dir_without_sops(tmp_path: Path):
fp = tmp_path / "bundle.tar.gz"
fp.write_text("x", encoding="utf-8")
with pytest.raises(RuntimeError):
manifest._prepare_bundle_dir(str(fp), sops_mode=False)
def test_tar_dir_to_with_progress_writes_progress_when_tty(monkeypatch, tmp_path: Path):
src = tmp_path / "dir"
src.mkdir()
(src / "a.txt").write_text("a", encoding="utf-8")
(src / "b.txt").write_text("b", encoding="utf-8")
out = tmp_path / "out.tar.gz"
writes: list[bytes] = []
monkeypatch.setattr(manifest.os, "isatty", lambda fd: True)
monkeypatch.setattr(manifest.os, "write", lambda fd, b: writes.append(b) or len(b))
manifest._tar_dir_to_with_progress(str(src), str(out), desc="tarring")
assert out.exists()
assert writes # progress was written
assert writes[-1].endswith(b"\n")
def test_encrypt_manifest_out_dir_to_sops_handles_missing_tmp_cleanup(
monkeypatch, tmp_path: Path
):
src_dir = tmp_path / "manifest"
src_dir.mkdir()
(src_dir / "x.txt").write_text("x", encoding="utf-8")
out = tmp_path / "manifest.tar.gz.sops"
monkeypatch.setattr(manifest, "require_sops_cmd", lambda: None)
def fake_encrypt(in_fp, out_fp, *args, **kwargs):
Path(out_fp).write_text("enc", encoding="utf-8")
monkeypatch.setattr(manifest, "encrypt_file_binary", fake_encrypt)
# Simulate race where tmp tar is already removed.
monkeypatch.setattr(
manifest.os, "unlink", lambda p: (_ for _ in ()).throw(FileNotFoundError())
)
res = manifest._encrypt_manifest_out_dir_to_sops(str(src_dir), str(out), ["ABC"]) # type: ignore[arg-type]
assert str(res).endswith(".sops")
assert out.exists()
def test_manifest_applies_jinjaturtle_to_jinjifyable_managed_file(
monkeypatch, tmp_path: Path
):
# Create a minimal bundle with just an apt_config snapshot.
bundle = tmp_path / "bundle"
(bundle / "artifacts" / "apt_config" / "etc" / "apt").mkdir(parents=True)
(bundle / "artifacts" / "apt_config" / "etc" / "apt" / "foo.ini").write_text(
"key=VALUE\n", encoding="utf-8"
)
state = {
"schema_version": 1,
"inventory": {"packages": {}},
"roles": {
"services": [],
"packages": [],
"apt_config": {
"role_name": "apt_config",
"managed_files": [
{
"path": "/etc/apt/foo.ini",
"src_rel": "etc/apt/foo.ini",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "apt_config",
}
],
"managed_dirs": [],
"excluded": [],
"notes": [],
},
},
}
(bundle / "state.json").write_text(
__import__("json").dumps(state), encoding="utf-8"
)
monkeypatch.setattr(manifest, "find_jinjaturtle_cmd", lambda: "jinjaturtle")
class _Res:
template_text = "key={{ foo }}\n"
vars_text = "foo: 123\n"
monkeypatch.setattr(manifest, "run_jinjaturtle", lambda *a, **k: _Res())
out_dir = tmp_path / "out"
manifest.manifest(str(bundle), str(out_dir), jinjaturtle="on")
tmpl = out_dir / "roles" / "apt_config" / "templates" / "etc" / "apt" / "foo.ini.j2"
assert tmpl.exists()
assert "{{ foo }}" in tmpl.read_text(encoding="utf-8")
defaults = out_dir / "roles" / "apt_config" / "defaults" / "main.yml"
txt = defaults.read_text(encoding="utf-8")
assert "foo: 123" in txt
# Non-templated file should not exist under files/.
assert not (
out_dir / "roles" / "apt_config" / "files" / "etc" / "apt" / "foo.ini"
).exists()
def test_manifest_writes_firewall_runtime_role(tmp_path: Path):
bundle = tmp_path / "bundle"
out = tmp_path / "ansible"
(bundle / "artifacts" / "firewall_runtime" / "firewall").mkdir(
parents=True, exist_ok=True
)
(bundle / "artifacts" / "firewall_runtime" / "firewall" / "ipset.save").write_text(
"create blocklist hash:ip family inet\nadd blocklist 203.0.113.10\n",
encoding="utf-8",
)
(bundle / "artifacts" / "firewall_runtime" / "firewall" / "iptables.v4").write_text(
"*filter\n:INPUT DROP [0:0]\n-A INPUT -m set --match-set blocklist src -j DROP\nCOMMIT\n",
encoding="utf-8",
)
state = {
"schema_version": 3,
"host": {"hostname": "test", "os": "debian", "pkg_backend": "dpkg"},
"inventory": {"packages": {}},
"roles": {
"users": {
"role_name": "users",
"users": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
"services": [],
"packages": [],
"apt_config": {
"role_name": "apt_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"dnf_config": {
"role_name": "dnf_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"firewall_runtime": {
"role_name": "firewall_runtime",
"packages": ["ipset", "iptables"],
"ipset_save": "firewall/ipset.save",
"ipset_sets": ["blocklist"],
"iptables_v4_save": "firewall/iptables.v4",
"iptables_v6_save": None,
"notes": [],
},
"etc_custom": {
"role_name": "etc_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"usr_local_custom": {
"role_name": "usr_local_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"extra_paths": {
"role_name": "extra_paths",
"include_patterns": [],
"exclude_patterns": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
},
}
(bundle / "state.json").write_text(json.dumps(state, indent=2), encoding="utf-8")
manifest.manifest(str(bundle), str(out))
tasks = (out / "roles" / "firewall_runtime" / "tasks" / "main.yml").read_text(
encoding="utf-8"
)
assert "ipset restore -exist" in tasks
assert "iptables-restore /etc/enroll/firewall/iptables.v4" in tasks
assert "ipset flush {{ item }}" in tasks
defaults = (out / "roles" / "firewall_runtime" / "defaults" / "main.yml").read_text(
encoding="utf-8"
)
assert "firewall_runtime_ipset_sets:" in defaults
assert "- blocklist" in defaults
assert "firewall_runtime_restore_iptables: true" in defaults
pb = (out / "playbook.yml").read_text(encoding="utf-8")
assert "role: firewall_runtime" in pb
assert (
out / "roles" / "firewall_runtime" / "files" / "firewall" / "ipset.save"
).exists()
assert "- users" in pb
assert "- etc_custom" in pb
assert "- curl" in pb
assert "- foo" in pb

96
tests/test_manifest_symlinks.py
View file

@ -1,96 +0,0 @@
import json
from pathlib import Path
import enroll.manifest as manifest
def test_manifest_emits_symlink_tasks_and_vars(tmp_path: Path):
bundle = tmp_path / "bundle"
out = tmp_path / "ansible"
state = {
"host": {"hostname": "test", "os": "debian", "pkg_backend": "dpkg"},
"inventory": {"packages": {}},
"roles": {
"users": {
"role_name": "users",
"users": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
"services": [
{
"unit": "nginx.service",
"role_name": "nginx",
"packages": ["nginx"],
"active_state": "active",
"sub_state": "running",
"unit_file_state": "enabled",
"condition_result": None,
"managed_files": [],
"managed_links": [
{
"path": "/etc/nginx/sites-enabled/default",
"target": "../sites-available/default",
"reason": "enabled_symlink",
}
],
"excluded": [],
"notes": [],
}
],
"packages": [],
"apt_config": {
"role_name": "apt_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"dnf_config": {
"role_name": "dnf_config",
"managed_files": [],
"excluded": [],
"notes": [],
},
"etc_custom": {
"role_name": "etc_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"usr_local_custom": {
"role_name": "usr_local_custom",
"managed_files": [],
"excluded": [],
"notes": [],
},
"extra_paths": {
"role_name": "extra_paths",
"include_patterns": [],
"exclude_patterns": [],
"managed_files": [],
"managed_links": [],
"excluded": [],
"notes": [],
},
},
}
bundle.mkdir(parents=True, exist_ok=True)
(bundle / "artifacts").mkdir(parents=True, exist_ok=True)
(bundle / "state.json").write_text(json.dumps(state), encoding="utf-8")
manifest.manifest(str(bundle), str(out))
tasks = (out / "roles" / "nginx" / "tasks" / "main.yml").read_text(encoding="utf-8")
assert "- name: Ensure managed symlinks exist" in tasks
assert 'loop: "{{ nginx_managed_links | default([]) }}"' in tasks
defaults = (out / "roles" / "nginx" / "defaults" / "main.yml").read_text(
encoding="utf-8"
)
# The role defaults should include the converted link mapping.
assert "nginx_managed_links:" in defaults
assert "dest: /etc/nginx/sites-enabled/default" in defaults
assert "src: ../sites-available/default" in defaults

416
tests/test_misc_coverage.py
View file

@ -1,416 +0,0 @@
from __future__ import annotations
import json
import os
import stat
import subprocess
import sys
import types
from pathlib import Path
from types import SimpleNamespace
import pytest
from enroll.cache import _safe_component, new_harvest_cache_dir
from enroll.ignore import IgnorePolicy
from enroll.sopsutil import (
SopsError,
_pgp_arg,
decrypt_file_binary_to,
encrypt_file_binary,
)
def test_safe_component_sanitizes_and_bounds_length():
assert _safe_component(" ") == "unknown"
assert _safe_component("a/b c") == "a_b_c"
assert _safe_component("x" * 200) == "x" * 64
def test_new_harvest_cache_dir_uses_xdg_cache_home(tmp_path: Path, monkeypatch):
monkeypatch.setenv("XDG_CACHE_HOME", str(tmp_path / "xdg"))
hc = new_harvest_cache_dir(hint="my host/01")
assert hc.dir.exists()
assert "my_host_01" in hc.dir.name
assert str(hc.dir).startswith(str(tmp_path / "xdg"))
# best-effort: ensure directory is not world-readable on typical FS
try:
mode = stat.S_IMODE(hc.dir.stat().st_mode)
assert mode & 0o077 == 0
except OSError:
pass
def test_ignore_policy_denies_binary_and_sensitive_content(tmp_path: Path):
p_bin = tmp_path / "binfile"
p_bin.write_bytes(b"abc\x00def")
assert IgnorePolicy().deny_reason(str(p_bin)) == "binary_like"
p_secret = tmp_path / "secret.conf"
p_secret.write_text("password=foo\n", encoding="utf-8")
assert IgnorePolicy().deny_reason(str(p_secret)) == "sensitive_content"
# dangerous mode disables heuristic scanning (but still checks file-ness/size)
assert IgnorePolicy(dangerous=True).deny_reason(str(p_secret)) is None
def test_ignore_policy_denies_usr_local_shadow_by_glob():
# This should short-circuit before stat() (path doesn't need to exist).
assert IgnorePolicy().deny_reason("/usr/local/etc/shadow") == "denied_path"
def test_sops_pgp_arg_and_encrypt_decrypt_roundtrip(tmp_path: Path, monkeypatch):
assert _pgp_arg([" ABC ", "DEF"]) == "ABC,DEF"
with pytest.raises(SopsError):
_pgp_arg([])
# Stub out sops and subprocess.
import enroll.sopsutil as s
monkeypatch.setattr(s, "require_sops_cmd", lambda: "sops")
class R:
def __init__(self, rc: int, out: bytes, err: bytes = b""):
self.returncode = rc
self.stdout = out
self.stderr = err
calls = []
def fake_run(cmd, capture_output, check):
calls.append(cmd)
# Return a deterministic payload so we can assert file writes.
if "--encrypt" in cmd:
return R(0, b"ENCRYPTED")
if "--decrypt" in cmd:
return R(0, b"PLAINTEXT")
return R(1, b"", b"bad")
monkeypatch.setattr(s.subprocess, "run", fake_run)
src = tmp_path / "src.bin"
src.write_bytes(b"x")
enc = tmp_path / "out.sops"
dec = tmp_path / "out.bin"
encrypt_file_binary(src, enc, pgp_fingerprints=["ABC"], mode=0o600)
assert enc.read_bytes() == b"ENCRYPTED"
decrypt_file_binary_to(enc, dec, mode=0o644)
assert dec.read_bytes() == b"PLAINTEXT"
# Sanity: we invoked encrypt and decrypt.
assert any("--encrypt" in c for c in calls)
assert any("--decrypt" in c for c in calls)
def test_cache_dir_defaults_to_home_cache(monkeypatch, tmp_path: Path):
# Ensure default path uses ~/.cache when XDG_CACHE_HOME is unset.
from enroll.cache import enroll_cache_dir
monkeypatch.delenv("XDG_CACHE_HOME", raising=False)
monkeypatch.setattr(Path, "home", lambda: tmp_path)
p = enroll_cache_dir()
assert str(p).startswith(str(tmp_path))
assert p.name == "enroll"
def test_harvest_cache_state_json_property(tmp_path: Path):
from enroll.cache import HarvestCache
hc = HarvestCache(tmp_path / "h1")
assert hc.state_json == hc.dir / "state.json"
def test_cache_dir_security_rejects_symlink(tmp_path: Path):
from enroll.cache import _ensure_dir_secure
real = tmp_path / "real"
real.mkdir()
link = tmp_path / "link"
link.symlink_to(real, target_is_directory=True)
with pytest.raises(RuntimeError, match="Refusing to use symlink"):
_ensure_dir_secure(link)
def test_cache_dir_chmod_failures_are_ignored(monkeypatch, tmp_path: Path):
from enroll import cache
# Make the cache base path deterministic and writable.
monkeypatch.setattr(cache, "enroll_cache_dir", lambda: tmp_path)
# Force os.chmod to fail to cover the "except OSError: pass" paths.
monkeypatch.setattr(
os, "chmod", lambda *a, **k: (_ for _ in ()).throw(OSError("nope"))
)
hc = cache.new_harvest_cache_dir()
assert hc.dir.exists()
assert hc.dir.is_dir()
def test_stat_triplet_falls_back_to_numeric_ids(monkeypatch, tmp_path: Path):
from enroll.fsutil import stat_triplet
import pwd
import grp
p = tmp_path / "x"
p.write_text("x", encoding="utf-8")
# Force username/group resolution failures.
monkeypatch.setattr(
pwd, "getpwuid", lambda _uid: (_ for _ in ()).throw(KeyError("no user"))
)
monkeypatch.setattr(
grp, "getgrgid", lambda _gid: (_ for _ in ()).throw(KeyError("no group"))
)
owner, group, mode = stat_triplet(str(p))
assert owner.isdigit()
assert group.isdigit()
assert len(mode) == 4
def test_ignore_policy_iter_effective_lines_removes_block_comments():
from enroll.ignore import IgnorePolicy
pol = IgnorePolicy()
data = b"""keep1
/*
drop me
*/
keep2
"""
assert list(pol.iter_effective_lines(data)) == [b"keep1", b"keep2"]
def test_ignore_policy_deny_reason_dir_variants(tmp_path: Path):
from enroll.ignore import IgnorePolicy
pol = IgnorePolicy()
# denied by glob
assert pol.deny_reason_dir("/etc/shadow") == "denied_path"
# symlink rejected
d = tmp_path / "d"
d.mkdir()
link = tmp_path / "l"
link.symlink_to(d, target_is_directory=True)
assert pol.deny_reason_dir(str(link)) == "symlink"
# not a directory
f = tmp_path / "f"
f.write_text("x", encoding="utf-8")
assert pol.deny_reason_dir(str(f)) == "not_directory"
# ok
assert pol.deny_reason_dir(str(d)) is None
def test_run_jinjaturtle_parses_outputs(monkeypatch, tmp_path: Path):
# Fully unit-test enroll.jinjaturtle.run_jinjaturtle by stubbing subprocess.run.
from enroll.jinjaturtle import run_jinjaturtle
def fake_run(cmd, **kwargs): # noqa: ARG001
# cmd includes "-d <defaults> -t <template>"
d_idx = cmd.index("-d") + 1
t_idx = cmd.index("-t") + 1
defaults = Path(cmd[d_idx])
template = Path(cmd[t_idx])
defaults.write_text("---\nfoo: 1\n", encoding="utf-8")
template.write_text("value={{ foo }}\n", encoding="utf-8")
return SimpleNamespace(returncode=0, stdout="ok", stderr="")
monkeypatch.setattr(subprocess, "run", fake_run)
src = tmp_path / "src.ini"
src.write_text("foo=1\n", encoding="utf-8")
res = run_jinjaturtle("/bin/jinjaturtle", str(src), role_name="role1")
assert "foo: 1" in res.vars_text
assert "value=" in res.template_text
def test_run_jinjaturtle_raises_on_failure(monkeypatch, tmp_path: Path):
from enroll.jinjaturtle import run_jinjaturtle
def fake_run(cmd, **kwargs): # noqa: ARG001
return SimpleNamespace(returncode=2, stdout="out", stderr="bad")
monkeypatch.setattr(subprocess, "run", fake_run)
src = tmp_path / "src.ini"
src.write_text("x", encoding="utf-8")
with pytest.raises(RuntimeError, match="jinjaturtle failed"):
run_jinjaturtle("/bin/jinjaturtle", str(src), role_name="role1")
def test_require_sops_cmd_errors_when_missing(monkeypatch):
from enroll.sopsutil import require_sops_cmd, SopsError
monkeypatch.setattr("enroll.sopsutil.shutil.which", lambda _: None)
with pytest.raises(SopsError, match="not found on PATH"):
require_sops_cmd()
def test_get_enroll_version_reports_unknown_on_metadata_failure(monkeypatch):
import enroll.version as v
fake_meta = types.ModuleType("importlib.metadata")
def boom():
raise RuntimeError("boom")
fake_meta.packages_distributions = boom
fake_meta.version = lambda _dist: boom()
monkeypatch.setitem(sys.modules, "importlib.metadata", fake_meta)
assert v.get_enroll_version() == "unknown"
def test_get_enroll_version_returns_unknown_if_importlib_metadata_unavailable(
monkeypatch,
):
import builtins
import enroll.version as v
real_import = builtins.__import__
def fake_import(
name, globals=None, locals=None, fromlist=(), level=0
): # noqa: A002
if name == "importlib.metadata":
raise ImportError("no metadata")
return real_import(name, globals, locals, fromlist, level)
monkeypatch.setattr(builtins, "__import__", fake_import)
assert v.get_enroll_version() == "unknown"
def test_compare_harvests_and_format_report(tmp_path: Path):
from enroll.diff import compare_harvests, format_report
old = tmp_path / "old"
new = tmp_path / "new"
(old / "artifacts").mkdir(parents=True)
(new / "artifacts").mkdir(parents=True)
def write_state(base: Path, state: dict) -> None:
base.mkdir(parents=True, exist_ok=True)
(base / "state.json").write_text(json.dumps(state, indent=2), encoding="utf-8")
# Old bundle: pkg a@1.0, pkg b@1.0, one service, one user, one managed file.
old_state = {
"schema_version": 3,
"host": {"hostname": "h1"},
"inventory": {"packages": {"a": {"version": "1.0"}, "b": {"version": "1.0"}}},
"roles": {
"services": [
{
"unit": "svc.service",
"role_name": "svc",
"packages": ["a"],
"active_state": "inactive",
"sub_state": "dead",
"unit_file_state": "enabled",
"condition_result": None,
"managed_files": [
{
"path": "/etc/foo.conf",
"src_rel": "etc/foo.conf",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "modified_conffile",
}
],
}
],
"packages": [],
"users": {
"role_name": "users",
"users": [{"name": "alice", "shell": "/bin/sh"}],
},
"apt_config": {"role_name": "apt_config", "managed_files": []},
"etc_custom": {"role_name": "etc_custom", "managed_files": []},
"usr_local_custom": {"role_name": "usr_local_custom", "managed_files": []},
"extra_paths": {"role_name": "extra_paths", "managed_files": []},
},
}
(old / "artifacts" / "svc" / "etc").mkdir(parents=True, exist_ok=True)
(old / "artifacts" / "svc" / "etc" / "foo.conf").write_text("old", encoding="utf-8")
write_state(old, old_state)
# New bundle: pkg a@2.0, pkg c@1.0, service changed, user changed, file moved role+content.
new_state = {
"schema_version": 3,
"host": {"hostname": "h2"},
"inventory": {"packages": {"a": {"version": "2.0"}, "c": {"version": "1.0"}}},
"roles": {
"services": [
{
"unit": "svc.service",
"role_name": "svc",
"packages": ["a", "c"],
"active_state": "active",
"sub_state": "running",
"unit_file_state": "enabled",
"condition_result": None,
"managed_files": [],
}
],
"packages": [],
"users": {
"role_name": "users",
"users": [{"name": "alice", "shell": "/bin/bash"}, {"name": "bob"}],
},
"apt_config": {"role_name": "apt_config", "managed_files": []},
"etc_custom": {"role_name": "etc_custom", "managed_files": []},
"usr_local_custom": {"role_name": "usr_local_custom", "managed_files": []},
"extra_paths": {
"role_name": "extra_paths",
"managed_files": [
{
"path": "/etc/foo.conf",
"src_rel": "etc/foo.conf",
"owner": "root",
"group": "root",
"mode": "0600",
"reason": "user_include",
},
{
"path": "/etc/added.conf",
"src_rel": "etc/added.conf",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "user_include",
},
],
},
},
}
(new / "artifacts" / "extra_paths" / "etc").mkdir(parents=True, exist_ok=True)
(new / "artifacts" / "extra_paths" / "etc" / "foo.conf").write_text(
"new", encoding="utf-8"
)
(new / "artifacts" / "extra_paths" / "etc" / "added.conf").write_text(
"x", encoding="utf-8"
)
write_state(new, new_state)
report, changed = compare_harvests(str(old), str(new))
assert changed is True
txt = format_report(report, fmt="text")
assert "Packages" in txt
md = format_report(report, fmt="markdown")
assert "# enroll diff report" in md
js = format_report(report, fmt="json")
parsed = json.loads(js)
assert parsed["packages"]["added"] == ["c"]

186
tests/test_pathfilter.py
View file

@ -1,186 +0,0 @@
from __future__ import annotations
import os
from pathlib import Path
import enroll.pathfilter as pf
def test_compile_and_match_prefix_glob_and_regex(tmp_path: Path):
from enroll.pathfilter import PathFilter, compile_path_pattern
# prefix semantics: matches the exact path and subtree
p = compile_path_pattern("/etc/nginx")
assert p.kind == "prefix"
assert p.matches("/etc/nginx")
assert p.matches("/etc/nginx/nginx.conf")
assert not p.matches("/etc/nginx2/nginx.conf")
# glob semantics
g = compile_path_pattern("/etc/**/*.conf")
assert g.kind == "glob"
assert g.matches("/etc/nginx/nginx.conf")
assert not g.matches("/var/etc/nginx.conf")
# explicit glob
g2 = compile_path_pattern("glob:/home/*/.bashrc")
assert g2.kind == "glob"
assert g2.matches("/home/alice/.bashrc")
# regex semantics (search, not match)
r = compile_path_pattern(r"re:/home/[^/]+/\.ssh/authorized_keys$")
assert r.kind == "regex"
assert r.matches("/home/alice/.ssh/authorized_keys")
assert not r.matches("/home/alice/.ssh/authorized_keys2")
# invalid regex: never matches
bad = compile_path_pattern("re:[")
assert bad.kind == "regex"
assert not bad.matches("/etc/passwd")
# exclude wins
pf = PathFilter(exclude=["/etc/nginx"], include=["/etc/nginx/nginx.conf"])
assert pf.is_excluded("/etc/nginx/nginx.conf")
def test_expand_includes_respects_exclude_symlinks_and_caps(tmp_path: Path):
from enroll.pathfilter import PathFilter, compile_path_pattern, expand_includes
root = tmp_path / "root"
(root / "a").mkdir(parents=True)
(root / "a" / "one.txt").write_text("1", encoding="utf-8")
(root / "a" / "two.txt").write_text("2", encoding="utf-8")
(root / "b").mkdir()
(root / "b" / "secret.txt").write_text("s", encoding="utf-8")
# symlink file should be ignored
os.symlink(str(root / "a" / "one.txt"), str(root / "a" / "link.txt"))
exclude = PathFilter(exclude=[str(root / "b")])
pats = [
compile_path_pattern(str(root / "a")),
compile_path_pattern("glob:" + str(root / "**" / "*.txt")),
]
paths, notes = expand_includes(pats, exclude=exclude, max_files=2)
# cap should limit to 2 files
assert len(paths) == 2
assert any("cap" in n.lower() for n in notes)
# excluded dir should not contribute
assert all("/b/" not in p for p in paths)
# symlink ignored
assert all(not p.endswith("link.txt") for p in paths)
def test_expand_includes_notes_on_no_matches(tmp_path: Path):
from enroll.pathfilter import compile_path_pattern, expand_includes
pats = [compile_path_pattern(str(tmp_path / "does_not_exist"))]
paths, notes = expand_includes(pats, max_files=10)
assert paths == []
assert any("matched no files" in n.lower() for n in notes)
def test_expand_includes_supports_regex_with_inferred_root(tmp_path: Path):
"""Regex includes are expanded by walking an inferred literal prefix root."""
from enroll.pathfilter import compile_path_pattern, expand_includes
root = tmp_path / "root"
(root / "home" / "alice" / ".config" / "myapp").mkdir(parents=True)
target = root / "home" / "alice" / ".config" / "myapp" / "settings.ini"
target.write_text("x=1\n", encoding="utf-8")
# This is anchored and begins with an absolute path, so expand_includes should
# infer a narrow walk root instead of scanning '/'.
rex = rf"re:^{root}/home/[^/]+/\.config/myapp/.*$"
pat = compile_path_pattern(rex)
paths, notes = expand_includes([pat], max_files=10)
assert str(target) in paths
assert notes == []
def test_compile_path_pattern_normalises_relative_prefix():
from enroll.pathfilter import compile_path_pattern
p = compile_path_pattern("etc/ssh")
assert p.kind == "prefix"
assert p.value == "/etc/ssh"
def test_norm_abs_empty_string_is_root():
assert pf._norm_abs("") == "/"
def test_posix_match_invalid_pattern_fails_closed(monkeypatch):
# Force PurePosixPath.match to raise to cover the exception handler.
real_match = pf.PurePosixPath.match
def boom(self, pat):
raise ValueError("bad pattern")
monkeypatch.setattr(pf.PurePosixPath, "match", boom)
try:
assert pf._posix_match("/etc/hosts", "[bad") is False
finally:
monkeypatch.setattr(pf.PurePosixPath, "match", real_match)
def test_regex_literal_prefix_handles_escapes():
# Prefix stops at meta chars but includes escaped literals.
assert pf._regex_literal_prefix(r"^/etc/\./foo") == "/etc/./foo"
def test_expand_includes_maybe_add_file_skips_non_files(monkeypatch, tmp_path: Path):
# Drive the _maybe_add_file branch that rejects symlinks/non-files.
pats = [pf.compile_path_pattern(str(tmp_path / "missing"))]
monkeypatch.setattr(pf.os.path, "isfile", lambda p: False)
monkeypatch.setattr(pf.os.path, "islink", lambda p: False)
monkeypatch.setattr(pf.os.path, "isdir", lambda p: False)
paths, notes = pf.expand_includes(pats, max_files=10)
assert paths == []
assert any("matched no files" in n for n in notes)
def test_expand_includes_prunes_excluded_dirs(monkeypatch):
include = [pf.compile_path_pattern("/root/**")]
exclude = pf.PathFilter(exclude=["/root/skip/**"])
# Simulate filesystem walk:
# /root has dirnames ['skip', 'keep'] but skip should be pruned.
monkeypatch.setattr(
pf.os.path,
"isdir",
lambda p: p in {"/root", "/root/keep", "/root/skip"},
)
monkeypatch.setattr(pf.os.path, "islink", lambda p: False)
monkeypatch.setattr(pf.os.path, "isfile", lambda p: True)
def walk(root, followlinks=False):
assert root == "/root"
yield ("/root", ["skip", "keep"], [])
yield ("/root/keep", [], ["a.txt"])
# If pruning works, we should never walk into /root/skip.
monkeypatch.setattr(pf.os, "walk", walk)
paths, _notes = pf.expand_includes(include, exclude=exclude, max_files=10)
assert "/root/keep/a.txt" in paths
assert not any(p.startswith("/root/skip") for p in paths)
def test_expand_includes_respects_max_files(monkeypatch):
include = [pf.compile_path_pattern("/root/**")]
monkeypatch.setattr(pf.os.path, "isdir", lambda p: p == "/root")
monkeypatch.setattr(pf.os.path, "islink", lambda p: False)
monkeypatch.setattr(pf.os.path, "isfile", lambda p: True)
monkeypatch.setattr(
pf.os,
"walk",
lambda root, followlinks=False: [("/root", [], ["a", "b", "c"])],
)
paths, notes = pf.expand_includes(include, max_files=2)
assert len(paths) == 2
assert "/root/c" not in paths

93
tests/test_platform.py
View file

@ -1,93 +0,0 @@
from __future__ import annotations
from pathlib import Path
import enroll.platform as platform
def test_read_os_release_parses_kv_and_strips_quotes(tmp_path: Path):
p = tmp_path / "os-release"
p.write_text(
"""
# comment
ID=fedora
ID_LIKE=\"rhel centos\"
NAME=\"Fedora Linux\"
EMPTY=
NOEQUALS
""",
encoding="utf-8",
)
osr = platform._read_os_release(str(p))
assert osr["ID"] == "fedora"
assert osr["ID_LIKE"] == "rhel centos"
assert osr["NAME"] == "Fedora Linux"
assert osr["EMPTY"] == ""
assert "NOEQUALS" not in osr
def test_detect_platform_prefers_os_release(monkeypatch):
monkeypatch.setattr(
platform,
"_read_os_release",
lambda path="/etc/os-release": {"ID": "fedora", "ID_LIKE": "rhel"},
)
# If os-release is decisive we shouldn't need which()
monkeypatch.setattr(platform.shutil, "which", lambda exe: None)
info = platform.detect_platform()
assert info.os_family == "redhat"
assert info.pkg_backend == "rpm"
def test_detect_platform_fallbacks_to_dpkg_when_unknown(monkeypatch):
monkeypatch.setattr(platform, "_read_os_release", lambda path="/etc/os-release": {})
monkeypatch.setattr(
platform.shutil, "which", lambda exe: "/usr/bin/dpkg" if exe == "dpkg" else None
)
info = platform.detect_platform()
assert info.os_family == "debian"
assert info.pkg_backend == "dpkg"
def test_get_backend_unknown_prefers_rpm_if_present(monkeypatch):
monkeypatch.setattr(
platform.shutil, "which", lambda exe: "/usr/bin/rpm" if exe == "rpm" else None
)
b = platform.get_backend(
platform.PlatformInfo(os_family="unknown", pkg_backend="unknown", os_release={})
)
assert isinstance(b, platform.RpmBackend)
def test_rpm_backend_modified_paths_labels_conffiles(monkeypatch):
b = platform.RpmBackend()
# Pretend rpm -V says both files changed, but only one is a config file.
monkeypatch.setattr(b, "_modified_files", lambda pkg: {"/etc/foo.conf", "/etc/bar"})
monkeypatch.setattr(b, "_config_files", lambda pkg: {"/etc/foo.conf"})
out = b.modified_paths("mypkg", ["/etc/foo.conf", "/etc/bar", "/etc/dnf/dnf.conf"])
assert out["/etc/foo.conf"] == "modified_conffile"
assert out["/etc/bar"] == "modified_packaged_file"
# Package-manager config paths are excluded.
assert "/etc/dnf/dnf.conf" not in out
def test_specific_paths_for_hints_differs_between_backends():
# We can exercise this without instantiating DpkgBackend (which reads dpkg status)
class Dummy(platform.PackageBackend):
name = "dummy"
pkg_config_prefixes = ("/etc/apt/",)
d = Dummy()
assert d.is_pkg_config_path("/etc/apt/sources.list")
assert not d.is_pkg_config_path("/etc/ssh/sshd_config")
r = platform.RpmBackend()
paths = set(r.specific_paths_for_hints({"nginx"}))
assert "/etc/sysconfig/nginx" in paths
assert "/etc/sysconfig/nginx.conf" in paths

72
tests/test_platform_backends.py
View file

@ -1,72 +0,0 @@
from __future__ import annotations
from collections import defaultdict
def test_dpkg_backend_modified_paths_marks_conffiles_and_packaged(monkeypatch):
from enroll.platform import DpkgBackend
# Provide fake conffiles md5sums.
monkeypatch.setattr(
"enroll.debian.parse_status_conffiles",
lambda: {"mypkg": {"/etc/mypkg.conf": "aaaa"}},
)
monkeypatch.setattr(
"enroll.debian.read_pkg_md5sums",
lambda _pkg: {"etc/other.conf": "bbbb"},
)
# Fake file_md5 values (avoids touching /etc).
def fake_md5(p: str):
if p == "/etc/mypkg.conf":
return "zzzz" # differs from conffile baseline
if p == "/etc/other.conf":
return "cccc" # differs from packaged baseline
if p == "/etc/apt/sources.list":
return "bbbb"
return None
monkeypatch.setattr("enroll.platform.file_md5", fake_md5)
b = DpkgBackend()
out = b.modified_paths(
"mypkg",
["/etc/mypkg.conf", "/etc/other.conf", "/etc/apt/sources.list"],
)
assert out["/etc/mypkg.conf"] == "modified_conffile"
assert out["/etc/other.conf"] == "modified_packaged_file"
# pkg config paths (like /etc/apt/...) are excluded.
assert "/etc/apt/sources.list" not in out
def test_rpm_backend_modified_paths_caches_queries(monkeypatch):
from enroll.platform import RpmBackend
calls = defaultdict(int)
def fake_modified(_pkg=None):
calls["modified"] += 1
return {"/etc/foo.conf", "/etc/bar.conf"}
def fake_config(_pkg=None):
calls["config"] += 1
return {"/etc/foo.conf"}
monkeypatch.setattr("enroll.rpm.rpm_modified_files", fake_modified)
monkeypatch.setattr("enroll.rpm.rpm_config_files", fake_config)
b = RpmBackend()
etc = ["/etc/foo.conf", "/etc/bar.conf", "/etc/baz.conf"]
out1 = b.modified_paths("ignored", etc)
out2 = b.modified_paths("ignored", etc)
assert out1 == out2
assert out1["/etc/foo.conf"] == "modified_conffile"
assert out1["/etc/bar.conf"] == "modified_packaged_file"
assert "/etc/baz.conf" not in out1
# Caches should mean we only queried rpm once.
assert calls["modified"] == 1
assert calls["config"] == 1

567
tests/test_remote.py
View file

@ -1,567 +0,0 @@
from __future__ import annotations
import io
import tarfile
from pathlib import Path
import pytest
def _make_tgz_bytes(files: dict[str, bytes]) -> bytes:
bio = io.BytesIO()
with tarfile.open(fileobj=bio, mode="w:gz") as tf:
for name, content in files.items():
ti = tarfile.TarInfo(name=name)
ti.size = len(content)
tf.addfile(ti, io.BytesIO(content))
return bio.getvalue()
def test_safe_extract_tar_rejects_path_traversal(tmp_path: Path):
from enroll.remote import _safe_extract_tar
# Build an unsafe tar with ../ traversal
bio = io.BytesIO()
with tarfile.open(fileobj=bio, mode="w:gz") as tf:
ti = tarfile.TarInfo(name="../evil")
ti.size = 1
tf.addfile(ti, io.BytesIO(b"x"))
bio.seek(0)
with tarfile.open(fileobj=bio, mode="r:gz") as tf:
with pytest.raises(RuntimeError, match="Unsafe tar member path"):
_safe_extract_tar(tf, tmp_path)
def test_safe_extract_tar_rejects_symlinks(tmp_path: Path):
from enroll.remote import _safe_extract_tar
bio = io.BytesIO()
with tarfile.open(fileobj=bio, mode="w:gz") as tf:
ti = tarfile.TarInfo(name="link")
ti.type = tarfile.SYMTYPE
ti.linkname = "/etc/passwd"
tf.addfile(ti)
bio.seek(0)
with tarfile.open(fileobj=bio, mode="r:gz") as tf:
with pytest.raises(RuntimeError, match="Refusing to extract"):
_safe_extract_tar(tf, tmp_path)
def test_remote_harvest_happy_path(tmp_path: Path, monkeypatch):
import sys
import enroll.remote as r
# Avoid building a real zipapp; just create a file.
def fake_build(_td: Path) -> Path:
p = _td / "enroll.pyz"
p.write_bytes(b"PYZ")
return p
monkeypatch.setattr(r, "_build_enroll_pyz", fake_build)
# Prepare a tiny harvest bundle tar stream from the "remote".
tgz = _make_tgz_bytes({"state.json": b'{"ok": true}\n'})
# Track each SSH exec_command call with whether a PTY was requested.
calls: list[tuple[str, bool]] = []
class _Chan:
def __init__(self, out: bytes = b"", err: bytes = b"", rc: int = 0):
self._out = out
self._err = err
self._out_i = 0
self._err_i = 0
self._rc = rc
self._closed = False
def recv_ready(self) -> bool:
return (not self._closed) and self._out_i < len(self._out)
def recv(self, n: int) -> bytes:
if self._closed:
return b""
chunk = self._out[self._out_i : self._out_i + n]
self._out_i += len(chunk)
return chunk
def recv_stderr_ready(self) -> bool:
return (not self._closed) and self._err_i < len(self._err)
def recv_stderr(self, n: int) -> bytes:
if self._closed:
return b""
chunk = self._err[self._err_i : self._err_i + n]
self._err_i += len(chunk)
return chunk
def exit_status_ready(self) -> bool:
return self._closed or (
self._out_i >= len(self._out) and self._err_i >= len(self._err)
)
def recv_exit_status(self) -> int:
return self._rc
def shutdown_write(self) -> None:
return
def close(self) -> None:
self._closed = True
class _Stdout:
def __init__(self, payload: bytes = b"", rc: int = 0, err: bytes = b""):
self._bio = io.BytesIO(payload)
# _ssh_run reads stdout/stderr via the underlying channel.
self.channel = _Chan(out=payload, err=err, rc=rc)
def read(self, n: int = -1) -> bytes:
return self._bio.read(n)
class _Stderr:
def __init__(self, payload: bytes = b""):
self._bio = io.BytesIO(payload)
def read(self, n: int = -1) -> bytes:
return self._bio.read(n)
class _SFTP:
def __init__(self):
self.put_calls: list[tuple[str, str]] = []
def put(self, local: str, remote: str) -> None:
self.put_calls.append((local, remote))
def close(self) -> None:
return
class FakeSSH:
def __init__(self):
self._sftp = _SFTP()
def load_system_host_keys(self):
return
def set_missing_host_key_policy(self, _policy):
return
def connect(self, **kwargs):
# Accept any connect parameters.
return
def open_sftp(self):
return self._sftp
def exec_command(self, cmd: str, *, get_pty: bool = False, **_kwargs):
calls.append((cmd, bool(get_pty)))
# The tar stream uses exec_command directly.
if cmd.startswith("tar -cz -C"):
return (None, _Stdout(tgz, rc=0), _Stderr(b""))
# _ssh_run path: id -un, mktemp -d, chmod, sudo harvest, sudo chown, rm -rf
if cmd == "id -un":
return (None, _Stdout(b"alice\n"), _Stderr())
if cmd == "mktemp -d":
return (None, _Stdout(b"/tmp/enroll-remote-123\n"), _Stderr())
if cmd.startswith("chmod 700"):
return (None, _Stdout(b""), _Stderr())
if cmd.startswith("sudo -n") and " harvest " in cmd:
if not get_pty:
msg = b"sudo: sorry, you must have a tty to run sudo\n"
return (None, _Stdout(b"", rc=1, err=msg), _Stderr(msg))
return (None, _Stdout(b"", rc=0), _Stderr(b""))
if cmd.startswith("sudo -S") and " harvest " in cmd:
return (None, _Stdout(b""), _Stderr())
if " harvest " in cmd:
return (None, _Stdout(b""), _Stderr())
if cmd.startswith("sudo -n") and " chown -R" in cmd:
if not get_pty:
msg = b"sudo: sorry, you must have a tty to run sudo\n"
return (None, _Stdout(b"", rc=1, err=msg), _Stderr(msg))
return (None, _Stdout(b"", rc=0), _Stderr(b""))
if cmd.startswith("rm -rf"):
return (None, _Stdout(b""), _Stderr())
return (None, _Stdout(b""), _Stderr(b"unknown"))
def close(self):
return
import types
class RejectPolicy:
pass
FakeParamiko = types.SimpleNamespace(SSHClient=FakeSSH, RejectPolicy=RejectPolicy)
# Provide a fake paramiko module.
monkeypatch.setitem(sys.modules, "paramiko", FakeParamiko)
out_dir = tmp_path / "out"
state_path = r.remote_harvest(
ask_become_pass=False,
local_out_dir=out_dir,
remote_host="example.com",
remote_port=2222,
remote_user=None,
include_paths=["/etc/nginx/nginx.conf"],
exclude_paths=["/etc/shadow"],
dangerous=True,
no_sudo=False,
)
assert state_path == out_dir / "state.json"
assert state_path.exists()
assert b"ok" in state_path.read_bytes()
# Ensure we attempted remote harvest with sudo and passed include/exclude and dangerous.
joined = "\n".join([c for c, _pty in calls])
assert "sudo" in joined
assert "--dangerous" in joined
assert "--include-path" in joined
assert "--exclude-path" in joined
# Ensure we fall back to PTY only when sudo reports it is required.
assert any(c == "id -un" and pty is False for c, pty in calls)
sudo_harvest = [
(c, pty) for c, pty in calls if c.startswith("sudo -n") and " harvest " in c
]
assert any(pty is False for _c, pty in sudo_harvest)
assert any(pty is True for _c, pty in sudo_harvest)
sudo_chown = [
(c, pty) for c, pty in calls if c.startswith("sudo -n") and " chown -R" in c
]
assert any(pty is False for _c, pty in sudo_chown)
assert any(pty is True for _c, pty in sudo_chown)
assert any(c.startswith("tar -cz -C") and pty is False for c, pty in calls)
def test_remote_harvest_no_sudo_does_not_request_pty_or_chown(
tmp_path: Path, monkeypatch
):
"""When --no-sudo is used we should not request a PTY nor run sudo chown."""
import sys
import enroll.remote as r
monkeypatch.setattr(
r,
"_build_enroll_pyz",
lambda td: (Path(td) / "enroll.pyz").write_bytes(b"PYZ")
or (Path(td) / "enroll.pyz"),
)
tgz = _make_tgz_bytes({"state.json": b"{}"})
calls: list[tuple[str, bool]] = []
class _Chan:
def __init__(self, out: bytes = b"", err: bytes = b"", rc: int = 0):
self._out = out
self._err = err
self._out_i = 0
self._err_i = 0
self._rc = rc
self._closed = False
def recv_ready(self) -> bool:
return (not self._closed) and self._out_i < len(self._out)
def recv(self, n: int) -> bytes:
if self._closed:
return b""
chunk = self._out[self._out_i : self._out_i + n]
self._out_i += len(chunk)
return chunk
def recv_stderr_ready(self) -> bool:
return (not self._closed) and self._err_i < len(self._err)
def recv_stderr(self, n: int) -> bytes:
if self._closed:
return b""
chunk = self._err[self._err_i : self._err_i + n]
self._err_i += len(chunk)
return chunk
def exit_status_ready(self) -> bool:
return self._closed or (
self._out_i >= len(self._out) and self._err_i >= len(self._err)
)
def recv_exit_status(self) -> int:
return self._rc
def shutdown_write(self) -> None:
return
def close(self) -> None:
self._closed = True
class _Stdout:
def __init__(self, payload: bytes = b"", rc: int = 0, err: bytes = b""):
self._bio = io.BytesIO(payload)
# _ssh_run reads stdout/stderr via the underlying channel.
self.channel = _Chan(out=payload, err=err, rc=rc)
def read(self, n: int = -1) -> bytes:
return self._bio.read(n)
class _Stderr:
def __init__(self, payload: bytes = b""):
self._bio = io.BytesIO(payload)
def read(self, n: int = -1) -> bytes:
return self._bio.read(n)
class _SFTP:
def put(self, _local: str, _remote: str) -> None:
return
def close(self) -> None:
return
class FakeSSH:
def __init__(self):
self._sftp = _SFTP()
def load_system_host_keys(self):
return
def set_missing_host_key_policy(self, _policy):
return
def connect(self, **_kwargs):
return
def open_sftp(self):
return self._sftp
def exec_command(self, cmd: str, *, get_pty: bool = False, **_kwargs):
calls.append((cmd, bool(get_pty)))
if cmd == "mktemp -d":
return (None, _Stdout(b"/tmp/enroll-remote-456\n"), _Stderr())
if cmd.startswith("chmod 700"):
return (None, _Stdout(b""), _Stderr())
if cmd.startswith("tar -cz -C"):
return (None, _Stdout(tgz, rc=0), _Stderr())
if " harvest " in cmd:
return (None, _Stdout(b""), _Stderr())
if cmd.startswith("rm -rf"):
return (None, _Stdout(b""), _Stderr())
return (None, _Stdout(b""), _Stderr())
def close(self):
return
import types
class RejectPolicy:
pass
monkeypatch.setitem(
sys.modules,
"paramiko",
types.SimpleNamespace(SSHClient=FakeSSH, RejectPolicy=RejectPolicy),
)
out_dir = tmp_path / "out"
r.remote_harvest(
ask_become_pass=False,
local_out_dir=out_dir,
remote_host="example.com",
remote_user="alice",
no_sudo=True,
)
joined = "\n".join([c for c, _pty in calls])
assert "sudo" not in joined
assert "sudo chown" not in joined
assert any(" harvest " in c and pty is False for c, pty in calls)
def test_remote_harvest_sudo_password_retry_uses_sudo_s_and_writes_password(
tmp_path: Path, monkeypatch
):
"""If sudo requires a password, we should fall back from -n to -S and feed stdin."""
import sys
import types
import enroll.remote as r
# Avoid building a real zipapp; just create a file.
monkeypatch.setattr(
r,
"_build_enroll_pyz",
lambda td: (Path(td) / "enroll.pyz").write_bytes(b"PYZ")
or (Path(td) / "enroll.pyz"),
)
tgz = _make_tgz_bytes({"state.json": b'{"ok": true}\n'})
calls: list[tuple[str, bool]] = []
stdin_by_cmd: dict[str, list[str]] = {}
class _Chan:
def __init__(self, out: bytes = b"", err: bytes = b"", rc: int = 0):
self._out = out
self._err = err
self._out_i = 0
self._err_i = 0
self._rc = rc
self._closed = False
def recv_ready(self) -> bool:
return (not self._closed) and self._out_i < len(self._out)
def recv(self, n: int) -> bytes:
if self._closed:
return b""
chunk = self._out[self._out_i : self._out_i + n]
self._out_i += len(chunk)
return chunk
def recv_stderr_ready(self) -> bool:
return (not self._closed) and self._err_i < len(self._err)
def recv_stderr(self, n: int) -> bytes:
if self._closed:
return b""
chunk = self._err[self._err_i : self._err_i + n]
self._err_i += len(chunk)
return chunk
def exit_status_ready(self) -> bool:
return self._closed or (
self._out_i >= len(self._out) and self._err_i >= len(self._err)
)
def recv_exit_status(self) -> int:
return self._rc
def shutdown_write(self) -> None:
return
def close(self) -> None:
self._closed = True
class _Stdout:
def __init__(self, payload: bytes = b"", rc: int = 0, err: bytes = b""):
self._bio = io.BytesIO(payload)
# _ssh_run reads stdout/stderr via the underlying channel.
self.channel = _Chan(out=payload, err=err, rc=rc)
def read(self, n: int = -1) -> bytes:
return self._bio.read(n)
class _Stderr:
def __init__(self, payload: bytes = b""):
self._bio = io.BytesIO(payload)
def read(self, n: int = -1) -> bytes:
return self._bio.read(n)
class _Stdin:
def __init__(self, cmd: str):
self._cmd = cmd
stdin_by_cmd.setdefault(cmd, [])
def write(self, s: str) -> None:
stdin_by_cmd[self._cmd].append(s)
def flush(self) -> None:
return
class _SFTP:
def put(self, _local: str, _remote: str) -> None:
return
def close(self) -> None:
return
class FakeSSH:
def __init__(self):
self._sftp = _SFTP()
def load_system_host_keys(self):
return
def set_missing_host_key_policy(self, _policy):
return
def connect(self, **_kwargs):
return
def open_sftp(self):
return self._sftp
def exec_command(self, cmd: str, *, get_pty: bool = False, **_kwargs):
calls.append((cmd, bool(get_pty)))
# Tar stream
if cmd.startswith("tar -cz -C"):
return (_Stdin(cmd), _Stdout(tgz, rc=0), _Stderr(b""))
if cmd == "mktemp -d":
return (_Stdin(cmd), _Stdout(b"/tmp/enroll-remote-789\n"), _Stderr())
if cmd.startswith("chmod 700"):
return (_Stdin(cmd), _Stdout(b""), _Stderr())
# First attempt: sudo -n fails, prompting is not allowed.
if cmd.startswith("sudo -n") and " harvest " in cmd:
return (
_Stdin(cmd),
_Stdout(b"", rc=1, err=b"sudo: a password is required\n"),
_Stderr(b"sudo: a password is required\n"),
)
# Retry: sudo -S succeeds and should have been fed the password via stdin.
if cmd.startswith("sudo -S") and " harvest " in cmd:
return (_Stdin(cmd), _Stdout(b"", rc=0), _Stderr(b""))
# chown succeeds passwordlessly (e.g., sudo timestamp is warm).
if cmd.startswith("sudo -n") and " chown -R" in cmd:
return (_Stdin(cmd), _Stdout(b"", rc=0), _Stderr(b""))
if cmd.startswith("rm -rf"):
return (_Stdin(cmd), _Stdout(b"", rc=0), _Stderr(b""))
# Fallback for unexpected commands.
return (_Stdin(cmd), _Stdout(b"", rc=0), _Stderr(b""))
def close(self):
return
class RejectPolicy:
pass
monkeypatch.setitem(
sys.modules,
"paramiko",
types.SimpleNamespace(SSHClient=FakeSSH, RejectPolicy=RejectPolicy),
)
out_dir = tmp_path / "out"
state_path = r.remote_harvest(
ask_become_pass=True,
getpass_fn=lambda _prompt="": "s3cr3t",
local_out_dir=out_dir,
remote_host="example.com",
remote_user="alice",
no_sudo=False,
)
assert state_path.exists()
assert b"ok" in state_path.read_bytes()
# Ensure we attempted with sudo -n first, then sudo -S.
sudo_n = [c for c, _pty in calls if c.startswith("sudo -n") and " harvest " in c]
sudo_s = [c for c, _pty in calls if c.startswith("sudo -S") and " harvest " in c]
assert len(sudo_n) == 1
assert len(sudo_s) == 1
# Ensure the password was written to stdin for the -S invocation.
assert stdin_by_cmd.get(sudo_s[0]) == ["s3cr3t\n"]

178
tests/test_rpm.py
View file

@ -1,178 +0,0 @@
from __future__ import annotations
import enroll.rpm as rpm
def test_rpm_owner_returns_none_when_unowned(monkeypatch):
monkeypatch.setattr(
rpm,
"_run",
lambda cmd, allow_fail=False, merge_err=False: (
1,
"file /etc/x is not owned by any package\n",
),
)
assert rpm.rpm_owner("/etc/x") is None
def test_rpm_owner_parses_name(monkeypatch):
monkeypatch.setattr(
rpm, "_run", lambda cmd, allow_fail=False, merge_err=False: (0, "bash\n")
)
assert rpm.rpm_owner("/bin/bash") == "bash"
def test_strip_arch_strips_known_arches():
assert rpm._strip_arch("vim-enhanced.x86_64") == "vim-enhanced"
assert rpm._strip_arch("foo.noarch") == "foo"
assert rpm._strip_arch("weird.token") == "weird.token"
def test_list_manual_packages_prefers_dnf_repoquery(monkeypatch):
monkeypatch.setattr(
rpm.shutil, "which", lambda exe: "/usr/bin/dnf" if exe == "dnf" else None
)
def fake_run(cmd, allow_fail=False, merge_err=False):
# First repoquery form returns usable output.
if cmd[:3] == ["dnf", "-q", "repoquery"]:
return 0, "vim-enhanced.x86_64\nhtop\nvim-enhanced.x86_64\n"
raise AssertionError(f"unexpected cmd: {cmd}")
monkeypatch.setattr(rpm, "_run", fake_run)
pkgs = rpm.list_manual_packages()
assert pkgs == ["htop", "vim-enhanced"]
def test_list_manual_packages_falls_back_to_history(monkeypatch):
monkeypatch.setattr(
rpm.shutil, "which", lambda exe: "/usr/bin/dnf" if exe == "dnf" else None
)
def fake_run(cmd, allow_fail=False, merge_err=False):
# repoquery fails
if cmd[:3] == ["dnf", "-q", "repoquery"]:
return 1, ""
if cmd[:3] == ["dnf", "-q", "history"]:
return (
0,
"Installed Packages\nvim-enhanced.x86_64\nLast metadata expiration check: 0:01:00 ago\n",
)
raise AssertionError(f"unexpected cmd: {cmd}")
monkeypatch.setattr(rpm, "_run", fake_run)
pkgs = rpm.list_manual_packages()
assert pkgs == ["vim-enhanced"]
def test_build_rpm_etc_index_uses_fallback_when_rpm_output_mismatches(monkeypatch):
# Two files in /etc, one owned, one unowned.
monkeypatch.setattr(
rpm, "_walk_etc_files", lambda: ["/etc/owned.conf", "/etc/unowned.conf"]
)
# Simulate chunk query producing unexpected extra line (mismatch) -> triggers per-file fallback.
monkeypatch.setattr(
rpm,
"_run",
lambda cmd, allow_fail=False, merge_err=False: (0, "ownedpkg\nEXTRA\nTHIRD\n"),
)
monkeypatch.setattr(
rpm, "rpm_owner", lambda p: "ownedpkg" if p == "/etc/owned.conf" else None
)
owned, owner_map, topdir_to_pkgs, pkg_to_etc = rpm.build_rpm_etc_index()
assert owned == {"/etc/owned.conf"}
assert owner_map["/etc/owned.conf"] == "ownedpkg"
assert "owned.conf" in topdir_to_pkgs
assert pkg_to_etc["ownedpkg"] == ["/etc/owned.conf"]
def test_build_rpm_etc_index_parses_chunk_output(monkeypatch):
monkeypatch.setattr(
rpm, "_walk_etc_files", lambda: ["/etc/ssh/sshd_config", "/etc/notowned"]
)
def fake_run(cmd, allow_fail=False, merge_err=False):
# One output line per input path.
return 0, "openssh-server\nfile /etc/notowned is not owned by any package\n"
monkeypatch.setattr(rpm, "_run", fake_run)
owned, owner_map, topdir_to_pkgs, pkg_to_etc = rpm.build_rpm_etc_index()
assert "/etc/ssh/sshd_config" in owned
assert "/etc/notowned" not in owned
assert owner_map["/etc/ssh/sshd_config"] == "openssh-server"
assert "ssh" in topdir_to_pkgs
assert "openssh-server" in topdir_to_pkgs["ssh"]
assert pkg_to_etc["openssh-server"] == ["/etc/ssh/sshd_config"]
def test_rpm_config_files_and_modified_files_parsing(monkeypatch):
monkeypatch.setattr(
rpm,
"_run",
lambda cmd, allow_fail=False, merge_err=False: (
0,
"/etc/foo.conf\n/usr/bin/tool\n",
),
)
assert rpm.rpm_config_files("mypkg") == {"/etc/foo.conf", "/usr/bin/tool"}
# rpm -V returns only changed/missing files
out = "S.5....T. c /etc/foo.conf\nmissing /etc/bar\n"
monkeypatch.setattr(
rpm, "_run", lambda cmd, allow_fail=False, merge_err=False: (1, out)
)
assert rpm.rpm_modified_files("mypkg") == {"/etc/foo.conf", "/etc/bar"}
def test_list_manual_packages_uses_yum_fallback(monkeypatch):
# No dnf, yum present.
monkeypatch.setattr(
rpm.shutil, "which", lambda exe: "/usr/bin/yum" if exe == "yum" else None
)
def fake_run(cmd, allow_fail=False, merge_err=False):
assert cmd[:3] == ["yum", "-q", "history"]
return 0, "Installed Packages\nvim-enhanced.x86_64\nhtop\n"
monkeypatch.setattr(rpm, "_run", fake_run)
assert rpm.list_manual_packages() == ["htop", "vim-enhanced"]
def test_list_installed_packages_parses_epoch_and_sorts(monkeypatch):
out = (
"bash\t0\t5.2.26\t1.el9\tx86_64\n"
"bash\t1\t5.2.26\t1.el9\taarch64\n"
"coreutils\t(none)\t9.1\t2.el9\tx86_64\n"
)
monkeypatch.setattr(
rpm, "_run", lambda cmd, allow_fail=False, merge_err=False: (0, out)
)
pkgs = rpm.list_installed_packages()
assert pkgs["bash"][0]["arch"] == "aarch64" # sorted by arch then version
assert pkgs["bash"][0]["version"].startswith("1:")
assert pkgs["coreutils"][0]["version"] == "9.1-2.el9"
def test_rpm_config_files_returns_empty_on_failure(monkeypatch):
monkeypatch.setattr(
rpm, "_run", lambda cmd, allow_fail=False, merge_err=False: (1, "")
)
assert rpm.rpm_config_files("missing") == set()
def test_rpm_owner_strips_epoch_prefix_when_present(monkeypatch):
# Defensive: rpm output might include epoch-like token.
monkeypatch.setattr(
rpm,
"_run",
lambda cmd, allow_fail=False, merge_err=False: (0, "1:bash\n"),
)
assert rpm.rpm_owner("/bin/bash") == "bash"

31
tests/test_rpm_run.py
View file

@ -1,31 +0,0 @@
from __future__ import annotations
import types
import pytest
import enroll.rpm as rpm
def test_run_raises_on_nonzero_returncode_when_not_allow_fail(monkeypatch):
def fake_run(cmd, check, text, stdout, stderr):
return types.SimpleNamespace(returncode=1, stdout="OUT", stderr="ERR")
monkeypatch.setattr(rpm.subprocess, "run", fake_run)
with pytest.raises(RuntimeError) as e:
rpm._run(["rpm", "-q"]) # type: ignore[attr-defined]
assert "Command failed" in str(e.value)
assert "ERR" in str(e.value)
assert "OUT" in str(e.value)
def test_run_merge_err_includes_stderr_in_stdout(monkeypatch):
def fake_run(cmd, check, text, stdout, stderr):
# When merge_err is True, stderr is redirected to STDOUT, so we only
# rely on stdout in our wrapper.
return types.SimpleNamespace(returncode=0, stdout="COMBINED", stderr=None)
monkeypatch.setattr(rpm.subprocess, "run", fake_run)
rc, out = rpm._run(["rpm", "-q"], merge_err=True)
assert rc == 0
assert out == "COMBINED"

8
tests/test_secrets.py Normal file
View file

@ -0,0 +1,8 @@
from enroll.secrets import SecretPolicy
def test_secret_policy_denies_common_backup_files():
pol = SecretPolicy()
assert pol.deny_reason("/etc/shadow-") == "denied_path"
assert pol.deny_reason("/etc/passwd-") == "denied_path"
assert pol.deny_reason("/etc/group-") == "denied_path"

121
tests/test_systemd.py
View file

@ -1,121 +0,0 @@
from __future__ import annotations
import pytest
def test_list_enabled_services_and_timers_filters_templates(monkeypatch):
import enroll.systemd as s
def fake_run(cmd: list[str]) -> str:
if "--type=service" in cmd:
return "\n".join(
[
"nginx.service enabled",
"getty@.service enabled", # template
"foo@bar.service enabled", # instance units are included
"ssh.service enabled",
]
)
if "--type=timer" in cmd:
return "\n".join(
[
"apt-daily.timer enabled",
"foo@.timer enabled", # template
]
)
raise AssertionError("unexpected")
monkeypatch.setattr(s, "_run", fake_run)
assert s.list_enabled_services() == [
"foo@bar.service",
"nginx.service",
"ssh.service",
]
assert s.list_enabled_timers() == ["apt-daily.timer"]
def test_get_unit_info_parses_fields(monkeypatch):
import enroll.systemd as s
class P:
def __init__(self, rc: int, out: str, err: str = ""):
self.returncode = rc
self.stdout = out
self.stderr = err
def fake_run(cmd, check, text, capture_output):
assert cmd[0:2] == ["systemctl", "show"]
return P(
0,
"\n".join(
[
"FragmentPath=/lib/systemd/system/nginx.service",
"DropInPaths=/etc/systemd/system/nginx.service.d/override.conf /etc/systemd/system/nginx.service.d/extra.conf",
"EnvironmentFiles=-/etc/default/nginx /etc/nginx/env",
"ExecStart={ path=/usr/sbin/nginx ; argv[]=/usr/sbin/nginx -g daemon off; }",
"ActiveState=active",
"SubState=running",
"UnitFileState=enabled",
"ConditionResult=yes",
]
),
)
monkeypatch.setattr(s.subprocess, "run", fake_run)
ui = s.get_unit_info("nginx.service")
assert ui.fragment_path == "/lib/systemd/system/nginx.service"
assert "/etc/default/nginx" in ui.env_files
assert "/etc/nginx/env" in ui.env_files
assert "/usr/sbin/nginx" in ui.exec_paths
assert ui.active_state == "active"
def test_get_unit_info_raises_unit_query_error(monkeypatch):
import enroll.systemd as s
class P:
def __init__(self, rc: int, out: str, err: str):
self.returncode = rc
self.stdout = out
self.stderr = err
def fake_run(cmd, check, text, capture_output):
return P(1, "", "no such unit")
monkeypatch.setattr(s.subprocess, "run", fake_run)
with pytest.raises(s.UnitQueryError) as ei:
s.get_unit_info("missing.service")
assert "missing.service" in str(ei.value)
assert ei.value.unit == "missing.service"
def test_get_timer_info_parses_fields(monkeypatch):
import enroll.systemd as s
class P:
def __init__(self, rc: int, out: str, err: str = ""):
self.returncode = rc
self.stdout = out
self.stderr = err
def fake_run(cmd, text, capture_output):
return P(
0,
"\n".join(
[
"FragmentPath=/lib/systemd/system/apt-daily.timer",
"DropInPaths=",
"EnvironmentFiles=-/etc/default/apt",
"Unit=apt-daily.service",
"ActiveState=active",
"SubState=waiting",
"UnitFileState=enabled",
"ConditionResult=yes",
]
),
)
monkeypatch.setattr(s.subprocess, "run", fake_run)
ti = s.get_timer_info("apt-daily.timer")
assert ti.trigger_unit == "apt-daily.service"
assert "/etc/default/apt" in ti.env_files

182
tests/test_validate.py
View file

@ -1,182 +0,0 @@
from __future__ import annotations
import json
import sys
from pathlib import Path
import pytest
import enroll.cli as cli
from enroll.validate import validate_harvest
def _base_state() -> dict:
return {
"enroll": {"version": "0.0.test", "harvest_time": 0},
"host": {
"hostname": "testhost",
"os": "unknown",
"pkg_backend": "dpkg",
"os_release": {},
},
"inventory": {"packages": {}},
"roles": {
"users": {
"role_name": "users",
"users": [],
"managed_dirs": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
"services": [],
"packages": [],
"apt_config": {
"role_name": "apt_config",
"managed_dirs": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
"dnf_config": {
"role_name": "dnf_config",
"managed_dirs": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
"etc_custom": {
"role_name": "etc_custom",
"managed_dirs": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
"usr_local_custom": {
"role_name": "usr_local_custom",
"managed_dirs": [],
"managed_files": [],
"excluded": [],
"notes": [],
},
"extra_paths": {
"role_name": "extra_paths",
"include_patterns": [],
"exclude_patterns": [],
"managed_dirs": [],
"managed_files": [],
"managed_links": [],
"excluded": [],
"notes": [],
},
},
}
def _write_bundle(tmp_path: Path, state: dict) -> Path:
bundle = tmp_path / "bundle"
bundle.mkdir(parents=True)
(bundle / "artifacts").mkdir()
(bundle / "state.json").write_text(json.dumps(state, indent=2), encoding="utf-8")
return bundle
def test_validate_ok_bundle(tmp_path: Path):
state = _base_state()
state["roles"]["etc_custom"]["managed_files"].append(
{
"path": "/etc/hosts",
"src_rel": "etc/hosts",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "custom_specific_path",
}
)
bundle = _write_bundle(tmp_path, state)
art = bundle / "artifacts" / "etc_custom" / "etc" / "hosts"
art.parent.mkdir(parents=True, exist_ok=True)
art.write_text("127.0.0.1 localhost\n", encoding="utf-8")
res = validate_harvest(str(bundle))
assert res.ok
assert res.errors == []
def test_validate_missing_artifact_is_error(tmp_path: Path):
state = _base_state()
state["roles"]["etc_custom"]["managed_files"].append(
{
"path": "/etc/hosts",
"src_rel": "etc/hosts",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "custom_specific_path",
}
)
bundle = _write_bundle(tmp_path, state)
res = validate_harvest(str(bundle))
assert not res.ok
assert any("missing artifact" in e for e in res.errors)
def test_validate_schema_error_is_reported(tmp_path: Path):
state = _base_state()
state["host"]["os"] = "not_a_real_os"
bundle = _write_bundle(tmp_path, state)
res = validate_harvest(str(bundle))
assert not res.ok
assert any(e.startswith("schema /host/os") for e in res.errors)
def test_cli_validate_exits_1_on_validation_error(monkeypatch, tmp_path: Path):
state = _base_state()
state["roles"]["etc_custom"]["managed_files"].append(
{
"path": "/etc/hosts",
"src_rel": "etc/hosts",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "custom_specific_path",
}
)
bundle = _write_bundle(tmp_path, state)
monkeypatch.setattr(sys, "argv", ["enroll", "validate", str(bundle)])
with pytest.raises(SystemExit) as e:
cli.main()
assert e.value.code == 1
def test_cli_validate_exits_1_on_validation_warning_with_flag(
monkeypatch, tmp_path: Path
):
state = _base_state()
state["roles"]["etc_custom"]["managed_files"].append(
{
"path": "/etc/hosts",
"src_rel": "etc/hosts",
"owner": "root",
"group": "root",
"mode": "0644",
"reason": "custom_specific_path",
}
)
bundle = _write_bundle(tmp_path, state)
art = bundle / "artifacts" / "etc_custom" / "etc" / "hosts"
art.parent.mkdir(parents=True, exist_ok=True)
art.write_text("127.0.0.1 localhost\n", encoding="utf-8")
art2 = bundle / "artifacts" / "etc_custom" / "etc" / "hosts2"
art2.write_text("hello\n", encoding="utf-8")
monkeypatch.setattr(
sys, "argv", ["enroll", "validate", str(bundle), "--fail-on-warnings"]
)
with pytest.raises(SystemExit) as e:
cli.main()
assert e.value.code == 1

36
tests/test_version_extra.py
View file

@ -1,36 +0,0 @@
from __future__ import annotations
import sys
import types
def test_get_enroll_version_returns_unknown_when_import_fails(monkeypatch):
from enroll.version import get_enroll_version
# Ensure both the module cache and the parent package attribute are redirected.
import importlib
dummy = types.ModuleType("importlib.metadata")
# Missing attributes will cause ImportError when importing names.
monkeypatch.setitem(sys.modules, "importlib.metadata", dummy)
monkeypatch.setattr(importlib, "metadata", dummy, raising=False)
assert get_enroll_version() == "unknown"
def test_get_enroll_version_uses_packages_distributions(monkeypatch):
# Restore the real module for this test.
monkeypatch.delitem(sys.modules, "importlib.metadata", raising=False)
import importlib.metadata
from enroll.version import get_enroll_version
monkeypatch.setattr(
importlib.metadata,
"packages_distributions",
lambda: {"enroll": ["enroll-dist"]},
)
monkeypatch.setattr(importlib.metadata, "version", lambda dist: "9.9.9")
assert get_enroll_version() == "9.9.9"