Compare commits

...
Sign in to create a new pull request.

25 commits
erb ... main

Author SHA1 Message Date
a712b19a45
Remove Fediverse link
All checks were successful
CI / test (push) Successful in 44s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m23s
Lint / test (push) Successful in 36s
2026-07-24 10:58:23 +10:00
d3694240da
0.7.0
All checks were successful
CI / test (push) Successful in 52s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m40s
Lint / test (push) Successful in 50s
2026-07-05 10:54:37 +10:00
f9332879ba
More hardening
All checks were successful
CI / test (push) Successful in 1m21s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m33s
Lint / test (push) Successful in 41s
2026-07-03 12:23:52 +10:00
c6ce0311ed
Forbid the ansible globals
All checks were successful
CI / test (push) Successful in 51s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m36s
Lint / test (push) Successful in 42s
2026-07-02 14:52:09 +10:00
054b90ebde
Don't reference built-in globals via loop-items in Jinja2 into keys
All checks were successful
CI / test (push) Successful in 52s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m31s
Lint / test (push) Successful in 43s
2026-07-02 14:22:07 +10:00
575c2b79f9
Fix catching of defusedxml exception
All checks were successful
CI / test (push) Successful in 48s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m29s
Lint / test (push) Successful in 39s
2026-07-01 12:23:19 +10:00
d4fd42522d
Fix bandit alert
All checks were successful
CI / test (push) Successful in 49s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m32s
Lint / test (push) Successful in 40s
2026-06-29 20:53:15 +10:00
42321a8ec9
Catch malformed config errors gracefully
Some checks failed
CI / test (push) Successful in 49s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m28s
Lint / test (push) Failing after 39s
2026-06-29 20:47:05 +10:00
3573e8e750
Make var name collapse underscores 2026-06-29 20:46:07 +10:00
5ae85ad11e
Fix tests
All checks were successful
CI / test (push) Successful in 47s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m25s
Lint / test (push) Successful in 37s
2026-06-29 14:44:39 +10:00
bc4af135b4
Fix tests
Some checks failed
CI / test (push) Successful in 49s
CI / test (debian, docker.io/library/debian:13, python3) (push) Failing after 1m22s
Lint / test (push) Successful in 37s
2026-06-29 14:31:39 +10:00
0e052a073a
Harden output 2026-06-29 14:30:46 +10:00
70be0f7e33
Hardening: use unsafe for ansible vars, ensure API use of JinjaTurtle uses safe XML parsing, avoid symlinks
Some checks failed
CI / test (push) Successful in 48s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m27s
Lint / test (push) Failing after 39s
2026-06-29 08:51:53 +10:00
4a1f2ac15e
More JSON defenses 2026-06-28 20:33:12 +10:00
661320558c
Go back to just jinja2 2026-06-25 17:07:58 +10:00
094c4d2274
Fixes
All checks were successful
CI / test (push) Successful in 47s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m19s
Lint / test (push) Successful in 37s
2026-06-24 17:42:18 +10:00
a9d56b66c5
Fixes for releasing
All checks were successful
CI / test (push) Successful in 46s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m18s
Lint / test (push) Successful in 38s
2026-06-23 18:03:29 +10:00
c5375b180f
0.5.6
All checks were successful
CI / test (push) Successful in 47s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m20s
Lint / test (push) Successful in 38s
2026-06-23 17:15:02 +10:00
383a529016
fix test
All checks were successful
CI / test (push) Successful in 46s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m19s
Lint / test (push) Successful in 40s
2026-06-23 15:57:17 +10:00
a5f8e7c481
Attempt to fix python bin
Some checks failed
CI / test (push) Successful in 47s
CI / test (debian, docker.io/library/debian:13, python3) (push) Successful in 1m22s
Lint / test (push) Failing after 39s
2026-06-23 15:42:42 +10:00
15770c8e3d
Upgrade to poetry2 and update dependencies
Some checks failed
CI / test (push) Failing after 53s
Lint / test (push) Failing after 42s
2026-06-23 14:30:42 +10:00
a5ca5f2974
Try to prevent what could lead to execution of embedded jinja in original files when converting 2026-06-23 14:20:05 +10:00
f5de32b778
Rewrite the README
All checks were successful
CI / test (push) Successful in 1m6s
Lint / test (push) Successful in 1m6s
2026-06-20 20:17:28 +10:00
13d1a2b972
0.5.5 2026-06-20 18:28:19 +10:00
00d0064362
Fixes for newlines in yaml 2026-06-20 18:27:22 +10:00
33 changed files with 2702 additions and 776 deletions

View file

@ -7,6 +7,14 @@ jobs:
test:
runs-on: docker
strategy:
fail-fast: false
matrix:
include:
- distro: debian
image: docker.io/library/debian:13
python: python3
steps:
- name: Checkout
uses: actions/checkout@v4
@ -17,13 +25,29 @@ jobs:
DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends python3-venv pipx
- name: Install Poetry
env:
PYTHON_BIN: ${{ matrix.python }}
POETRY_VERSION: "2.4.1"
run: |
pipx install poetry==1.8.3
/root/.local/bin/poetry --version
set -eux
if ! command -v pipx >/dev/null 2>&1; then
"${PYTHON_BIN}" -m pip install --user pipx
fi
PIPX_BIN="$(command -v pipx || true)"
if [ -z "${PIPX_BIN}" ]; then
PIPX_BIN="${HOME}/.local/bin/pipx"
fi
"${PIPX_BIN}" install --python "${PYTHON_BIN}" "poetry==${POETRY_VERSION}"
echo "$HOME/.local/bin" >> "$GITHUB_PATH"
export PATH="$HOME/.local/bin:$PATH"
poetry --version
poetry --version | grep -E "Poetry \(version 2\."
- name: Install project deps (including test extras)
env:
PYTHON_BIN: ${{ matrix.python }}
run: |
poetry env use "${PYTHON_BIN}"
poetry install --with dev
- name: Run test script

View file

@ -61,6 +61,23 @@ rsync -a --delete \
"${SRC}/" "${WORK}/"
cd "${WORK}"
# This project's pyproject.toml uses the PEP 621 [project] table, which needs
# poetry-core >= 2.0. Debian bookworm and Ubuntu jammy/noble ship an older
# poetry-core that cannot parse it. On those, swap in the legacy Poetry 1.x
# formatted metadata (kept in sync at pyproject.poetry1.toml) for the build.
# trixie and newer ship poetry-core 2.x and keep the PEP 621 file.
if [ -f pyproject.poetry1.toml ]; then
core_ver="$(python3 -c 'import poetry.core as c; print(c.__version__)' 2>/dev/null || echo 0)"
core_major="${core_ver%%.*}"
if [ "${core_major:-0}" -lt 2 ]; then
echo "poetry-core ${core_ver} < 2.0: using legacy pyproject.poetry1.toml"
cp pyproject.poetry1.toml pyproject.toml
else
echo "poetry-core ${core_ver} >= 2.0: using PEP 621 pyproject.toml"
fi
fi
if [ -n "${SUITE:-}" ]; then
export DEBEMAIL="mig@mig5.net"
export DEBFULLNAME="Miguel Jacq"

249
README.md
View file

@ -4,55 +4,134 @@
<img src="https://git.mig5.net/mig5/jinjaturtle/raw/branch/main/jinjaturtle.svg" alt="JinjaTurtle logo" width="240" />
</div>
JinjaTurtle is a command-line tool to help you generate Jinja2 templates and
Ansible inventory from a native configuration file (or files) of a piece of
software.
JinjaTurtle is a command-line tool that helps turn existing native
configuration files into reusable configuration-management templates.
By default it generates:
- a **Jinja2** template; and
- an **Ansible defaults YAML** file containing the variables used by that
template.
JinjaTurtle does not try to replace configuration-management tools. Its job is
to speed up the boring first pass: take a real config file, discover the values
inside it, replace those values with variables, and write the corresponding
variable data beside the template.
## How it works
* The config file(s) is/are examined
* Parameter key names are generated based on the parameter names in the
config file. In keeping with Ansible best practices, you pass a prefix
for the key names, which should typically match the name of your Ansible
role.
* A Jinja2 file is generated from the file with those parameter key names
injected as the `{{ variable }}` names.
* An Ansible inventory YAML file is generated with those key names and the
*values* taken from the original config file as the default vars.
JinjaTurtle examines a source config file and keeps the original structure as
much as possible.
By default, the Jinja2 template and the Ansible inventory are printed to
stdout. However, it is possible to output the results to new files.
For the default Jinja2/Ansible mode:
1. The config file is parsed.
2. Variable names are generated from the config keys and paths.
3. Those variable names are prefixed with `--role-name`, which should usually
match your Ansible role name.
4. A Jinja2 template is generated with values replaced by `{{ variable }}`
expressions.
5. An Ansible defaults YAML file is generated with those variables and the
original values.
By default, the generated variable data and template are printed to stdout. Use
`--defaults-output` and `--template-output` to write them to files.
## Jinja2 / Ansible example
Say you have a `php.ini` file and you are inside an Ansible role with
`defaults/` and `templates/` directories:
```shell
jinjaturtle php.ini \
--role-name php \
--defaults-output defaults/main.yml \
--template-output templates/php.ini.j2
```
Given a source value such as:
```ini
memory_limit = 256M
```
JinjaTurtle will produce a template value like:
```jinja2
memory_limit = {{ php_memory_limit }}
```
and defaults data like:
```yaml
php_memory_limit: 256M
```
## What sort of config files can it handle?
TOML, YAML, INI, JSON and XML-style config files should be okay. There are always
going to be some edge cases in very complex files that are difficult to work
with, though, so you may still find that you need to tweak the results.
JinjaTurtle supports common structured and semi-structured config formats:
For XML and YAML files, JinjaTurtle will attempt to generate 'for' loops
and lists in the Ansible yaml if the config file looks homogenous enough to
support it. However, if it lacks the confidence in this, it will fall back to
using scalar-style flattened attributes.
- TOML
- YAML
- INI-style files
- JSON
- XML
- Postfix `main.cf`
- systemd unit files, such as `*.service`, `*.socket`, `*.timer`, and related
unit types
- OpenSSH-style config files, including `ssh_config`, `sshd_config`, and common
`*.conf` snippets detected as SSH config
You may need or wish to tidy up the config to suit your needs.
For ambiguous extensions such as `*.conf`, JinjaTurtle uses lightweight content
sniffing. You can always force a handler with `--format`.
The goal here is really to *speed up* converting files into Ansible/Jinja2,
but not necessarily to make it perfect.
For YAML, XML, TOML, INI-style, and other supported structured files,
JinjaTurtle will attempt to generate loops when a repeated structure looks
homogeneous enough. If it is not confident, it falls back to flattened scalar
variables.
Some very complex files will still need manual cleanup. The goal is to speed up
conversion into Jinja2 templates, not to guarantee a perfect final module without
review.
## JSON, quoting, and type preservation
JinjaTurtle tries to preserve rendered config types.
For JSON, it uses JSON-aware expressions rather than plain string substitution.
This avoids generating invalid JSON such as:
```json
{"enabled": True}
```
when the correct rendered JSON should be:
```json
{"enabled": true}
```
This uses Ansible-style JSON filters.
## Can I convert multiple files at once?
Certainly! Pass the folder name instead of a specific file name, and JinjaTurtle
will convert any files it understands in that folder, storing all the various
vars in the destination defaults yaml file, and converting each file into a
Jinja2 template per file type.
Yes. Pass a directory instead of a single file and JinjaTurtle will convert the
files it understands in that directory.
If all the files had the same 'type', there'll be one Jinja2 template.
```shell
jinjaturtle ./config-dir \
--role-name myrole \
--defaults-output defaults/main.yml \
--template-output templates/
```
You can also pass `--recursive` to recurse into subfolders.
Use `--recursive` to recurse into subdirectories.
Note: when using 'folder' mode and multiple files of the same type, their vars
will be listed under an 'items' parent key in the yaml, each with an `id` key.
You'll then want to use a `loop` in Ansible later, e.g:
In folder mode, variables for multiple files of the same type are grouped under
an `items`-style structure in the generated YAML so that the resulting templates
can be used with loops in Ansible.
For example:
```yaml
- name: Render configs
@ -93,9 +172,9 @@ sudo dnf upgrade --refresh
sudo dnf install jinjaturtle
```
### From PyPi
### From PyPI
```
```bash
pip install jinjaturtle
```
@ -103,61 +182,97 @@ pip install jinjaturtle
Clone the repo and then run inside the clone:
```
```bash
poetry install
```
### AppImage
Download the AppImage from the Releases and make it executable, and put it
on your `$PATH`.
## How to run it
Say you have a `php.ini` file and you are in a directory structure like an
Ansible role (with subfolders `defaults` and `templates`):
```shell
jinjaturtle php.ini \
--role-name php \
--defaults-output defaults/main.yml \
--template-output templates/php.ini.j2
```
## Full usage info
```
usage: jinjaturtle [-h] -r ROLE_NAME [-f {json,ini,toml,yaml,xml,postfix,systemd}] [-d DEFAULTS_OUTPUT] [-t TEMPLATE_OUTPUT] config
```text
usage: jinjaturtle [-h] [-r ROLE_NAME] [--recursive]
[-f {ini,json,toml,yaml,xml,postfix,systemd,ssh}]
[-d DEFAULTS_OUTPUT] [-t TEMPLATE_OUTPUT]
config
Convert a config file into Ansible inventory and a Jinja2 template.
Convert a config file into an Ansible defaults file and Jinja2 template.
positional arguments:
config Path to the source configuration file.
config Path to a config file OR a folder containing supported
config files. Supported: .toml, .yaml/.yml, .json,
.ini/.cfg/.conf, .xml, ssh_config/sshd_config
options:
-h, --help show this help message and exit
-r, --role-name ROLE_NAME
Ansible role name, used as variable prefix (e.g. cometbft).
-f, --format {ini,json,toml,xml}
Force config format instead of auto-detecting from filename.
Role name / variable prefix. In Jinja2 mode this is
usually the Ansible role name. Defaults to jinjaturtle.
--recursive When CONFIG is a folder, recurse into subfolders.
-f, --format {ini,json,toml,yaml,xml,postfix,systemd,ssh}
Force config format instead of auto-detecting from
filename.
-d, --defaults-output DEFAULTS_OUTPUT
Path to write defaults/main.yml. If omitted, default vars are printed to stdout.
Path to write the generated variable YAML. If omitted,
it is printed to stdout.
-t, --template-output TEMPLATE_OUTPUT
Path to write the Jinja2 config template. If omitted, template is printed to stdout.
Path to write the generated config template. If omitted,
it is printed to stdout.
```
## Additional supported formats
JinjaTurtle can also template some common "bespoke" config formats:
JinjaTurtle also templates some common bespoke config formats:
- **Postfix main.cf** (`main.cf`) → `--format postfix`
- **systemd unit files** (`*.service`, `*.socket`, etc.) → `--format systemd`
- **OpenSSH config** (`ssh_config`, `sshd_config`, and detected snippets) →
`--format ssh`
For ambiguous extensions like `*.conf`, JinjaTurtle uses lightweight content sniffing; you can always force a specific handler via `--format`.
For ambiguous extensions like `*.conf`, JinjaTurtle uses lightweight content
sniffing. You can always force a specific handler with `--format`.
## Security model
JinjaTurtle is frequently pointed at config files that were *harvested* from
real systems, where some content may be influenced by an untrusted party (a
hostname, a login banner, a `GECOS` comment, a "Managed by ..." note). It is
therefore designed so that source content cannot turn into executable template
code.
Two guarantees matter:
1. **Values are data, never code.** Every config *value* is replaced with a
`{{ variable }}` placeholder in the template, and the original value is stored
separately in the defaults data. When the template is later rendered,
the placeholder prints the value as a literal string; Jinja2 does not
recursively render the *contents* of a variable, so a payload sitting inside
a value is inert.
2. **Verbatim text is neutralised.** To preserve formatting, JinjaTurtle copies
comments, blank lines, headers and any unrecognised lines from the source
into the template. Any template metacharacters in that copied text
(`{{ }}`, `{% %}`, `{# #}` ) are escaped so they render as the literal
characters the author wrote, rather than executing.
### Consumer responsibilities
The value guarantee above relies on the downstream renderer being single-pass,
which is the normal case:
- **Ansible**: rendering a template with `template:`/`ansible.builtin.template`
is single-pass. For defence in depth, treat the generated defaults as
untrusted input — Ansible already does not re-template variable *contents* by
default. If you build your own var structures from this data and pass them
through additional templating, mark untrusted values with the `!unsafe` tag so
they are never re-evaluated.
In short: render JinjaTurtle output exactly once. Do not feed it back through
another templating pass.
**IMPORTANT**: Always review both the original config files, then the resulting
templates generated by JinjaTurtle, before integrating them into your config
management system!
## Found a bug, have a suggestion?
You can e-mail me (see the pyproject.toml for details) or contact me on the Fediverse:
https://goto.mig5.net/@mig5
You can e-mail me; see `pyproject.toml` for details.

24
debian/changelog vendored
View file

@ -1,3 +1,27 @@
jinjaturtle (0.7.0) unstable; urgency=medium
* Much hardening.
-- Miguel Jacq <mig@mig5.net> Sun, 5 Jul 2026 10:54:00 +1000
jinjaturtle (0.5.7) unstable; urgency=medium
* More hardening measures
-- Miguel Jacq <mig@mig5.net> Wed, 24 Jun 2026 16:13:00 +1000
jinjaturtle (0.5.6) unstable; urgency=medium
* Try to prevent what could lead to execution of embedded jinja in original files when converting
-- Miguel Jacq <mig@mig5.net> Tue, 23 Jun 2026 16:47:00 +1000
jinjaturtle (0.5.5) unstable; urgency=medium
* erb support
-- Miguel Jacq <mig@mig5.net> Sat, 20 Jun 2026 18:27:00 +1000
jinjaturtle (0.5.4) unstable; urgency=medium
* Make templates more faithful to the original file in terms of indentation, newlines, no deserialisation of things like < or >.

239
poetry.lock generated
View file

@ -1,4 +1,4 @@
# This file is automatically @generated by Poetry 1.8.3 and should not be changed by hand.
# This file is automatically @generated by Poetry 2.4.1 and should not be changed by hand.
[[package]]
name = "certifi"
@ -6,6 +6,7 @@ version = "2026.6.17"
description = "Python package for providing Mozilla's CA Bundle."
optional = false
python-versions = ">=3.7"
groups = ["dev"]
files = [
{file = "certifi-2026.6.17-py3-none-any.whl", hash = "sha256:2227dcbaafe0d2f59279d1762ddddc37783ed4354594f194ffc31d20f41fc3db"},
{file = "certifi-2026.6.17.tar.gz", hash = "sha256:024c88eeec92ca068db80f02b8b07c9cef7b9fe261d1d535abfd5abd6f6af432"},
@ -17,6 +18,7 @@ version = "3.4.7"
description = "The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet."
optional = false
python-versions = ">=3.7"
groups = ["dev"]
files = [
{file = "charset_normalizer-3.4.7-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:cdd68a1fb318e290a2077696b7eb7a21a49163c455979c639bf5a5dcdc46617d"},
{file = "charset_normalizer-3.4.7-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e17b8d5d6a8c47c85e68ca8379def1303fd360c3e22093a807cd34a71cd082b8"},
@ -155,6 +157,8 @@ version = "0.4.6"
description = "Cross-platform colored terminal text."
optional = false
python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7"
groups = ["dev"]
markers = "sys_platform == \"win32\""
files = [
{file = "colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6"},
{file = "colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44"},
@ -162,124 +166,110 @@ files = [
[[package]]
name = "coverage"
version = "7.14.1"
version = "7.14.3"
description = "Code coverage measurement for Python"
optional = false
python-versions = ">=3.10"
groups = ["dev"]
files = [
{file = "coverage-7.14.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:3e3680291c4a1d0dadfa84a2c459576a4af5133abb617905714339a0c73138cf"},
{file = "coverage-7.14.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:a5274669f37f2343635a347b91a60777621341ab3378e9c6ac9335eee704bddf"},
{file = "coverage-7.14.1-cp310-cp310-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:cfe5a5fec635799ef33428f1e5e61bafa45a92a96190ba731561ba558ccc214d"},
{file = "coverage-7.14.1-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:62a9f70b52e0b5a95cfef4a5c5641b06983cadc5e538a3feeb5c00211f523ac2"},
{file = "coverage-7.14.1-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3c18ebc343e15be53049b3a2dce38fe82d58f37e20ab9094b3a39c0aa4f6bb47"},
{file = "coverage-7.14.1-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:b84ffdf877644e7096aa936991efeed873f7f3df57b9cd001312b7668ab08550"},
{file = "coverage-7.14.1-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e854312c4103f2ad4c0dc023b69b77ebfd2c89db5f86c4c94dc2353f9a92167e"},
{file = "coverage-7.14.1-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:c643734307300234fafa36bf2a040a7235f8f177ea1fd6ec1423aea6fb7b929f"},
{file = "coverage-7.14.1-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:84ac9499e48700399a5dd0ea7085b5091961fec52c68d66b4ec0d3cf7f4441b1"},
{file = "coverage-7.14.1-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:7f02d09f70776579b926d889a4c9c235070a1f47c40458aeaca563fae5acfdb5"},
{file = "coverage-7.14.1-cp310-cp310-musllinux_1_2_riscv64.whl", hash = "sha256:ce66d8e46da2bb5ee313a745cbd2e391d319176c1f7a9451bfcd3a2fb920859b"},
{file = "coverage-7.14.1-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:c912c259304cfb5ee584481cfb7ce1ff932b4d61e6c9140b8f19cb7b5ed82332"},
{file = "coverage-7.14.1-cp310-cp310-win32.whl", hash = "sha256:1238cb94638e610e972c60dac68e813f868dc7d6e982535270558443058d9d59"},
{file = "coverage-7.14.1-cp310-cp310-win_amd64.whl", hash = "sha256:fc459e5d73be2d6332fcfe8dbf3d8994671fe33c700f4565988ecfa511547253"},
{file = "coverage-7.14.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:478b5bcd63c2e1357c5c7e16c070690df7b07f676b1c114d7b93e533c664309f"},
{file = "coverage-7.14.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:a24a81f9715ee42ef59a316cc11611c98fe23920f7c81861315c9f3ff4a230f4"},
{file = "coverage-7.14.1-cp311-cp311-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:196a13319ad88d6d8ef5ab489ec4f44ddde2143c0c7d5b27786f6c3ffd56a7e1"},
{file = "coverage-7.14.1-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:3d452fd08b5c72c5167c93e6867b5c08500bd40f2a21e1e854a500550b6cc36f"},
{file = "coverage-7.14.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:23bf7fa51ac02e07fc7c96849b82946da47ae862dc8f86d183b2a4864fc38129"},
{file = "coverage-7.14.1-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:bcaa50684dcaadfa599ac48f81103c756d791cfd85c97203d2217c593d48b860"},
{file = "coverage-7.14.1-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:4ea1c034f95c9b056e856b794630b17f9fa3d57e4800ff1e503d3be0f9c9078c"},
{file = "coverage-7.14.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:c7e057326434e441306226fbeb5d1aaf14a2637efe97ba668306635835f32ad7"},
{file = "coverage-7.14.1-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:59baf88468dbc8d63b1887afd92bda52e40bb1561696e5819670601403810cec"},
{file = "coverage-7.14.1-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:d34d75f892b3ab73ba11cab5442cce7b3e168fd64162b16f0e1e0d09c508edef"},
{file = "coverage-7.14.1-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:3a56abc20a472baf0304c455721bc601477440d28ecfde8a03dde79ede07e0df"},
{file = "coverage-7.14.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:6a3cb83d1552c0cd1b4906655b6a33fd4a8473229633a901c6b73bf86914dee9"},
{file = "coverage-7.14.1-cp311-cp311-win32.whl", hash = "sha256:10274a1fbeb8ec5d72966e17bb198a3104257aca4ac09d98667c5f8aca8c8548"},
{file = "coverage-7.14.1-cp311-cp311-win_amd64.whl", hash = "sha256:87ebdf787d4888e3f3f2d523eadc6e18c6d18c6d0eb173801a189641627fb37e"},
{file = "coverage-7.14.1-cp311-cp311-win_arm64.whl", hash = "sha256:dd34767fa19848d35659ffc0a75314f58c7af3f1cd87ec521e8292a1238398a3"},
{file = "coverage-7.14.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:a06c76364a9360e33d6d23769aefdf7f66f38e2ffb60ceb1baaa4989d83b695c"},
{file = "coverage-7.14.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fad54e871165f6ec2f536063ac74c3104508a12963e64072ba44bd822de52b0c"},
{file = "coverage-7.14.1-cp312-cp312-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:84b535f00655ecafe1d929d1fb00ed5d6fa3051ea643ab2c161a3887b86f294b"},
{file = "coverage-7.14.1-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:6b6b0853b895fe0e98cbfc580d1ec3393d9302b4b1e96a77b3f5c91fdab899e6"},
{file = "coverage-7.14.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:442cc9c952b2df400cda54bb04ab87330cf2cd08a8692cbbea36773531eb6f37"},
{file = "coverage-7.14.1-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:8270544c361ed405a27a060dbc9ed2c124b084d96dfdc2d9a2510482aef981ad"},
{file = "coverage-7.14.1-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:48b283b1dd6372e8de2a7a9a4c4d5dc06f4d4fd209b876f3c88a7a205a0c8f84"},
{file = "coverage-7.14.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:5b0c99ba93a07d56f6df340bb79be53202a082b2fdb81bfe6190b741a3470d54"},
{file = "coverage-7.14.1-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:e471bc5769ff073b058cfadb0d736b56ce067c8560eabeb0da88462df98c23e7"},
{file = "coverage-7.14.1-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:f497a1ea81d4cd7c10ddcaa685135b9aabd291af3d55775a9ddf3cb7a364cdd9"},
{file = "coverage-7.14.1-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:2222be86d0b54f5dd5a38f45f17f315f737245e857bf0bdedc70734f84a13c02"},
{file = "coverage-7.14.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:85e85586565842f6932abebd4c18bcb1074223dc0b3576e7d173ca710622813a"},
{file = "coverage-7.14.1-cp312-cp312-win32.whl", hash = "sha256:4a28fd227808366b196a75476dced2eb35b351d6766ba9c858dc93319e87f4f1"},
{file = "coverage-7.14.1-cp312-cp312-win_amd64.whl", hash = "sha256:54acdb6674a4661768d7bf7db32dfb9f46ab1d764f8aba6df75ce1a6a088724e"},
{file = "coverage-7.14.1-cp312-cp312-win_arm64.whl", hash = "sha256:99cd41ff91afd94896fea3bc002706b6ae4ce95727d06e4a0f39c0a8d8bd8b1a"},
{file = "coverage-7.14.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:be9f2c802dcfce3f71298303aa5dad0dce440a76c52f2f60dacd8656dab78793"},
{file = "coverage-7.14.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:6223a72fd0e4c7156353ec0f08a5f93623e1d3034d0e2683b9bb8ea674131b1d"},
{file = "coverage-7.14.1-cp313-cp313-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:7279d2110a28cebc738b6459ecda2771735a4c18465fbbd36b3288fe5ed92247"},
{file = "coverage-7.14.1-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:9eeb3fcbc13ba40dfbdb22d01d196a28e9cef9ed4c29b60061a1e0e823a9929d"},
{file = "coverage-7.14.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5f0cfc27c539f07cf5c0a4cfe211d0b6cae039f8f40526dbaa71944e64b50a7b"},
{file = "coverage-7.14.1-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:221c70f316241a78e77e607c227cefc8808d4e08f28d99c04f35694690e940be"},
{file = "coverage-7.14.1-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:da028256b04ec30e5e0114b6f76172938c313991f0a2d3d894271315cf5d5e43"},
{file = "coverage-7.14.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:76a085d7005236a767e3426148b2c407e53ad61695c562f8a81da2d373324901"},
{file = "coverage-7.14.1-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:b553d04b5e778a8e56d57eb134aff42a92718ecba45e79c4764ecfa40efd92ff"},
{file = "coverage-7.14.1-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:46f714d2fb8ae2f4f29f23ada7f1e79b759fff5a70f94a1dac23af204c3ec9e4"},
{file = "coverage-7.14.1-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:1896f5e19ff3f0431c7ce2172adc54890fd97f86b59ced8ca1649145d9ffe35d"},
{file = "coverage-7.14.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:62fd185ef9df3c33d1c8178c5af105f762afbad96038de9a4ae100aa6297ca33"},
{file = "coverage-7.14.1-cp313-cp313-win32.whl", hash = "sha256:ab4af6352741a604c431c6072fce5bee33bf0f20dc7a56618d6bf6bb89e9810c"},
{file = "coverage-7.14.1-cp313-cp313-win_amd64.whl", hash = "sha256:7af486dabe8954d03b087f0021540897afe084f04e16ff5579e08cc46f871416"},
{file = "coverage-7.14.1-cp313-cp313-win_arm64.whl", hash = "sha256:2224f89ffd0c5605ccce1ed7a584da162bc7c55f601ab1c946bc9de31a486b42"},
{file = "coverage-7.14.1-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:de286598cc65d2b489411174b1faec2f5a7775fb3201fd925db2a76b4030f37d"},
{file = "coverage-7.14.1-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:042c46ded7c288aeb07cf14a28b6c1e10b78fcba40171c3fa1e939377eeef0b5"},
{file = "coverage-7.14.1-cp313-cp313t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:f4ddbe407477f04c45115d1a4e5bc480f753553b534d338d4c3358b1cdd0ea52"},
{file = "coverage-7.14.1-cp313-cp313t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:d13e6725992e2d2fd7d81d4f5241952d13740121dfd501da09201be39b2c003a"},
{file = "coverage-7.14.1-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f747dc8edcfe740130f28f32f3995e955494285717e86ee25af51db2219df08a"},
{file = "coverage-7.14.1-cp313-cp313t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:ced2f09ef276fd58611a1ef502164ad266d2b75174e5a40cabbdb4033f9f6cf2"},
{file = "coverage-7.14.1-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b84800013769a78ccb9ef4659402e26d06867e337b61ec365f77ad008adea80e"},
{file = "coverage-7.14.1-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:ea8cd6ca0ee9f616aaef3afc6882e32c2cbf18b00d96313ffd76af650574034d"},
{file = "coverage-7.14.1-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:aa5e304a873fabddc11e484e9b6b738bd38bd7bed17b09aa84eecf5332e8b8bb"},
{file = "coverage-7.14.1-cp313-cp313t-musllinux_1_2_ppc64le.whl", hash = "sha256:5a1c5215be81035e629d5bc756650634d0bf31991038db7a0eccb90f025ce16d"},
{file = "coverage-7.14.1-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:79058c47dae6788504b5effb319961bcd72d7240551464b91d474bc0ed186d69"},
{file = "coverage-7.14.1-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:370c5afae3fa0658e11694a32b24c2778f6bc2d17718121f94ee185e69f26b54"},
{file = "coverage-7.14.1-cp313-cp313t-win32.whl", hash = "sha256:3758dd0a7f1fa57365ef2e781df0f0731d38b6e3772259d13dae4bd8a958d4b1"},
{file = "coverage-7.14.1-cp313-cp313t-win_amd64.whl", hash = "sha256:6ff665fb023a77386fe11685190cee1f60a7d635994a30d9b0a061533d470fce"},
{file = "coverage-7.14.1-cp313-cp313t-win_arm64.whl", hash = "sha256:17a5a241e5997621a956a7f402a7433ef4221e5152809b785bec79e2323799f1"},
{file = "coverage-7.14.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:d5ed429d0b8edaac649e889b4ffcedb6c80b06629a3f93050e3dddfb99235bee"},
{file = "coverage-7.14.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:8011224a62280e50dab346960c03cf47aca1a1e09e608c0fb33fd6e0cc8e9500"},
{file = "coverage-7.14.1-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:12c42ec1e14f553c4f817e989365982e646e27211f10a0f717855b94a79c8906"},
{file = "coverage-7.14.1-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:06144cd511cf2624873a035c5069cf297144f6e77a73ee3d7a55b605ec5efb42"},
{file = "coverage-7.14.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:a311d8e1da24be5c1ccf85cbfb06315dbaa1703d5a1eab3f6432c72b837917c8"},
{file = "coverage-7.14.1-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:c79cead5b5bc584d9c71451cb984d0e3a84e0c0937379c8efcbf27c8d661b851"},
{file = "coverage-7.14.1-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:dcbf65f1f66a26cdd88c35cf68fb4729c5d1cd2e88added72420541dfb212034"},
{file = "coverage-7.14.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:fd86572566fb40189a8260446158235159bc7a82dfbc87a3b39cf4fb57fcec1c"},
{file = "coverage-7.14.1-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:7771b601718fdde84832c3a434ca9bbf4ae9adbc49d84198b4110700c3c77c36"},
{file = "coverage-7.14.1-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:39b21e212c55af06fa375e3dbf90a8a8e38792f3a910c580066d23563830ddd5"},
{file = "coverage-7.14.1-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:f2302660e32562a532b442480121aef8aa61a5bdb20b30bf0adab29f10a5a4b4"},
{file = "coverage-7.14.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:03a6f93c1ec3b7f2e77b5dbcc5573a2c21f12529a5c6bbe0f16f72303cc2fa4d"},
{file = "coverage-7.14.1-cp314-cp314-win32.whl", hash = "sha256:8a3ce026d73290f42f08dafecbd82c193a74df280461fbf97300fec51fd133ee"},
{file = "coverage-7.14.1-cp314-cp314-win_amd64.whl", hash = "sha256:114c95ef29302423b87d159075805f4ab973254a2638a5d7d046c94887cc87d7"},
{file = "coverage-7.14.1-cp314-cp314-win_arm64.whl", hash = "sha256:a07891c3f4805442b31b71e84ba3cf29ed1aa9a428284e06deeb4b23e5b46343"},
{file = "coverage-7.14.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:1101a5ebb083aecb625ebb6209d4105b58f647b093cb2dc8122d7b33f743cfe1"},
{file = "coverage-7.14.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:851b9e1e4e8a4608e77c79714b2e77c0970d2ed7202a05e92ae407817481887b"},
{file = "coverage-7.14.1-cp314-cp314t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:d5b89cdfb2ee051b71e8c3c70bd81a9eff81100f736a269136fe1a68efe00474"},
{file = "coverage-7.14.1-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:0177614a0370f227888b4e436a7c55686d6a9f90eb1ade2b624ba685a1686e86"},
{file = "coverage-7.14.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2d69af5dea2de76fc485a83032a630523f985198b7e25be901ec60181587b01e"},
{file = "coverage-7.14.1-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:35ab22d91de736e8966b980dc355cbcdd2c6dbbcfe275f9a2991bc8a91b3df65"},
{file = "coverage-7.14.1-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:357d4e32935c36588aaba057d734fa32428c360c9fc2e4442afbf1b646beee6e"},
{file = "coverage-7.14.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:51bd64741cc6fa065abd300ede1afe5a5291ece9c31da8b24884deda48bcc3f8"},
{file = "coverage-7.14.1-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:9132cd363a68a4c3daa7c8704a654b1e39d3360f6f5b8ddd470608a945236c07"},
{file = "coverage-7.14.1-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:07c6290b1697b862c0478eab545eec949a0d0e4d6d03497f446d706da3b4f2de"},
{file = "coverage-7.14.1-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:5ea0c297e27133853b4d8a3eb799bff5a2dbd9f2f41537a240d337ac9b4df890"},
{file = "coverage-7.14.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:01b7733daad0237daa01ef80fe2dfceffc911e6a17fa7b55d14aa8214eaaaecd"},
{file = "coverage-7.14.1-cp314-cp314t-win32.whl", hash = "sha256:6adc5a36984624a70bf11d7184e20fa0a49aa7c47ffab43804106a1a695ea22e"},
{file = "coverage-7.14.1-cp314-cp314t-win_amd64.whl", hash = "sha256:ddf799247318f34dbcd2efa8c95a8d0642674e926bb1774cf9b63dfd2a389d1c"},
{file = "coverage-7.14.1-cp314-cp314t-win_arm64.whl", hash = "sha256:145986fe66647eb489f18d9a997567a3fd358584c4b5a808769113abc07466af"},
{file = "coverage-7.14.1-py3-none-any.whl", hash = "sha256:a252f21c27e38347e60111a3266b03827422a7d5525951aceee313aa68bab1d2"},
{file = "coverage-7.14.1.tar.gz", hash = "sha256:30c08f7d90415aa98b3c990385dea2939b0da55f38515e5b369b83655f8523be"},
{file = "coverage-7.14.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:360bec1f58e7243e3405d3bdf7a1a8115aa9b448d54dc7cd6f7b7e0e9406b62e"},
{file = "coverage-7.14.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:ed68faa5e85de2f3e400bc3f122e5c82735a58c8bb24b9f63a2215954ba17b2d"},
{file = "coverage-7.14.3-cp310-cp310-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:830c1fca669c572dec37ce9c838224ee45aac5be0f6961edf871e82e49d6537c"},
{file = "coverage-7.14.3-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:a64caee2193563601dbaaa55fe2dcf597debef04a2f8f1fa8a07aa4bb7ac7a1e"},
{file = "coverage-7.14.3-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0096fd7559178f0cc9cf088f2dbd2a02ef85bacaa69732c633517286b4494610"},
{file = "coverage-7.14.3-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:6197e5a00183c11a8ce7c6abd18be1a9189fd8399084ffc95196f4f0db4f2137"},
{file = "coverage-7.14.3-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:7dfe427045520d6abca33687dfef767b4f635015893a1816c5decb12eb72ce18"},
{file = "coverage-7.14.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:9a3f142070eb7b82fc4085a55d887396f9c4e21250bccebe2ba22502c45b9647"},
{file = "coverage-7.14.3-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:64b2055bb6e0dc945af35cdeceb3633e6ed9273475ef3af85592410fd6803803"},
{file = "coverage-7.14.3-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:1551b4caac3e3ec9f2bfcec6bf3776e01c0edbdd2e240431a50ca1a1aac72c27"},
{file = "coverage-7.14.3-cp310-cp310-musllinux_1_2_riscv64.whl", hash = "sha256:583d50d59142f8549470bd6390471d0fe8b8c8d69d6a0f28ac71e05380cef640"},
{file = "coverage-7.14.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:e0bb8a6bc7015efdf8a928753b25da1b9ca2d6f24ef04d2ee0688e486f32aae7"},
{file = "coverage-7.14.3-cp310-cp310-win32.whl", hash = "sha256:d48400185564042287dc487c1f016a3397f18ab4f4c5d5ec36edc218f7ffa35b"},
{file = "coverage-7.14.3-cp310-cp310-win_amd64.whl", hash = "sha256:eadea7aba74e40adee867a8c0eec17b820b061d308a4b014f7a0e118c2b0aa61"},
{file = "coverage-7.14.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:e574801e1d643561594aa021206c46d80b257e9853087090ba97bed8b0a509d3"},
{file = "coverage-7.14.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:f82b6bb7d75a2613e85d07cefa3a8c973d0544a8993337f6e2728e4a1e94c305"},
{file = "coverage-7.14.3-cp311-cp311-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:a2335ea5fed26af2e831094964fa3f8fae60b45f7e37fcc2d3b615b2add3ad87"},
{file = "coverage-7.14.3-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:fbb8c3a98e779013786ae01d229662aeacbc77100efbd3f2f245219ace5af700"},
{file = "coverage-7.14.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ac082660de8f429ba0ea363595abb838998570b9a7546777c60f413ab902bbde"},
{file = "coverage-7.14.3-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:8ac012839ff7e396030f1e94e10553a431d14e4de2ab65cb3acb72bbd5628ca2"},
{file = "coverage-7.14.3-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:5952f8c1bda2a5347154450379316e6dfa4d934d62ca35f6784451e6f55074fb"},
{file = "coverage-7.14.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:8cf0f2509acb4619e2471a1951089054dd58ebea7a912066d2ea56dd4c24ca4a"},
{file = "coverage-7.14.3-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:2e41fd3aab806770008279a93879b0924b16247e09ab537c043d08bbca53b4ab"},
{file = "coverage-7.14.3-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:f0a47095963cfe054e0df178daca95aec21e680d6076da807c3add28dfe920f7"},
{file = "coverage-7.14.3-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:a090cbf9521e78ffdb2fcf448b72902afe9f5923ff6a12d5c0d0120200348af9"},
{file = "coverage-7.14.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:4d310baf69a4fbe8a098ce727e4808a34866ac718a6f759ae659cbd3221358bc"},
{file = "coverage-7.14.3-cp311-cp311-win32.whl", hash = "sha256:74fdd718d88fe144f4579b8747873a07ec3f04cb837d5faec5a25d9e22fa31a8"},
{file = "coverage-7.14.3-cp311-cp311-win_amd64.whl", hash = "sha256:cc96aa922e21d4bc5d5ed3c915cef27dfcbc13686f47d5e378d647fbfba655a2"},
{file = "coverage-7.14.3-cp311-cp311-win_arm64.whl", hash = "sha256:c66f9f9d4f1e9712eb9b1de5310f881d4e2188cfcba5065e1a8490f38687f2c4"},
{file = "coverage-7.14.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:3d74ff26299c4879ce3a4d826f9d3d4d556fd285fde7bbce3c0ef5a8ab1cec24"},
{file = "coverage-7.14.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:96150a9cf3468ea20f0bc5d0e21b3df8972c31480ef90fa7614b773cc6429665"},
{file = "coverage-7.14.3-cp312-cp312-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:27d07a46500ba23515b838dbcf52512026af04090755cf6cc64166d88c9b9a1a"},
{file = "coverage-7.14.3-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:621e13c6108234d7960aaf5762ab5c3c00f33c30c15af06dcbff0c73bf112727"},
{file = "coverage-7.14.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4b60ca6d8af70473491a15a343cbabab2e8f9ea66a4376e81c7aa24876a6f977"},
{file = "coverage-7.14.3-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:c90a7cdd5e380e1ce02f19792e2ac2fbfbf177e35a27e69fd3e873b30d895c0c"},
{file = "coverage-7.14.3-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:5d788e5fd55347eef06ca0732c77d04a264de67e8ff24631270cdff3767a60cf"},
{file = "coverage-7.14.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:62c7f79db2851c95ef020e5d28b97afde3daf9f7febcd35b53e05638f729063f"},
{file = "coverage-7.14.3-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:90f7608aeb5d9b60b523b9fb2a4ee1973867cc4865a3f26fe6c7577073b70205"},
{file = "coverage-7.14.3-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:1e3b91f9c4740aeb571ecf82e5e8d8e4ab62d34fcb5a5d4e5baa38c6f7d2857c"},
{file = "coverage-7.14.3-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:c946099774a7699de03cbd0ff0a64e21aed4525eed9d959adde4afe6d15758ef"},
{file = "coverage-7.14.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:16b206e521feb8b7133a45754643dead0538489cf8b783b90cf5f4e3299625fd"},
{file = "coverage-7.14.3-cp312-cp312-win32.whl", hash = "sha256:ea3169c7116eb6cdf7608c6c7da9ecfcb3da40688e3a510fac2d1d2bafd6dc35"},
{file = "coverage-7.14.3-cp312-cp312-win_amd64.whl", hash = "sha256:7ea52fc08f007bcc494d4bb3df3851e95843d881860ba38fe2c64dc100db5e7d"},
{file = "coverage-7.14.3-cp312-cp312-win_arm64.whl", hash = "sha256:8cec0ad652ec57790970d817490105bd917d783c2f7b38d6b58a0ca312e1a336"},
{file = "coverage-7.14.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:47968988b367990ae4ab17523790c38cd125e02c6bfd379b6022be2d40bdc38c"},
{file = "coverage-7.14.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:0ee68f5c34812780f3a7063382c0a9fcbb99985b7ddcdcaa626e4f3fb2e0783a"},
{file = "coverage-7.14.3-cp313-cp313-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:fa9e5c6857a7e80fa22ace5cf3550ae392bbfc322f1d8dd2d2d5a8be38cec027"},
{file = "coverage-7.14.3-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:98a0859b0e98e43e1178a9402e19c8127766b14f7109a374d976e5a62c0e5c73"},
{file = "coverage-7.14.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:69918344541ed9c8368566c2adc03c0e33d4550d7faa87d1b35e49b6a3286ea9"},
{file = "coverage-7.14.3-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:b7f300ac92cd4b570724c8ffbbd0c130fee298d2447f41d5a3abf58976fae1de"},
{file = "coverage-7.14.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:11a7ec9f97ab950f4c5af62229befc7faf208fdbc0116d3902d7e306cf2c5abd"},
{file = "coverage-7.14.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a571bd889cd36c5922ce8e42e059f9d37d02301531d11374afa4c87a578625d5"},
{file = "coverage-7.14.3-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:de76caefc8deabb0dd1678b6a980be97d14c8d87e213ac194dbf8b09e96d63fb"},
{file = "coverage-7.14.3-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:d20a15c622194234161535459affa8f7905830391c9ccfa060d495dbfe3a1c7f"},
{file = "coverage-7.14.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:b488bd4b23397db62e7a9459129d01ff06a846582a732efd24834b24a6ada498"},
{file = "coverage-7.14.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:6a3693b4153394d265f44fb855fdc80e72403024d4d6f91c4871b334d028e4e0"},
{file = "coverage-7.14.3-cp313-cp313-win32.whl", hash = "sha256:338b19131ab1a6b767b462bfcbaa692e7ae22f24463e39d49b02a83410ff6b37"},
{file = "coverage-7.14.3-cp313-cp313-win_amd64.whl", hash = "sha256:b3d77f7f196abdef7e01415de1bce09f216189e83e58159cfeef2b92d0464994"},
{file = "coverage-7.14.3-cp313-cp313-win_arm64.whl", hash = "sha256:e6230e688c7c3e65cedd41a774eb4ec221adc6bfee13768231015b702d5e4150"},
{file = "coverage-7.14.3-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:605ab2b566a22bd94834529d66d295c364aba84afd3e5498285c7a524017b1fc"},
{file = "coverage-7.14.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:a3c2134809e80fac091bfed18a6991b5a5eb5df5ae32b17ac4f4f99864b73dd7"},
{file = "coverage-7.14.3-cp314-cp314-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:c02efd507227bde9969cab0db8f48890eb3b5dcad6afac57a4792df4133543ce"},
{file = "coverage-7.14.3-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:1bb93c2aa61d2a5b38f1526546d95cf4132cb681e541a337bf8dfd092be816e5"},
{file = "coverage-7.14.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f502e948e03e866538048bba081c075caaa62e5bda6ea5b7432e45f587eb462a"},
{file = "coverage-7.14.3-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:9973ef2463f8e6cfb61a6324126bb3e17d67a85f22f58d856e583ea2e3ca6501"},
{file = "coverage-7.14.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:9be4e7d4c5ca0427889f8f9d614bd630c2be741b1de7699bca3b2b6c0e41003e"},
{file = "coverage-7.14.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a574912f3bde4b0619f6e97d01aa590b70998859244793769eb3a6df78ee56d3"},
{file = "coverage-7.14.3-cp314-cp314-musllinux_1_2_i686.whl", hash = "sha256:e343fb086c9cd780b38622fea7c369acd64c1a0724312149b5d769c387a2b1f5"},
{file = "coverage-7.14.3-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:3c68df8e61f1e09633fefc7538297145623957a048534368c9d212782aa5e845"},
{file = "coverage-7.14.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:3e5b550a128419373c2f6cec28a244207013ef15f5cbcff6a5ca09d1dfaaf027"},
{file = "coverage-7.14.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2bfc4dd0a912329eccc7484a7d0b2a38032b38c40663b1e1ac595f10c457954b"},
{file = "coverage-7.14.3-cp314-cp314-win32.whl", hash = "sha256:0423d64c013057a06e70f070f073cec4b0cbc7d2b27f3c7007292f2ff1d52965"},
{file = "coverage-7.14.3-cp314-cp314-win_amd64.whl", hash = "sha256:92c22e19ce64ca3f2ad751f16f14df1468b4c231bd6af97185063a9c292a0cb3"},
{file = "coverage-7.14.3-cp314-cp314-win_arm64.whl", hash = "sha256:41de778bd41780586e2b04912079c73089ab5d839624e28db3bdb26de638da92"},
{file = "coverage-7.14.3-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:8427f370ca67db4c975d2a26acfc0e5783ca0b52444dbc50278ace0f35445949"},
{file = "coverage-7.14.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:d8e88f335544a47e22ae2e45b344772925ec65166555c958720d5ed971880891"},
{file = "coverage-7.14.3-cp314-cp314t-manylinux1_i686.manylinux_2_28_i686.manylinux_2_5_i686.whl", hash = "sha256:beaab199b9e5ceaf5a225e16a9d4df136f2a1eae0a5c20de1e277c8a5225f388"},
{file = "coverage-7.14.3-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", hash = "sha256:b3ff255799f5a1676c71c1c32ec01fd043aa09d57b3d95764b24992757184784"},
{file = "coverage-7.14.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:878832eaac515b62decfa76965aed558775f86bf1fc8cca76993c0c84ae31aed"},
{file = "coverage-7.14.3-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:611e62cb9386096d81b63e0a05330750268617231e7bd598e1fe77482a2c58a5"},
{file = "coverage-7.14.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:02c41de2a88011b893050fc9830267d927a50a215f7ad5ec17349db7090ccf26"},
{file = "coverage-7.14.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:526ce9721116af23b1065089f0b75046fe521e7772ab94b641cd66b7a0421889"},
{file = "coverage-7.14.3-cp314-cp314t-musllinux_1_2_i686.whl", hash = "sha256:e4ed44705ca4bead6fc977a8b741f2145608289b33c8a9b42a95d0f15aedbf4d"},
{file = "coverage-7.14.3-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:2415902f385a23dcc4ccd26e0ba803249a169af6a930c003a4c715eeb9a5444e"},
{file = "coverage-7.14.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:b75ee850fc2d7c831e883220c445b035f2224de2ba6103f1e56dbd237ab913f7"},
{file = "coverage-7.14.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:dc9b4e35e7c3920e925ba7f14886fd5fbe481232754624e832ddba66c7535635"},
{file = "coverage-7.14.3-cp314-cp314t-win32.whl", hash = "sha256:7b27c822a8161afbe48e99f1adfb098d270ae7e0f7d7b0555ce110529bdb69cc"},
{file = "coverage-7.14.3-cp314-cp314t-win_amd64.whl", hash = "sha256:39e1dbbb6ff2c338e0196a482558a792a1de3aa64261196f5cdb3da016ad9cda"},
{file = "coverage-7.14.3-cp314-cp314t-win_arm64.whl", hash = "sha256:68520c90babfa2d560eca6d497921ed3a4f469623bd709733124491b2aa8ef3f"},
{file = "coverage-7.14.3-py3-none-any.whl", hash = "sha256:fb7e18afb6e903c1a92401a2f0501ac277dca527bb9ca6fe1f691a8a0026a0e8"},
{file = "coverage-7.14.3.tar.gz", hash = "sha256:1a7563a443f3d53fdeb040ec8c9f7466aed7ca3dc5891aa09d3ca3625fa4387f"},
]
[package.dependencies]
tomli = {version = "*", optional = true, markers = "python_full_version <= \"3.11.0a6\" and extra == \"toml\""}
[package.extras]
toml = ["tomli"]
toml = ["tomli ; python_full_version <= \"3.11.0a6\""]
[[package]]
name = "defusedxml"
@ -287,6 +277,7 @@ version = "0.7.1"
description = "XML bomb protection for Python stdlib modules"
optional = false
python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
groups = ["main"]
files = [
{file = "defusedxml-0.7.1-py2.py3-none-any.whl", hash = "sha256:a352e7e428770286cc899e2542b6cdaedb2b4953ff269a210103ec58f6198a61"},
{file = "defusedxml-0.7.1.tar.gz", hash = "sha256:1bb3032db185915b62d7c6209c5a8792be6a32ab2fedacc84e01b52c51aa3e69"},
@ -298,6 +289,7 @@ version = "5.0"
description = "A library for working with .desktop files"
optional = false
python-versions = ">=3.10"
groups = ["dev"]
files = [
{file = "desktop_entry_lib-5.0-py3-none-any.whl", hash = "sha256:e60a0c2c5e42492dbe5378e596b1de87d1b1c4dc74d1f41998a164ee27a1226f"},
{file = "desktop_entry_lib-5.0.tar.gz", hash = "sha256:9a621bac1819fe21021356e41fec0ac096ed56e6eb5dcfe0639cd8654914b864"},
@ -312,6 +304,8 @@ version = "1.3.1"
description = "Backport of PEP 654 (exception groups)"
optional = false
python-versions = ">=3.7"
groups = ["dev"]
markers = "python_version == \"3.10\""
files = [
{file = "exceptiongroup-1.3.1-py3-none-any.whl", hash = "sha256:a7a39a3bd276781e98394987d3a5701d0c4edffb633bb7a5144577f82c773598"},
{file = "exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219"},
@ -329,6 +323,7 @@ version = "3.18"
description = "Internationalized Domain Names in Applications (IDNA)"
optional = false
python-versions = ">=3.9"
groups = ["dev"]
files = [
{file = "idna-3.18-py3-none-any.whl", hash = "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2"},
{file = "idna-3.18.tar.gz", hash = "sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848"},
@ -343,6 +338,7 @@ version = "2.3.0"
description = "brain-dead simple config-ini parsing"
optional = false
python-versions = ">=3.10"
groups = ["dev"]
files = [
{file = "iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12"},
{file = "iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730"},
@ -354,6 +350,7 @@ version = "3.1.6"
description = "A very fast and expressive template engine."
optional = false
python-versions = ">=3.7"
groups = ["main"]
files = [
{file = "jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67"},
{file = "jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d"},
@ -371,6 +368,7 @@ version = "3.0.3"
description = "Safely add untrusted strings to HTML/XML markup."
optional = false
python-versions = ">=3.9"
groups = ["main"]
files = [
{file = "markupsafe-3.0.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559"},
{file = "markupsafe-3.0.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419"},
@ -469,6 +467,7 @@ version = "26.2"
description = "Core utilities for Python packages"
optional = false
python-versions = ">=3.8"
groups = ["dev"]
files = [
{file = "packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e"},
{file = "packaging-26.2.tar.gz", hash = "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661"},
@ -480,6 +479,7 @@ version = "1.6.0"
description = "plugin and hook calling mechanisms for python"
optional = false
python-versions = ">=3.9"
groups = ["dev"]
files = [
{file = "pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746"},
{file = "pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3"},
@ -495,6 +495,7 @@ version = "2.20.0"
description = "Pygments is a syntax highlighting package written in Python."
optional = false
python-versions = ">=3.9"
groups = ["dev"]
files = [
{file = "pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176"},
{file = "pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f"},
@ -509,6 +510,7 @@ version = "4.2"
description = "Generate AppImages from your Python projects"
optional = false
python-versions = ">=3.9"
groups = ["dev"]
files = [
{file = "pyproject_appimage-4.2-py3-none-any.whl", hash = "sha256:d6892643db5759dc06531a4546bdab404a519c63814c060f8749979a8625d9cc"},
{file = "pyproject_appimage-4.2.tar.gz", hash = "sha256:6b6387250cb1e6ecbb08a13f5810749396ebe8637f2f35bf2296bfdd5e65cd6e"},
@ -525,6 +527,7 @@ version = "8.4.2"
description = "pytest: simple powerful testing with Python"
optional = false
python-versions = ">=3.9"
groups = ["dev"]
files = [
{file = "pytest-8.4.2-py3-none-any.whl", hash = "sha256:872f880de3fc3a5bdc88a11b39c9710c3497a547cfa9320bc3c5e62fbf272e79"},
{file = "pytest-8.4.2.tar.gz", hash = "sha256:86c0d0b93306b961d58d62a4db4879f27fe25513d4b969df351abdddb3c30e01"},
@ -548,6 +551,7 @@ version = "5.0.0"
description = "Pytest plugin for measuring coverage."
optional = false
python-versions = ">=3.8"
groups = ["dev"]
files = [
{file = "pytest-cov-5.0.0.tar.gz", hash = "sha256:5837b58e9f6ebd335b0f8060eecce69b662415b16dc503883a02f45dfeb14857"},
{file = "pytest_cov-5.0.0-py3-none-any.whl", hash = "sha256:4f0764a1219df53214206bf1feea4633c3b558a2925c8b59f144f682861ce652"},
@ -566,6 +570,7 @@ version = "6.0.3"
description = "YAML parser and emitter for Python"
optional = false
python-versions = ">=3.8"
groups = ["main"]
files = [
{file = "PyYAML-6.0.3-cp38-cp38-macosx_10_13_x86_64.whl", hash = "sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f"},
{file = "PyYAML-6.0.3-cp38-cp38-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4"},
@ -648,6 +653,7 @@ version = "2.34.2"
description = "Python HTTP for Humans."
optional = false
python-versions = ">=3.10"
groups = ["dev"]
files = [
{file = "requests-2.34.2-py3-none-any.whl", hash = "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0"},
{file = "requests-2.34.2.tar.gz", hash = "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed"},
@ -669,6 +675,7 @@ version = "2.4.1"
description = "A lil' TOML parser"
optional = false
python-versions = ">=3.8"
groups = ["main", "dev"]
files = [
{file = "tomli-2.4.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:f8f0fc26ec2cc2b965b7a3b87cd19c5c6b8c5e5f436b984e85f486d652285c30"},
{file = "tomli-2.4.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:4ab97e64ccda8756376892c53a72bd1f964e519c77236368527f758fbc36a53a"},
@ -718,6 +725,7 @@ files = [
{file = "tomli-2.4.1-py3-none-any.whl", hash = "sha256:0d85819802132122da43cb86656f8d1f8c6587d54ae7dcaf30e90533028b49fe"},
{file = "tomli-2.4.1.tar.gz", hash = "sha256:7c7e1a961a0b2f2472c1ac5b69affa0ae1132c39adcb67aba98568702b9cc23f"},
]
markers = {main = "python_version == \"3.10\"", dev = "python_full_version <= \"3.11.0a6\""}
[[package]]
name = "typing-extensions"
@ -725,6 +733,8 @@ version = "4.15.0"
description = "Backported and Experimental Type Hints for Python 3.9+"
optional = false
python-versions = ">=3.9"
groups = ["dev"]
markers = "python_version == \"3.10\""
files = [
{file = "typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548"},
{file = "typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466"},
@ -736,18 +746,19 @@ version = "2.7.0"
description = "HTTP library with thread-safe connection pooling, file post, and more."
optional = false
python-versions = ">=3.10"
groups = ["dev"]
files = [
{file = "urllib3-2.7.0-py3-none-any.whl", hash = "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897"},
{file = "urllib3-2.7.0.tar.gz", hash = "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c"},
]
[package.extras]
brotli = ["brotli (>=1.2.0)", "brotlicffi (>=1.2.0.0)"]
brotli = ["brotli (>=1.2.0) ; platform_python_implementation == \"CPython\"", "brotlicffi (>=1.2.0.0) ; platform_python_implementation != \"CPython\""]
h2 = ["h2 (>=4,<5)"]
socks = ["pysocks (>=1.5.6,!=1.5.7,<2.0)"]
zstd = ["backports-zstd (>=1.0.0)"]
zstd = ["backports-zstd (>=1.0.0) ; python_version < \"3.14\""]
[metadata]
lock-version = "2.0"
python-versions = "^3.10"
content-hash = "026c4acd254e889b70bb8c25ffb5e6323eee86380f54f2d8ef02f59ae9307529"
lock-version = "2.1"
python-versions = ">=3.10,<4.0"
content-hash = "161dfd9b44e1063656bacea8e8fb08a2614361a8262d8f665240e4f391732e1b"

View file

@ -1,36 +1,40 @@
[tool.poetry]
[project]
name = "jinjaturtle"
version = "0.5.4"
version = "0.7.0"
description = "Convert config files into Ansible defaults and Jinja2 templates."
authors = ["Miguel Jacq <mig@mig5.net>"]
authors = [
{ name = "Miguel Jacq", email = "mig@mig5.net" },
]
license = "GPL-3.0-or-later"
readme = "README.md"
packages = [{ include = "jinjaturtle", from = "src" }]
requires-python = ">=3.10,<4.0"
keywords = ["ansible", "jinja2", "config", "toml", "ini", "yaml", "json", "devops"]
dependencies = [
"PyYAML (>=6.0,<7.0)",
"defusedxml (>=0.7.1,<0.8.0)",
"jinja2 (>=3.1.6,<4.0.0)",
"tomli (>=2.0.0,<3.0.0) ; python_version < '3.11'",
]
[project.urls]
homepage = "https://git.mig5.net/mig5/jinjaturtle"
repository = "https://git.mig5.net/mig5/jinjaturtle"
[tool.poetry.dependencies]
python = "^3.10"
PyYAML = "^6.0"
tomli = { version = "^2.0.0", python = "<3.11" }
defusedxml = "^0.7.1"
jinja2 = "^3.1.6"
[tool.poetry.scripts]
[project.scripts]
jinjaturtle = "jinjaturtle.cli:main"
[tool.poetry]
packages = [{ include = "jinjaturtle", from = "src" }]
[tool.poetry.group.dev.dependencies]
pytest = "^8"
pytest-cov = "^5"
pyproject-appimage = "^4.2"
[build-system]
requires = ["poetry-core>=1.0.0"]
requires = ["poetry-core>=2.0.0"]
build-backend = "poetry.core.masonry.api"
[tool.pyproject-appimage]
script = "jinjaturtle"
output = "JinjaTurtle.AppImage"
[tool.poetry.dev-dependencies]
pytest = "^8"
pytest-cov = "^5"
pyproject-appimage = "^4.2"

View file

@ -7,11 +7,6 @@ filedust -y .
# Publish to Pypi
poetry build
poetry publish
# Make AppImage
poetry run pyproject-appimage
mv JinjaTurtle.AppImage dist/
# Sign packages
for file in `ls -1 dist/`; do qubes-gpg-client --batch --armor --detach-sign dist/$file > dist/$file.asc; done
@ -52,7 +47,6 @@ REMOTE="ashpool.mig5.net:/opt/repo_rpm"
DISTS=(
fedora:43
fedora:42
)
for dist in ${DISTS[@]}; do
@ -87,6 +81,9 @@ for dist in ${DISTS[@]}; do
qubes-gpg-client --local-user "$KEYID" --detach-sign --armor "$RPM_REPO/repodata/repomd.xml" > "$RPM_REPO/repodata/repomd.xml.asc"
done
# If we got this far, we can publish to PyPI
poetry publish
echo "==> Syncing repo to server..."
rsync -aHPvz --exclude=.git --delete "$REPO_ROOT/" "$REMOTE/"

View file

@ -1,4 +1,4 @@
%global upstream_version 0.5.4
%global upstream_version 0.7.0
Name: jinjaturtle
Version: %{upstream_version}
@ -42,6 +42,14 @@ Convert config files into Ansible defaults and Jinja2 templates.
%{_bindir}/jinjaturtle
%changelog
* Sun Jul 05 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Much hardening
* Wed Jun 24 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- More hardening
* Tue Jun 23 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Try to prevent what could lead to execution of embedded jinja in original files when converting
* Sat Jun 20 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- erb support
* Sat Jun 20 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Make templates more faithful to the original file in terms of indentation, newlines, no deserialisation of things like < or >.
- More test coverage
@ -51,7 +59,7 @@ Convert config files into Ansible defaults and Jinja2 templates.
- Fix indentation problems with nested dicts
* Fri Jun 19 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Empty dicts and lists are now emitted as leaf defaults.
* Tue May 11 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
* Mon May 11 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Support ssh configs
* Tue Jan 06 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Support converting systemd files and postfix main.cf

View file

@ -3,6 +3,7 @@ from __future__ import annotations
import argparse
import sys
from defusedxml import defuse_stdlib
from defusedxml.common import DefusedXmlException
from pathlib import Path
from . import j2
@ -12,11 +13,12 @@ from .core import (
flatten_config,
generate_ansible_yaml,
generate_jinja2_template,
generate_puppet_hiera_yaml,
generate_erb_template,
ConfigParseError,
)
from .multi import process_directory
from .safety import TemplateSafetyError
from .output_safety import OutputPathError, ensure_safe_directory, write_text_safely
def _build_arg_parser() -> argparse.ArgumentParser:
@ -58,24 +60,47 @@ def _build_arg_parser() -> argparse.ArgumentParser:
"--template-output",
help="Path to write the generated config template. If omitted, template is printed to stdout.",
)
ap.add_argument(
"--template-engine",
choices=[j2.NAME, "erb"],
default=j2.NAME,
help="Template syntax to generate (default: jinja2). Use erb for Puppet templates.",
)
ap.add_argument(
"--puppet-class",
help=(
"Puppet class/Hiera namespace to use with --template-engine erb. "
"Defaults to --role-name. This lets tools use a file-specific "
"variable prefix while writing Hiera keys under the real Puppet class."
),
)
return ap
def _main(argv: list[str] | None = None) -> int:
try:
return _run(argv)
except TemplateSafetyError as exc:
# The output safety gate refused to emit a template because it contained
# a construct JinjaTurtle never produces -- i.e. attacker-influenced
# source text became live template code. Fail closed with a clear
# message and a non-zero exit code; never write the unsafe template.
print(
f"jinjaturtle: refusing to generate unsafe template: {exc}", file=sys.stderr
)
return 2
except OutputPathError as exc:
print(f"jinjaturtle: refusing unsafe output path: {exc}", file=sys.stderr)
return 2
except DefusedXmlException as exc:
# defusedxml rejected the XML because it attempted a DTD, entity
# expansion, or external reference (XXE / billion-laughs class attack).
# This is a deliberately-blocked attack, not a benign malformed file, so
# core.parse_config lets it propagate unchanged rather than folding it
# into ConfigParseError. Report it as a refused unsafe input with a
# non-zero exit code instead of leaking an internal traceback.
print(
"jinjaturtle: refusing unsafe XML input: the document uses a DTD, "
f"entity expansion, or external reference ({exc.__class__.__name__}). "
"This is blocked to prevent XXE / entity-expansion attacks.",
file=sys.stderr,
)
return 2
except ConfigParseError as exc:
# The source file could not be parsed as its (detected or forced)
# format. This is expected for malformed/attacker-influenced input;
# fail cleanly with a non-zero exit code instead of a traceback.
print(f"jinjaturtle: {exc}", file=sys.stderr)
return 1
def _run(argv: list[str] | None = None) -> int:
defuse_stdlib()
parser = _build_arg_parser()
args = parser.parse_args(argv)
@ -90,36 +115,23 @@ def _main(argv: list[str] | None = None) -> int:
# Write defaults
if args.defaults_output:
Path(args.defaults_output).write_text(defaults_yaml, encoding="utf-8")
write_text_safely(Path(args.defaults_output), defaults_yaml)
else:
print("# defaults/main.yml")
print(defaults_yaml, end="")
# Optionally translate folder-mode templates to ERB. Folder mode keeps
# the existing data shape; single-file mode below is the preferred
# Puppet path because it can produce class-parameter Hiera keys.
if args.template_engine == "erb":
from .erb import translate_jinja2_to_erb
for o in outputs:
o.template = translate_jinja2_to_erb(
o.template,
role_prefix=args.role_name,
puppet_class=args.puppet_class or args.role_name,
)
template_ext = "erb" if args.template_engine == "erb" else j2.TEMPLATE_EXTENSION
template_ext = j2.TEMPLATE_EXTENSION
# Write templates
if args.template_output:
out_path = Path(args.template_output)
if len(outputs) == 1 and not out_path.is_dir():
out_path.write_text(outputs[0].template, encoding="utf-8")
write_text_safely(out_path, outputs[0].template)
else:
out_path.mkdir(parents=True, exist_ok=True)
ensure_safe_directory(out_path)
for o in outputs:
(out_path / f"config.{o.fmt}.{template_ext}").write_text(
o.template, encoding="utf-8"
write_text_safely(
out_path / f"config.{o.fmt}.{template_ext}", o.template
)
else:
for o in outputs:
@ -145,51 +157,28 @@ def _main(argv: list[str] | None = None) -> int:
# Flatten config (excluding loop paths if loops are detected)
flat_items = flatten_config(fmt, parsed, loop_candidates)
if args.template_engine == "erb":
ansible_yaml = generate_puppet_hiera_yaml(
args.role_name,
flat_items,
loop_candidates,
puppet_class=args.puppet_class or args.role_name,
)
template_str = generate_erb_template(
fmt,
parsed,
args.role_name,
original_text=config_text,
loop_candidates=loop_candidates,
flat_items=flat_items,
puppet_class=args.puppet_class or args.role_name,
)
else:
# Generate defaults YAML (with loop collections if detected)
ansible_yaml = generate_ansible_yaml(
args.role_name, flat_items, loop_candidates
)
# Generate defaults YAML (with loop collections if detected)
ansible_yaml = generate_ansible_yaml(args.role_name, flat_items, loop_candidates)
# Generate template (with loops if detected)
template_str = generate_jinja2_template(
fmt,
parsed,
args.role_name,
original_text=config_text,
loop_candidates=loop_candidates,
)
# Generate template (with loops if detected)
template_str = generate_jinja2_template(
fmt,
parsed,
args.role_name,
original_text=config_text,
loop_candidates=loop_candidates,
)
if args.defaults_output:
Path(args.defaults_output).write_text(ansible_yaml, encoding="utf-8")
write_text_safely(Path(args.defaults_output), ansible_yaml)
else:
print("# defaults/main.yml")
print(ansible_yaml, end="")
if args.template_output:
Path(args.template_output).write_text(template_str, encoding="utf-8")
write_text_safely(Path(args.template_output), template_str)
else:
print(
"# config.erb"
if args.template_engine == "erb"
else f"# config.{j2.TEMPLATE_EXTENSION}"
)
print(f"# config.{j2.TEMPLATE_EXTENSION}")
print(template_str, end="")
return 0
@ -199,4 +188,8 @@ def main() -> None:
"""
Console-script entry point.
"""
_main(sys.argv[1:])
sys.exit(_main(sys.argv[1:]))
if __name__ == "__main__":
main()

View file

@ -8,7 +8,10 @@ import re
import yaml
from .loop_analyzer import LoopAnalyzer, LoopCandidate
from .erb import puppet_class_name, puppet_local_var_name, translate_jinja2_to_erb
from .safety import (
verify_jinja2_template_safe,
verify_no_live_jinja_in_json_keys,
)
from .handlers import (
BaseHandler,
IniHandler,
@ -30,6 +33,22 @@ class QuotedString(str):
pass
class AnsibleUnsafeString(str):
"""Marker type emitted with Ansible's !unsafe YAML tag.
Ansible recursively templates string values by default. Source-derived
config values that contain Jinja delimiters must therefore be marked
unsafe in defaults/main.yml, otherwise a harvested value such as
``{{ lookup('pipe', 'id') }}`` becomes executable on the Ansible
controller when the generated role is applied.
"""
pass
_JINJA_STARTS = ("{{", "{%", "{#")
def _fallback_str_representer(dumper: yaml.SafeDumper, data: Any):
"""
Fallback for objects the dumper doesn't know about.
@ -49,7 +68,33 @@ def _quoted_str_representer(dumper: yaml.SafeDumper, data: QuotedString):
return dumper.represent_scalar("tag:yaml.org,2002:str", str(data), style='"')
def _ansible_unsafe_str_representer(dumper: yaml.SafeDumper, data: AnsibleUnsafeString):
return dumper.represent_scalar("!unsafe", str(data), style="'")
def _needs_ansible_unsafe(value: str) -> bool:
return any(marker in value for marker in _JINJA_STARTS)
def _mark_ansible_unsafe_values(obj: Any) -> Any:
"""Recursively mark mapping/list values containing Jinja as !unsafe.
Mapping keys are intentionally left alone: they are variable names or YAML
structure, not Ansible-templated values. Values nested in folder-mode item
lists, including source-derived ``id`` values, are protected.
"""
if isinstance(obj, dict):
return {k: _mark_ansible_unsafe_values(v) for k, v in obj.items()}
if isinstance(obj, list):
return [_mark_ansible_unsafe_values(v) for v in obj]
if isinstance(obj, str) and _needs_ansible_unsafe(obj):
return AnsibleUnsafeString(obj)
return obj
_TurtleDumper.add_representer(QuotedString, _quoted_str_representer)
_TurtleDumper.add_representer(AnsibleUnsafeString, _ansible_unsafe_str_representer)
# Use our fallback for any unknown object types
_TurtleDumper.add_representer(None, _fallback_str_representer)
@ -81,8 +126,9 @@ def dump_yaml(data: Any, *, sort_keys: bool = True) -> str:
This is used by both the single-file and multi-file code paths.
"""
safe_data = _mark_ansible_unsafe_values(data)
return yaml.dump(
data,
safe_data,
Dumper=_TurtleDumper,
sort_keys=sort_keys,
default_flow_style=False,
@ -259,6 +305,66 @@ def detect_format(path: Path, explicit: str | None = None) -> str:
return "ini"
class ConfigParseError(Exception):
"""Raised when a source config file cannot be parsed as its format.
Each underlying parser (json, tomllib, PyYAML, defusedxml/ElementTree,
configparser) raises its own exception type on malformed input. Without a
single normalised error, a malformed file -- which is entirely expected when
JinjaTurtle is pointed at harvested, attacker-influenceable config -- would
escape as an unhandled traceback (e.g. ``xml.etree.ElementTree.ParseError``
on an XML file whose element name is not well-formed). ``parse_config``
converts every such failure into this one type so the CLI can fail closed
with a clean message and a non-zero exit code, and so library callers (such
as Enroll, which falls back to copying the raw file) have a single, stable
exception to catch.
Note: defusedxml's *security* exceptions (``EntitiesForbidden``,
``DTDForbidden``, ...) are intentionally NOT folded into this type. They
signal an attempted XXE/entity-expansion attack rather than a benign
malformed file, and must propagate unchanged so callers can tell the two
apart.
"""
def _build_malformed_config_errors() -> tuple[type[BaseException], ...]:
"""Return the concrete "this file is malformed" exception types to catch.
Deliberately specific. In particular we must avoid catching plain
``ValueError``: defusedxml's ``EntitiesForbidden``/``DTDForbidden`` subclass
``ValueError``, and those are security signals that must NOT be swallowed.
"""
import configparser
import json
from xml.etree.ElementTree import ParseError as _XMLParseError # nosec
import yaml as _yaml
errs: list[type[BaseException]] = [
_XMLParseError,
json.JSONDecodeError,
configparser.Error,
_yaml.YAMLError,
UnicodeDecodeError,
]
try:
import tomllib
errs.append(tomllib.TOMLDecodeError)
except ModuleNotFoundError: # pragma: no cover - Python < 3.11 fallback
try:
import tomli # type: ignore
errs.append(tomli.TOMLDecodeError)
except ModuleNotFoundError:
pass
return tuple(errs)
_MALFORMED_CONFIG_ERRORS = _build_malformed_config_errors()
def parse_config(path: Path, fmt: str | None = None) -> tuple[str, Any]:
"""
Parse config file into a Python object.
@ -267,9 +373,38 @@ def parse_config(path: Path, fmt: str | None = None) -> tuple[str, Any]:
handler = _HANDLERS.get(fmt)
if handler is None:
raise ValueError(f"Unsupported config format: {fmt}")
parsed = handler.parse(path)
# Make sure datetime objects are treated as strings (TOML, YAML)
parsed = _stringify_timestamps(parsed)
try:
parsed = handler.parse(path)
# Make sure datetime objects are treated as strings (TOML, YAML). This
# walks the parsed object recursively, so keep it inside the try where a
# RecursionError from a pathological structure is normalised below.
parsed = _stringify_timestamps(parsed)
except ConfigParseError:
raise
except _MALFORMED_CONFIG_ERRORS as exc:
# Normalise the per-parser "this file is malformed" errors into one
# json.JSONDecodeError / tomllib.TOMLDecodeError (ValueError
# subclasses), PyYAML's YAMLError, configparser.Error, and
# xml.etree.ElementTree.ParseError (raised by defusedxml on XML whose
# structure/element name is not well-formed). A bad input file is
# expected when parsing harvested config, so fail closed with a clean
# error instead of an unhandled traceback.
#
# IMPORTANT: this deliberately does NOT catch defusedxml's security
# exceptions (EntitiesForbidden, DTDForbidden, ...). Those signal an
# attempted XXE/entity-expansion attack and must propagate unchanged so
# callers (and tests) can distinguish "malformed" from "malicious".
raise ConfigParseError(f"could not parse {path} as {fmt}: {exc}") from exc
except RecursionError as exc:
# A deeply-nested or self-referential structure (e.g. a recursive YAML
# anchor) can exhaust the Python stack while walking the parsed object.
# The YAML handler already rejects reference cycles up front; this is a
# format-agnostic backstop so any such input fails closed with a clean
# message instead of a stack-overflow traceback.
raise ConfigParseError(
f"could not parse {path} as {fmt}: input is too deeply nested "
"or self-referential"
) from exc
return fmt, parsed
@ -378,93 +513,28 @@ def generate_jinja2_template(
# Check if handler supports loop-aware generation
if hasattr(handler, "generate_jinja2_template_with_loops") and loop_candidates:
return handler.generate_jinja2_template_with_loops(
template = handler.generate_jinja2_template_with_loops(
parsed, role_prefix, original_text, loop_candidates
)
else:
# Fallback to original scalar-only generation
template = handler.generate_jinja2_template(
parsed, role_prefix, original_text=original_text
)
# Fallback to original scalar-only generation
return handler.generate_jinja2_template(
parsed, role_prefix, original_text=original_text
)
# Defence in depth: independently verify that the finished template contains
# only JinjaTurtle-emitted constructs. If any handler failed to neutralise
# verbatim source text, the un-escaped payload shows up here as a live tag
# and generation aborts instead of emitting an injectable template.
verify_jinja2_template_safe(template)
# Format-specific backstop: JinjaTurtle never emits Jinja inside a JSON object
# key, so a live construct in key position means source key text leaked into
# the template unescaped. This is independent of per-handler escaping.
if fmt == "json":
verify_no_live_jinja_in_json_keys(template)
def _template_variable_names(
role_prefix: str,
flat_items: list[tuple[tuple[str, ...], Any]],
loop_candidates: list[LoopCandidate] | None = None,
) -> set[str]:
names = {make_var_name(role_prefix, path) for path, _value in flat_items}
if loop_candidates:
for candidate in loop_candidates:
names.add(make_var_name(role_prefix, candidate.path))
return names
def generate_puppet_hiera_yaml(
role_prefix: str,
flat_items: list[tuple[tuple[str, ...], Any]],
loop_candidates: list[LoopCandidate] | None = None,
*,
puppet_class: str | None = None,
) -> str:
"""Create Puppet Hiera data suitable for Automatic Parameter Lookup.
``role_prefix`` remains the source variable prefix used by JinjaTurtle while
``puppet_class`` is the Puppet class/Hiera namespace. In the normal case
they are the same, so ``php_memory_limit`` becomes ``php::memory_limit``.
Enroll may pass a file-specific role prefix and a separate Puppet class to
avoid parameter-name collisions inside one generated Puppet module.
"""
klass = puppet_class_name(puppet_class or role_prefix)
data: dict[str, Any] = {}
for path, value in flat_items:
generated = make_var_name(role_prefix, path)
local = puppet_local_var_name(role_prefix, generated, puppet_class=klass)
data[f"{klass}::{local}"] = value
if loop_candidates:
for candidate in loop_candidates:
generated = make_var_name(role_prefix, candidate.path)
local = puppet_local_var_name(role_prefix, generated, puppet_class=klass)
data[f"{klass}::{local}"] = candidate.items
return dump_yaml(data, sort_keys=True)
def generate_erb_template(
fmt: str,
parsed: Any,
role_prefix: str,
*,
original_text: str | None = None,
loop_candidates: list[LoopCandidate] | None = None,
flat_items: list[tuple[tuple[str, ...], Any]] | None = None,
puppet_class: str | None = None,
) -> str:
"""Generate a Puppet ERB template from JinjaTurtle's renderer-neutral data.
The first implementation intentionally translates the Jinja2 subset emitted
by JinjaTurtle's existing format handlers. This keeps parsing, formatting
preservation, and loop detection identical between Jinja2 and ERB output
while still producing Puppet-native ``@parameter`` references.
"""
jinja_template = generate_jinja2_template(
fmt,
parsed,
role_prefix,
original_text=original_text,
loop_candidates=loop_candidates,
)
names = _template_variable_names(role_prefix, flat_items or [], loop_candidates)
return translate_jinja2_to_erb(
jinja_template,
role_prefix=role_prefix,
puppet_class=puppet_class or role_prefix,
variable_names=names,
)
return template
def _stringify_timestamps(obj: Any) -> Any:

View file

@ -1,218 +0,0 @@
from __future__ import annotations
import re
def _safe_name(raw: str, *, fallback: str = "var") -> str:
text = re.sub(r"[^A-Za-z0-9_]+", "_", str(raw or fallback)).strip("_").lower()
text = re.sub(r"_+", "_", text)
if not text:
text = fallback
if not re.match(r"^[a-z_]", text):
text = f"{fallback}_{text}"
return text
def puppet_class_name(raw: str) -> str:
"""Return a conservative Puppet class/Hiera namespace name."""
text = _safe_name(raw, fallback="jinjaturtle")
if not re.match(r"^[a-z]", text):
text = f"jinjaturtle_{text}"
return text
def _role_prefix_name(raw: str) -> str:
return _safe_name(raw, fallback="jinjaturtle")
def puppet_local_var_name(
role_prefix: str,
jinja_var_name: str,
*,
puppet_class: str | None = None,
) -> str:
"""Map a generated JinjaTurtle variable to a Puppet class parameter.
For the common case where ``--role-name php`` also means class ``php``, a
generated Jinja variable such as ``php_memory_limit`` becomes Puppet local
parameter ``memory_limit`` and Hiera key ``php::memory_limit``.
Enroll sometimes needs a file-specific variable prefix to avoid collisions
inside a generated module. When ``puppet_class`` differs from
``role_prefix`` we keep the full generated variable name as the local
parameter and only use ``puppet_class`` as the Hiera namespace.
"""
var_name = _safe_name(jinja_var_name, fallback="value")
prefix = _role_prefix_name(role_prefix)
klass = puppet_class_name(puppet_class or role_prefix)
if klass == prefix and var_name.startswith(prefix + "_"):
stripped = var_name[len(prefix) + 1 :]
return stripped or var_name
return var_name
class ErbTranslator:
"""Translate the Jinja2 subset emitted by JinjaTurtle into Puppet ERB."""
_TOKEN_RE = re.compile(r"({{.*?}}|{%.*?%})", re.S)
def __init__(
self,
*,
role_prefix: str,
puppet_class: str | None = None,
variable_names: set[str] | None = None,
) -> None:
self.role_prefix = role_prefix
self.puppet_class = puppet_class or role_prefix
self.variable_names = set(variable_names or set())
self.loop_stack: list[tuple[str, str, str]] = []
self.needs_json = False
def translate(self, template_text: str) -> str:
parts = self._TOKEN_RE.split(template_text)
out: list[str] = []
for token in parts:
if not token:
continue
if token.startswith("{{") and token.endswith("}}"):
expr = token[2:-2].strip()
out.append(f"<%= {self.expr_to_ruby(expr)} %>")
continue
if token.startswith("{%") and token.endswith("%}"):
stmt = token[2:-2].strip()
out.append(self.statement_to_erb(stmt))
continue
out.append(token)
rendered = "".join(out)
if self.needs_json and "require 'json'" not in rendered:
rendered = "<% require 'json' -%>\n" + rendered
return rendered
def local_var(self, name: str) -> str:
return puppet_local_var_name(
self.role_prefix, name, puppet_class=self.puppet_class
)
def ruby_value(self, expr: str) -> str:
expr = expr.strip()
if expr in {"true", "True"}:
return "true"
if expr in {"false", "False"}:
return "false"
if expr in {"none", "None", "null"}:
return "nil"
if re.match(r"^[A-Za-z_][A-Za-z0-9_]*$", expr):
if any(expr == loop_var for loop_var, _idx, _coll in self.loop_stack):
return expr
return f"@{self.local_var(expr)}"
m = re.match(r"^([A-Za-z_][A-Za-z0-9_]*)\.([A-Za-z_][A-Za-z0-9_]*)$", expr)
if m:
base, key = m.groups()
if any(base == loop_var for loop_var, _idx, _coll in self.loop_stack):
return f"{base}[{key!r}]"
return f"@{self.local_var(base)}[{key!r}]"
return expr
def expr_to_ruby(self, expr: str) -> str:
expr = expr.strip()
# JinjaTurtle emits these YAML-preserving ternaries for booleans/nulls.
m = re.match(
r"^(['\"])(true|false)\1\s+if\s+([A-Za-z_][A-Za-z0-9_\.]*)\s+else\s+(['\"])(true|false)\4$",
expr,
)
if m:
truthy = m.group(2)
cond = self.ruby_value(m.group(3))
falsy = m.group(5)
return f"{cond} ? {truthy!r} : {falsy!r}"
m = re.match(
r"^(['\"])(null)\1\s+if\s+([A-Za-z_][A-Za-z0-9_\.]*)\s+is\s+none\s+else\s+([A-Za-z_][A-Za-z0-9_\.]*)$",
expr,
)
if m:
value = self.ruby_value(m.group(3))
fallback = self.ruby_value(m.group(4))
return f"{value}.nil? ? 'null' : {fallback}"
if "|" in expr:
base, *filters = [part.strip() for part in expr.split("|")]
ruby = self.ruby_value(base)
for filt in filters:
if filt.startswith("lower"):
ruby = f"{ruby}.to_s.downcase"
elif filt.startswith("to_json") or filt.startswith("tojson"):
self.needs_json = True
if "indent" in filt:
ruby = f"JSON.pretty_generate({ruby})"
else:
ruby = f"JSON.generate({ruby})"
return ruby
return self.ruby_value(expr)
def statement_to_erb(self, stmt: str) -> str:
if stmt.endswith("-"):
stmt = stmt[:-1].rstrip()
if stmt.startswith("for "):
m = re.match(
r"^for\s+([A-Za-z_][A-Za-z0-9_]*)\s+in\s+([A-Za-z_][A-Za-z0-9_]*)$",
stmt,
)
if m:
loop_var, collection = m.groups()
idx_var = f"__jt_idx_{len(self.loop_stack)}"
collection_ruby = self.ruby_value(collection)
self.loop_stack.append((loop_var, idx_var, collection_ruby))
return f"<% {collection_ruby}.each_with_index do |{loop_var}, {idx_var}| -%>"
if stmt == "endfor":
if self.loop_stack:
self.loop_stack.pop()
return "<% end %>"
if stmt.startswith("if "):
cond = stmt[3:].strip()
if cond == "not loop.last" and self.loop_stack:
_loop_var, idx_var, collection_ruby = self.loop_stack[-1]
return f"<% if {idx_var} < ({collection_ruby}.length - 1) -%>"
m = re.match(r"^([A-Za-z_][A-Za-z0-9_\.]*)\s+is\s+defined$", cond)
if m:
return f"<% unless {self.ruby_value(m.group(1))}.nil? -%>"
m = re.match(r"^([A-Za-z_][A-Za-z0-9_\.]*)\s+is\s+none$", cond)
if m:
return f"<% if {self.ruby_value(m.group(1))}.nil? -%>"
return f"<% if {self.expr_to_ruby(cond)} -%>"
if stmt == "else":
return "<% else -%>"
if stmt.startswith("elif "):
return f"<% elsif {self.expr_to_ruby(stmt[5:].strip())} -%>"
if stmt == "endif":
return "<% end -%>"
# Preserve unknown Jinja statements visibly as an ERB comment so the
# generated template does not contain invalid Jinja syntax.
return f"<%# Unsupported JinjaTurtle statement: {stmt} %>"
def translate_jinja2_to_erb(
template_text: str,
*,
role_prefix: str,
puppet_class: str | None = None,
variable_names: set[str] | None = None,
) -> str:
return ErbTranslator(
role_prefix=role_prefix,
puppet_class=puppet_class,
variable_names=variable_names,
).translate(template_text)

98
src/jinjaturtle/escape.py Normal file
View file

@ -0,0 +1,98 @@
from __future__ import annotations
"""Neutralise template metacharacters in text copied verbatim from source files.
JinjaTurtle preserves formatting by copying parts of the *original* config file
straight into the generated template: comments, blank lines, section headers,
and any line it does not recognise as ``key = value``. Config *values* are
always replaced with ``{{ var }}`` placeholders and parked in the defaults data,
so a payload inside a value is inert. Verbatim text is different: if the source
contains ``{{ ... }}``, ``{% ... %}`` or ``{# ... #}`` (Jinja2), or ``<%= %>`` /
``<% %>`` (ERB), that text becomes *live template code* in the output and is
executed when Ansible later renders the template.
Because JinjaTurtle is frequently fed harvested, attacker-influenceable config
(hostnames, banners, GECOS-derived comments, "Managed by" notes), this is a
template-injection / SSTI vector that can lead to remote code execution on the
configuration-management control node.
The functions here render those metacharacters as literal text so they survive a
later template render as the characters the source author actually wrote, rather
than as executable template syntax.
Design notes:
* We only ever escape text that originates from the *source file*. We never
pass JinjaTurtle's own generated placeholders (``{{ role_var }}``) through
these helpers, so the placeholders keep working.
* Jinja2 literal text is wrapped in a single ``{% raw %} ... {% endraw %}``
block. ``raw`` disables *all* tag interpretation inside it -- expressions,
statements and ``{# #}`` comments alike -- so one wrap neutralises every
Jinja construct. The only way to break out of a raw block is a literal
``{% endraw %}`` in the source, so we defang the token ``endraw`` (in any
internal spacing) before wrapping.
"""
import re
# Jinja2 delimiters we must neutralise. Engine-default; JinjaTurtle never
# configures custom delimiters.
_JINJA_MARKERS = ("{{", "}}", "{%", "%}", "{#", "#}")
# ERB delimiters. Longer markers first so "<%=" matches before "<%".
_ERB_OPEN_MARKERS = ("<%=", "<%-", "<%#", "<%")
_ERB_CLOSE_MARKERS = ("-%>", "%>")
# Matches a Jinja2 endraw tag in any internal spacing and with any
# whitespace-control marker on either side. Jinja2 accepts "-", "+", or no
# marker adjacent to the "%}"/"{%" of a block tag (e.g. "{%endraw%}",
# "{% endraw %}", "{%- endraw -%}", "{%+ endraw +%}"), and ALL of these close
# a raw block. The control marker must be matched so a "{%+ endraw %}" in
# attacker-influenced source text cannot survive defanging and break out of our
# {% raw %} wrapper. [-+]? appears on both sides accordingly.
_ENDRAW_RE = re.compile(r"{%[-+]?\s*endraw\s*[-+]?%}")
# Sentinel inserted between "end" and "raw" to break the endraw keyword without
# changing the visible characters. We use a Jinja comment-free approach: insert
# the two halves across a raw boundary so the literal text still reads "endraw"
# to a human but is never a valid tag. See escape_jinja_literal for usage.
def contains_jinja_markup(text: str) -> bool:
"""Return True if *text* contains any Jinja2 delimiter."""
return any(m in text for m in _JINJA_MARKERS)
def _defang_endraw(text: str) -> str:
"""Rewrite any literal ``{% endraw %}`` so it cannot close our raw wrapper.
We turn each endraw tag into ``{% endraw %}{{ '{% endraw %}' }}{% raw %}``...
no -- that would re-introduce live tags. Instead we keep everything literal:
we break the keyword by emitting the tag's text in two raw segments split
inside the word ``endraw``. The result, when later rendered, reproduces the
exact original characters ``{% endraw %}`` while never being a parseable tag.
"""
def _replace(match: re.Match[str]) -> str:
tag = match.group(0)
# Split the keyword "endraw" as "end" + "raw"; close and reopen the raw
# block between them. Each half is plain text inside a raw block, so the
# reconstructed output is byte-identical to the original tag, but at no
# point does the token "{% endraw %}" exist contiguously to close raw.
idx = tag.lower().index("endraw")
head = tag[: idx + 3] # up to and including "end"
tail = tag[idx + 3 :] # "raw...%}"
return f"{head}{{% endraw %}}{{% raw %}}{tail}"
return _ENDRAW_RE.sub(_replace, text)
def escape_jinja_literal(text: str) -> str:
"""Make *text* render as literal characters under a later Jinja2 render.
Text with no Jinja metacharacters is returned unchanged so the common case
stays byte-for-byte identical to the source. Otherwise the text is wrapped
in a single ``{% raw %}`` block, with any embedded ``endraw`` defanged.
"""
if not text or not contains_jinja_markup(text):
return text
return "{% raw %}" + _defang_endraw(text) + "{% endraw %}"

View file

@ -1,5 +1,6 @@
from __future__ import annotations
import re
from pathlib import Path
from typing import Any, Iterable
@ -57,8 +58,20 @@ class BaseHandler:
role_prefix_section_subsection_key
Sanitises parts to lowercase [a-z0-9_] and strips extras.
Consecutive separators are collapsed to a single underscore. This is
required for correctness, not just aesthetics: a source key such as
``log..level`` or ``cache--size`` would otherwise sanitise to a name
containing a double underscore (``log__level``). The output safety gate
in ``safety.py`` deliberately rejects *any* ``__`` in a generated
identifier because ``__`` is the gateway to every Jinja2 SSTI gadget
(``__class__``/``__globals__``/...). Emitting a dunder here would make
JinjaTurtle's own gate reject JinjaTurtle's own placeholder, aborting
generation on entirely benign config. Collapsing runs keeps every
generated name a plain single-underscore-delimited identifier that the
gate accepts.
"""
role_prefix = role_prefix.strip().lower()
role_prefix = re.sub(r"_+", "_", role_prefix.strip().lower())
clean_parts: list[str] = []
for part in path:
@ -70,7 +83,9 @@ class BaseHandler:
cleaned_chars.append(c.lower())
else:
cleaned_chars.append("_")
cleaned_part = "".join(cleaned_chars).strip("_")
# Collapse runs of underscores (from adjacent separators) to a
# single "_" so the result can never contain a forbidden "__".
cleaned_part = re.sub(r"_+", "_", "".join(cleaned_chars)).strip("_")
if cleaned_part:
clean_parts.append(cleaned_part)

View file

@ -6,6 +6,7 @@ from typing import Any
from . import BaseHandler
from .. import j2
from ..escape import escape_jinja_literal
class IniHandler(BaseHandler):
@ -84,16 +85,18 @@ class IniHandler(BaseHandler):
line = raw_line
stripped = line.lstrip()
# Blank or pure comment: keep as-is
# Blank or pure comment: keep formatting, but neutralise any
# template metacharacters so attacker-controlled comment text cannot
# become live template code in the output.
if not stripped or stripped[0] in {"#", ";"}:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
# Section header
if stripped.startswith("[") and "]" in stripped:
header_inner = stripped[1 : stripped.index("]")]
current_section = header_inner.strip()
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
# Work without newline so we can re-attach it exactly
@ -108,8 +111,9 @@ class IniHandler(BaseHandler):
eq_index = content.find("=")
if eq_index == -1:
# Not a simple key=value line: leave untouched
out_lines.append(raw_line)
# Not a simple key=value line: leave content intact but escape
# any template metacharacters.
out_lines.append(escape_jinja_literal(raw_line))
continue
before_eq = content[:eq_index]
@ -117,7 +121,7 @@ class IniHandler(BaseHandler):
key = before_eq.strip()
if not key:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
# Whitespace after '='
@ -146,8 +150,16 @@ class IniHandler(BaseHandler):
else:
replacement_value = j2.variable(var_name)
# ``before_eq`` (key + surrounding whitespace) and ``comment_part``
# both originate from the source file and may carry template
# metacharacters; escape each independently so the safe
# ``replacement_value`` placeholder between them is preserved.
new_content = (
before_eq + "=" + leading_ws + replacement_value + comment_part
escape_jinja_literal(before_eq)
+ "="
+ leading_ws
+ replacement_value
+ escape_jinja_literal(comment_part)
)
out_lines.append(new_content + newline)

View file

@ -7,7 +7,8 @@ from typing import Any
from . import DictLikeHandler
from .. import j2
from ..loop_analyzer import LoopCandidate
from ..escape import escape_jinja_literal
from ..loop_analyzer import LoopCandidate, is_safe_loop_field_key
class JsonHandler(DictLikeHandler):
@ -109,10 +110,18 @@ class JsonHandler(DictLikeHandler):
chunks: list[str] = []
pos = 0
for path, start, end in spans:
chunks.append(text[pos:start])
# Text between scalar values (object keys, structural punctuation,
# whitespace, and any comment-like trailing text) is copied verbatim
# from the source file. Like every other text-emitting handler, this
# verbatim text must be neutralised: if it contains Jinja2 markup it
# would otherwise become live template code at apply time. The value
# itself is replaced with a safe placeholder below. ``escape_jinja_literal``
# is a no-op on text without Jinja markers, so benign JSON is unchanged
# byte-for-byte and a later render reproduces the original characters.
chunks.append(escape_jinja_literal(text[pos:start]))
chunks.append(self._json_value_expr(self.make_var_name(role_prefix, path)))
pos = end
chunks.append(text[pos:])
chunks.append(escape_jinja_literal(text[pos:]))
return "".join(chunks)
def _collect_json_scalar_spans(
@ -213,7 +222,12 @@ class JsonHandler(DictLikeHandler):
def _walk(obj: Any, path: tuple[str, ...] = ()) -> Any:
if isinstance(obj, dict):
return {k: _walk(v, path + (str(k),)) for k, v in obj.items()}
# Keys are emitted verbatim into the template, so neutralise any
# Jinja markup in them (see _generate_json_template_from_text).
return {
escape_jinja_literal(str(k)): _walk(v, path + (str(k),))
for k, v in obj.items()
}
if isinstance(obj, list):
return [_walk(v, path + (str(i),)) for i, v in enumerate(obj)]
# scalar - use marker that will be replaced with to_json
@ -261,7 +275,12 @@ class JsonHandler(DictLikeHandler):
return f"__LOOP_DICT__{collection_var}__{item_var}__"
if isinstance(obj, dict):
return {k: _walk(v, current_path + (str(k),)) for k, v in obj.items()}
# Keys are emitted verbatim into the template, so neutralise any
# Jinja markup in them (see _generate_json_template_from_text).
return {
escape_jinja_literal(str(k)): _walk(v, current_path + (str(k),))
for k, v in obj.items()
}
if isinstance(obj, list):
# Check if this list is a loop candidate
if current_path in loop_paths:
@ -364,8 +383,21 @@ class JsonHandler(DictLikeHandler):
] # first line has no indent; we prepend `inner` when emitting
for i, key in enumerate(keys):
comma = "," if i < len(keys) - 1 else ""
# Defence in depth: never interpolate a raw source key into an
# ``item_var.key`` reference. A key such as ``a }}{{ x`` would break
# out of the value placeholder and inject a live construct that the
# output-safety gate cannot distinguish from a legitimate variable.
if not is_safe_loop_field_key(key):
raise ValueError(
f"refusing to emit loop-item field reference for unsafe key: {key!r}"
)
# The literal key text is emitted verbatim into the template; escape any
# Jinja markup in it. The value side ({item_var}.{key}) is constrained by
# the output safety gate's dotted-name allowlist, which fails closed on
# anything that is not a plain identifier path.
dict_lines.append(
f'{field}"{key}": ' f"{j2.to_json(f'{item_var}.{key}')}{comma}"
f'{field}"{escape_jinja_literal(str(key))}": '
f"{j2.to_json(f'{item_var}.{key}')}{comma}"
)
# Comma between *items* goes after the closing brace.
dict_lines.append(f"{inner}}}{j2.if_not_loop_last()},{j2.endif()}")

View file

@ -5,6 +5,7 @@ from typing import Any
from . import BaseHandler
from .. import j2
from ..escape import escape_jinja_literal
class PostfixMainHandler(BaseHandler):
@ -108,16 +109,16 @@ class PostfixMainHandler(BaseHandler):
stripped = content.strip()
if not stripped:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
i += 1
continue
if stripped.startswith("#"):
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
i += 1
continue
if "=" not in content:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
i += 1
continue
@ -127,7 +128,7 @@ class PostfixMainHandler(BaseHandler):
key = before_eq.strip()
if not key:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
i += 1
continue
@ -162,16 +163,21 @@ class PostfixMainHandler(BaseHandler):
var = self.make_var_name(role_prefix, (key,))
v = value
# ``before_eq`` (key) and ``comment_part`` are source-derived and may
# contain template metacharacters; escape each around the safe
# placeholder.
safe_before = escape_jinja_literal(before_eq)
safe_comment = escape_jinja_literal(comment_part)
quoted = len(v) >= 2 and v[0] == v[-1] and v[0] in {'"', "'"}
if quoted:
replacement = (
f"{before_eq}={leading_ws}{j2.quoted_variable(var)}"
f"{comment_part}{newline}"
f"{safe_before}={leading_ws}{j2.quoted_variable(var)}"
f"{safe_comment}{newline}"
)
else:
replacement = (
f"{before_eq}={leading_ws}{j2.variable(var)}"
f"{comment_part}{newline}"
f"{safe_before}={leading_ws}{j2.variable(var)}"
f"{safe_comment}{newline}"
)
out_lines.append(replacement)

View file

@ -7,6 +7,7 @@ from typing import Any
from . import BaseHandler
from .. import j2
from ..escape import escape_jinja_literal
_SECTION_KEYWORDS = {"host", "match"}
@ -255,12 +256,12 @@ class SshConfigHandler(BaseHandler):
out_lines: list[str] = []
for ln in parsed.lines:
if ln.kind != "kv":
out_lines.append(ln.raw)
out_lines.append(escape_jinja_literal(ln.raw))
continue
path = self._path_for_line(ln)
if not path:
out_lines.append(ln.raw)
out_lines.append(escape_jinja_literal(ln.raw))
continue
var = self.make_var_name(role_prefix, path)
@ -270,9 +271,12 @@ class SshConfigHandler(BaseHandler):
else:
replacement_value = j2.variable(var)
# ``before_value`` (keyword + spacing) and ``comment`` are
# source-derived; escape around the safe placeholder.
rendered = (
f"{ln.before_value}{replacement_value}"
f"{ln.whitespace_before_comment}{ln.comment}{ln.newline}"
f"{escape_jinja_literal(ln.before_value)}{replacement_value}"
f"{ln.whitespace_before_comment}"
f"{escape_jinja_literal(ln.comment)}{ln.newline}"
)
out_lines.append(rendered)

View file

@ -6,6 +6,7 @@ from typing import Any
from . import BaseHandler
from .. import j2
from ..escape import escape_jinja_literal
@dataclass
@ -157,7 +158,13 @@ class SystemdUnitHandler(BaseHandler):
out_lines: list[str] = []
for ln in parsed.lines:
if ln.kind != "kv" or not ln.section or not ln.key:
out_lines.append(ln.raw)
# Verbatim lines (blank/comment/section/unrecognised "raw")
# originate from the source file. Escape template
# metacharacters so they cannot become live template code.
# This is the only handler that emits unrecognised lines, which
# is where Jinja *statement* injection (``{% ... %}``) was
# possible, so escaping here is essential.
out_lines.append(escape_jinja_literal(ln.raw))
continue
path: tuple[str, ...] = (ln.section, ln.key)
@ -166,16 +173,18 @@ class SystemdUnitHandler(BaseHandler):
var = self.make_var_name(role_prefix, path)
v = (ln.value or "").strip()
safe_before = escape_jinja_literal(ln.before_eq)
safe_comment = escape_jinja_literal(ln.comment)
quoted = len(v) >= 2 and v[0] == v[-1] and v[0] in {'"', "'"}
if quoted:
repl = (
f"{ln.before_eq}={ln.leading_ws_after_eq}"
f"{j2.quoted_variable(var)}{ln.comment}"
f"{safe_before}={ln.leading_ws_after_eq}"
f"{j2.quoted_variable(var)}{safe_comment}"
)
else:
repl = (
f"{ln.before_eq}={ln.leading_ws_after_eq}"
f"{j2.variable(var)}{ln.comment}"
f"{safe_before}={ln.leading_ws_after_eq}"
f"{j2.variable(var)}{safe_comment}"
)
newline = "\n" if ln.raw.endswith("\n") else ""

View file

@ -5,7 +5,8 @@ from typing import Any
from . import DictLikeHandler
from .. import j2
from ..loop_analyzer import LoopCandidate
from ..escape import escape_jinja_literal
from ..loop_analyzer import LoopCandidate, is_safe_loop_field_key
try:
import tomllib
@ -77,19 +78,25 @@ class TomlHandler(DictLikeHandler):
def emit_kv(path: tuple[str, ...], key: str, value: Any) -> None:
var_name = self.make_var_name(role_prefix, path + (key,))
if isinstance(value, str):
lines.append(f"{key} = {self._toml_quoted_expr(var_name)}")
lines.append(
f"{escape_jinja_literal(str(key))} = {self._toml_quoted_expr(var_name)}"
)
elif isinstance(value, bool):
# Booleans need | lower filter (Python True/False → TOML true/false)
lines.append(f"{key} = {self._toml_value_expr(var_name, value)}")
lines.append(
f"{escape_jinja_literal(str(key))} = {self._toml_value_expr(var_name, value)}"
)
else:
lines.append(f"{key} = {self._toml_value_expr(var_name, value)}")
lines.append(
f"{escape_jinja_literal(str(key))} = {self._toml_value_expr(var_name, value)}"
)
def walk(obj: dict[str, Any], path: tuple[str, ...] = ()) -> None:
scalar_items = {k: v for k, v in obj.items() if not isinstance(v, dict)}
nested_items = {k: v for k, v in obj.items() if isinstance(v, dict)}
if path:
header = ".".join(path)
header = ".".join(escape_jinja_literal(str(p)) for p in path)
lines.append(f"[{header}]")
for key, val in scalar_items.items():
@ -130,10 +137,14 @@ class TomlHandler(DictLikeHandler):
def emit_kv(path: tuple[str, ...], key: str, value: Any) -> None:
var_name = self.make_var_name(role_prefix, path + (key,))
if isinstance(value, str):
lines.append(f"{key} = {self._toml_quoted_expr(var_name)}")
lines.append(
f"{escape_jinja_literal(str(key))} = {self._toml_quoted_expr(var_name)}"
)
elif isinstance(value, bool):
# Booleans need | lower filter (Python True/False → TOML true/false)
lines.append(f"{key} = {self._toml_value_expr(var_name, value)}")
lines.append(
f"{escape_jinja_literal(str(key))} = {self._toml_value_expr(var_name, value)}"
)
elif isinstance(value, list):
# Check if this list is a loop candidate
if path + (key,) in loop_paths:
@ -157,19 +168,26 @@ class TomlHandler(DictLikeHandler):
elif candidate.item_schema in ("simple_dict", "nested"):
# Dict list loop - TOML array of tables
# This is complex for TOML, using simplified approach
lines.append(f"{key} = " f"{j2.to_json(var_name)}")
lines.append(
f"{escape_jinja_literal(str(key))} = "
f"{j2.to_json(var_name)}"
)
else:
# Not a loop, treat as regular variable
lines.append(f"{key} = {self._toml_value_expr(var_name, value)}")
lines.append(
f"{escape_jinja_literal(str(key))} = {self._toml_value_expr(var_name, value)}"
)
else:
lines.append(f"{key} = {self._toml_value_expr(var_name, value)}")
lines.append(
f"{escape_jinja_literal(str(key))} = {self._toml_value_expr(var_name, value)}"
)
def walk(obj: dict[str, Any], path: tuple[str, ...] = ()) -> None:
scalar_items = {k: v for k, v in obj.items() if not isinstance(v, dict)}
nested_items = {k: v for k, v in obj.items() if isinstance(v, dict)}
if path:
header = ".".join(path)
header = ".".join(escape_jinja_literal(str(p)) for p in path)
lines.append(f"[{header}]")
for key, val in scalar_items.items():
@ -217,7 +235,7 @@ class TomlHandler(DictLikeHandler):
# Blank or pure comment
if not stripped or stripped.startswith("#"):
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
# Table header: [server] or [server.tls] or [[array.of.tables]]
@ -230,7 +248,7 @@ class TomlHandler(DictLikeHandler):
inner = inner.strip("[]") # handle [[table]] as well
parts = [p.strip() for p in inner.split(".") if p.strip()]
current_table = tuple(parts)
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
# Try key = value
@ -245,7 +263,7 @@ class TomlHandler(DictLikeHandler):
eq_index = content.find("=")
if eq_index == -1:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
before_eq = content[:eq_index]
@ -253,7 +271,7 @@ class TomlHandler(DictLikeHandler):
key = before_eq.strip()
if not key:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
# Whitespace after '='
@ -289,19 +307,23 @@ class TomlHandler(DictLikeHandler):
nested_var = self.make_var_name(role_prefix, nested_path)
if isinstance(sub_val, str):
inner_bits.append(
f"{sub_key} = {self._toml_quoted_expr(nested_var)}"
f"{escape_jinja_literal(str(sub_key))} = {self._toml_quoted_expr(nested_var)}"
)
elif isinstance(sub_val, bool):
inner_bits.append(
f"{sub_key} = {self._toml_value_expr(nested_var, sub_val)}"
f"{escape_jinja_literal(str(sub_key))} = {self._toml_value_expr(nested_var, sub_val)}"
)
else:
inner_bits.append(
f"{sub_key} = {self._toml_value_expr(nested_var, sub_val)}"
f"{escape_jinja_literal(str(sub_key))} = {self._toml_value_expr(nested_var, sub_val)}"
)
replacement_value = "{ " + ", ".join(inner_bits) + " }"
new_content = (
before_eq + "=" + leading_ws + replacement_value + comment_part
escape_jinja_literal(before_eq)
+ "="
+ leading_ws
+ replacement_value
+ escape_jinja_literal(comment_part)
)
out_lines.append(new_content + newline)
continue
@ -327,7 +349,11 @@ class TomlHandler(DictLikeHandler):
replacement_value = j2.variable(var_name)
new_content = (
before_eq + "=" + leading_ws + replacement_value + comment_part
escape_jinja_literal(before_eq)
+ "="
+ leading_ws
+ replacement_value
+ escape_jinja_literal(comment_part)
)
out_lines.append(new_content + newline)
@ -355,7 +381,7 @@ class TomlHandler(DictLikeHandler):
if not stripped or stripped.startswith("#"):
# Only output if we're not skipping
if not skip_until_next_table:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
# Table header: [server] or [server.tls] or [[array.of.tables]]
@ -404,25 +430,39 @@ class TomlHandler(DictLikeHandler):
out_lines.append(
f"{j2.for_start(item_var, collection_var)}\n"
)
out_lines.append(f"[[{'.'.join(table_path)}]]\n")
out_lines.append(
f"[[{'.'.join(escape_jinja_literal(str(p)) for p in table_path)}]]\n"
)
# Add fields from sample item
for key, value in sample_item.items():
if key == "_key":
continue
# Defence in depth: the loop analyzer refuses a
# dict-loop whose items contain a non-identifier
# key, so ``key`` is always a plain identifier
# here. Never interpolate a raw key into an
# ``item_var.key`` reference: a key such as
# ``a }}{{ x`` would break out of the placeholder
# and inject a live construct.
if not is_safe_loop_field_key(key):
raise ValueError(
"refusing to emit loop-item field "
f"reference for unsafe key: {key!r}"
)
if isinstance(value, str):
out_lines.append(
f"{key} = "
f"{escape_jinja_literal(str(key))} = "
f"{self._toml_quoted_expr(f'{item_var}.{key}')}\n"
)
elif isinstance(value, bool):
out_lines.append(
f"{key} = "
f"{escape_jinja_literal(str(key))} = "
f"{self._toml_value_expr(f'{item_var}.{key}', value)}\n"
)
else:
out_lines.append(
f"{key} = "
f"{escape_jinja_literal(str(key))} = "
f"{self._toml_value_expr(f'{item_var}.{key}', value)}\n"
)
@ -437,7 +477,7 @@ class TomlHandler(DictLikeHandler):
skip_until_next_table = False
current_table = table_path
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
# If we're inside a skipped array-of-tables section, skip this line
@ -456,7 +496,7 @@ class TomlHandler(DictLikeHandler):
eq_index = content.find("=")
if eq_index == -1:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
before_eq = content[:eq_index]
@ -464,7 +504,7 @@ class TomlHandler(DictLikeHandler):
key = before_eq.strip()
if not key:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
# Whitespace after '='
@ -501,7 +541,11 @@ class TomlHandler(DictLikeHandler):
replacement_value = j2.to_json(collection_var)
new_content = (
before_eq + "=" + leading_ws + replacement_value + comment_part
escape_jinja_literal(before_eq)
+ "="
+ leading_ws
+ replacement_value
+ escape_jinja_literal(comment_part)
)
out_lines.append(new_content + newline)
continue
@ -526,19 +570,23 @@ class TomlHandler(DictLikeHandler):
nested_var = self.make_var_name(role_prefix, nested_path)
if isinstance(sub_val, str):
inner_bits.append(
f"{sub_key} = {self._toml_quoted_expr(nested_var)}"
f"{escape_jinja_literal(str(sub_key))} = {self._toml_quoted_expr(nested_var)}"
)
elif isinstance(sub_val, bool):
inner_bits.append(
f"{sub_key} = {self._toml_value_expr(nested_var, sub_val)}"
f"{escape_jinja_literal(str(sub_key))} = {self._toml_value_expr(nested_var, sub_val)}"
)
else:
inner_bits.append(
f"{sub_key} = {self._toml_value_expr(nested_var, sub_val)}"
f"{escape_jinja_literal(str(sub_key))} = {self._toml_value_expr(nested_var, sub_val)}"
)
replacement_value = "{ " + ", ".join(inner_bits) + " }"
new_content = (
before_eq + "=" + leading_ws + replacement_value + comment_part
escape_jinja_literal(before_eq)
+ "="
+ leading_ws
+ replacement_value
+ escape_jinja_literal(comment_part)
)
out_lines.append(new_content + newline)
continue
@ -564,7 +612,11 @@ class TomlHandler(DictLikeHandler):
replacement_value = j2.variable(var_name)
new_content = (
before_eq + "=" + leading_ws + replacement_value + comment_part
escape_jinja_literal(before_eq)
+ "="
+ leading_ws
+ replacement_value
+ escape_jinja_literal(comment_part)
)
out_lines.append(new_content + newline)

View file

@ -3,10 +3,12 @@ from __future__ import annotations
from collections import Counter, defaultdict
from pathlib import Path
from typing import Any
import xml.etree.ElementTree as ET # nosec
import xml.etree.ElementTree as ET # nosec B405 - safe trees only; parsing uses defusedxml
import defusedxml.ElementTree as DET
from .base import BaseHandler
from .. import j2
from ..escape import escape_jinja_literal
from ..loop_analyzer import LoopCandidate
@ -19,11 +21,11 @@ class XmlHandler(BaseHandler):
def parse(self, path: Path) -> ET.Element:
text = path.read_text(encoding="utf-8")
parser = ET.XMLParser(
target=ET.TreeBuilder(insert_comments=False)
) # nosec B314
parser.feed(text)
root = parser.close()
# Security must live in the handler, not only in the CLI entry point:
# callers may import JinjaTurtle as a library and invoke parse_config()
# directly. defusedxml rejects DTD/entity abuse and also discards
# comments by default, matching the previous TreeBuilder behaviour.
root = DET.fromstring(text)
return root
def flatten(self, parsed: Any) -> list[tuple[tuple[str, ...], Any]]:
@ -230,6 +232,37 @@ class XmlHandler(BaseHandler):
walk(root, ())
# Internal marker prefixes used by JinjaTurtle's own comment nodes. These
# must NOT be escaped (they are converted into real Jinja control structures
# downstream). Source-file comments have none of these prefixes.
_MARKER_PREFIXES = ("LOOP:", "IF:", "ENDIF:")
def _is_jt_marker(self, comment_text: str) -> bool:
stripped = (comment_text or "").lstrip()
return any(stripped.startswith(p) for p in self._MARKER_PREFIXES)
def _escape_source_comments(self, root: ET.Element) -> None:
"""Escape template metacharacters in comments preserved from the source.
XML comments are re-emitted verbatim by ``ET.tostring`` (the tree is
parsed with ``insert_comments=True``). Attacker-controlled comment text
such as ``<!-- {{ cmd.run('id') }} -->`` would otherwise become live
template code. Element/attribute *names* cannot carry Jinja delimiters
(XML naming rules forbid the characters and the parser rejects them), and
text/attribute *values* are already replaced with ``{{ var }}``
placeholders, so comments (and the prolog, handled separately) are the
only XML injection vector.
JinjaTurtle's own internal marker comments are left untouched so they can
be converted into real loops/conditionals later.
"""
# ET represents comments with a callable tag (ET.Comment). Iterate all
# descendants and escape comment text that is not one of our markers.
for elem in root.iter():
if elem.tag is ET.Comment:
if not self._is_jt_marker(elem.text or ""):
elem.text = escape_jinja_literal(elem.text or "")
def _generate_xml_template_from_text(self, role_prefix: str, text: str) -> str:
"""Generate scalar-only Jinja2 template."""
prolog, body = self._split_xml_prolog(text)
@ -240,12 +273,16 @@ class XmlHandler(BaseHandler):
self._apply_jinja_to_xml_tree(role_prefix, root)
# Neutralise template metacharacters in any comments preserved from the
# source file before serialising.
self._escape_source_comments(root)
indent = getattr(ET, "indent", None)
if indent is not None:
indent(root, space=" ") # type: ignore[arg-type]
xml_body = ET.tostring(root, encoding="unicode")
return prolog + xml_body
return escape_jinja_literal(prolog) + xml_body
def _generate_xml_template_with_loops_from_text(
self,
@ -265,6 +302,11 @@ class XmlHandler(BaseHandler):
# Apply Jinja transformations (including loop markers)
self._apply_jinja_to_xml_tree(role_prefix, root, loop_candidates)
# Escape comments preserved from the source. JinjaTurtle's own
# LOOP/IF/ENDIF marker comments are recognised and left intact so they
# can be converted into real Jinja control structures below.
self._escape_source_comments(root)
# Convert to string
indent = getattr(ET, "indent", None)
if indent is not None:
@ -275,7 +317,7 @@ class XmlHandler(BaseHandler):
# Post-process to replace loop markers with actual Jinja loops
xml_body = self._insert_xml_loops(xml_body, role_prefix, loop_candidates, root)
return prolog + xml_body
return escape_jinja_literal(prolog) + xml_body
def _insert_xml_loops(
self,

View file

@ -6,9 +6,41 @@ from typing import Any
from .dict import DictLikeHandler
from .. import j2
from ..escape import escape_jinja_literal
from ..loop_analyzer import is_safe_loop_field_key
from ..loop_analyzer import LoopCandidate
def _reject_recursive_structure(obj: Any) -> None:
"""Raise ``yaml.YAMLError`` if *obj* contains a reference cycle.
A recursive YAML anchor (``a: &a [*a]``) produces a container that contains
itself. Every consumer in JinjaTurtle walks the parsed object depth-first,
so a cycle would raise ``RecursionError`` deep in unrelated code. Detect it
up front by tracking the ``id()`` of containers on the current descent path;
a repeat means a cycle. ``yaml.YAMLError`` is raised so ``parse_config``
normalises it into a clean ``ConfigParseError`` like any other malformed
input, rather than surfacing a stack-overflow traceback.
"""
on_path: set[int] = set()
def walk(node: Any) -> None:
if isinstance(node, (dict, list)):
marker = id(node)
if marker in on_path:
raise yaml.YAMLError(
"recursive/self-referential YAML structure is not supported"
)
on_path.add(marker)
children = node.values() if isinstance(node, dict) else node
for child in children:
walk(child)
on_path.discard(marker)
walk(obj)
class YamlHandler(DictLikeHandler):
"""
YAML handler that can generate both scalar templates and loop-based templates.
@ -19,7 +51,16 @@ class YamlHandler(DictLikeHandler):
def parse(self, path: Path) -> Any:
text = path.read_text(encoding="utf-8")
return yaml.safe_load(text) or {}
parsed = yaml.safe_load(text) or {}
# PyYAML's safe_load happily builds *recursive* structures from an anchor
# that references itself (e.g. ``a: &a [*a]``). Downstream flattening,
# timestamp-stringifying and template generation all walk the parsed
# object recursively and would blow the Python stack (RecursionError) on
# such input. JinjaTurtle is regularly pointed at harvested,
# attacker-influenceable config, so reject a self-referential document
# cleanly here rather than crashing later.
_reject_recursive_structure(parsed)
return parsed
def generate_jinja2_template(
self,
@ -102,7 +143,7 @@ class YamlHandler(DictLikeHandler):
indent = len(raw_line) - len(stripped)
if not stripped or stripped.startswith("#"):
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
while stack and indent < stack[-1][0]:
@ -112,7 +153,7 @@ class YamlHandler(DictLikeHandler):
key_part, rest = stripped.split(":", 1)
key = key_part.strip()
if not key:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
rest_stripped = rest.lstrip(" \t")
@ -125,7 +166,7 @@ class YamlHandler(DictLikeHandler):
stack.append((indent, path, "map"))
if not has_value:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
value_part, comment_part = self._split_inline_comment(
@ -147,8 +188,8 @@ class YamlHandler(DictLikeHandler):
replacement = self._yaml_scalar_expr(var_name, raw_value)
leading = rest[: len(rest) - len(rest.lstrip(" \t"))]
new_rest = f"{leading}{replacement}{comment_part}"
new_stripped = f"{key}:{new_rest}"
new_rest = f"{leading}{replacement}{escape_jinja_literal(comment_part)}"
new_stripped = f"{escape_jinja_literal(key)}:{new_rest}"
out_lines.append(
" " * indent
+ new_stripped
@ -185,7 +226,7 @@ class YamlHandler(DictLikeHandler):
else:
replacement = self._yaml_scalar_expr(var_name, raw_value)
new_stripped = f"- {replacement}{comment_part}"
new_stripped = f"- {replacement}{escape_jinja_literal(comment_part)}"
out_lines.append(
" " * indent
+ new_stripped
@ -193,7 +234,7 @@ class YamlHandler(DictLikeHandler):
)
continue
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
return "".join(out_lines)
@ -275,7 +316,7 @@ class YamlHandler(DictLikeHandler):
next_line = next_significant_line(line_index)
if next_line is None:
skip_until_indent = None
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
else:
next_indent, next_stripped = next_line
still_in_collection = next_indent > skip_until_indent or (
@ -284,12 +325,12 @@ class YamlHandler(DictLikeHandler):
)
if not still_in_collection:
skip_until_indent = None
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
if is_comment:
if indent <= skip_until_indent:
skip_until_indent = None
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
# Comments/blank lines indented beneath the replaced
# collection are considered part of that collection and
# cannot be placed safely inside a generated loop.
@ -303,7 +344,7 @@ class YamlHandler(DictLikeHandler):
# Blank or comment lines
if is_blank or is_comment:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
# Adjust stack based on indent
@ -315,7 +356,7 @@ class YamlHandler(DictLikeHandler):
key_part, rest = stripped.split(":", 1)
key = key_part.strip()
if not key:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
rest_stripped = rest.lstrip(" \t")
@ -353,7 +394,7 @@ class YamlHandler(DictLikeHandler):
continue
if not has_value:
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
continue
# Scalar value - replace with variable
@ -376,8 +417,8 @@ class YamlHandler(DictLikeHandler):
replacement = self._yaml_scalar_expr(var_name, raw_value)
leading = rest[: len(rest) - len(rest.lstrip(" \t"))]
new_rest = f"{leading}{replacement}{comment_part}"
new_stripped = f"{key}:{new_rest}"
new_rest = f"{leading}{replacement}{escape_jinja_literal(comment_part)}"
new_stripped = f"{escape_jinja_literal(key)}:{new_rest}"
out_lines.append(
" " * indent
+ new_stripped
@ -439,7 +480,7 @@ class YamlHandler(DictLikeHandler):
else:
replacement = self._yaml_scalar_expr(var_name, raw_value)
new_stripped = f"- {replacement}{comment_part}"
new_stripped = f"- {replacement}{escape_jinja_literal(comment_part)}"
out_lines.append(
" " * indent
+ new_stripped
@ -447,7 +488,7 @@ class YamlHandler(DictLikeHandler):
)
continue
out_lines.append(raw_line)
out_lines.append(escape_jinja_literal(raw_line))
return "".join(out_lines)
@ -480,7 +521,7 @@ class YamlHandler(DictLikeHandler):
lines: list[str] = []
if not is_list:
key = candidate.path[-1] if candidate.path else "items"
lines.append(f"{indent_str}{key}:")
lines.append(f"{indent_str}{escape_jinja_literal(str(key))}:")
item_lines: list[str] = []
if candidate.items:
@ -510,10 +551,10 @@ class YamlHandler(DictLikeHandler):
# into the rendered YAML and nesting the next top-level key.
lines.append(f"{j2.for_start(item_var, collection_var)}{item_lines[0]}")
lines.extend(item_lines[1:])
lines.append(j2.for_end())
lines.append(j2.for_end(keep_trailing_newline=True))
else:
lines.append(j2.for_start(item_var, collection_var))
lines.append(j2.for_end())
lines.append(j2.for_end(keep_trailing_newline=True))
return "\n".join(lines)
@ -546,15 +587,31 @@ class YamlHandler(DictLikeHandler):
# Special key for dict collections - output as comment or skip
continue
# Defence in depth: the loop analyzer already refuses a dict-loop
# whose items contain a non-identifier key (see _analyze_dict_schema),
# so ``key`` should always be a plain identifier here. Assert it
# rather than interpolate a raw key into a Jinja reference: a key such
# as ``a }}{{ x`` would otherwise close the placeholder and inject a
# live construct that the output gate cannot distinguish from a
# legitimate variable.
if not is_safe_loop_field_key(key):
raise ValueError(
f"refusing to emit loop-item field reference for unsafe key: {key!r}"
)
if first_key and is_list_item:
# First key gets the list marker
value_expr = self._yaml_value_expr(f"{loop_var}.{key}", value)
lines.append(f"{indent_str}- {key}: {value_expr}")
lines.append(
f"{indent_str}- {escape_jinja_literal(str(key))}: {value_expr}"
)
first_key = False
else:
# Subsequent keys are indented
sub_indent = indent + 2 if is_list_item else indent
value_expr = self._yaml_value_expr(f"{loop_var}.{key}", value)
lines.append(f"{' ' * sub_indent}{key}: {value_expr}")
lines.append(
f"{' ' * sub_indent}{escape_jinja_literal(str(key))}: {value_expr}"
)
return lines

View file

@ -2,7 +2,6 @@ from __future__ import annotations
from typing import Any
NAME = "jinja2"
TEMPLATE_EXTENSION = "j2"
JSON_VALUE_FILTER = "to_json(ensure_ascii=False)"
@ -43,10 +42,19 @@ def to_json(
return filtered(value, f"to_json({', '.join(args)})")
def statement(value: str, *, trim_right: bool = False) -> str:
def statement(
value: str,
*,
trim_right: bool = False,
keep_trailing_newline: bool = False,
) -> str:
"""Return a Jinja2 statement tag for an already-built statement body."""
if trim_right and keep_trailing_newline:
raise ValueError("trim_right and keep_trailing_newline are mutually exclusive")
if trim_right:
return f"{{% {value} -%}}"
if keep_trailing_newline:
return f"{{% {value} +%}}"
return f"{{% {value} %}}"
@ -66,8 +74,12 @@ def for_start(item_var: str, collection_var: str) -> str:
return statement(f"for {item_var} in {collection_var}")
def for_end(*, trim_right: bool = False) -> str:
return statement("endfor", trim_right=trim_right)
def for_end(*, trim_right: bool = False, keep_trailing_newline: bool = False) -> str:
return statement(
"endfor",
trim_right=trim_right,
keep_trailing_newline=keep_trailing_newline,
)
def yaml_scalar_expression(var_name: str, raw_value: str | None = None) -> str:

View file

@ -7,10 +7,34 @@ instead of flattened scalar variables.
from __future__ import annotations
import re
from collections import Counter
from typing import Any, Literal
# A dict-loop emits per-item field references of the form ``loopvar.<field>``.
# ``<field>`` is derived from a source key, which is attacker-influenceable when
# JinjaTurtle is fed harvested config. The output-safety gate only accepts a
# reference whose every hop matches this identifier class, so a key must reduce
# to exactly this shape before it can be used as a loop-item field. Anything else
# (a key containing ``}}``, quotes, ``.``, ``__``, ...) must not be turned into a
# loop; the caller falls back to scalar generation, where every value goes through
# make_var_name() and all verbatim text is escaped.
_SAFE_LOOP_FIELD_RE = re.compile(r"(?!\w*__)[A-Za-z_][A-Za-z0-9_]*\Z")
def is_safe_loop_field_key(key: Any) -> bool:
"""Return True if *key* is safe to emit as a ``loopvar.<key>`` field access.
The check mirrors the output-safety gate's identifier class (single
identifier, no double underscore). A key that does not match cannot be
expressed as a dotted loop-item reference without risking template
injection, so a loop candidate containing such a key is rejected upstream.
"""
return bool(_SAFE_LOOP_FIELD_RE.match(str(key)))
class LoopCandidate:
"""
Represents a detected loop opportunity in the config structure.
@ -265,6 +289,19 @@ class LoopAnalyzer:
if not dicts:
return "heterogeneous"
# Security: a dict-loop emits ``loopvar.<key>`` field references. If any
# item key is not a plain identifier, it cannot be expressed safely as a
# dotted reference (a key such as ``a }}{{ x`` would break out of the
# placeholder and inject a live construct). Refuse the loop so the caller
# falls back to scalar generation, which escapes all verbatim text and
# routes every value through make_var_name().
for d in dicts:
for k in d.keys():
if k == "_key":
continue
if not is_safe_loop_field_key(k):
return "heterogeneous"
# Get key sets from each dict
key_sets = [set(d.keys()) for d in dicts]

View file

@ -22,15 +22,26 @@ Notes:
from collections import Counter, defaultdict
from copy import deepcopy
import os
import configparser
import stat
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Iterable
import xml.etree.ElementTree as ET # nosec
from . import j2
from .core import dump_yaml, flatten_config, make_var_name, parse_config
from .core import (
dump_yaml,
flatten_config,
make_var_name,
parse_config,
ConfigParseError,
)
from .handlers.xml import XmlHandler
from .safety import verify_jinja2_template_safe, verify_no_live_jinja_in_json_keys
from .escape import escape_jinja_literal
SUPPORTED_SUFFIXES: dict[str, set[str]] = {
@ -42,8 +53,23 @@ SUPPORTED_SUFFIXES: dict[str, set[str]] = {
}
def _lstat(path: Path) -> os.stat_result:
return path.lstat()
def is_supported_file(path: Path) -> bool:
if not path.is_file():
"""Return True only for real regular files with supported suffixes.
pathlib.Path.is_file() follows symlinks. Folder mode must not follow
attacker-controlled symlinks when run over an untrusted tree, especially if
an administrator accidentally runs the CLI as root.
"""
try:
st = _lstat(path)
except FileNotFoundError:
return False
if not stat.S_ISREG(st.st_mode):
return False
suffix = path.suffix.lower()
for exts in SUPPORTED_SUFFIXES.values():
@ -53,11 +79,16 @@ def is_supported_file(path: Path) -> bool:
def iter_supported_files(root: Path, recursive: bool) -> list[Path]:
if not root.exists():
try:
st = _lstat(root)
except FileNotFoundError:
raise FileNotFoundError(str(root))
if root.is_file():
if stat.S_ISLNK(st.st_mode):
raise ValueError(f"refusing to follow symlink: {root}")
if stat.S_ISREG(st.st_mode):
return [root] if is_supported_file(root) else []
if not root.is_dir():
if not stat.S_ISDIR(st.st_mode):
return []
it = root.rglob("*") if recursive else root.glob("*")
@ -160,6 +191,11 @@ def _yaml_render_union(
if isinstance(union_obj, dict):
for key, val in union_obj.items():
key_path = path + (str(key),)
# The key text is copied verbatim into the template; escape it so an
# attacker-influenced key (e.g. ``{{ 7*7 }}``) cannot become live
# template code. ``key_path`` (used only to build sanitised var
# names) keeps the original key.
safe_key = escape_jinja_literal(str(key))
cond_var = (
defined_var_name(role_prefix, key_path)
if key_path in optional_containers
@ -170,13 +206,13 @@ def _yaml_render_union(
value = _yaml_scalar_placeholder(role_prefix, key_path, val)
if cond_var:
lines.append(f"{ind}{j2.if_defined(cond_var)}")
lines.append(f"{ind}{key}: {value}")
lines.append(f"{ind}{safe_key}: {value}")
if cond_var:
lines.append(f"{ind}{j2.endif()}")
else:
if cond_var:
lines.append(f"{ind}{j2.if_defined(cond_var)}")
lines.append(f"{ind}{key}:")
lines.append(f"{ind}{safe_key}:")
lines.extend(
_yaml_render_union(
role_prefix,
@ -214,6 +250,7 @@ def _yaml_render_union(
first = True
for k, v in item.items():
kp = item_path + (str(k),)
safe_k = escape_jinja_literal(str(k))
k_cond = (
defined_var_name(role_prefix, kp)
if kp in optional_containers
@ -224,14 +261,14 @@ def _yaml_render_union(
if first:
if k_cond:
lines.append(f"{ind}{j2.if_defined(k_cond)}")
lines.append(f"{ind}- {k}: {value}")
lines.append(f"{ind}- {safe_k}: {value}")
if k_cond:
lines.append(f"{ind}{j2.endif()}")
first = False
else:
if k_cond:
lines.append(f"{ind} {j2.if_defined(k_cond)}")
lines.append(f"{ind} {k}: {value}")
lines.append(f"{ind} {safe_k}: {value}")
if k_cond:
lines.append(f"{ind} {j2.endif()}")
else:
@ -239,7 +276,7 @@ def _yaml_render_union(
if first:
if k_cond:
lines.append(f"{ind}{j2.if_defined(k_cond)}")
lines.append(f"{ind}- {k}:")
lines.append(f"{ind}- {safe_k}:")
lines.extend(
_yaml_render_union(
role_prefix,
@ -255,7 +292,7 @@ def _yaml_render_union(
else:
if k_cond:
lines.append(f"{ind} {j2.if_defined(k_cond)}")
lines.append(f"{ind} {k}:")
lines.append(f"{ind} {safe_k}:")
lines.extend(
_yaml_render_union(
role_prefix,
@ -301,6 +338,7 @@ def _toml_render_union(
def emit_kv(path: tuple[str, ...], key: str, value: Any) -> None:
var_name = make_var_name(role_prefix, path + (key,))
safe_key = escape_jinja_literal(str(key))
cond = (
defined_var_name(role_prefix, path + (key,))
if (path + (key,)) in optional_containers
@ -309,11 +347,11 @@ def _toml_render_union(
if cond:
lines.append(f"{j2.if_defined(cond)}")
if isinstance(value, str):
lines.append(f"{key} = {j2.quoted_variable(var_name)}")
lines.append(f"{safe_key} = {j2.quoted_variable(var_name)}")
elif isinstance(value, bool):
lines.append(f"{key} = {j2.lower(var_name)}")
lines.append(f"{safe_key} = {j2.lower(var_name)}")
else:
lines.append(f"{key} = {j2.variable(var_name)}")
lines.append(f"{safe_key} = {j2.variable(var_name)}")
if cond:
lines.append(j2.endif())
@ -326,7 +364,7 @@ def _toml_render_union(
)
if cond:
lines.append(f"{j2.if_defined(cond)}")
lines.append(f"[{'.'.join(path)}]")
lines.append(f"[{'.'.join(escape_jinja_literal(str(p)) for p in path)}]")
scalar_items = {k: v for k, v in obj.items() if not isinstance(v, dict)}
nested_items = {k: v for k, v in obj.items() if isinstance(v, dict)}
@ -412,10 +450,11 @@ def _ini_render_union(
)
if sec_cond:
lines.append(f"{j2.if_defined(sec_cond)}")
lines.append(f"[{section}]")
lines.append(f"[{escape_jinja_literal(str(section))}]")
for key, raw_val in union.items(section, raw=True):
path = (section, key)
var = make_var_name(role_prefix, path)
safe_key = escape_jinja_literal(str(key))
key_cond = (
defined_var_name(role_prefix, path) if path in optional_keys else None
)
@ -424,9 +463,9 @@ def _ini_render_union(
if key_cond:
lines.append(f"{j2.if_defined(key_cond)}")
if quoted:
lines.append(f"{key} = {j2.quoted_variable(var)}")
lines.append(f"{safe_key} = {j2.quoted_variable(var)}")
else:
lines.append(f"{key} = {j2.variable(var)}")
lines.append(f"{safe_key} = {j2.variable(var)}")
if key_cond:
lines.append(j2.endif())
lines.append("")
@ -599,7 +638,13 @@ def process_directory(
# Parse and group by format
grouped: dict[str, list[tuple[Path, Any]]] = defaultdict(list)
for p in files:
fmt, parsed = parse_config(p, None)
try:
fmt, parsed = parse_config(p, None)
except ConfigParseError as exc:
# One malformed file should not abort processing of an entire
# directory. Skip it with a warning; the rest still generate.
print(f"jinjaturtle: skipping {p}: {exc}", file=sys.stderr)
continue
if fmt not in FOLDER_SUPPORTED_FORMATS:
# Directory mode only supports a subset of formats for now.
continue
@ -769,4 +814,12 @@ def process_directory(
defaults_doc[out.list_var] = out.items
defaults_yaml = dump_yaml(defaults_doc, sort_keys=True)
# Defence in depth: folder-mode union templates are built by their own
# renderers (not core.generate_jinja2_template), so gate each one here too.
# Any un-neutralised source text that became a live tag aborts generation.
for out in outputs:
verify_jinja2_template_safe(out.template)
if out.fmt == "json":
verify_no_live_jinja_in_json_keys(out.template)
return defaults_yaml, outputs

View file

@ -0,0 +1,191 @@
from __future__ import annotations
"""Safer file-output helpers for the JinjaTurtle CLI.
The CLI is often used by administrators. A plain Path.write_text() follows a
final-path symlink and can therefore be dangerous when a root-run invocation
writes into an attacker-writable tree. These helpers validate path components,
refuse symlink parents, reject root-run output through untrusted parent
components, write through a temporary file in the target directory, and replace
the final path atomically. Existing final-path symlinks are refused rather than
followed.
"""
import os
from pathlib import Path
import stat
import tempfile
class OutputPathError(OSError):
"""Raised when a requested output path is unsafe."""
# Keep a reference to the real euid getter. Tests can monkeypatch
# _effective_uid directly without changing process-wide os.geteuid behaviour.
_OS_GETEUID = getattr(os, "geteuid", None)
def _effective_uid() -> int | None:
if _OS_GETEUID is None:
return None
try:
return int(_OS_GETEUID())
except OSError:
return None
def _absolute(path: Path) -> Path:
expanded = path.expanduser()
return expanded if expanded.is_absolute() else Path.cwd() / expanded
def _chmod_private(path: Path) -> None:
try:
os.chmod(path, 0o700)
except OSError:
# Best-effort; mkdir(mode=0o700) is already used for normal filesystems.
pass
def _assert_trusted_root_parent(path: Path, st: os.stat_result) -> None:
"""Reject root-run output through attacker-controlled parent directories.
A root-run JinjaTurtle process may write files that an administrator later
applies as configuration-management input. Parent directories controlled by
an unprivileged user are therefore not acceptable output anchors. Root-owned
sticky shared directories such as /tmp are allowed as a boundary, but any
existing child below them must be root-owned and not writable by group/other.
"""
if _effective_uid() != 0:
return
if not stat.S_ISDIR(st.st_mode):
raise OutputPathError(f"output parent is not a directory: {path}")
if st.st_uid != 0:
raise OutputPathError(
f"output parent is not owned by root; refusing root-run output: {path}"
)
writable_by_group_or_other = st.st_mode & (stat.S_IWGRP | stat.S_IWOTH)
sticky = st.st_mode & stat.S_ISVTX
if writable_by_group_or_other and not sticky:
raise OutputPathError(
"output parent is writable by group/other; "
f"refusing root-run output: {path}"
)
def _assert_existing_directory_component(path: Path) -> None:
try:
st = path.lstat()
except OSError as exc:
raise OutputPathError(f"unable to inspect output parent: {path}") from exc
if stat.S_ISLNK(st.st_mode):
raise OutputPathError(f"refusing to use symlink parent: {path}")
if not stat.S_ISDIR(st.st_mode):
raise OutputPathError(f"output parent is not a directory: {path}")
_assert_trusted_root_parent(path, st)
def _mkdir_safe_dir_tree(path: Path) -> Path:
"""Create/validate a directory tree one component at a time.
pathlib.mkdir(parents=True) can traverse a symlink inserted after a parent
pre-check. Walking one component at a time keeps every existing component
checked before it is used, and newly-created components are immediately
re-inspected.
"""
out = _absolute(path)
parts = out.parts
if not parts:
return out
if out.is_absolute():
cur = Path(parts[0])
rest = parts[1:]
_assert_existing_directory_component(cur)
else:
# _absolute() currently always returns an absolute path, but keep this
# branch for clarity if that helper is ever relaxed.
cur = Path.cwd()
rest = parts
_assert_existing_directory_component(cur)
for part in rest:
cur = cur / part
if os.path.lexists(cur):
_assert_existing_directory_component(cur)
continue
try:
os.mkdir(cur, 0o700)
except FileExistsError:
_assert_existing_directory_component(cur)
continue
_chmod_private(cur)
_assert_existing_directory_component(cur)
return out
def _check_existing_output_file(path: Path) -> None:
try:
st = path.lstat()
except FileNotFoundError:
return
if stat.S_ISLNK(st.st_mode):
raise OutputPathError(f"refusing to write through symlink: {path}")
if not stat.S_ISREG(st.st_mode):
raise OutputPathError(f"refusing to replace non-regular file: {path}")
def _check_parent_components(parent: Path) -> None:
"""Require every existing parent component to be a trusted real directory."""
_mkdir_safe_dir_tree(parent)
def ensure_safe_directory(path: Path) -> None:
"""Create or validate a directory tree without accepting unsafe parents."""
_mkdir_safe_dir_tree(path)
def write_text_safely(path: Path, text: str, *, encoding: str = "utf-8") -> None:
"""Write text without following a final-path symlink.
The target's parent is created/validated component by component. Existing
parent symlinks are refused for every user. When running as root, existing
parent components must also be root-owned and not writable by group/other,
except for sticky shared boundaries such as /tmp. Existing final-path
symlinks are refused rather than followed.
"""
path = _absolute(path)
parent = _mkdir_safe_dir_tree(path.parent)
_check_existing_output_file(path)
fd = -1
tmp_name: str | None = None
try:
fd, tmp_name = tempfile.mkstemp(
prefix=f".{path.name}.", suffix=".tmp", dir=str(parent), text=True
)
with os.fdopen(fd, "w", encoding=encoding) as f:
fd = -1
f.write(text)
f.flush()
os.fsync(f.fileno())
os.chmod(tmp_name, 0o600)
_check_parent_components(path.parent)
_check_existing_output_file(path)
os.replace(tmp_name, path)
tmp_name = None
finally:
if fd >= 0:
os.close(fd)
if tmp_name is not None:
try:
os.unlink(tmp_name)
except FileNotFoundError:
pass

382
src/jinjaturtle/safety.py Normal file
View file

@ -0,0 +1,382 @@
from __future__ import annotations
"""Output safety gate for generated templates (defence in depth).
JinjaTurtle's first line of defence is per-handler escaping: every piece of
verbatim source text is meant to be wrapped/neutralised before it reaches the
template (see ``escape.py``). That model is correct but *fragile*: it relies on
every handler remembering to escape at every site, and on the escaper being
exactly right for every delimiter form. A single forgotten call site -- or a
new handler, or a missed delimiter variant -- silently reopens a
template-injection / SSTI path that can become remote code execution on the
configuration-management control node when the template is later rendered.
This module adds a second, independent line of defence that does **not** depend
on getting every escape right. It inspects the *finished* template and proves a
single global property:
Every *live* template construct in the output is one that JinjaTurtle itself
legitimately emits. Anything else can only have originated from
un-neutralised source text, so generation fails loudly instead of emitting a
dangerous template.
The check is positive/allowlist-based, which is the safe direction: unknown
constructs are rejected, not ignored. It runs at the single choke points in
``core.py`` (``generate_jinja2_template``) so it covers every current handler and
every future one automatically.
Why this is robust against the escaper being wrong
---------------------------------------------------
We tokenise with Jinja2's own lexer. The lexer emits a JinjaTurtle
``{% raw %} ... {% endraw %}`` wrapper as ``raw_begin`` / inert ``data`` /
``raw_end``: the wrapped literal text is *not* tokenised as live tags. So the
verifier only ever sees, as live constructs, the tags Jinja2 would actually
execute. If an escaped block was mis-wrapped such that a payload escapes the
raw wrapper (the historical ``{%+ endraw %}`` bug), that payload now appears as a
*live* token here and is rejected -- the gate catches the failure even though
the escaper produced it.
"""
import re
__all__ = [
"TemplateSafetyError",
"verify_jinja2_template_safe",
"verify_erb_template_safe",
"verify_no_live_jinja_in_json_keys",
]
class TemplateSafetyError(Exception):
"""Raised when a generated template contains a construct JinjaTurtle would
never emit, indicating that un-neutralised source text became live template
code. Generation must abort rather than emit the template."""
# --------------------------------------------------------------------------- #
# Allowlist grammar for JinjaTurtle-emitted Jinja2.
#
# JinjaTurtle emits a deliberately tiny subset of Jinja2. Each pattern below
# describes the *full body* of a tag (the text between ``{%``/``%}`` or
# ``{{``/``}}``), already stripped of surrounding whitespace and of any ``-``/
# ``+`` whitespace-control markers. Identifiers (variable names, loop vars,
# keys) are restricted to a conservative character class; crucially this class
# excludes characters needed for SSTI gadgets (quotes, parentheses, brackets,
# arithmetic/operator characters, ``%``, ``|`` except in the known filter forms,
# attribute access beyond a single dotted hop, etc.).
# --------------------------------------------------------------------------- #
# A single identifier. Word characters only, and -- critically -- no
# double-underscore anywhere. ``__`` is the gateway to every classic Jinja2
# SSTI gadget (``__class__``, ``__init__``, ``__globals__``, ``__builtins__``),
# and JinjaTurtle never emits a dunder, so forbidding ``__`` here removes the
# entire attribute-traversal escape class even if such a token reached output.
_NAME = r"(?!\w*__)[A-Za-z_][A-Za-z0-9_]*"
# A dotted reference for loop-item field access. JinjaTurtle emits at most
# three hops (``loopvar``, ``loopvar.field``, ``loopvar.key.subkey``), so cap the
# depth rather than allow arbitrary chains.
_DOTTED = rf"{_NAME}(?:\.{_NAME}){{0,2}}"
# Filters JinjaTurtle is known to emit inside ``{{ ... }}`` expressions.
_KNOWN_FILTERS = (
r"lower",
r"to_json\(ensure_ascii=(?:True|False)\)",
r"to_json\(indent=\d+,\s*ensure_ascii=(?:True|False)\)",
r"tojson",
)
_FILTER_ALT = "|".join(_KNOWN_FILTERS)
# Expression bodies allowed inside ``{{ ... }}``.
_EXPR_PATTERNS = tuple(
re.compile(p)
for p in (
# Plain variable / dotted loop-field reference.
rf"^{_DOTTED}$",
# Filtered reference: ``name | filter`` (one known filter).
rf"^{_DOTTED}\s*\|\s*(?:{_FILTER_ALT})$",
# YAML-preserving boolean ternary emitted by j2.yaml_*_expression:
# 'true' if NAME else 'false' / "true" if NAME else "false"
rf"^(['\"])(?:true|false)\1\s+if\s+{_DOTTED}\s+else\s+(['\"])(?:true|false)\2$",
# YAML-preserving null ternary:
# 'null' if NAME is none else NAME
rf"^(['\"])null\1\s+if\s+{_DOTTED}\s+is\s+none\s+else\s+{_DOTTED}$",
)
)
# Statement bodies allowed inside ``{% ... %}``.
_STMT_PATTERNS = tuple(
re.compile(p)
for p in (
rf"^for\s+{_NAME}\s+in\s+{_DOTTED}$",
r"^endfor$",
rf"^if\s+{_DOTTED}\s+is\s+defined$",
rf"^if\s+{_DOTTED}\s+is\s+none$",
r"^if\s+not\s+loop\.last$",
r"^else$",
r"^endif$",
# ``elif`` is emitted only for the same shapes as the if-conditions
# above; keep it conservative.
rf"^elif\s+{_DOTTED}\s+is\s+(?:defined|none)$",
)
)
def _strip_ws_control(body: str) -> str:
"""Remove a leading/trailing Jinja whitespace-control marker and spaces."""
body = body.strip()
if body[:1] in "-+":
body = body[1:]
if body[-1:] in "-+":
body = body[:-1]
return body.strip()
# Built-in Jinja2/Ansible globals that JinjaTurtle never emits as a reference
# head. The allowlist grammar necessarily accepts any bare identifier (it cannot
# tell an injected global from a legitimate generated variable), so these names
# are rejected explicitly: their presence as the head of a live reference means
# source text leaked into a live construct. This is a targeted backstop and does
# not replace the grammar check.
_FORBIDDEN_REFERENCE_HEADS = frozenset(
{
"range",
"dict",
"lipsum",
"cycler",
"joiner",
"namespace",
"config",
"self",
"cycle",
"request",
"get_flashed_messages",
"url_for",
# Ansible global set
"lookup",
"q",
"query",
"now",
"omit",
"undef",
}
)
def _reference_head(body: str) -> str:
"""Return the leading identifier (before any dot/filter) of an expression."""
body = _strip_ws_control(body)
m = re.match(r"[A-Za-z_][A-Za-z0-9_]*", body)
return m.group(0) if m else ""
def _expr_is_allowed(body: str) -> bool:
body = _strip_ws_control(body)
if _reference_head(body) in _FORBIDDEN_REFERENCE_HEADS:
return False
return any(p.match(body) for p in _EXPR_PATTERNS)
def _stmt_is_allowed(body: str) -> bool:
body = _strip_ws_control(body)
# A ``for ... in <collection>`` whose collection is a forbidden global is
# rejected for the same reason as an expression head.
m = re.match(r"for\s+[A-Za-z_]\w*\s+in\s+([A-Za-z_][A-Za-z0-9_]*)", body)
if m and m.group(1) in _FORBIDDEN_REFERENCE_HEADS:
return False
return any(p.match(body) for p in _STMT_PATTERNS)
def verify_jinja2_template_safe(template_text: str) -> None:
"""Validate that *template_text* contains only JinjaTurtle-emitted Jinja2.
Raises :class:`TemplateSafetyError` on the first live construct that is not
in the allowlist grammar. Text inside JinjaTurtle's own ``{% raw %}``
wrappers is treated as inert (the lexer does not tokenise it as tags), so
legitimately-escaped source content passes.
"""
# Import lazily so the dependency is only needed when generating Jinja2.
import jinja2
env = jinja2.Environment(autoescape=True)
try:
tokens = list(env.lex(template_text))
except jinja2.TemplateSyntaxError as exc:
# A syntax error means our own raw-wrapping did not fully contain the
# source text (e.g. an early ``endraw`` breakout left dangling tags).
# That is precisely a safety failure, not a benign parse hiccup.
raise TemplateSafetyError(
f"generated template does not lex as the JinjaTurtle subset: {exc}"
) from exc
# Walk the token stream. Jinja2 yields raw blocks as single
# ``raw_begin``/``raw_end`` tokens with inert ``data`` between them, so we
# never see wrapped literal text as live tags. We collect the raw inner
# text of each live ``{% %}`` / ``{{ }}`` construct (preserving original
# spacing) and check it against the allowlist.
mode: str | None = None # None, "block", or "variable"
body_parts: list[str] = []
def _flush(kind: str, lineno: int) -> None:
body = "".join(body_parts)
if kind == "variable":
ok = _expr_is_allowed(body)
else:
ok = _stmt_is_allowed(body)
if not ok:
shown = body.strip()
wrapped = (
"{{ " + shown + " }}" if kind == "variable" else "{% " + shown + " %}"
)
raise TemplateSafetyError(
"refusing to emit template: unexpected live "
f"{'expression' if kind == 'variable' else 'statement'} "
f"{wrapped!r} at line {lineno}. This construct is not one "
"JinjaTurtle emits, so it likely came from un-neutralised "
"source text (possible template injection)."
)
for lineno, tok_type, value in tokens:
if tok_type in ("variable_begin", "block_begin"):
mode = "variable" if tok_type == "variable_begin" else "block"
body_parts = []
elif tok_type in ("variable_end", "block_end"):
if mode is not None:
_flush(mode, lineno)
mode = None
body_parts = []
elif mode is not None:
# Preserve the original token text (including its own whitespace
# tokens) so the reconstructed body matches the source spacing.
body_parts.append(value if isinstance(value, str) else str(value))
# raw_begin / raw_end / data tokens outside a tag are inert: skip.
# --------------------------------------------------------------------------- #
# JSON-key gate (defence in depth, format-specific).
#
# JinjaTurtle never emits a live Jinja construct inside a JSON *object key*: keys
# are copied verbatim from the source and (after escape.py) are wrapped in
# ``{% raw %}`` if they contain markup, so a key never lexes as a live tag. A
# live construct in key position can therefore only mean source key text leaked
# into the template unescaped (the json-handler blind spot). This gate is
# independent of the escaper: it inspects the finished template, replaces every
# *live* Jinja construct with an inert sentinel (raw-wrapped literal text stays
# literal), and rejects any sentinel that lands in a JSON key slot.
# --------------------------------------------------------------------------- #
# Sentinel byte that cannot occur in normal generated template text.
_LIVE_SENTINEL = "\x00"
# A JSON key is a double-quoted string immediately followed (after optional
# whitespace) by a colon. We only need to detect a sentinel *inside* such a
# string, so match a quoted run that ends in `":` and look for the sentinel.
_JSON_KEY_RE = re.compile(r'"((?:[^"\\]|\\.)*)"\s*:', re.S)
_KEY_JINJA_DELIMS = ("{{", "}}", "{%", "%}", "{#", "#}")
def _contains_jinja_delim(text: str) -> bool:
"""True if *text* contains any Jinja delimiter (live or escaped-literal)."""
return any(d in text for d in _KEY_JINJA_DELIMS)
def verify_no_live_jinja_in_json_keys(template_text: str) -> None:
"""Reject a JSON template that carries Jinja markup in an object key.
JinjaTurtle never templates a JSON *object key*: keys come straight from the
source and a key is an identifier/string, never a value placeholder. Any
Jinja in key position therefore means attacker-influenced source key text
reached the template. This gate fails closed on it, independent of whether a
handler left the markup *live* (a raw ``{{ ... }}`` in the key) or *escaped*
it into a ``{% raw %}`` wrapper -- both indicate a key that should never have
contained templating, so generation aborts rather than emitting it.
Detection is done on the lexer token stream: we reconstruct the document with
every live tag collapsed to a sentinel and every ``{% raw %}``-wrapped region
also marked, then reject a sentinel that lands inside a JSON key string.
"""
import jinja2
env = jinja2.Environment(autoescape=True)
try:
tokens = list(env.lex(template_text))
except jinja2.TemplateSyntaxError as exc:
raise TemplateSafetyError(
f"generated JSON template does not lex as the JinjaTurtle subset: {exc}"
) from exc
out: list[str] = []
in_tag = False
for _lineno, tok_type, value in tokens:
if tok_type in ("variable_begin", "block_begin", "comment_begin"):
# A live construct: collapse to a sentinel so it is detectable if it
# sits in key position. (raw_begin/raw_end are *not* live; the data
# inside a raw block is preserved verbatim below, so an escaped key
# still shows its literal Jinja delimiters to the key check.)
in_tag = True
out.append(_LIVE_SENTINEL)
elif tok_type in ("variable_end", "block_end", "comment_end"):
in_tag = False
elif not in_tag:
# data, whitespace, raw_begin/raw_end markers, and inert raw content.
out.append(value if isinstance(value, str) else str(value))
reconstructed = "".join(out)
for match in _JSON_KEY_RE.finditer(reconstructed):
key_text = match.group(1)
if _LIVE_SENTINEL in key_text or _contains_jinja_delim(key_text):
raise TemplateSafetyError(
"refusing to emit JSON template: Jinja markup appears inside a "
"JSON object key. JinjaTurtle never templates keys, so this "
"indicates attacker-influenced source key text (possible "
"template injection)."
)
#
# JinjaTurtle's ERB output is produced by translating the (already-verified)
# Jinja2 subset, so the Jinja2 gate is the primary guarantee. As an independent
# ERB-side backstop we confirm that every ERB tag body is one the translator
# emits, and that no Jinja2 delimiters survived into the ERB output (which would
# indicate a raw block the translator failed to recognise -- the historical
# ``{%+ raw %}`` blind spot).
# --------------------------------------------------------------------------- #
_ERB_TAG_RE = re.compile(r"<%[-=#]?(.*?)[-]?%>", re.S)
_JINJA_DELIMS = ("{{", "}}", "{%", "%}", "{#", "#}")
# Bodies the ErbTranslator emits. Kept permissive for Ruby method chains it
# constructs (``@var``, ``.each_with_index``, ``JSON.generate(...)`` etc.) but
# anchored so arbitrary attacker text cannot masquerade as one.
_ERB_STMT_PATTERNS = tuple(
re.compile(p)
for p in (
r"^require 'json'$",
r"^end$",
r"^else$",
r"^@?[A-Za-z_][\w@\.\[\]'\"]*\.each_with_index do \|[A-Za-z_]\w*, __jt_idx_\d+\| $",
r"^if .+$",
r"^elsif .+$",
r"^unless .+\.nil\?$",
r"^# Unsupported JinjaTurtle statement: .*$",
)
)
def verify_erb_template_safe(template_text: str) -> None:
"""Validate that *template_text* contains no leftover Jinja2 delimiters.
The translator is the security-relevant step for ERB; this backstop ensures
no live Jinja construct survived translation (which would mean a raw block
was not recognised and source text passed through untouched).
"""
for delim in _JINJA_DELIMS:
if delim in template_text:
raise TemplateSafetyError(
"refusing to emit ERB template: it still contains the Jinja2 "
f"delimiter {delim!r}, which means source text was not fully "
"translated/neutralised (possible template injection)."
)

View file

@ -132,57 +132,3 @@ def test_cli_folder_single_output_file_when_one_format(tmp_path):
assert defaults_path.is_file()
assert template_path.is_file()
assert "to_json" in template_path.read_text(encoding="utf-8")
def test_cli_erb_outputs_puppet_hiera_and_erb_template(tmp_path):
cfg = tmp_path / "app.ini"
cfg.write_text("[main]\nport = 8080\n", encoding="utf-8")
data_out = tmp_path / "common.yaml"
template_out = tmp_path / "app.ini.erb"
exit_code = cli._main(
[
str(cfg),
"-r",
"php",
"--template-engine",
"erb",
"--defaults-output",
str(data_out),
"--template-output",
str(template_out),
]
)
assert exit_code == 0
assert "php::main_port: '8080'" in data_out.read_text(encoding="utf-8")
assert "port = <%= @main_port %>" in template_out.read_text(encoding="utf-8")
def test_cli_erb_can_use_separate_puppet_class_namespace(tmp_path):
cfg = tmp_path / "app.ini"
cfg.write_text("[main]\nport = 8080\n", encoding="utf-8")
data_out = tmp_path / "node.yaml"
template_out = tmp_path / "app.ini.erb"
exit_code = cli._main(
[
str(cfg),
"-r",
"php_etc_app_ini",
"--template-engine",
"erb",
"--puppet-class",
"php",
"--defaults-output",
str(data_out),
"--template-output",
str(template_out),
]
)
assert exit_code == 0
data = data_out.read_text(encoding="utf-8")
template = template_out.read_text(encoding="utf-8")
assert "php::php_etc_app_ini_main_port: '8080'" in data
assert "port = <%= @php_etc_app_ini_main_port %>" in template

View file

@ -0,0 +1,205 @@
"""Regression tests for malformed-input handling (``ConfigParseError``).
Bug: every parse-layer handler (xml/json/toml/yaml/ini) used to let its
underlying parser's exception escape as an unhandled traceback when given a
malformed file. Pointing JinjaTurtle (or Enroll, which calls it as a library)
at harvested, attacker-influenceable config makes malformed input an entirely
expected condition, so it must fail cleanly instead of crashing.
These tests pin down that:
* ``parse_config`` raises the normalised ``ConfigParseError`` for malformed
XML/JSON/TOML/INI;
* defusedxml's *security* exceptions (XXE / ``EntitiesForbidden``) are NOT
swallowed by that normalisation -- they must still propagate so a caller can
distinguish "malformed" from "malicious";
* an unrelated error (e.g. the TOML "tomllib missing" ``RuntimeError``) is not
captured by the normalisation either;
* the CLI exits non-zero with a clean message (no traceback) on malformed
input; and
* folder mode skips an unparseable file instead of aborting the whole run.
"""
from __future__ import annotations
import subprocess
import sys
from pathlib import Path
import pytest
from jinjaturtle.core import ConfigParseError, parse_config
def _run_cli(args: list[str]) -> subprocess.CompletedProcess[str]:
return subprocess.run(
[sys.executable, "-m", "jinjaturtle.cli", *args],
capture_output=True,
text=True,
)
@pytest.mark.parametrize(
"fmt,filename,content",
[
# Element name is not well-formed XML -> expat ParseError historically
# escaped as an uncaught traceback.
("xml", "bad.xml", "<root><{{tag}}/></root>"),
("xml", "bad2.xml", "<root><unclosed></root>"),
("json", "bad.json", "{not valid json"),
("toml", "bad.toml", "x = = ="),
# A bare key with no value is invalid INI for configparser.
("ini", "bad.ini", "[s]\nthis line has no equals and no colon\n"),
],
)
def test_parse_config_raises_configparseerror_on_malformed(
tmp_path: Path, fmt: str, filename: str, content: str
) -> None:
src = tmp_path / filename
src.write_text(content, encoding="utf-8")
with pytest.raises(ConfigParseError):
parse_config(src, fmt=fmt)
def test_parse_config_does_not_swallow_xxe_security_exception(tmp_path: Path) -> None:
"""defusedxml's EntitiesForbidden must propagate, not become ConfigParseError.
EntitiesForbidden subclasses ValueError, so a naive ``except ValueError`` /
``except Exception`` in parse_config would mask an XXE attempt as a benign
"malformed file". The normalisation is deliberately scoped to exclude it.
"""
from defusedxml.common import EntitiesForbidden
xxe = (
'<?xml version="1.0"?>\n'
'<!DOCTYPE foo [ <!ENTITY xxe SYSTEM "file:///etc/passwd"> ]>\n'
"<foo>&xxe;</foo>\n"
)
src = tmp_path / "xxe.xml"
src.write_text(xxe, encoding="utf-8")
with pytest.raises(EntitiesForbidden):
parse_config(src, fmt="xml")
def test_parse_config_does_not_swallow_unrelated_runtimeerror(
tmp_path: Path, monkeypatch
) -> None:
"""A non-malformation error (tomllib missing) must propagate unchanged."""
import jinjaturtle.handlers.toml as toml_module
monkeypatch.setattr(toml_module, "tomllib", None)
src = tmp_path / "x.toml"
src.write_text('a = "b"\n', encoding="utf-8")
with pytest.raises(RuntimeError) as exc:
parse_config(src, fmt="toml")
assert "tomllib/tomli is required" in str(exc.value)
def test_cli_fails_cleanly_on_malformed_xml(tmp_path: Path) -> None:
"""The CLI must exit non-zero with a clean message, not a traceback."""
src = tmp_path / "bad.xml"
src.write_text("<root><{{tag}}/></root>", encoding="utf-8")
res = _run_cli([str(src), "-f", "xml", "-r", "demo"])
assert res.returncode != 0
# Clean, user-facing message -- not a Python traceback.
assert "could not parse" in res.stderr
assert "Traceback (most recent call last)" not in res.stderr
@pytest.mark.parametrize(
"content",
[
# Classic XXE: external SYSTEM entity used to read a local file.
(
'<?xml version="1.0"?>\n'
'<!DOCTYPE root [<!ENTITY xxe SYSTEM "file:///etc/hostname">]>\n'
"<root>&xxe;</root>\n"
),
# Billion-laughs style internal entity expansion.
(
'<?xml version="1.0"?>\n'
'<!DOCTYPE lolz [<!ENTITY lol "lol"><!ENTITY lol2 "&lol;&lol;&lol;">]>\n'
"<lolz>&lol2;</lolz>\n"
),
# External parameter entity (SSRF / out-of-band XXE vector).
(
'<?xml version="1.0"?>\n'
'<!DOCTYPE root [<!ENTITY % ext SYSTEM "http://127.0.0.1:9/x.dtd"> %ext;]>\n'
"<root>x</root>\n"
),
],
)
def test_cli_refuses_xxe_cleanly(tmp_path: Path, content: str) -> None:
"""defusedxml's security refusal must surface as a clean non-zero exit.
``EntitiesForbidden``/``DTDForbidden``/``ExternalReferenceForbidden`` subclass
``DefusedXmlException`` (and ``ValueError``). ``parse_config`` deliberately
lets them propagate rather than folding them into ``ConfigParseError`` so a
blocked attack stays distinguishable from a benign malformed file. The CLI
must therefore catch ``DefusedXmlException`` itself and exit cleanly instead
of leaking a Python traceback.
"""
src = tmp_path / "xxe.xml"
src.write_text(content, encoding="utf-8")
res = _run_cli([str(src), "-f", "xml", "-r", "demo"])
assert res.returncode == 2
assert "refusing unsafe XML input" in res.stderr
assert "Traceback (most recent call last)" not in res.stderr
def test_folder_mode_skips_unparseable_file(tmp_path: Path) -> None:
"""One malformed file should not abort processing of the whole directory."""
from jinjaturtle.multi import process_directory
good = tmp_path / "good.json"
good.write_text('{ "host": "localhost" }', encoding="utf-8")
bad = tmp_path / "bad.json"
bad.write_text("{not valid json", encoding="utf-8")
defaults_yaml, outputs = process_directory(tmp_path, False, "demo")
# The good file still produced output despite the bad sibling: its content
# and source id appear, and a template was generated.
assert "localhost" in defaults_yaml
assert "good.json" in defaults_yaml
assert outputs
@pytest.mark.parametrize(
"content",
[
"a: &a [*a]\n", # self-referential sequence alias
"root: &r\n child: *r\n", # self-referential mapping alias
],
)
def test_recursive_yaml_is_rejected_cleanly(tmp_path: Path, content: str) -> None:
"""A recursive YAML anchor must fail as a clean ConfigParseError.
PyYAML's safe_load builds a self-referential object from a recursive anchor;
JinjaTurtle's downstream walks (flatten, timestamp-stringify, template gen)
would otherwise blow the stack with a RecursionError traceback. Harvested
config is attacker-influenceable, so this must fail closed.
"""
src = tmp_path / "cyclic.yaml"
src.write_text(content, encoding="utf-8")
with pytest.raises(ConfigParseError):
parse_config(src, "yaml")
def test_recursive_yaml_cli_exits_nonzero_without_traceback(tmp_path: Path) -> None:
src = tmp_path / "cyclic.yaml"
src.write_text("a: &a [*a]\n", encoding="utf-8")
res = _run_cli([str(src), "-f", "yaml", "-r", "role"])
assert res.returncode != 0
assert "Traceback" not in res.stderr
assert "RecursionError" not in res.stderr
def test_shared_noncyclic_yaml_aliases_still_work(tmp_path: Path) -> None:
"""A shared (acyclic) anchor referenced multiple times is a DAG, not a
cycle, and must still parse and template normally."""
src = tmp_path / "shared.yaml"
src.write_text("base: &b\n x: 1\na: *b\nc: *b\n", encoding="utf-8")
fmt, parsed = parse_config(src, "yaml")
assert fmt == "yaml"
assert parsed["a"] == {"x": 1}
assert parsed["c"] == {"x": 1}

View file

@ -191,3 +191,43 @@ def test_flatten_config_unsupported_format():
flatten_config("bogusfmt", parsed=None)
assert "Unsupported format" in str(exc.value)
def test_make_var_name_collapses_adjacent_separators():
"""Regression: adjacent separators must not produce a forbidden ``__``.
A source key such as ``log..level`` or ``cache--size`` previously sanitised
to a name containing a double underscore (``..._log__level``). The output
safety gate in safety.py rejects *any* ``__`` in a generated identifier
(it is the gateway to Jinja2 SSTI gadgets), so emitting one made JinjaTurtle
reject its own placeholder and abort generation on entirely benign config.
make_var_name now collapses runs of underscores to a single ``_``.
"""
for raw_key in ("log..level", "cache--size", "a...b", "x.-.y", "a..b--c"):
name = make_var_name("role", ("main", raw_key))
assert "__" not in name, f"{raw_key!r} produced {name!r}"
# Still a valid Ansible/Jinja identifier.
assert name.replace("_", "").isalnum() or name == "role"
# The double-underscore-free collapse is stable and predictable.
assert make_var_name("role", ("main", "log..level")) == "role_main_log_level"
assert make_var_name("role", ("main", "cache--size")) == "role_main_cache_size"
# A leading-digit-free prefix with its own repeated separators is collapsed too.
assert make_var_name("My__Role", ("k",)) == "my_role_k"
def test_make_var_name_collapsed_names_pass_output_safety_gate():
"""The names make_var_name emits must be accepted by the safety gate.
This binds the two modules together: whatever identifier make_var_name
produces for a hostile-looking key must lex as the JinjaTurtle subset, so a
real template built from it is not refused.
"""
from jinjaturtle.safety import verify_jinja2_template_safe
for raw_key in ("log..level", "cache--size", "a...b", "weird..key..name"):
var = make_var_name("demo", ("section", raw_key))
# Build the kind of expression a handler would emit for this variable.
template = "{{ " + var + " }}"
# Must not raise TemplateSafetyError.
verify_jinja2_template_safe(template)

View file

@ -0,0 +1,327 @@
"""Security regression tests for template injection (SSTI).
JinjaTurtle copies parts of the source config (comments, unrecognised lines,
structural keys) verbatim into the generated template. If that text contains
Jinja2/ERB delimiters it must be neutralised, otherwise attacker-influenced
config content becomes live template code that executes when Ansible later
renders the template.
These tests render the *generated* template the way a downstream tool would and
assert that an injected payload never executes. A tripwire object is exposed
under every name a payload might reference; if the rendered output ever contains
the tripwire sentinel, an injected expression executed and the test fails.
"""
from __future__ import annotations
import subprocess
import sys
from pathlib import Path
import jinja2
import pytest
import yaml as pyyaml
from jinjaturtle.escape import (
escape_jinja_literal,
)
TRIP = "__TRIPWIRE_FIRED__"
class _UnsafeAwareLoader(pyyaml.SafeLoader):
pass
def _construct_unsafe(loader: _UnsafeAwareLoader, node: pyyaml.Node):
return loader.construct_scalar(node)
_UnsafeAwareLoader.add_constructor("!unsafe", _construct_unsafe)
def _safe_load_defaults(text: str):
return pyyaml.load(text, Loader=_UnsafeAwareLoader)
class _Boom:
"""Returns the tripwire sentinel for any access/call an SSTI payload makes."""
def run(self, *a, **k):
return TRIP
def __call__(self, *a, **k):
return TRIP
def __getitem__(self, k):
return self
def __getattr__(self, n):
return _Boom()
def __str__(self):
return TRIP
def _render_jinja(template_text: str, defaults: dict) -> str:
env = jinja2.Environment(undefined=jinja2.ChainableUndefined)
env.filters.setdefault("to_json", lambda v, **k: __import__("json").dumps(v))
env.filters.setdefault("lower", lambda v: str(v).lower())
ctx = dict(defaults or {})
for name in ("salt", "cmd", "os", "subprocess", "cycler", "lipsum", "namespace"):
ctx.setdefault(name, _Boom())
return env.from_string(template_text).render(**ctx)
def _run_jinjaturtle(tmp_path: Path, source_name: str, body: str, fmt: str):
src = tmp_path / source_name
src.write_text(body, encoding="utf-8")
tpl = tmp_path / "out.tpl"
dfl = tmp_path / "defaults.yml"
res = subprocess.run(
[
sys.executable,
"-m",
"jinjaturtle.cli",
str(src),
"-f",
fmt,
"--role-name",
"role",
"-t",
str(tpl),
"-d",
str(dfl),
],
capture_output=True,
text=True,
)
assert res.returncode == 0, f"generation failed: {res.stderr}"
defaults = _safe_load_defaults(dfl.read_text()) or {}
return tpl.read_text(), defaults
# A representative payload for each format, placed where the format allows
# attacker-controlled verbatim text (comments / unrecognised lines).
FORMAT_CASES = [
(
"ini",
"evil.ini",
"[s]\n"
"good = ok ; {{ salt['cmd.run']('id') }}\n"
"# {{ cmd.run('whoami') }}\n",
),
(
"yaml",
"evil.yaml",
"server:\n" " motd: ok\n" " # {{ salt['cmd.run']('id') }}\n",
),
(
"toml",
"evil.toml",
"[s]\n" 'good = "ok"\n' "# {{ cmd.run('id') }}\n",
),
(
"xml",
"evil.xml",
"<config>\n"
" <!-- {{ salt['cmd.run']('id') }} -->\n"
' <server name="ok"><motd>ok</motd></server>\n'
"</config>\n",
),
(
"postfix",
"main.cf",
"myhostname = mail.example.com\n" "# {{ salt['cmd.run']('id') }}\n",
),
(
"systemd",
"evil.service",
"[Unit]\n"
"Description=ok\n"
"# {{ cmd.run('id') }}\n"
"RawLineNoEquals {% for x in ().__class__.__bases__ %}\n"
"[Service]\n"
"ExecStart=/bin/true\n",
),
(
"ssh",
"sshd_config",
"# {{ salt['cmd.run']('id') }}\n" "Port 22\n" "PermitRootLogin no\n",
),
]
@pytest.mark.parametrize("fmt,name,body", FORMAT_CASES)
def test_comment_payload_does_not_execute(tmp_path, fmt, name, body):
template_text, defaults = _run_jinjaturtle(tmp_path, name, body, fmt)
rendered = _render_jinja(template_text, defaults)
assert TRIP not in rendered, f"injected payload executed for {fmt}:\n{rendered}"
@pytest.mark.parametrize("fmt,name,body", FORMAT_CASES)
def test_generated_template_is_renderable(tmp_path, fmt, name, body):
# A correct escape must still produce a syntactically valid template.
template_text, defaults = _run_jinjaturtle(tmp_path, name, body, fmt)
# Should not raise a TemplateSyntaxError.
_render_jinja(template_text, defaults)
# --- JSON object-key injection ------------------------------------------------
#
# The JSON handler copies the text *between* scalar values (object keys,
# punctuation) verbatim. A key is never a value placeholder, so Jinja markup in a
# key can only come from attacker-influenced source text. The output gate
# (verify_no_live_jinja_in_json_keys) must fail closed on it -- including the
# "benign-looking name" form (e.g. ``{{ ansible_hostname }}``) that the generic
# allowlist would otherwise accept as an ordinary variable reference, and which
# could leak an in-scope variable's value into the rendered config at apply time.
JSON_KEY_INJECTION_BODIES = [
# benign-looking variable reference (the residual bypass: passes the generic
# allowlist but must still be rejected in *key* position)
'{ "{{ ansible_hostname }}": "v" }',
# dotted reference (e.g. dumping another host's vars)
'{ "{{ hostvars.localhost }}": 1 }',
# self-referencing a sibling-derived variable name
'{ "{{ role_port }}": "x", "port": 8080 }',
# classic gadget (already rejected historically; kept as a guard)
'{ "{{ cycler.__init__.__globals__ }}": 1 }',
# statement injection in a key
'{ "{% for x in y %}k{% endfor %}": 1 }',
# nested object key
'{ "ok": { "{{ ansible_hostname }}": 2 } }',
]
@pytest.mark.parametrize("body", JSON_KEY_INJECTION_BODIES)
def test_json_key_injection_fails_closed_cli(tmp_path, body):
src = tmp_path / "evil.json"
src.write_text(body, encoding="utf-8")
out = tmp_path / "out.j2"
res = subprocess.run(
[
sys.executable,
"-m",
"jinjaturtle.cli",
str(src),
"-f",
"json",
"--role-name",
"role",
"-t",
str(out),
],
capture_output=True,
text=True,
)
assert res.returncode == 2, f"expected fail-closed, got rc={res.returncode}"
assert "refusing to generate unsafe template" in res.stderr
assert not out.exists(), "no template may be written when the gate refuses"
def test_json_benign_keys_still_generate(tmp_path):
src = tmp_path / "ok.json"
src.write_text('{ "host": "localhost", "port": 8080 }', encoding="utf-8")
out = tmp_path / "out.j2"
res = subprocess.run(
[
sys.executable,
"-m",
"jinjaturtle.cli",
str(src),
"-f",
"json",
"--role-name",
"demo",
"-t",
str(out),
],
capture_output=True,
text=True,
)
assert res.returncode == 0, res.stderr
template_text = out.read_text()
# Keys stay literal; values become placeholders.
assert '"host":' in template_text
assert "demo_host" in template_text
# --- Unit-level guarantees for the escaper itself ---------------------------
SSTI_PAYLOADS = [
"{{ 7*7 }}",
"{{ salt['cmd.run']('id') }}",
"{% set x = cycler.__init__.__globals__ %}{{ x }}",
"{# comment payload #}",
"text {% endraw %} breakout {{ evil }}",
"nested {% endraw %} spacing {{ evil }}",
"{%- endraw -%}{{ evil }}",
# Whitespace-control markers: Jinja2 accepts "-", "+" or none adjacent to a
# tag's delimiters, and every variant closes a {% raw %} block. The "+"
# forms in particular were a raw-wrapper breakout vector (the defang regex
# historically only matched "-"), so all combinations must be neutralised.
"{%+ endraw %}{{ evil }}",
"{% endraw +%}{{ evil }}",
"{%+ endraw +%}{{ evil }}",
"{%- endraw +%}{{ evil }}",
"{%+ endraw -%}{{ evil }}",
# Full breakout attempt: close raw early, inject live code, re-open raw to
# swallow our trailing {% endraw %} so the template would otherwise compile.
"{%+ endraw %}{{ evil }}{%+ raw %}",
"mixed {{ a }} and {% b %} and {# c #}",
"}}{{ orphan delimiters %}{%",
]
@pytest.mark.parametrize("payload", SSTI_PAYLOADS)
def test_escape_jinja_literal_renders_back_to_original(payload):
"""Escaped text must render to the exact original characters, inertly."""
env = jinja2.Environment(undefined=jinja2.ChainableUndefined)
escaped = escape_jinja_literal(payload)
rendered = env.from_string(escaped).render(evil="EVIL", x="X", a="A")
assert rendered == payload
assert TRIP not in rendered
def test_escape_jinja_literal_noop_on_plain_text():
for plain in ["", "hello world", "# a normal comment", "port = 8080", "key: value"]:
assert escape_jinja_literal(plain) == plain
def test_escape_jinja_literal_actually_blocks_execution():
env = jinja2.Environment(undefined=jinja2.ChainableUndefined)
payload = "{{ boom.run('x') }}"
escaped = escape_jinja_literal(payload)
rendered = env.from_string(escaped).render(boom=_Boom())
assert TRIP not in rendered
# Sanity: the *unescaped* payload would have fired the tripwire.
fired = env.from_string(payload).render(boom=_Boom())
assert TRIP in fired
@pytest.mark.parametrize(
"endraw",
[
"{% endraw %}",
"{% endraw -%}",
"{% endraw +%}",
"{%- endraw %}",
"{%- endraw -%}",
"{%- endraw +%}",
"{%+ endraw %}",
"{%+ endraw -%}",
"{%+ endraw +%}",
],
)
def test_endraw_whitespace_control_cannot_break_out(endraw):
"""Every whitespace-control form of endraw closes a {% raw %} block in
Jinja2, so each must be defanged. A payload that closes raw early, injects
a live tripwire call, then re-opens raw to balance the wrapper must still
render inertly back to its original characters."""
env = jinja2.Environment(undefined=jinja2.ChainableUndefined)
payload = f"{endraw}{{{{ boom.run('x') }}}}{{%+ raw %}}"
escaped = escape_jinja_literal(payload)
rendered = env.from_string(escaped).render(boom=_Boom())
assert TRIP not in rendered
assert rendered == payload

View file

@ -0,0 +1,233 @@
"""Regression tests for the output safety gate (``jinjaturtle.safety``).
The gate is JinjaTurtle's second, independent line of defence against template
injection. Where the per-handler escaper neutralises verbatim source text, the
gate inspects the *finished* template and refuses to emit it if any live
construct is not one JinjaTurtle itself produces. These tests cover:
* the gate's allow/deny grammar (unit level);
* the two concrete injection findings that motivated it -- JSON object keys
and folder-mode union keys copied into templates unescaped;
* end-to-end CLI fail-closed behaviour (non-zero exit, no file written).
"""
from __future__ import annotations
import json
import subprocess
import sys
from pathlib import Path
import pytest
from jinjaturtle import core
from jinjaturtle.multi import process_directory
from jinjaturtle.safety import (
TemplateSafetyError,
verify_jinja2_template_safe,
verify_erb_template_safe,
)
# --------------------------------------------------------------------------- #
# Unit: the allow/deny grammar.
# --------------------------------------------------------------------------- #
LEGIT_TEMPLATES = [
"{{ demo_memory_limit }}",
"{{ demo_x | to_json(ensure_ascii=False) }}",
"{{ demo_a | to_json(indent=2, ensure_ascii=False) }}",
"{{ demo_name | lower }}",
"{{ 'true' if demo_flag else 'false' }}",
'{{ "true" if demo_flag else "false" }}',
"{{ 'null' if demo_v is none else demo_v }}",
"{% for server in demo_servers %}{{ server.name }}{% endfor %}",
"{{ server.config.port }}",
"{% if demo_x is defined %}{{ demo_x }}{% endif %}",
"{% if demo_x is none %}x{% endif %}",
"{% if not loop.last %},{% endif %}",
"plain text with no tags",
"{% raw %}# literal {{ not_code }} {% if x %}{% endraw %}",
"{% raw %}{{ 7*7 }}{% endraw %}: value", # escaped key (folder-mode fix)
]
@pytest.mark.parametrize("template", LEGIT_TEMPLATES)
def test_gate_allows_jinjaturtle_constructs(template):
# Must not raise.
verify_jinja2_template_safe(template)
MALICIOUS_TEMPLATES = [
"{{ 7*7 }}",
"{{ cycler.__init__.__globals__ }}",
"{{ cycler.__init__.__globals__.os.popen('id').read() }}",
"{{ salt['cmd.run']('id') }}",
"{{ self.__init__ }}",
"{% for x in ().__class__.__base__.__subclasses__() %}{{ x }}{% endfor %}",
"{{ config.items() }}",
'{{ request["application"] }}',
"{% set x = 1 %}",
"{{ lipsum.__globals__ }}",
"{{ a.__class__.__mro__ }}",
"{{ ''.join(['a','b']) }}",
"{% include 'x' %}",
"{% import 'x' as y %}",
]
@pytest.mark.parametrize("template", MALICIOUS_TEMPLATES)
def test_gate_blocks_injection(template):
with pytest.raises(TemplateSafetyError):
verify_jinja2_template_safe(template)
def test_gate_blocks_double_underscore_anywhere():
# The no-dunder rule is the backbone of blocking attribute-traversal SSTI.
with pytest.raises(TemplateSafetyError):
verify_jinja2_template_safe("{{ demo__x }}")
with pytest.raises(TemplateSafetyError):
verify_jinja2_template_safe("{{ a.b__c }}")
def test_gate_rejects_unlexable_breakout_as_safety_error():
# A raw-wrapper breakout that leaves dangling tags must surface as a
# TemplateSafetyError, not a raw Jinja2 syntax error.
broken = "{% raw %}{%+ endraw %}{{ 7*7 }}" # unbalanced on purpose
with pytest.raises(TemplateSafetyError):
verify_jinja2_template_safe(broken)
# --------------------------------------------------------------------------- #
# Finding 1: JSON object keys copied verbatim into the template.
# --------------------------------------------------------------------------- #
JSON_KEY_PAYLOADS = [
'{ "{{ 7*7 }}": "v" }',
'{ "{{cycler.__init__.__globals__}}": 1 }',
"{ \"ok\": { \"{{ salt['cmd.run']('id') }}\": 2 } }",
]
@pytest.mark.parametrize("src", JSON_KEY_PAYLOADS)
def test_json_key_injection_is_blocked(src):
parsed = json.loads(src)
with pytest.raises(TemplateSafetyError):
core.generate_jinja2_template("json", parsed, "demo", original_text=src)
def test_json_benign_template_still_generates():
src = '{ "name": "app", "port": 8080 }'
parsed = json.loads(src)
out = core.generate_jinja2_template("json", parsed, "demo", original_text=src)
assert "demo_name" in out
assert "demo_port" in out
# --------------------------------------------------------------------------- #
# Finding 2: folder-mode union renderers copied keys verbatim.
# --------------------------------------------------------------------------- #
def _write(tmp: Path, name: str, text: str) -> None:
(tmp / name).write_text(text, encoding="utf-8")
def test_folder_yaml_key_injection_is_neutralised(tmp_path):
# The malicious key must be escaped (root-cause fix), so the union template
# generates successfully AND passes the gate, rendering the key inertly.
_write(tmp_path, "a.yaml", '"{{ 7*7 }}": value\nnormalkey: ok\n')
_write(tmp_path, "b.yaml", "normalkey: ok2\n")
_defaults, outputs = process_directory(tmp_path, False, "demo")
template = "\n".join(o.template for o in outputs)
# Escaped, not live:
assert "{% raw %}{{ 7*7 }}{% endraw %}" in template
# And the gate (run inside process_directory) did not reject it.
def test_folder_ini_section_and_key_injection_neutralised(tmp_path):
_write(
tmp_path,
"a.ini",
"[{{ 7*7 }}]\n{{ evil }} = x\n",
)
_write(tmp_path, "b.ini", "[normal]\nk = y\n")
_defaults, outputs = process_directory(tmp_path, False, "demo")
template = "\n".join(o.template for o in outputs)
# Section header and key are escaped, not live.
assert "{{ 7*7 }}" not in _strip_raw_blocks(template)
assert "{{ evil }}" not in _strip_raw_blocks(template)
def test_folder_toml_key_injection_neutralised(tmp_path):
_write(tmp_path, "a.toml", '"{{ 7*7 }}" = "v"\nok = "y"\n')
_write(tmp_path, "b.toml", 'ok = "z"\n')
_defaults, outputs = process_directory(tmp_path, False, "demo")
template = "\n".join(o.template for o in outputs)
assert "{{ 7*7 }}" not in _strip_raw_blocks(template)
def _strip_raw_blocks(text: str) -> str:
"""Remove the *contents* of raw blocks so we can assert no LIVE payload
remains outside them."""
import re
return re.sub(
r"{%[-+]?\s*raw\s*[-+]?%}.*?{%[-+]?\s*endraw\s*[-+]?%}",
"",
text,
flags=re.S,
)
# --------------------------------------------------------------------------- #
# ERB gate.
# --------------------------------------------------------------------------- #
def test_erb_gate_blocks_leftover_jinja_delimiters():
with pytest.raises(TemplateSafetyError):
verify_erb_template_safe("ok <%= @x %> but {{ leftover }} here")
def test_erb_gate_allows_clean_erb():
verify_erb_template_safe("memory = <%= @memory_limit %>\n")
# --------------------------------------------------------------------------- #
# End-to-end CLI: fail closed.
# --------------------------------------------------------------------------- #
def _run_cli(args, env_extra=None):
import os
env = {**os.environ, "PYTHONPATH": "src"}
if env_extra:
env.update(env_extra)
return subprocess.run(
[sys.executable, "-m", "jinjaturtle.cli", *args],
capture_output=True,
text=True,
env=env,
)
def test_cli_fails_closed_on_injection(tmp_path):
bad = tmp_path / "evil.json"
bad.write_text('{ "{{ 7*7 }}": "v" }', encoding="utf-8")
out = tmp_path / "out.j2"
result = _run_cli([str(bad), "-r", "demo", "-f", "json", "-t", str(out)])
assert result.returncode == 2
assert "refusing to generate unsafe template" in result.stderr
# No template file may be written when the gate refuses.
assert not out.exists()
def test_cli_succeeds_on_benign_input(tmp_path):
good = tmp_path / "ok.json"
good.write_text('{ "name": "app" }', encoding="utf-8")
out = tmp_path / "out.j2"
result = _run_cli([str(good), "-r", "demo", "-f", "json", "-t", str(out)])
assert result.returncode == 0
assert out.exists()

View file

@ -0,0 +1,136 @@
from __future__ import annotations
from pathlib import Path
import os
import pytest
import yaml
from defusedxml.common import EntitiesForbidden
from jinjaturtle import cli
from jinjaturtle.core import generate_ansible_yaml, parse_config, flatten_config
from jinjaturtle.multi import is_supported_file, iter_supported_files, process_directory
from jinjaturtle.output_safety import OutputPathError, write_text_safely
class UnsafeAwareLoader(yaml.SafeLoader):
pass
def _unsafe(loader: UnsafeAwareLoader, node: yaml.Node):
return loader.construct_scalar(node)
UnsafeAwareLoader.add_constructor("!unsafe", _unsafe)
def test_jinja_values_are_emitted_as_ansible_unsafe(tmp_path: Path):
src = tmp_path / "app.ini"
src.write_text("[main]\ncmd = {{ lookup('pipe','id') }}\n", encoding="utf-8")
fmt, parsed = parse_config(src)
defaults_yaml = generate_ansible_yaml("role", flatten_config(fmt, parsed))
assert "role_main_cmd: !unsafe" in defaults_yaml
assert "{{ lookup(''pipe'',''id'') }}" in defaults_yaml
loaded = yaml.load(defaults_yaml, Loader=UnsafeAwareLoader)
assert loaded["role_main_cmd"] == "{{ lookup('pipe','id') }}"
def test_folder_mode_marks_nested_jinja_values_and_ids_unsafe(tmp_path: Path):
src = tmp_path / "src"
src.mkdir()
# Filename ids are source-derived values too.
(src / "{{ bad }}.yaml").write_text(
"message: \"{{ lookup('pipe','id') }}\"\n", encoding="utf-8"
)
defaults_yaml, _outputs = process_directory(
src, recursive=False, role_prefix="role"
)
assert "id: !unsafe" in defaults_yaml
assert "role_message: !unsafe" in defaults_yaml
loaded = yaml.load(defaults_yaml, Loader=UnsafeAwareLoader)
assert loaded["role_items"][0]["id"] == "{{ bad }}.yaml"
assert loaded["role_items"][0]["role_message"] == "{{ lookup('pipe','id') }}"
def test_xml_parser_rejects_entities_when_called_as_library(tmp_path: Path):
src = tmp_path / "bad.xml"
src.write_text(
"<!DOCTYPE root [<!ENTITY xxe SYSTEM 'file:///etc/passwd'>]><root>&xxe;</root>",
encoding="utf-8",
)
with pytest.raises(EntitiesForbidden):
parse_config(src, "xml")
def test_folder_mode_does_not_follow_symlinked_files(tmp_path: Path):
real = tmp_path / "secret.ini"
real.write_text("[main]\nsecret=yes\n", encoding="utf-8")
root = tmp_path / "root"
root.mkdir()
link = root / "link.ini"
link.symlink_to(real)
assert not is_supported_file(link)
assert iter_supported_files(root, recursive=False) == []
assert iter_supported_files(root, recursive=True) == []
@pytest.mark.skipif(not hasattr(os, "symlink"), reason="symlinks unavailable")
def test_cli_refuses_to_write_through_final_symlink(tmp_path: Path):
target = tmp_path / "target.txt"
target.write_text("keep\n", encoding="utf-8")
link = tmp_path / "out.yml"
link.symlink_to(target)
with pytest.raises(OutputPathError):
write_text_safely(link, "replace\n")
assert target.read_text(encoding="utf-8") == "keep\n"
assert link.is_symlink()
def test_cli_refuses_symlinked_output_parent(tmp_path: Path):
real_dir = tmp_path / "real"
real_dir.mkdir()
link_dir = tmp_path / "linkdir"
link_dir.symlink_to(real_dir, target_is_directory=True)
with pytest.raises(OutputPathError):
write_text_safely(link_dir / "out.yml", "data\n")
assert not (real_dir / "out.yml").exists()
def test_cli_reports_unsafe_output_path_without_overwriting_symlink(tmp_path: Path):
cfg = tmp_path / "app.ini"
cfg.write_text("[main]\nname = ok\n", encoding="utf-8")
target = tmp_path / "target.yml"
target.write_text("keep\n", encoding="utf-8")
link = tmp_path / "defaults.yml"
link.symlink_to(target)
exit_code = cli._main([str(cfg), "--defaults-output", str(link)])
assert exit_code == 2
assert target.read_text(encoding="utf-8") == "keep\n"
def test_cli_refuses_root_output_through_group_writable_parent(
tmp_path: Path, monkeypatch
):
from jinjaturtle import output_safety
unsafe_dir = tmp_path / "unsafe"
unsafe_dir.mkdir()
unsafe_dir.chmod(0o777)
monkeypatch.setattr(output_safety, "_effective_uid", lambda: 0)
with pytest.raises(OutputPathError):
write_text_safely(unsafe_dir / "out.yml", "data\n")
assert not (unsafe_dir / "out.yml").exists()

View file

@ -10,7 +10,6 @@ from jinjaturtle.core import (
analyze_loops,
flatten_config,
generate_ansible_yaml,
generate_erb_template,
generate_jinja2_template,
)
from jinjaturtle.handlers.yaml import YamlHandler
@ -178,17 +177,15 @@ def test_yaml_loop_preserves_blank_separator_after_list(tmp_path: Path):
"blank",
"require:\n - rubocop-performance\n - rubocop-rspec\n\nAllCops:\n NewCops: enable\n",
"\n - rubocop-rspec\n\nAllCops:",
"<% end %>\nAllCops:",
),
(
"no_blank",
"require:\n - rubocop-performance\n - rubocop-rspec\nAllCops:\n NewCops: enable\n",
"\n - rubocop-rspec\nAllCops:",
"<% end %>AllCops:",
),
]
for label, text, rendered_expected, erb_expected in cases:
for label, text, rendered_expected in cases:
path = tmp_path / f"{label}.yml"
path.write_text(text, encoding="utf-8")
@ -205,19 +202,9 @@ def test_yaml_loop_preserves_blank_separator_after_list(tmp_path: Path):
rendered = Template(template).render(**defaults)
assert rendered_expected in rendered
erb_template = generate_erb_template(
fmt,
parsed,
"role",
original_text=text,
loop_candidates=loop_candidates,
flat_items=flat_items,
)
assert erb_expected in erb_template
def test_yaml_loop_preserves_following_top_level_comments(tmp_path: Path):
from jinja2 import Template
from jinja2 import Environment, Template
from jinjaturtle.core import analyze_loops
@ -248,10 +235,15 @@ def test_yaml_loop_preserves_following_top_level_comments(tmp_path: Path):
fmt, parsed, "role", original_text=text, loop_candidates=loop_candidates
)
rendered = Template(template).render(**defaults)
ansible_rendered = (
Environment(trim_blocks=True).from_string(template).render(**defaults)
)
assert "# Offense count: 2" in rendered
assert "# This cop supports unsafe autocorrection" in rendered
assert "spec.rb\n\n# Offense count: 2" in ansible_rendered
assert yaml.safe_load(rendered) == yaml.safe_load(text)
assert yaml.safe_load(ansible_rendered) == yaml.safe_load(text)
def test_yaml_scalar_loop_preserves_quoted_list_items(tmp_path: Path):