mig5/jinjaturtle
1
0
Fork 0
Code Issues Pull requests Projects Releases 16 Packages Wiki Activity Actions

Go back to just jinja2

Browse source
This commit is contained in:
Miguel Jacq 2026-06-25 17:07:58 +10:00
parent 094c4d2274
commit 661320558c
Signed by: mig5
GPG key ID: 03906B4110AAD3B8
9 changed files with 26 additions and 664 deletions
Download patch file Download diff file Expand all files Collapse all files

125
README.md
View file

@ -13,9 +13,6 @@ By default it generates:
- an **Ansible defaults YAML** file containing the variables used by that
template.
It can also generate **ERB** templates and **Puppet Hiera-style YAML** data for
Puppet workflows.
JinjaTurtle does not try to replace configuration-management tools. Its job is
to speed up the boring first pass: take a real config file, discover the values
inside it, replace those values with variables, and write the corresponding
@ -37,16 +34,6 @@ For the default Jinja2/Ansible mode:
5. An Ansible defaults YAML file is generated with those variables and the
original values.
For ERB/Puppet mode:
1. The same parse/flatten/loop analysis is used.
2. An ERB template is generated with Puppet-style instance variables such as
`<%= @memory_limit %>`.
3. The variables file is written as Puppet Hiera-style data, such as
`php::memory_limit: 256M`.
4. If `--puppet-class` is supplied, that class name is used as the Hiera
namespace while `--role-name` remains the local variable prefix.
By default, the generated variable data and template are printed to stdout. Use
`--defaults-output` and `--template-output` to write them to files.
@ -80,78 +67,6 @@ and defaults data like:
php_memory_limit: 256M
```
## ERB / Puppet example
Use `--template-engine erb` when you want Puppet ERB output:
```shell
jinjaturtle php.ini \
--role-name php \
--template-engine erb \
--defaults-output data/common.yaml \
--template-output templates/php.ini.erb
```
Given the same source value:
```ini
memory_limit = 256M
```
JinjaTurtle will produce an ERB template value like:
```erb
memory_limit = <%= @memory_limit %>
```
and Hiera-style data like:
```yaml
php::memory_limit: 256M
```
The `--defaults-output` option name is retained for CLI compatibility, but in
ERB mode the file is intended to be Puppet Hiera data rather than Ansible role
defaults.
JinjaTurtle does **not** generate Puppet classes or `file` resources. A Puppet
module, should still declare the class parameters and call the template, for
example with Puppet's `template()` function.
## Using `--puppet-class`
Most direct usage can simply use the same value for the role name and Puppet
class name:
```shell
jinjaturtle php.ini --role-name php --template-engine erb
```
This creates Hiera keys such as:
```yaml
php::memory_limit: 256M
```
and local ERB variables such as:
```erb
<%= @memory_limit %>
```
For generated systems, it can be useful to make `--role-name` more specific
while keeping the Hiera keys under the real Puppet class. For example:
```shell
jinjaturtle php.ini \
--role-name php_etc_php_ini \
--puppet-class php \
--template-engine erb
```
In that case the variable prefix can stay file-specific, while the Hiera data
is still written under `php::...`.
## What sort of config files can it handle?
JinjaTurtle supports common structured and semi-structured config formats:
@ -176,8 +91,8 @@ homogeneous enough. If it is not confident, it falls back to flattened scalar
variables.
Some very complex files will still need manual cleanup. The goal is to speed up
conversion into Jinja2 or ERB templates, not to guarantee a perfect final module
without review.
conversion into Jinja2 templates, not to guarantee a perfect final module without
review.
## JSON, quoting, and type preservation
@ -196,17 +111,7 @@ when the correct rendered JSON should be:
{"enabled": true}
```
In Jinja2 mode this uses Ansible-style JSON filters. In ERB mode it emits Ruby
JSON generation where required, for example:
```erb
<% require 'json' -%>
{
"enabled": <%= JSON.generate(@enabled) %>
}
```
That is expected for JSON ERB templates.
This uses Ansible-style JSON filters.
## Can I convert multiple files at once?
@ -287,8 +192,6 @@ poetry install
usage: jinjaturtle [-h] [-r ROLE_NAME] [--recursive]
[-f {ini,json,toml,yaml,xml,postfix,systemd,ssh}]
[-d DEFAULTS_OUTPUT] [-t TEMPLATE_OUTPUT]
[--template-engine {jinja2,erb}]
[--puppet-class PUPPET_CLASS]
config
Convert a config file into an Ansible defaults file and Jinja2 template.
@ -302,8 +205,7 @@ options:
-h, --help show this help message and exit
-r, --role-name ROLE_NAME
Role name / variable prefix. In Jinja2 mode this is
usually the Ansible role name. In ERB mode it is used
as the local variable prefix. Defaults to jinjaturtle.
usually the Ansible role name. Defaults to jinjaturtle.
--recursive When CONFIG is a folder, recurse into subfolders.
-f, --format {ini,json,toml,yaml,xml,postfix,systemd,ssh}
Force config format instead of auto-detecting from
@ -314,12 +216,6 @@ options:
-t, --template-output TEMPLATE_OUTPUT
Path to write the generated config template. If omitted,
it is printed to stdout.
--template-engine {jinja2,erb}
Template syntax to generate. Defaults to jinja2. Use
erb for Puppet templates.
--puppet-class PUPPET_CLASS
Puppet class / Hiera namespace to use with
--template-engine erb. Defaults to --role-name.
```
## Additional supported formats
@ -346,16 +242,16 @@ Two guarantees matter:
1. **Values are data, never code.** Every config *value* is replaced with a
`{{ variable }}` placeholder in the template, and the original value is stored
separately in the defaults/Hiera data. When the template is later rendered,
separately in the defaults data. When the template is later rendered,
the placeholder prints the value as a literal string; Jinja2 does not
recursively render the *contents* of a variable, so a payload sitting inside
a value (for example `motd = {{ salt['cmd.run']('id') }}`) is inert.
a value is inert.
2. **Verbatim text is neutralised.** To preserve formatting, JinjaTurtle copies
comments, blank lines, headers and any unrecognised lines from the source
into the template. Any template metacharacters in that copied text
(`{{ }}`, `{% %}`, `{# #}` for Jinja2; `<% %>` for ERB) are escaped so they
render as the literal characters the author wrote, rather than executing.
(`{{ }}`, `{% %}`, `{# #}` ) are escaped so they render as the literal
characters the author wrote, rather than executing.
### Consumer responsibilities
@ -368,11 +264,6 @@ which is the normal case:
default. If you build your own var structures from this data and pass them
through additional templating, mark untrusted values with the `!unsafe` tag so
they are never re-evaluated.
- **Salt**: use the generated file as a `file.managed` template
(`template: jinja`). Do **not** place JinjaTurtle output where Salt would
render it a second time as part of SLS/pillar rendering, which is a separate
Jinja pass and would re-evaluate any embedded expressions.
- **Puppet/ERB**: render with the standard `template()`/`epp()` flow.
In short: render JinjaTurtle output exactly once. Do not feed it back through
another templating pass.

61
src/jinjaturtle/cli.py
View file

@ -12,12 +12,10 @@ from .core import (
flatten_config,
generate_ansible_yaml,
generate_jinja2_template,
generate_puppet_hiera_yaml,
generate_erb_template,
)
from .multi import process_directory
from .safety import TemplateSafetyError, verify_erb_template_safe
from .safety import TemplateSafetyError
def _build_arg_parser() -> argparse.ArgumentParser:
@ -59,20 +57,6 @@ def _build_arg_parser() -> argparse.ArgumentParser:
"--template-output",
help="Path to write the generated config template. If omitted, template is printed to stdout.",
)
ap.add_argument(
"--template-engine",
choices=[j2.NAME, "erb"],
default=j2.NAME,
help="Template syntax to generate (default: jinja2). Use erb for Puppet templates.",
)
ap.add_argument(
"--puppet-class",
help=(
"Puppet class/Hiera namespace to use with --template-engine erb. "
"Defaults to --role-name. This lets tools use a file-specific "
"variable prefix while writing Hiera keys under the real Puppet class."
),
)
return ap
@ -110,21 +94,7 @@ def _run(argv: list[str] | None = None) -> int:
print("# defaults/main.yml")
print(defaults_yaml, end="")
# Optionally translate folder-mode templates to ERB. Folder mode keeps
# the existing data shape; single-file mode below is the preferred
# Puppet path because it can produce class-parameter Hiera keys.
if args.template_engine == "erb":
from .erb import translate_jinja2_to_erb
for o in outputs:
o.template = translate_jinja2_to_erb(
o.template,
role_prefix=args.role_name,
puppet_class=args.puppet_class or args.role_name,
)
verify_erb_template_safe(o.template)
template_ext = "erb" if args.template_engine == "erb" else j2.TEMPLATE_EXTENSION
template_ext = j2.TEMPLATE_EXTENSION
# Write templates
if args.template_output:
@ -161,27 +131,8 @@ def _run(argv: list[str] | None = None) -> int:
# Flatten config (excluding loop paths if loops are detected)
flat_items = flatten_config(fmt, parsed, loop_candidates)
if args.template_engine == "erb":
ansible_yaml = generate_puppet_hiera_yaml(
args.role_name,
flat_items,
loop_candidates,
puppet_class=args.puppet_class or args.role_name,
)
template_str = generate_erb_template(
fmt,
parsed,
args.role_name,
original_text=config_text,
loop_candidates=loop_candidates,
flat_items=flat_items,
puppet_class=args.puppet_class or args.role_name,
)
else:
# Generate defaults YAML (with loop collections if detected)
ansible_yaml = generate_ansible_yaml(
args.role_name, flat_items, loop_candidates
)
ansible_yaml = generate_ansible_yaml(args.role_name, flat_items, loop_candidates)
# Generate template (with loops if detected)
template_str = generate_jinja2_template(
@ -201,11 +152,7 @@ def _run(argv: list[str] | None = None) -> int:
if args.template_output:
Path(args.template_output).write_text(template_str, encoding="utf-8")
else:
print(
"# config.erb"
if args.template_engine == "erb"
else f"# config.{j2.TEMPLATE_EXTENSION}"
)
print(f"# config.{j2.TEMPLATE_EXTENSION}")
print(template_str, end="")
return 0

70
src/jinjaturtle/core.py
View file

@ -8,9 +8,7 @@ import re
import yaml
from .loop_analyzer import LoopAnalyzer, LoopCandidate
from .erb import puppet_class_name, puppet_local_var_name, translate_jinja2_to_erb
from .safety import (
verify_erb_template_safe,
verify_jinja2_template_safe,
)
from .handlers import (
@ -411,74 +409,6 @@ def _template_variable_names(
return names
def generate_puppet_hiera_yaml(
role_prefix: str,
flat_items: list[tuple[tuple[str, ...], Any]],
loop_candidates: list[LoopCandidate] | None = None,
*,
puppet_class: str | None = None,
) -> str:
"""Create Puppet Hiera data suitable for Automatic Parameter Lookup.
``role_prefix`` remains the source variable prefix used by JinjaTurtle while
``puppet_class`` is the Puppet class/Hiera namespace. In the normal case
they are the same, so ``php_memory_limit`` becomes ``php::memory_limit``.
"""
klass = puppet_class_name(puppet_class or role_prefix)
data: dict[str, Any] = {}
for path, value in flat_items:
generated = make_var_name(role_prefix, path)
local = puppet_local_var_name(role_prefix, generated, puppet_class=klass)
data[f"{klass}::{local}"] = value
if loop_candidates:
for candidate in loop_candidates:
generated = make_var_name(role_prefix, candidate.path)
local = puppet_local_var_name(role_prefix, generated, puppet_class=klass)
data[f"{klass}::{local}"] = candidate.items
return dump_yaml(data, sort_keys=True)
def generate_erb_template(
fmt: str,
parsed: Any,
role_prefix: str,
*,
original_text: str | None = None,
loop_candidates: list[LoopCandidate] | None = None,
flat_items: list[tuple[tuple[str, ...], Any]] | None = None,
puppet_class: str | None = None,
) -> str:
"""Generate a Puppet ERB template from JinjaTurtle's renderer-neutral data.
The first implementation intentionally translates the Jinja2 subset emitted
by JinjaTurtle's existing format handlers. This keeps parsing, formatting
preservation, and loop detection identical between Jinja2 and ERB output
while still producing Puppet-native ``@parameter`` references.
"""
jinja_template = generate_jinja2_template(
fmt,
parsed,
role_prefix,
original_text=original_text,
loop_candidates=loop_candidates,
)
names = _template_variable_names(role_prefix, flat_items or [], loop_candidates)
erb_template = translate_jinja2_to_erb(
jinja_template,
role_prefix=role_prefix,
puppet_class=puppet_class or role_prefix,
variable_names=names,
)
# Defence in depth: no live Jinja2 delimiter may survive translation.
verify_erb_template_safe(erb_template)
return erb_template
def _stringify_timestamps(obj: Any) -> Any:
"""
Recursively walk a parsed config and turn any datetime/date/time objects

243
src/jinjaturtle/erb.py
View file

@ -1,243 +0,0 @@
from __future__ import annotations
import re
from .escape import escape_erb_literal
def _safe_name(raw: str, *, fallback: str = "var") -> str:
text = re.sub(r"[^A-Za-z0-9_]+", "_", str(raw or fallback)).strip("_").lower()
text = re.sub(r"_+", "_", text)
if not text:
text = fallback
if not re.match(r"^[a-z_]", text):
text = f"{fallback}_{text}"
return text
def puppet_class_name(raw: str) -> str:
"""Return a conservative Puppet class/Hiera namespace name."""
text = _safe_name(raw, fallback="jinjaturtle")
if not re.match(r"^[a-z]", text):
text = f"jinjaturtle_{text}"
return text
def _role_prefix_name(raw: str) -> str:
return _safe_name(raw, fallback="jinjaturtle")
def puppet_local_var_name(
role_prefix: str,
jinja_var_name: str,
*,
puppet_class: str | None = None,
) -> str:
"""Map a generated JinjaTurtle variable to a Puppet class parameter.
For the common case where ``--role-name php`` also means class ``php``, a
generated Jinja variable such as ``php_memory_limit`` becomes Puppet local
parameter ``memory_limit`` and Hiera key ``php::memory_limit``.
When ``puppet_class`` differs from ``role_prefix`` we keep the full
generated variable name as the local parameter and only use ``puppet_class``
as the Hiera namespace.
"""
var_name = _safe_name(jinja_var_name, fallback="value")
prefix = _role_prefix_name(role_prefix)
klass = puppet_class_name(puppet_class or role_prefix)
if klass == prefix and var_name.startswith(prefix + "_"):
stripped = var_name[len(prefix) + 1 :]
return stripped or var_name
return var_name
class ErbTranslator:
"""Translate the Jinja2 subset emitted by JinjaTurtle into Puppet ERB."""
_TOKEN_RE = re.compile(r"({{.*?}}|{%.*?%})", re.S)
def __init__(
self,
*,
role_prefix: str,
puppet_class: str | None = None,
variable_names: set[str] | None = None,
) -> None:
self.role_prefix = role_prefix
self.puppet_class = puppet_class or role_prefix
self.variable_names = set(variable_names or set())
self.loop_stack: list[tuple[str, str, str]] = []
self.needs_json = False
# Matches a JinjaTurtle ``{% raw %} ... {% endraw %}`` block (non-greedy).
# JinjaTurtle emits raw blocks only to carry verbatim, security-escaped
# source text (comments and unrecognised lines), so the *contents* must be
# treated as literal output, never translated as Jinja tokens.
_RAW_BLOCK_RE = re.compile(
r"{%[-+]?\s*raw\s*[-+]?%}(.*?){%[-+]?\s*endraw\s*[-+]?%}", re.S
)
def translate(self, template_text: str) -> str:
# Split out raw blocks first. Their inner text is literal and must be
# carried through as literal ERB (with ERB delimiters re-escaped), rather
# than tokenised -- otherwise an escaped Jinja payload inside a comment
# would be "re-animated" into live ERB during translation.
segments = self._RAW_BLOCK_RE.split(template_text)
out: list[str] = []
# re.split with one capture group yields: [text, raw_inner, text, ...].
for idx, segment in enumerate(segments):
if idx % 2 == 1:
# Captured raw-block contents: emit as literal ERB text.
out.append(escape_erb_literal(segment))
else:
out.append(self._translate_tokens(segment))
rendered = "".join(out)
if self.needs_json and "require 'json'" not in rendered:
rendered = "<% require 'json' -%>\n" + rendered
return rendered
def _translate_tokens(self, template_text: str) -> str:
parts = self._TOKEN_RE.split(template_text)
out: list[str] = []
for token in parts:
if not token:
continue
if token.startswith("{{") and token.endswith("}}"):
expr = token[2:-2].strip()
out.append(f"<%= {self.expr_to_ruby(expr)} %>")
continue
if token.startswith("{%") and token.endswith("%}"):
stmt = token[2:-2].strip()
out.append(self.statement_to_erb(stmt))
continue
out.append(token)
return "".join(out)
def local_var(self, name: str) -> str:
return puppet_local_var_name(
self.role_prefix, name, puppet_class=self.puppet_class
)
def ruby_value(self, expr: str) -> str:
expr = expr.strip()
if expr in {"true", "True"}:
return "true"
if expr in {"false", "False"}:
return "false"
if expr in {"none", "None", "null"}:
return "nil"
if re.match(r"^[A-Za-z_][A-Za-z0-9_]*$", expr):
if any(expr == loop_var for loop_var, _idx, _coll in self.loop_stack):
return expr
return f"@{self.local_var(expr)}"
m = re.match(r"^([A-Za-z_][A-Za-z0-9_]*)\.([A-Za-z_][A-Za-z0-9_]*)$", expr)
if m:
base, key = m.groups()
if any(base == loop_var for loop_var, _idx, _coll in self.loop_stack):
return f"{base}[{key!r}]"
return f"@{self.local_var(base)}[{key!r}]"
return expr
def expr_to_ruby(self, expr: str) -> str:
expr = expr.strip()
# JinjaTurtle emits these YAML-preserving ternaries for booleans/nulls.
m = re.match(
r"^(['\"])(true|false)\1\s+if\s+([A-Za-z_][A-Za-z0-9_\.]*)\s+else\s+(['\"])(true|false)\4$",
expr,
)
if m:
truthy = m.group(2)
cond = self.ruby_value(m.group(3))
falsy = m.group(5)
return f"{cond} ? {truthy!r} : {falsy!r}"
m = re.match(
r"^(['\"])(null)\1\s+if\s+([A-Za-z_][A-Za-z0-9_\.]*)\s+is\s+none\s+else\s+([A-Za-z_][A-Za-z0-9_\.]*)$",
expr,
)
if m:
value = self.ruby_value(m.group(3))
fallback = self.ruby_value(m.group(4))
return f"{value}.nil? ? 'null' : {fallback}"
if "|" in expr:
base, *filters = [part.strip() for part in expr.split("|")]
ruby = self.ruby_value(base)
for filt in filters:
if filt.startswith("lower"):
ruby = f"{ruby}.to_s.downcase"
elif filt.startswith("to_json") or filt.startswith("tojson"):
self.needs_json = True
if "indent" in filt:
ruby = f"JSON.pretty_generate({ruby})"
else:
ruby = f"JSON.generate({ruby})"
return ruby
return self.ruby_value(expr)
def statement_to_erb(self, stmt: str) -> str:
if stmt.endswith(("-", "+")):
stmt = stmt[:-1].rstrip()
if stmt.startswith("for "):
m = re.match(
r"^for\s+([A-Za-z_][A-Za-z0-9_]*)\s+in\s+([A-Za-z_][A-Za-z0-9_]*)$",
stmt,
)
if m:
loop_var, collection = m.groups()
idx_var = f"__jt_idx_{len(self.loop_stack)}"
collection_ruby = self.ruby_value(collection)
self.loop_stack.append((loop_var, idx_var, collection_ruby))
return f"<% {collection_ruby}.each_with_index do |{loop_var}, {idx_var}| -%>"
if stmt == "endfor":
if self.loop_stack:
self.loop_stack.pop()
return "<% end %>"
if stmt.startswith("if "):
cond = stmt[3:].strip()
if cond == "not loop.last" and self.loop_stack:
_loop_var, idx_var, collection_ruby = self.loop_stack[-1]
return f"<% if {idx_var} < ({collection_ruby}.length - 1) -%>"
m = re.match(r"^([A-Za-z_][A-Za-z0-9_\.]*)\s+is\s+defined$", cond)
if m:
return f"<% unless {self.ruby_value(m.group(1))}.nil? -%>"
m = re.match(r"^([A-Za-z_][A-Za-z0-9_\.]*)\s+is\s+none$", cond)
if m:
return f"<% if {self.ruby_value(m.group(1))}.nil? -%>"
return f"<% if {self.expr_to_ruby(cond)} -%>"
if stmt == "else":
return "<% else -%>"
if stmt.startswith("elif "):
return f"<% elsif {self.expr_to_ruby(stmt[5:].strip())} -%>"
if stmt == "endif":
return "<% end -%>"
# Preserve unknown Jinja statements visibly as an ERB comment so the
# generated template does not contain invalid Jinja syntax.
return f"<%# Unsupported JinjaTurtle statement: {stmt} %>"
def translate_jinja2_to_erb(
template_text: str,
*,
role_prefix: str,
puppet_class: str | None = None,
variable_names: set[str] | None = None,
) -> str:
return ErbTranslator(
role_prefix=role_prefix,
puppet_class=puppet_class,
variable_names=variable_names,
).translate(template_text)

57
src/jinjaturtle/escape.py
View file

@ -9,7 +9,7 @@ always replaced with ``{{ var }}`` placeholders and parked in the defaults data,
so a payload inside a value is inert. Verbatim text is different: if the source
contains ``{{ ... }}``, ``{% ... %}`` or ``{# ... #}`` (Jinja2), or ``<%= %>`` /
``<% %>`` (ERB), that text becomes *live template code* in the output and is
executed when Salt/Ansible/Puppet later renders the template.
executed when Ansible later renders the template.
Because JinjaTurtle is frequently fed harvested, attacker-influenceable config
(hostnames, banners, GECOS-derived comments, "Managed by" notes), this is a
@ -30,14 +30,6 @@ Design notes:
Jinja construct. The only way to break out of a raw block is a literal
``{% endraw %}`` in the source, so we defang the token ``endraw`` (in any
internal spacing) before wrapping.
* The ERB translator (``erb.py``) is raw-aware: it copies the *contents* of a
JinjaTurtle raw block through as literal ERB text and re-escapes any ERB
delimiters found there. That keeps a Jinja-escaped comment inert after the
Jinja2 -> ERB translation step, instead of the payload being "re-animated"
as ERB.
* ``escape_erb_literal`` exists for completeness / direct ERB emission: it
rewrites each ERB delimiter into an ERB expression that prints the delimiter
characters literally.
"""
import re
@ -70,11 +62,6 @@ def contains_jinja_markup(text: str) -> bool:
return any(m in text for m in _JINJA_MARKERS)
def contains_erb_markup(text: str) -> bool:
"""Return True if *text* contains any ERB delimiter."""
return any(m in text for m in (*_ERB_OPEN_MARKERS, *_ERB_CLOSE_MARKERS))
def _defang_endraw(text: str) -> str:
"""Rewrite any literal ``{% endraw %}`` so it cannot close our raw wrapper.
@ -109,45 +96,3 @@ def escape_jinja_literal(text: str) -> str:
if not text or not contains_jinja_markup(text):
return text
return "{% raw %}" + _defang_endraw(text) + "{% endraw %}"
def escape_erb_literal(text: str) -> str:
"""Make *text* render as literal characters under a later ERB render.
ERB has no ``raw`` block, so each opening/closing delimiter is rewritten as
an ERB expression that prints the delimiter literally. Text with no ERB
metacharacters is returned unchanged.
"""
if not text or not contains_erb_markup(text):
return text
result: list[str] = []
i = 0
n = len(text)
while i < n:
matched = None
for marker in (*_ERB_CLOSE_MARKERS, *_ERB_OPEN_MARKERS):
if text.startswith(marker, i):
matched = marker
break
if matched is not None:
escaped = matched.replace("\\", "\\\\").replace('"', '\\"')
result.append('<%= "' + escaped + '" %>')
i += len(matched)
else:
result.append(text[i])
i += 1
return "".join(result)
def escape_literal(text: str, *, engine: str = "jinja2") -> str:
"""Escape verbatim *text* for the target template *engine*.
``engine`` is ``"jinja2"`` (default) or ``"erb"``. Unknown engines fall back
to Jinja2 escaping. In JinjaTurtle's pipeline ERB output is produced by
translating Jinja2 output, and the translator is raw-aware, so handlers can
always Jinja-escape and rely on the translator to keep the literal inert.
"""
if engine == "erb":
return escape_erb_literal(text)
return escape_jinja_literal(text)

4
src/jinjaturtle/safety.py
View file

@ -22,8 +22,8 @@ single global property:
The check is positive/allowlist-based, which is the safe direction: unknown
constructs are rejected, not ignored. It runs at the single choke points in
``core.py`` (``generate_jinja2_template`` / ``generate_erb_template``), so it
covers every current handler and every future one automatically.
``core.py`` (``generate_jinja2_template``) so it covers every current handler and
every future one automatically.
Why this is robust against the escaper being wrong
---------------------------------------------------

54
tests/test_cli.py
View file

@ -132,57 +132,3 @@ def test_cli_folder_single_output_file_when_one_format(tmp_path):
assert defaults_path.is_file()
assert template_path.is_file()
assert "to_json" in template_path.read_text(encoding="utf-8")
def test_cli_erb_outputs_puppet_hiera_and_erb_template(tmp_path):
cfg = tmp_path / "app.ini"
cfg.write_text("[main]\nport = 8080\n", encoding="utf-8")
data_out = tmp_path / "common.yaml"
template_out = tmp_path / "app.ini.erb"
exit_code = cli._main(
[
str(cfg),
"-r",
"php",
"--template-engine",
"erb",
"--defaults-output",
str(data_out),
"--template-output",
str(template_out),
]
)
assert exit_code == 0
assert "php::main_port: '8080'" in data_out.read_text(encoding="utf-8")
assert "port = <%= @main_port %>" in template_out.read_text(encoding="utf-8")
def test_cli_erb_can_use_separate_puppet_class_namespace(tmp_path):
cfg = tmp_path / "app.ini"
cfg.write_text("[main]\nport = 8080\n", encoding="utf-8")
data_out = tmp_path / "node.yaml"
template_out = tmp_path / "app.ini.erb"
exit_code = cli._main(
[
str(cfg),
"-r",
"php_etc_app_ini",
"--template-engine",
"erb",
"--puppet-class",
"php",
"--defaults-output",
str(data_out),
"--template-output",
str(template_out),
]
)
assert exit_code == 0
data = data_out.read_text(encoding="utf-8")
template = template_out.read_text(encoding="utf-8")
assert "php::php_etc_app_ini_main_port: '8080'" in data
assert "port = <%= @php_etc_app_ini_main_port %>" in template

47
tests/test_injection_security.py
View file

@ -3,8 +3,8 @@
JinjaTurtle copies parts of the source config (comments, unrecognised lines,
structural keys) verbatim into the generated template. If that text contains
Jinja2/ERB delimiters it must be neutralised, otherwise attacker-influenced
config content becomes live template code that executes when Salt/Ansible/Puppet
later renders the template.
config content becomes live template code that executes when Ansible later
renders the template.
These tests render the *generated* template the way a downstream tool would and
assert that an injected payload never executes. A tripwire object is exposed
@ -14,7 +14,6 @@ the tripwire sentinel, an injected expression executed and the test fails.
from __future__ import annotations
import re
import subprocess
import sys
from pathlib import Path
@ -25,8 +24,6 @@ import yaml as pyyaml
from jinjaturtle.escape import (
escape_jinja_literal,
escape_erb_literal,
escape_literal,
)
TRIP = "__TRIPWIRE_FIRED__"
@ -233,43 +230,3 @@ def test_endraw_whitespace_control_cannot_break_out(endraw):
rendered = env.from_string(escaped).render(boom=_Boom())
assert TRIP not in rendered
assert rendered == payload
@pytest.mark.parametrize(
"payload",
[
"<%= system('id') %>",
"<% require 'open3' %>",
"text <%= 1+1 %> more",
"-%> orphan <%",
],
)
def test_escape_erb_literal_removes_executable_tags(payload):
escaped = escape_erb_literal(payload)
# No raw executable ERB tag should survive that contains the original code.
live = re.findall(r"<%[-=#]?(.*?)-?%>", escaped, re.S)
for chunk in live:
assert "system" not in chunk
assert "require" not in chunk
# Numeric/expression payloads must be reduced to literal-string prints.
def test_escape_literal_dispatches_by_engine():
"""The public ``escape_literal`` wrapper routes to the right engine."""
payload = "{{ 7*7 }}"
# Default engine is Jinja2.
assert escape_literal(payload) == escape_jinja_literal(payload)
assert escape_literal(payload, engine="jinja2") == escape_jinja_literal(payload)
# An unknown engine falls back to the safer Jinja2 escaping.
assert escape_literal(payload, engine="nonsense") == escape_jinja_literal(payload)
# ERB routing.
erb_payload = "<%= system('id') %>"
assert escape_literal(erb_payload, engine="erb") == escape_erb_literal(erb_payload)
def test_escape_literal_jinja_output_is_inert():
"""End-to-end: text routed through escape_literal does not execute."""
escaped = escape_literal("{{ boom.run('x') }}")
env = jinja2.Environment(undefined=jinja2.ChainableUndefined)
rendered = env.from_string(escaped).render(boom=_Boom())
assert TRIP not in rendered

11
tests/test_yaml_handler.py
View file

@ -10,7 +10,6 @@ from jinjaturtle.core import (
analyze_loops,
flatten_config,
generate_ansible_yaml,
generate_erb_template,
generate_jinja2_template,
)
from jinjaturtle.handlers.yaml import YamlHandler
@ -205,16 +204,6 @@ def test_yaml_loop_preserves_blank_separator_after_list(tmp_path: Path):
rendered = Template(template).render(**defaults)
assert rendered_expected in rendered
erb_template = generate_erb_template(
fmt,
parsed,
"role",
original_text=text,
loop_candidates=loop_candidates,
flat_items=flat_items,
)
assert erb_expected in erb_template
def test_yaml_loop_preserves_following_top_level_comments(tmp_path: Path):
from jinja2 import Environment, Template