mig5/jinjaturtle
1
0
Fork 0
Code Issues Pull requests Projects Releases 16 Packages Wiki Activity Actions

Compare commits

base: mig5:f5de32b778503f6056069762211db685d1af34e6
Branches Tags
mig5:main
mig5:erb
mig5:0.7.0
mig5:0.5.6
mig5:0.5.5
mig5:0.5.3
mig5:0.5.0
mig5:0.4.0
mig5:0.3.5
mig5:0.3.4
mig5:0.3.3
mig5:0.3.2
mig5:0.3.1
mig5:0.1.4
mig5:0.1.3
mig5:0.1.2
mig5:0.1.1
mig5:0.1.0
..
compare: mig5:5ee084d3958f0cf2557dec5cb5fefffeaf8d00b8
Branches Tags
mig5:main
mig5:erb
mig5:0.7.0
mig5:0.5.6
mig5:0.5.5
mig5:0.5.3
mig5:0.5.0
mig5:0.4.0
mig5:0.3.5
mig5:0.3.4
mig5:0.3.3
mig5:0.3.2
mig5:0.3.1
mig5:0.1.4
mig5:0.1.3
mig5:0.1.2
mig5:0.1.1
mig5:0.1.0

No commits in common. "f5de32b778503f6056069762211db685d1af34e6" and "5ee084d3958f0cf2557dec5cb5fefffeaf8d00b8" have entirely different histories.
f5de32b778 ... 5ee084d395

19 changed files with 221 additions and 1052 deletions
Download patch file Download diff file Expand all files Collapse all files

319
README.md
View file

@ -4,230 +4,55 @@
<img src="https://git.mig5.net/mig5/jinjaturtle/raw/branch/main/jinjaturtle.svg" alt="JinjaTurtle logo" width="240" />
</div>
JinjaTurtle is a command-line tool that helps turn existing native
configuration files into reusable configuration-management templates.
By default it generates:
- a **Jinja2** template; and
- an **Ansible defaults YAML** file containing the variables used by that
template.
It can also generate **ERB** templates and **Puppet Hiera-style YAML** data for
Puppet workflows.
JinjaTurtle does not try to replace configuration-management tools. Its job is
to speed up the boring first pass: take a real config file, discover the values
inside it, replace those values with variables, and write the corresponding
variable data beside the template.
JinjaTurtle is a command-line tool to help you generate Jinja2 templates and
Ansible inventory from a native configuration file (or files) of a piece of
software.
## How it works
JinjaTurtle examines a source config file and keeps the original structure as
much as possible.
* The config file(s) is/are examined
* Parameter key names are generated based on the parameter names in the
config file. In keeping with Ansible best practices, you pass a prefix
for the key names, which should typically match the name of your Ansible
role.
* A Jinja2 file is generated from the file with those parameter key names
injected as the `{{ variable }}` names.
* An Ansible inventory YAML file is generated with those key names and the
*values* taken from the original config file as the default vars.
For the default Jinja2/Ansible mode:
1. The config file is parsed.
2. Variable names are generated from the config keys and paths.
3. Those variable names are prefixed with `--role-name`, which should usually
match your Ansible role name.
4. A Jinja2 template is generated with values replaced by `{{ variable }}`
expressions.
5. An Ansible defaults YAML file is generated with those variables and the
original values.
For ERB/Puppet mode:
1. The same parse/flatten/loop analysis is used.
2. An ERB template is generated with Puppet-style instance variables such as
`<%= @memory_limit %>`.
3. The variables file is written as Puppet Hiera-style data, such as
`php::memory_limit: 256M`.
4. If `--puppet-class` is supplied, that class name is used as the Hiera
namespace while `--role-name` remains the local variable prefix.
By default, the generated variable data and template are printed to stdout. Use
`--defaults-output` and `--template-output` to write them to files.
## Jinja2 / Ansible example
Say you have a `php.ini` file and you are inside an Ansible role with
`defaults/` and `templates/` directories:
```shell
jinjaturtle php.ini \
--role-name php \
--defaults-output defaults/main.yml \
--template-output templates/php.ini.j2
```
Given a source value such as:
```ini
memory_limit = 256M
```
JinjaTurtle will produce a template value like:
```jinja2
memory_limit = {{ php_memory_limit }}
```
and defaults data like:
```yaml
php_memory_limit: 256M
```
## ERB / Puppet example
Use `--template-engine erb` when you want Puppet ERB output:
```shell
jinjaturtle php.ini \
--role-name php \
--template-engine erb \
--defaults-output data/common.yaml \
--template-output templates/php.ini.erb
```
Given the same source value:
```ini
memory_limit = 256M
```
JinjaTurtle will produce an ERB template value like:
```erb
memory_limit = <%= @memory_limit %>
```
and Hiera-style data like:
```yaml
php::memory_limit: 256M
```
The `--defaults-output` option name is retained for CLI compatibility, but in
ERB mode the file is intended to be Puppet Hiera data rather than Ansible role
defaults.
JinjaTurtle does **not** generate Puppet classes or `file` resources. A Puppet
module, or another tool such as Enroll, should still declare the class
parameters and call the template, for example with Puppet's `template()`
function.
## Using `--puppet-class`
Most direct usage can simply use the same value for the role name and Puppet
class name:
```shell
jinjaturtle php.ini --role-name php --template-engine erb
```
This creates Hiera keys such as:
```yaml
php::memory_limit: 256M
```
and local ERB variables such as:
```erb
<%= @memory_limit %>
```
For generated systems, it can be useful to make `--role-name` more specific
while keeping the Hiera keys under the real Puppet class. For example:
```shell
jinjaturtle php.ini \
--role-name php_etc_php_ini \
--puppet-class php \
--template-engine erb
```
In that case the variable prefix can stay file-specific, while the Hiera data
is still written under `php::...`.
By default, the Jinja2 template and the Ansible inventory are printed to
stdout. However, it is possible to output the results to new files.
## What sort of config files can it handle?
JinjaTurtle supports common structured and semi-structured config formats:
TOML, YAML, INI, JSON and XML-style config files should be okay. There are always
going to be some edge cases in very complex files that are difficult to work
with, though, so you may still find that you need to tweak the results.
- TOML
- YAML
- INI-style files
- JSON
- XML
- Postfix `main.cf`
- systemd unit files, such as `*.service`, `*.socket`, `*.timer`, and related
unit types
- OpenSSH-style config files, including `ssh_config`, `sshd_config`, and common
`*.conf` snippets detected as SSH config
For XML and YAML files, JinjaTurtle will attempt to generate 'for' loops
and lists in the Ansible yaml if the config file looks homogenous enough to
support it. However, if it lacks the confidence in this, it will fall back to
using scalar-style flattened attributes.
For ambiguous extensions such as `*.conf`, JinjaTurtle uses lightweight content
sniffing. You can always force a handler with `--format`.
You may need or wish to tidy up the config to suit your needs.
For YAML, XML, TOML, INI-style, and other supported structured files,
JinjaTurtle will attempt to generate loops when a repeated structure looks
homogeneous enough. If it is not confident, it falls back to flattened scalar
variables.
Some very complex files will still need manual cleanup. The goal is to speed up
conversion into Jinja2 or ERB templates, not to guarantee a perfect final module
without review.
## JSON, quoting, and type preservation
JinjaTurtle tries to preserve rendered config types.
For JSON, it uses JSON-aware expressions rather than plain string substitution.
This avoids generating invalid JSON such as:
```json
{"enabled": True}
```
when the correct rendered JSON should be:
```json
{"enabled": true}
```
In Jinja2 mode this uses Ansible-style JSON filters. In ERB mode it emits Ruby
JSON generation where required, for example:
```erb
<% require 'json' -%>
{
"enabled": <%= JSON.generate(@enabled) %>
}
```
That is expected for JSON ERB templates.
The goal here is really to *speed up* converting files into Ansible/Jinja2,
but not necessarily to make it perfect.
## Can I convert multiple files at once?
Yes. Pass a directory instead of a single file and JinjaTurtle will convert the
files it understands in that directory.
Certainly! Pass the folder name instead of a specific file name, and JinjaTurtle
will convert any files it understands in that folder, storing all the various
vars in the destination defaults yaml file, and converting each file into a
Jinja2 template per file type.
```shell
jinjaturtle ./config-dir \
--role-name myrole \
--defaults-output defaults/main.yml \
--template-output templates/
```
If all the files had the same 'type', there'll be one Jinja2 template.
Use `--recursive` to recurse into subdirectories.
You can also pass `--recursive` to recurse into subfolders.
In folder mode, variables for multiple files of the same type are grouped under
an `items`-style structure in the generated YAML so that the resulting templates
can be used with loops in Ansible.
For example:
Note: when using 'folder' mode and multiple files of the same type, their vars
will be listed under an 'items' parent key in the yaml, each with an `id` key.
You'll then want to use a `loop` in Ansible later, e.g:
```yaml
- name: Render configs
@ -268,9 +93,9 @@ sudo dnf upgrade --refresh
sudo dnf install jinjaturtle
```
### From PyPI
### From PyPi
```bash
```
pip install jinjaturtle
```
@ -278,81 +103,61 @@ pip install jinjaturtle
Clone the repo and then run inside the clone:
```bash
```
poetry install
```
### AppImage
Download the AppImage from the Releases page, make it executable, and put it on
your `$PATH`.
Download the AppImage from the Releases and make it executable, and put it
on your `$PATH`.
## How to run it
Say you have a `php.ini` file and you are in a directory structure like an
Ansible role (with subfolders `defaults` and `templates`):
```shell
jinjaturtle php.ini \
--role-name php \
--defaults-output defaults/main.yml \
--template-output templates/php.ini.j2
```
## Full usage info
```text
usage: jinjaturtle [-h] [-r ROLE_NAME] [--recursive]
[-f {ini,json,toml,yaml,xml,postfix,systemd,ssh}]
[-d DEFAULTS_OUTPUT] [-t TEMPLATE_OUTPUT]
[--template-engine {jinja2,erb}]
[--puppet-class PUPPET_CLASS]
config
```
usage: jinjaturtle [-h] -r ROLE_NAME [-f {json,ini,toml,yaml,xml,postfix,systemd}] [-d DEFAULTS_OUTPUT] [-t TEMPLATE_OUTPUT] config
Convert a config file into an Ansible defaults file and Jinja2 template.
Convert a config file into Ansible inventory and a Jinja2 template.
positional arguments:
config Path to a config file OR a folder containing supported
config files. Supported: .toml, .yaml/.yml, .json,
.ini/.cfg/.conf, .xml, ssh_config/sshd_config
config Path to the source configuration file.
options:
-h, --help show this help message and exit
-r, --role-name ROLE_NAME
Role name / variable prefix. In Jinja2 mode this is
usually the Ansible role name. In ERB mode it is used
as the local variable prefix. Defaults to jinjaturtle.
--recursive When CONFIG is a folder, recurse into subfolders.
-f, --format {ini,json,toml,yaml,xml,postfix,systemd,ssh}
Force config format instead of auto-detecting from
filename.
Ansible role name, used as variable prefix (e.g. cometbft).
-f, --format {ini,json,toml,xml}
Force config format instead of auto-detecting from filename.
-d, --defaults-output DEFAULTS_OUTPUT
Path to write the generated variable YAML. If omitted,
it is printed to stdout.
Path to write defaults/main.yml. If omitted, default vars are printed to stdout.
-t, --template-output TEMPLATE_OUTPUT
Path to write the generated config template. If omitted,
it is printed to stdout.
--template-engine {jinja2,erb}
Template syntax to generate. Defaults to jinja2. Use
erb for Puppet templates.
--puppet-class PUPPET_CLASS
Puppet class / Hiera namespace to use with
--template-engine erb. Defaults to --role-name.
Path to write the Jinja2 config template. If omitted, template is printed to stdout.
```
## Additional supported formats
JinjaTurtle also templates some common bespoke config formats:
JinjaTurtle can also template some common "bespoke" config formats:
- **Postfix main.cf** (`main.cf`) → `--format postfix`
- **systemd unit files** (`*.service`, `*.socket`, etc.) → `--format systemd`
- **OpenSSH config** (`ssh_config`, `sshd_config`, and detected snippets) →
`--format ssh`
For ambiguous extensions like `*.conf`, JinjaTurtle uses lightweight content
sniffing. You can always force a specific handler with `--format`.
For ambiguous extensions like `*.conf`, JinjaTurtle uses lightweight content sniffing; you can always force a specific handler via `--format`.
## Relationship with Enroll
JinjaTurtle can be used directly, but it is also useful as a helper for tools
that generate configuration-management code.
Enroll can call JinjaTurtle when it is available on the `PATH`. In that setup,
Enroll decides where Puppet, Ansible, or Salt files belong, while JinjaTurtle
concentrates on converting harvested config files into templates plus variable
data.
## Found a bug, have a suggestion?
You can e-mail me; see `pyproject.toml` for details. You can also contact me on
the Fediverse:
You can e-mail me (see the pyproject.toml for details) or contact me on the Fediverse:
https://goto.mig5.net/@mig5

6
debian/changelog vendored
View file

@ -1,9 +1,3 @@
jinjaturtle (0.5.5) unstable; urgency=medium
* erb support
-- Miguel Jacq <mig@mig5.net> Sat, 20 Jun 2026 18:27:00 +1000
jinjaturtle (0.5.4) unstable; urgency=medium
* Make templates more faithful to the original file in terms of indentation, newlines, no deserialisation of things like < or >.

2
pyproject.toml
View file

@ -1,6 +1,6 @@
[tool.poetry]
name = "jinjaturtle"
version = "0.5.5"
version = "0.5.4"
description = "Convert config files into Ansible defaults and Jinja2 templates."
authors = ["Miguel Jacq <mig@mig5.net>"]
license = "GPL-3.0-or-later"

4
rpm/jinjaturtle.spec
View file

@ -1,4 +1,4 @@
%global upstream_version 0.5.5
%global upstream_version 0.5.4
Name: jinjaturtle
Version: %{upstream_version}
@ -43,8 +43,6 @@ Convert config files into Ansible defaults and Jinja2 templates.
%changelog
* Sat Jun 20 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- erb support
* Sat Jun 20 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}
- Make templates more faithful to the original file in terms of indentation, newlines, no deserialisation of things like < or >.
- More test coverage
* Fri Jun 19 2026 Miguel Jacq <mig@mig5.net> - %{version}-%{release}

69
src/jinjaturtle/cli.py
View file

@ -5,15 +5,12 @@ import sys
from defusedxml import defuse_stdlib
from pathlib import Path
from . import j2
from .core import (
parse_config,
analyze_loops,
flatten_config,
generate_ansible_yaml,
generate_jinja2_template,
generate_puppet_hiera_yaml,
generate_erb_template,
)
from .multi import process_directory
@ -56,21 +53,7 @@ def _build_arg_parser() -> argparse.ArgumentParser:
ap.add_argument(
"-t",
"--template-output",
help="Path to write the generated config template. If omitted, template is printed to stdout.",
)
ap.add_argument(
"--template-engine",
choices=[j2.NAME, "erb"],
default=j2.NAME,
help="Template syntax to generate (default: jinja2). Use erb for Puppet templates.",
)
ap.add_argument(
"--puppet-class",
help=(
"Puppet class/Hiera namespace to use with --template-engine erb. "
"Defaults to --role-name. This lets tools use a file-specific "
"variable prefix while writing Hiera keys under the real Puppet class."
),
help="Path to write the Jinja2 config template. If omitted, template is printed to stdout.",
)
return ap
@ -95,21 +78,6 @@ def _main(argv: list[str] | None = None) -> int:
print("# defaults/main.yml")
print(defaults_yaml, end="")
# Optionally translate folder-mode templates to ERB. Folder mode keeps
# the existing data shape; single-file mode below is the preferred
# Puppet path because it can produce class-parameter Hiera keys.
if args.template_engine == "erb":
from .erb import translate_jinja2_to_erb
for o in outputs:
o.template = translate_jinja2_to_erb(
o.template,
role_prefix=args.role_name,
puppet_class=args.puppet_class or args.role_name,
)
template_ext = "erb" if args.template_engine == "erb" else j2.TEMPLATE_EXTENSION
# Write templates
if args.template_output:
out_path = Path(args.template_output)
@ -118,16 +86,12 @@ def _main(argv: list[str] | None = None) -> int:
else:
out_path.mkdir(parents=True, exist_ok=True)
for o in outputs:
(out_path / f"config.{o.fmt}.{template_ext}").write_text(
(out_path / f"config.{o.fmt}.j2").write_text(
o.template, encoding="utf-8"
)
else:
for o in outputs:
name = (
f"config.{template_ext}"
if len(outputs) == 1
else f"config.{o.fmt}.{template_ext}"
)
name = "config.j2" if len(outputs) == 1 else f"config.{o.fmt}.j2"
print(f"# {name}")
print(o.template, end="")
@ -145,27 +109,8 @@ def _main(argv: list[str] | None = None) -> int:
# Flatten config (excluding loop paths if loops are detected)
flat_items = flatten_config(fmt, parsed, loop_candidates)
if args.template_engine == "erb":
ansible_yaml = generate_puppet_hiera_yaml(
args.role_name,
flat_items,
loop_candidates,
puppet_class=args.puppet_class or args.role_name,
)
template_str = generate_erb_template(
fmt,
parsed,
args.role_name,
original_text=config_text,
loop_candidates=loop_candidates,
flat_items=flat_items,
puppet_class=args.puppet_class or args.role_name,
)
else:
# Generate defaults YAML (with loop collections if detected)
ansible_yaml = generate_ansible_yaml(
args.role_name, flat_items, loop_candidates
)
ansible_yaml = generate_ansible_yaml(args.role_name, flat_items, loop_candidates)
# Generate template (with loops if detected)
template_str = generate_jinja2_template(
@ -185,11 +130,7 @@ def _main(argv: list[str] | None = None) -> int:
if args.template_output:
Path(args.template_output).write_text(template_str, encoding="utf-8")
else:
print(
"# config.erb"
if args.template_engine == "erb"
else f"# config.{j2.TEMPLATE_EXTENSION}"
)
print("# config.j2")
print(template_str, end="")
return 0

80
src/jinjaturtle/core.py
View file

@ -8,7 +8,6 @@ import re
import yaml
from .loop_analyzer import LoopAnalyzer, LoopCandidate
from .erb import puppet_class_name, puppet_local_var_name, translate_jinja2_to_erb
from .handlers import (
BaseHandler,
IniHandler,
@ -388,85 +387,6 @@ def generate_jinja2_template(
)
def _template_variable_names(
role_prefix: str,
flat_items: list[tuple[tuple[str, ...], Any]],
loop_candidates: list[LoopCandidate] | None = None,
) -> set[str]:
names = {make_var_name(role_prefix, path) for path, _value in flat_items}
if loop_candidates:
for candidate in loop_candidates:
names.add(make_var_name(role_prefix, candidate.path))
return names
def generate_puppet_hiera_yaml(
role_prefix: str,
flat_items: list[tuple[tuple[str, ...], Any]],
loop_candidates: list[LoopCandidate] | None = None,
*,
puppet_class: str | None = None,
) -> str:
"""Create Puppet Hiera data suitable for Automatic Parameter Lookup.
``role_prefix`` remains the source variable prefix used by JinjaTurtle while
``puppet_class`` is the Puppet class/Hiera namespace. In the normal case
they are the same, so ``php_memory_limit`` becomes ``php::memory_limit``.
Enroll may pass a file-specific role prefix and a separate Puppet class to
avoid parameter-name collisions inside one generated Puppet module.
"""
klass = puppet_class_name(puppet_class or role_prefix)
data: dict[str, Any] = {}
for path, value in flat_items:
generated = make_var_name(role_prefix, path)
local = puppet_local_var_name(role_prefix, generated, puppet_class=klass)
data[f"{klass}::{local}"] = value
if loop_candidates:
for candidate in loop_candidates:
generated = make_var_name(role_prefix, candidate.path)
local = puppet_local_var_name(role_prefix, generated, puppet_class=klass)
data[f"{klass}::{local}"] = candidate.items
return dump_yaml(data, sort_keys=True)
def generate_erb_template(
fmt: str,
parsed: Any,
role_prefix: str,
*,
original_text: str | None = None,
loop_candidates: list[LoopCandidate] | None = None,
flat_items: list[tuple[tuple[str, ...], Any]] | None = None,
puppet_class: str | None = None,
) -> str:
"""Generate a Puppet ERB template from JinjaTurtle's renderer-neutral data.
The first implementation intentionally translates the Jinja2 subset emitted
by JinjaTurtle's existing format handlers. This keeps parsing, formatting
preservation, and loop detection identical between Jinja2 and ERB output
while still producing Puppet-native ``@parameter`` references.
"""
jinja_template = generate_jinja2_template(
fmt,
parsed,
role_prefix,
original_text=original_text,
loop_candidates=loop_candidates,
)
names = _template_variable_names(role_prefix, flat_items or [], loop_candidates)
return translate_jinja2_to_erb(
jinja_template,
role_prefix=role_prefix,
puppet_class=puppet_class or role_prefix,
variable_names=names,
)
def _stringify_timestamps(obj: Any) -> Any:
"""
Recursively walk a parsed config and turn any datetime/date/time objects

218
src/jinjaturtle/erb.py
View file

@ -1,218 +0,0 @@
from __future__ import annotations
import re
def _safe_name(raw: str, *, fallback: str = "var") -> str:
text = re.sub(r"[^A-Za-z0-9_]+", "_", str(raw or fallback)).strip("_").lower()
text = re.sub(r"_+", "_", text)
if not text:
text = fallback
if not re.match(r"^[a-z_]", text):
text = f"{fallback}_{text}"
return text
def puppet_class_name(raw: str) -> str:
"""Return a conservative Puppet class/Hiera namespace name."""
text = _safe_name(raw, fallback="jinjaturtle")
if not re.match(r"^[a-z]", text):
text = f"jinjaturtle_{text}"
return text
def _role_prefix_name(raw: str) -> str:
return _safe_name(raw, fallback="jinjaturtle")
def puppet_local_var_name(
role_prefix: str,
jinja_var_name: str,
*,
puppet_class: str | None = None,
) -> str:
"""Map a generated JinjaTurtle variable to a Puppet class parameter.
For the common case where ``--role-name php`` also means class ``php``, a
generated Jinja variable such as ``php_memory_limit`` becomes Puppet local
parameter ``memory_limit`` and Hiera key ``php::memory_limit``.
Enroll sometimes needs a file-specific variable prefix to avoid collisions
inside a generated module. When ``puppet_class`` differs from
``role_prefix`` we keep the full generated variable name as the local
parameter and only use ``puppet_class`` as the Hiera namespace.
"""
var_name = _safe_name(jinja_var_name, fallback="value")
prefix = _role_prefix_name(role_prefix)
klass = puppet_class_name(puppet_class or role_prefix)
if klass == prefix and var_name.startswith(prefix + "_"):
stripped = var_name[len(prefix) + 1 :]
return stripped or var_name
return var_name
class ErbTranslator:
"""Translate the Jinja2 subset emitted by JinjaTurtle into Puppet ERB."""
_TOKEN_RE = re.compile(r"({{.*?}}|{%.*?%})", re.S)
def __init__(
self,
*,
role_prefix: str,
puppet_class: str | None = None,
variable_names: set[str] | None = None,
) -> None:
self.role_prefix = role_prefix
self.puppet_class = puppet_class or role_prefix
self.variable_names = set(variable_names or set())
self.loop_stack: list[tuple[str, str, str]] = []
self.needs_json = False
def translate(self, template_text: str) -> str:
parts = self._TOKEN_RE.split(template_text)
out: list[str] = []
for token in parts:
if not token:
continue
if token.startswith("{{") and token.endswith("}}"):
expr = token[2:-2].strip()
out.append(f"<%= {self.expr_to_ruby(expr)} %>")
continue
if token.startswith("{%") and token.endswith("%}"):
stmt = token[2:-2].strip()
out.append(self.statement_to_erb(stmt))
continue
out.append(token)
rendered = "".join(out)
if self.needs_json and "require 'json'" not in rendered:
rendered = "<% require 'json' -%>\n" + rendered
return rendered
def local_var(self, name: str) -> str:
return puppet_local_var_name(
self.role_prefix, name, puppet_class=self.puppet_class
)
def ruby_value(self, expr: str) -> str:
expr = expr.strip()
if expr in {"true", "True"}:
return "true"
if expr in {"false", "False"}:
return "false"
if expr in {"none", "None", "null"}:
return "nil"
if re.match(r"^[A-Za-z_][A-Za-z0-9_]*$", expr):
if any(expr == loop_var for loop_var, _idx, _coll in self.loop_stack):
return expr
return f"@{self.local_var(expr)}"
m = re.match(r"^([A-Za-z_][A-Za-z0-9_]*)\.([A-Za-z_][A-Za-z0-9_]*)$", expr)
if m:
base, key = m.groups()
if any(base == loop_var for loop_var, _idx, _coll in self.loop_stack):
return f"{base}[{key!r}]"
return f"@{self.local_var(base)}[{key!r}]"
return expr
def expr_to_ruby(self, expr: str) -> str:
expr = expr.strip()
# JinjaTurtle emits these YAML-preserving ternaries for booleans/nulls.
m = re.match(
r"^(['\"])(true|false)\1\s+if\s+([A-Za-z_][A-Za-z0-9_\.]*)\s+else\s+(['\"])(true|false)\4$",
expr,
)
if m:
truthy = m.group(2)
cond = self.ruby_value(m.group(3))
falsy = m.group(5)
return f"{cond} ? {truthy!r} : {falsy!r}"
m = re.match(
r"^(['\"])(null)\1\s+if\s+([A-Za-z_][A-Za-z0-9_\.]*)\s+is\s+none\s+else\s+([A-Za-z_][A-Za-z0-9_\.]*)$",
expr,
)
if m:
value = self.ruby_value(m.group(3))
fallback = self.ruby_value(m.group(4))
return f"{value}.nil? ? 'null' : {fallback}"
if "|" in expr:
base, *filters = [part.strip() for part in expr.split("|")]
ruby = self.ruby_value(base)
for filt in filters:
if filt.startswith("lower"):
ruby = f"{ruby}.to_s.downcase"
elif filt.startswith("to_json") or filt.startswith("tojson"):
self.needs_json = True
if "indent" in filt:
ruby = f"JSON.pretty_generate({ruby})"
else:
ruby = f"JSON.generate({ruby})"
return ruby
return self.ruby_value(expr)
def statement_to_erb(self, stmt: str) -> str:
if stmt.endswith(("-", "+")):
stmt = stmt[:-1].rstrip()
if stmt.startswith("for "):
m = re.match(
r"^for\s+([A-Za-z_][A-Za-z0-9_]*)\s+in\s+([A-Za-z_][A-Za-z0-9_]*)$",
stmt,
)
if m:
loop_var, collection = m.groups()
idx_var = f"__jt_idx_{len(self.loop_stack)}"
collection_ruby = self.ruby_value(collection)
self.loop_stack.append((loop_var, idx_var, collection_ruby))
return f"<% {collection_ruby}.each_with_index do |{loop_var}, {idx_var}| -%>"
if stmt == "endfor":
if self.loop_stack:
self.loop_stack.pop()
return "<% end %>"
if stmt.startswith("if "):
cond = stmt[3:].strip()
if cond == "not loop.last" and self.loop_stack:
_loop_var, idx_var, collection_ruby = self.loop_stack[-1]
return f"<% if {idx_var} < ({collection_ruby}.length - 1) -%>"
m = re.match(r"^([A-Za-z_][A-Za-z0-9_\.]*)\s+is\s+defined$", cond)
if m:
return f"<% unless {self.ruby_value(m.group(1))}.nil? -%>"
m = re.match(r"^([A-Za-z_][A-Za-z0-9_\.]*)\s+is\s+none$", cond)
if m:
return f"<% if {self.ruby_value(m.group(1))}.nil? -%>"
return f"<% if {self.expr_to_ruby(cond)} -%>"
if stmt == "else":
return "<% else -%>"
if stmt.startswith("elif "):
return f"<% elsif {self.expr_to_ruby(stmt[5:].strip())} -%>"
if stmt == "endif":
return "<% end -%>"
# Preserve unknown Jinja statements visibly as an ERB comment so the
# generated template does not contain invalid Jinja syntax.
return f"<%# Unsupported JinjaTurtle statement: {stmt} %>"
def translate_jinja2_to_erb(
template_text: str,
*,
role_prefix: str,
puppet_class: str | None = None,
variable_names: set[str] | None = None,
) -> str:
return ErbTranslator(
role_prefix=role_prefix,
puppet_class=puppet_class,
variable_names=variable_names,
).translate(template_text)

9
src/jinjaturtle/handlers/ini.py
View file

@ -5,7 +5,6 @@ from pathlib import Path
from typing import Any
from . import BaseHandler
from .. import j2
class IniHandler(BaseHandler):
@ -64,9 +63,9 @@ class IniHandler(BaseHandler):
var_name = self.make_var_name(role_prefix, path)
value = value.strip()
if len(value) >= 2 and value[0] == value[-1] and value[0] in {'"', "'"}:
lines.append(f"{key} = {j2.quoted_variable(var_name)}")
lines.append(f'{key} = "{{{{ {var_name} }}}}"')
else:
lines.append(f"{key} = {j2.variable(var_name)}")
lines.append(f"{key} = {{{{ {var_name} }}}}")
lines.append("")
return "\n".join(lines).rstrip() + "\n"
@ -142,9 +141,9 @@ class IniHandler(BaseHandler):
if use_quotes:
quote_char = raw_value[0]
replacement_value = j2.quoted_variable(var_name, quote_char)
replacement_value = f"{quote_char}{{{{ {var_name} }}}}{quote_char}"
else:
replacement_value = j2.variable(var_name)
replacement_value = f"{{{{ {var_name} }}}}"
new_content = (
before_eq + "=" + leading_ws + replacement_value + comment_part

22
src/jinjaturtle/handlers/json.py
View file

@ -6,7 +6,6 @@ from pathlib import Path
from typing import Any
from . import DictLikeHandler
from .. import j2
from ..loop_analyzer import LoopCandidate
@ -32,7 +31,7 @@ class JsonHandler(DictLikeHandler):
return self._generate_json_template(role_prefix, parsed)
JSON_INDENT = 2
JSON_VALUE_FILTER = j2.JSON_VALUE_FILTER
JSON_VALUE_FILTER = "to_json(ensure_ascii=False)"
def _leading_indent(self, s: str, idx: int) -> int:
"""Return the number of leading spaces on the line containing idx."""
@ -86,7 +85,7 @@ class JsonHandler(DictLikeHandler):
useful in HTML, but noisy in configuration files. JinjaTurtle generates
Ansible templates, so use Ansible's ``to_json`` filter instead.
"""
return j2.filtered(var_name, self.JSON_VALUE_FILTER)
return f"{{{{ {var_name} | {self.JSON_VALUE_FILTER} }}}}"
def _generate_json_template_from_text(self, role_prefix: str, text: str) -> str:
"""Replace JSON scalar values in-place, preserving original formatting.
@ -331,10 +330,10 @@ class JsonHandler(DictLikeHandler):
# a blank line between iterations under default Jinja whitespace settings.
return (
f"[\n"
f"{j2.for_start(item_var, collection_var)}"
f"{inner}{j2.to_json(item_var)}"
f"{j2.if_not_loop_last()},{j2.endif()}\n"
f"{j2.for_end()}{base}]"
f"{{% for {item_var} in {collection_var} %}}"
f"{inner}{{{{ {item_var} | to_json(ensure_ascii=False) }}}}"
f"{{% if not loop.last %}},{{% endif %}}\n"
f"{{% endfor %}}{base}]"
)
def _generate_json_dict_loop(
@ -365,15 +364,16 @@ class JsonHandler(DictLikeHandler):
for i, key in enumerate(keys):
comma = "," if i < len(keys) - 1 else ""
dict_lines.append(
f'{field}"{key}": ' f"{j2.to_json(f'{item_var}.{key}')}{comma}"
f'{field}"{key}": '
f"{{{{ {item_var}.{key} | to_json(ensure_ascii=False) }}}}{comma}"
)
# Comma between *items* goes after the closing brace.
dict_lines.append(f"{inner}}}{j2.if_not_loop_last()},{j2.endif()}")
dict_lines.append(f"{inner}}}{{% if not loop.last %}},{{% endif %}}")
dict_body = "\n".join(dict_lines)
# Put the `{% for %}` at the start of the first item line to avoid blank lines.
return (
f"[\n"
f"{j2.for_start(item_var, collection_var)}{inner}{dict_body}\n"
f"{j2.for_end()}{base}]"
f"{{% for {item_var} in {collection_var} %}}{inner}{dict_body}\n"
f"{{% endfor %}}{base}]"
)

9
src/jinjaturtle/handlers/postfix.py
View file

@ -4,7 +4,6 @@ from pathlib import Path
from typing import Any
from . import BaseHandler
from .. import j2
class PostfixMainHandler(BaseHandler):
@ -93,7 +92,7 @@ class PostfixMainHandler(BaseHandler):
lines: list[str] = []
for k, v in parsed.items():
var = self.make_var_name(role_prefix, (k,))
lines.append(f"{k} = {j2.variable(var)}")
lines.append(f"{k} = {{{{ {var} }}}}")
return "\n".join(lines).rstrip() + "\n"
return self._generate_from_text(role_prefix, original_text)
@ -165,13 +164,11 @@ class PostfixMainHandler(BaseHandler):
quoted = len(v) >= 2 and v[0] == v[-1] and v[0] in {'"', "'"}
if quoted:
replacement = (
f"{before_eq}={leading_ws}{j2.quoted_variable(var)}"
f"{comment_part}{newline}"
f'{before_eq}={leading_ws}"{{{{ {var} }}}}"{comment_part}{newline}'
)
else:
replacement = (
f"{before_eq}={leading_ws}{j2.variable(var)}"
f"{comment_part}{newline}"
f"{before_eq}={leading_ws}{{{{ {var} }}}}{comment_part}{newline}"
)
out_lines.append(replacement)

5
src/jinjaturtle/handlers/ssh.py
View file

@ -6,7 +6,6 @@ from pathlib import Path
from typing import Any
from . import BaseHandler
from .. import j2
_SECTION_KEYWORDS = {"host", "match"}
@ -266,9 +265,9 @@ class SshConfigHandler(BaseHandler):
var = self.make_var_name(role_prefix, path)
if ln.quoted and ln.value:
quote_char = ln.value[0]
replacement_value = j2.quoted_variable(var, quote_char)
replacement_value = f"{quote_char}{{{{ {var} }}}}{quote_char}"
else:
replacement_value = j2.variable(var)
replacement_value = f"{{{{ {var} }}}}"
rendered = (
f"{ln.before_value}{replacement_value}"

11
src/jinjaturtle/handlers/systemd.py
View file

@ -5,7 +5,6 @@ from pathlib import Path
from typing import Any
from . import BaseHandler
from .. import j2
@dataclass
@ -168,15 +167,9 @@ class SystemdUnitHandler(BaseHandler):
v = (ln.value or "").strip()
quoted = len(v) >= 2 and v[0] == v[-1] and v[0] in {'"', "'"}
if quoted:
repl = (
f"{ln.before_eq}={ln.leading_ws_after_eq}"
f"{j2.quoted_variable(var)}{ln.comment}"
)
repl = f'{ln.before_eq}={ln.leading_ws_after_eq}"{{{{ {var} }}}}"{ln.comment}'
else:
repl = (
f"{ln.before_eq}={ln.leading_ws_after_eq}"
f"{j2.variable(var)}{ln.comment}"
)
repl = f"{ln.before_eq}={ln.leading_ws_after_eq}{{{{ {var} }}}}{ln.comment}"
newline = "\n" if ln.raw.endswith("\n") else ""
out_lines.append(repl + newline)

95
src/jinjaturtle/handlers/toml.py
View file

@ -4,7 +4,6 @@ from pathlib import Path
from typing import Any
from . import DictLikeHandler
from .. import j2
from ..loop_analyzer import LoopCandidate
try:
@ -17,14 +16,6 @@ class TomlHandler(DictLikeHandler):
fmt = "toml"
flatten_lists = False # keep lists as scalars
def _toml_value_expr(self, var_name: str, value: Any | None = None) -> str:
if isinstance(value, bool):
return j2.lower(var_name)
return j2.variable(var_name)
def _toml_quoted_expr(self, var_name: str, quote: str = '"') -> str:
return j2.quoted_variable(var_name, quote)
def parse(self, path: Path) -> Any:
if tomllib is None:
raise RuntimeError(
@ -77,12 +68,12 @@ class TomlHandler(DictLikeHandler):
def emit_kv(path: tuple[str, ...], key: str, value: Any) -> None:
var_name = self.make_var_name(role_prefix, path + (key,))
if isinstance(value, str):
lines.append(f"{key} = {self._toml_quoted_expr(var_name)}")
lines.append(f'{key} = "{{{{ {var_name} }}}}"')
elif isinstance(value, bool):
# Booleans need | lower filter (Python True/False → TOML true/false)
lines.append(f"{key} = {self._toml_value_expr(var_name, value)}")
lines.append(f"{key} = {{{{ {var_name} | lower }}}}")
else:
lines.append(f"{key} = {self._toml_value_expr(var_name, value)}")
lines.append(f"{key} = {{{{ {var_name} }}}}")
def walk(obj: dict[str, Any], path: tuple[str, ...] = ()) -> None:
scalar_items = {k: v for k, v in obj.items() if not isinstance(v, dict)}
@ -130,10 +121,10 @@ class TomlHandler(DictLikeHandler):
def emit_kv(path: tuple[str, ...], key: str, value: Any) -> None:
var_name = self.make_var_name(role_prefix, path + (key,))
if isinstance(value, str):
lines.append(f"{key} = {self._toml_quoted_expr(var_name)}")
lines.append(f'{key} = "{{{{ {var_name} }}}}"')
elif isinstance(value, bool):
# Booleans need | lower filter (Python True/False → TOML true/false)
lines.append(f"{key} = {self._toml_value_expr(var_name, value)}")
lines.append(f"{key} = {{{{ {var_name} | lower }}}}")
elif isinstance(value, list):
# Check if this list is a loop candidate
if path + (key,) in loop_paths:
@ -148,21 +139,24 @@ class TomlHandler(DictLikeHandler):
# Scalar list loop
lines.append(
f"{key} = ["
f"{j2.for_start(item_var, collection_var)}"
f"{j2.variable(item_var)}"
f"{j2.if_not_loop_last()}, {j2.endif()}"
f"{j2.for_end()}"
f"{{% for {item_var} in {collection_var} %}}"
f"{{{{ {item_var} }}}}"
f"{{% if not loop.last %}}, {{% endif %}}"
f"{{% endfor %}}"
f"]"
)
elif candidate.item_schema in ("simple_dict", "nested"):
# Dict list loop - TOML array of tables
# This is complex for TOML, using simplified approach
lines.append(f"{key} = " f"{j2.to_json(var_name)}")
lines.append(
f"{key} = "
f"{{{{ {var_name} | to_json(ensure_ascii=False) }}}}"
)
else:
# Not a loop, treat as regular variable
lines.append(f"{key} = {self._toml_value_expr(var_name, value)}")
lines.append(f"{key} = {{{{ {var_name} }}}}")
else:
lines.append(f"{key} = {self._toml_value_expr(var_name, value)}")
lines.append(f"{key} = {{{{ {var_name} }}}}")
def walk(obj: dict[str, Any], path: tuple[str, ...] = ()) -> None:
scalar_items = {k: v for k, v in obj.items() if not isinstance(v, dict)}
@ -288,17 +282,13 @@ class TomlHandler(DictLikeHandler):
nested_path = path + (sub_key,)
nested_var = self.make_var_name(role_prefix, nested_path)
if isinstance(sub_val, str):
inner_bits.append(
f"{sub_key} = {self._toml_quoted_expr(nested_var)}"
)
inner_bits.append(f'{sub_key} = "{{{{ {nested_var} }}}}"')
elif isinstance(sub_val, bool):
inner_bits.append(
f"{sub_key} = {self._toml_value_expr(nested_var, sub_val)}"
f"{sub_key} = {{{{ {nested_var} | lower }}}}"
)
else:
inner_bits.append(
f"{sub_key} = {self._toml_value_expr(nested_var, sub_val)}"
)
inner_bits.append(f"{sub_key} = {{{ {nested_var} }}}")
replacement_value = "{ " + ", ".join(inner_bits) + " }"
new_content = (
before_eq + "=" + leading_ws + replacement_value + comment_part
@ -320,11 +310,11 @@ class TomlHandler(DictLikeHandler):
if use_quotes:
quote_char = raw_value[0]
replacement_value = self._toml_quoted_expr(var_name, quote_char)
replacement_value = f"{quote_char}{{{{ {var_name} }}}}{quote_char}"
elif is_bool:
replacement_value = j2.lower(var_name)
replacement_value = f"{{{{ {var_name} | lower }}}}"
else:
replacement_value = j2.variable(var_name)
replacement_value = f"{{{{ {var_name} }}}}"
new_content = (
before_eq + "=" + leading_ws + replacement_value + comment_part
@ -402,7 +392,7 @@ class TomlHandler(DictLikeHandler):
# Build loop
out_lines.append(
f"{j2.for_start(item_var, collection_var)}\n"
f"{{% for {item_var} in {collection_var} %}}\n"
)
out_lines.append(f"[[{'.'.join(table_path)}]]\n")
@ -412,21 +402,14 @@ class TomlHandler(DictLikeHandler):
continue
if isinstance(value, str):
out_lines.append(
f"{key} = "
f"{self._toml_quoted_expr(f'{item_var}.{key}')}\n"
)
elif isinstance(value, bool):
out_lines.append(
f"{key} = "
f"{self._toml_value_expr(f'{item_var}.{key}', value)}\n"
f'{key} = "{{{{ {item_var}.{key} }}}}"\n'
)
else:
out_lines.append(
f"{key} = "
f"{self._toml_value_expr(f'{item_var}.{key}', value)}\n"
f"{key} = {{{{ {item_var}.{key} }}}}\n"
)
out_lines.append(f"{j2.for_end()}\n")
out_lines.append("{% endfor %}\n")
# Skip all content until the next different table
skip_until_next_table = True
@ -490,15 +473,17 @@ class TomlHandler(DictLikeHandler):
# Scalar list loop
replacement_value = (
f"["
f"{j2.for_start(item_var, collection_var)}"
f"{j2.variable(item_var)}"
f"{j2.if_not_loop_last()}, {j2.endif()}"
f"{j2.for_end()}"
f"{{% for {item_var} in {collection_var} %}}"
f"{{{{ {item_var} }}}}"
f"{{% if not loop.last %}}, {{% endif %}}"
f"{{% endfor %}}"
f"]"
)
else:
# Dict/nested loop - use to_json filter for complex arrays
replacement_value = j2.to_json(collection_var)
replacement_value = (
f"{{{{ {collection_var} | to_json(ensure_ascii=False) }}}}"
)
new_content = (
before_eq + "=" + leading_ws + replacement_value + comment_part
@ -525,17 +510,13 @@ class TomlHandler(DictLikeHandler):
nested_path = path + (sub_key,)
nested_var = self.make_var_name(role_prefix, nested_path)
if isinstance(sub_val, str):
inner_bits.append(
f"{sub_key} = {self._toml_quoted_expr(nested_var)}"
)
inner_bits.append(f'{sub_key} = "{{{{ {nested_var} }}}}"')
elif isinstance(sub_val, bool):
inner_bits.append(
f"{sub_key} = {self._toml_value_expr(nested_var, sub_val)}"
f"{sub_key} = {{{{ {nested_var} | lower }}}}"
)
else:
inner_bits.append(
f"{sub_key} = {self._toml_value_expr(nested_var, sub_val)}"
)
inner_bits.append(f"{sub_key} = {{{{ {nested_var} }}}}")
replacement_value = "{ " + ", ".join(inner_bits) + " }"
new_content = (
before_eq + "=" + leading_ws + replacement_value + comment_part
@ -557,11 +538,11 @@ class TomlHandler(DictLikeHandler):
if use_quotes:
quote_char = raw_value[0]
replacement_value = self._toml_quoted_expr(var_name, quote_char)
replacement_value = f"{quote_char}{{{{ {var_name} }}}}{quote_char}"
elif is_bool:
replacement_value = j2.lower(var_name)
replacement_value = f"{{{{ {var_name} | lower }}}}"
else:
replacement_value = j2.variable(var_name)
replacement_value = f"{{{{ {var_name} }}}}"
new_content = (
before_eq + "=" + leading_ws + replacement_value + comment_part

25
src/jinjaturtle/handlers/xml.py
View file

@ -6,7 +6,6 @@ from typing import Any
import xml.etree.ElementTree as ET # nosec
from .base import BaseHandler
from .. import j2
from ..loop_analyzer import LoopCandidate
@ -173,7 +172,7 @@ class XmlHandler(BaseHandler):
for attr_name in list(elem.attrib.keys()):
attr_path = path + (f"@{attr_name}",)
var_name = self.make_var_name(role_prefix, attr_path)
elem.set(attr_name, j2.variable(var_name))
elem.set(attr_name, f"{{{{ {var_name} }}}}")
# Children
children = [c for c in list(elem) if isinstance(c.tag, str)]
@ -186,7 +185,7 @@ class XmlHandler(BaseHandler):
else:
text_path = path + ("value",)
var_name = self.make_var_name(role_prefix, text_path)
elem.text = j2.variable(var_name)
elem.text = f"{{{{ {var_name} }}}}"
# Handle children - check for loops first
counts = Counter(child.tag for child in children)
@ -340,12 +339,12 @@ class XmlHandler(BaseHandler):
# Build loop
result_lines.append(
f"{indent_str}{j2.for_start(item_var, collection_var)}"
f"{indent_str}{{% for {item_var} in {collection_var} %}}"
)
# Add each line of the sample with proper indentation
for sample_line in sample_lines:
result_lines.append(f"{indent_str} {sample_line}")
result_lines.append(f"{indent_str}{j2.for_end()}")
result_lines.append(f"{indent_str}{{% endfor %}}")
else:
# Keep the marker if we can't find the candidate
result_lines.append(line)
@ -361,11 +360,11 @@ class XmlHandler(BaseHandler):
end = line.find("-->", start)
condition = line[start:end]
indent = len(line) - len(line.lstrip())
final_lines.append(f"{' ' * indent}{j2.if_defined(condition)}")
final_lines.append(f"{' ' * indent}{{% if {condition} is defined %}}")
# Replace <!--ENDIF:field--> with {% endif %}
elif "<!--ENDIF:" in line:
indent = len(line) - len(line.lstrip())
final_lines.append(f"{' ' * indent}{j2.endif()}")
final_lines.append(f"{' ' * indent}{{% endif %}}")
else:
final_lines.append(line)
@ -417,13 +416,13 @@ class XmlHandler(BaseHandler):
# Attribute - these come from element attributes
attr_name = key[1:] # Remove @ prefix
# Use simple variable reference - attributes should always exist
elem.set(attr_name, j2.variable(f"{loop_var}.{attr_name}"))
elem.set(attr_name, f"{{{{ {loop_var}.{attr_name} }}}}")
elif key == "_text":
# Simple text content - use ._text accessor for dict-based items
elem.text = j2.variable(f"{loop_var}._text")
elem.text = f"{{{{ {loop_var}._text }}}}"
elif key == "value":
# Text with attributes/children
elem.text = j2.variable(f"{loop_var}.value")
elem.text = f"{{{{ {loop_var}.value }}}}"
elif key == "_key":
# This is the dict key (for dict collections), skip in XML
pass
@ -432,13 +431,13 @@ class XmlHandler(BaseHandler):
# Create a conditional wrapper comment
child = ET.Element(key)
if "_text" in value:
child.text = j2.variable(f"{loop_var}.{key}._text")
child.text = f"{{{{ {loop_var}.{key}._text }}}}"
else:
# More complex nested structure
for sub_key, sub_val in value.items():
if not sub_key.startswith("_"):
grandchild = ET.SubElement(child, sub_key)
grandchild.text = j2.variable(f"{loop_var}.{key}.{sub_key}")
grandchild.text = f"{{{{ {loop_var}.{key}.{sub_key} }}}}"
# Wrap the child in a Jinja if statement (will be done via text replacement)
# For now, add a marker comment before the element
@ -453,7 +452,7 @@ class XmlHandler(BaseHandler):
marker = ET.Comment(f"IF:{loop_var}.{key}")
elem.append(marker)
child = ET.SubElement(elem, key)
child.text = j2.variable(f"{loop_var}.{key}")
child.text = f"{{{{ {loop_var}.{key} }}}}"
end_marker = ET.Comment(f"ENDIF:{key}")
elem.append(end_marker)

94
src/jinjaturtle/handlers/yaml.py
View file

@ -5,7 +5,6 @@ from pathlib import Path
from typing import Any
from .dict import DictLikeHandler
from .. import j2
from ..loop_analyzer import LoopCandidate
@ -68,10 +67,19 @@ class YamlHandler(DictLikeHandler):
consumers are stricter than PyYAML, so emit explicit YAML spelling for
values that were originally YAML booleans/nulls.
"""
return j2.yaml_scalar_expression(var_name, raw_value)
raw = (raw_value or "").strip().lower()
if raw in {"true", "false"}:
return f"{{{{ 'true' if {var_name} else 'false' }}}}"
if raw in {"null", "~"}:
return f"{{{{ 'null' if {var_name} is none else {var_name} }}}}"
return f"{{{{ {var_name} }}}}"
def _yaml_value_expr(self, value_expr: str, sample_value: Any | None = None) -> str:
return j2.yaml_value_expression(value_expr, sample_value)
if isinstance(sample_value, bool):
return f"{{{{ 'true' if {value_expr} else 'false' }}}}"
if sample_value is None:
return f"{{{{ 'null' if {value_expr} is none else {value_expr} }}}}"
return f"{{{{ {value_expr} }}}}"
def _surrounding_quote(self, raw_value: str) -> str | None:
if (
@ -142,7 +150,7 @@ class YamlHandler(DictLikeHandler):
if use_quotes:
q = raw_value[0]
replacement = j2.quoted_variable(var_name, q)
replacement = f"{q}{{{{ {var_name} }}}}{q}"
else:
replacement = self._yaml_scalar_expr(var_name, raw_value)
@ -181,7 +189,7 @@ class YamlHandler(DictLikeHandler):
if use_quotes:
q = raw_value[0]
replacement = j2.quoted_variable(var_name, q)
replacement = f"{q}{{{{ {var_name} }}}}{q}"
else:
replacement = self._yaml_scalar_expr(var_name, raw_value)
@ -227,37 +235,27 @@ class YamlHandler(DictLikeHandler):
def current_path() -> tuple[str, ...]:
return stack[-1][1] if stack else ()
def first_sequence_item_style(
def first_sequence_item_quote(
start_index: int, parent_indent: int
) -> tuple[str | None, int | None]:
) -> str | None:
for future_line in lines[start_index + 1 :]:
future_stripped = future_line.lstrip()
future_indent = len(future_line) - len(future_stripped)
if not future_stripped or future_stripped.startswith("#"):
continue
if future_indent < parent_indent:
return None, None
return None
if future_stripped.startswith("- "):
value_part, _comment_part = self._split_inline_comment(
future_stripped[2:], {"#"}
)
return self._surrounding_quote(value_part.strip()), future_indent
return self._surrounding_quote(value_part.strip())
if future_indent <= parent_indent:
return None, None
return None, None
def next_significant_line(index: int) -> tuple[int, str] | None:
for future_line in lines[index + 1 :]:
future_stripped = future_line.lstrip()
if not future_stripped.strip() or future_stripped.startswith("#"):
continue
return len(future_line) - len(future_stripped), future_stripped
return None
return None
for line_index, raw_line in enumerate(lines):
stripped = raw_line.lstrip()
is_blank = not stripped.strip()
is_comment = stripped.startswith("#")
indent = len(raw_line) - len(stripped)
# If we're skipping lines inside a collection replaced by a loop,
@ -271,22 +269,7 @@ class YamlHandler(DictLikeHandler):
# Stop only when a non-list item at the parent indentation appears,
# or when indentation moves above the parent collection.
if skip_until_indent is not None:
if is_blank:
next_line = next_significant_line(line_index)
if next_line is None:
skip_until_indent = None
out_lines.append(raw_line)
else:
next_indent, next_stripped = next_line
still_in_collection = next_indent > skip_until_indent or (
next_indent == skip_until_indent
and next_stripped.startswith("- ")
)
if not still_in_collection:
skip_until_indent = None
out_lines.append(raw_line)
continue
if is_comment:
if not stripped or stripped.startswith("#"):
if indent <= skip_until_indent:
skip_until_indent = None
out_lines.append(raw_line)
@ -302,7 +285,7 @@ class YamlHandler(DictLikeHandler):
continue # Skip this line
# Blank or comment lines
if is_blank or is_comment:
if not stripped or stripped.startswith("#"):
out_lines.append(raw_line)
continue
@ -332,19 +315,13 @@ class YamlHandler(DictLikeHandler):
# Find the matching candidate
candidate = next(c for c in loop_candidates if c.path == path)
scalar_quote, item_indent = first_sequence_item_style(
line_index, indent
)
if candidate.item_schema != "scalar":
scalar_quote = None
if candidate.item_schema == "scalar":
scalar_quote = first_sequence_item_quote(line_index, indent)
# Generate loop
loop_str = self._generate_yaml_loop(
candidate,
role_prefix,
indent,
scalar_quote=scalar_quote,
item_indent=item_indent,
candidate, role_prefix, indent, scalar_quote=scalar_quote
)
out_lines.append(loop_str)
@ -371,7 +348,7 @@ class YamlHandler(DictLikeHandler):
if use_quotes:
q = raw_value[0]
replacement = j2.quoted_variable(var_name, q)
replacement = f"{q}{{{{ {var_name} }}}}{q}"
else:
replacement = self._yaml_scalar_expr(var_name, raw_value)
@ -435,7 +412,7 @@ class YamlHandler(DictLikeHandler):
if use_quotes:
q = raw_value[0]
replacement = j2.quoted_variable(var_name, q)
replacement = f"{q}{{{{ {var_name} }}}}{q}"
else:
replacement = self._yaml_scalar_expr(var_name, raw_value)
@ -458,7 +435,6 @@ class YamlHandler(DictLikeHandler):
indent: int,
is_list: bool = False,
scalar_quote: str | None = None,
item_indent: int | None = None,
) -> str:
"""
Generate a Jinja2 for loop for a YAML collection.
@ -485,17 +461,13 @@ class YamlHandler(DictLikeHandler):
item_lines: list[str] = []
if candidate.items:
sample_item = candidate.items[0]
effective_item_indent = (
item_indent if item_indent is not None else indent + 2
)
if is_list:
effective_item_indent = indent
item_indent_str = " " * effective_item_indent
item_indent = indent + 2 if not is_list else indent
item_indent_str = " " * item_indent
if candidate.item_schema == "scalar":
value_expr = self._yaml_value_expr(item_var, sample_item)
if scalar_quote and isinstance(sample_item, str):
value_expr = j2.quoted_variable(item_var, scalar_quote)
value_expr = f"{scalar_quote}{{{{ {item_var} }}}}{scalar_quote}"
item_lines.append(f"{item_indent_str}- {value_expr}")
elif candidate.item_schema in ("simple_dict", "nested"):
item_lines = self._dict_to_yaml_lines(
@ -508,14 +480,14 @@ class YamlHandler(DictLikeHandler):
# rendering a blank line after the parent key. Keeping the control
# tag itself at column zero prevents its indentation from leaking
# into the rendered YAML and nesting the next top-level key.
lines.append(f"{j2.for_start(item_var, collection_var)}{item_lines[0]}")
lines.append(f"{{% for {item_var} in {collection_var} %}}{item_lines[0]}")
lines.extend(item_lines[1:])
lines.append(j2.for_end(keep_trailing_newline=True))
lines.append("{% endfor %}")
else:
lines.append(j2.for_start(item_var, collection_var))
lines.append(j2.for_end(keep_trailing_newline=True))
lines.append(f"{{% for {item_var} in {collection_var} %}}")
lines.append("{% endfor %}")
return "\n".join(lines)
return "\n".join(lines) + "\n"
def _dict_to_yaml_lines(
self,

102
src/jinjaturtle/j2.py
View file

@ -1,102 +0,0 @@
from __future__ import annotations
from typing import Any
NAME = "jinja2"
TEMPLATE_EXTENSION = "j2"
JSON_VALUE_FILTER = "to_json(ensure_ascii=False)"
def expression(value: str) -> str:
"""Return a Jinja2 output expression for an already-built expression body."""
return f"{{{{ {value} }}}}"
def variable(name: str) -> str:
"""Return a Jinja2 output expression for a variable name."""
return expression(name)
def quoted_variable(name: str, quote: str = '"') -> str:
"""Return a quoted Jinja2 variable placeholder."""
return f"{quote}{variable(name)}{quote}"
def filtered(value: str, filter_expression: str) -> str:
"""Return a Jinja2 output expression with one filter expression applied."""
return expression(f"{value} | {filter_expression}")
def lower(value: str) -> str:
"""Return a Jinja2 expression that renders a value through ``| lower``."""
return filtered(value, "lower")
def to_json(
value: str, *, indent: int | None = None, ensure_ascii: bool = False
) -> str:
"""Return a Jinja2 expression using Ansible's ``to_json`` filter."""
args: list[str] = []
if indent is not None:
args.append(f"indent={indent}")
args.append(f"ensure_ascii={ensure_ascii}")
return filtered(value, f"to_json({', '.join(args)})")
def statement(
value: str,
*,
trim_right: bool = False,
keep_trailing_newline: bool = False,
) -> str:
"""Return a Jinja2 statement tag for an already-built statement body."""
if trim_right and keep_trailing_newline:
raise ValueError("trim_right and keep_trailing_newline are mutually exclusive")
if trim_right:
return f"{{% {value} -%}}"
if keep_trailing_newline:
return f"{{% {value} +%}}"
return f"{{% {value} %}}"
def if_defined(name: str) -> str:
return statement(f"if {name} is defined")
def if_not_loop_last() -> str:
return statement("if not loop.last")
def endif() -> str:
return statement("endif")
def for_start(item_var: str, collection_var: str) -> str:
return statement(f"for {item_var} in {collection_var}")
def for_end(*, trim_right: bool = False, keep_trailing_newline: bool = False) -> str:
return statement(
"endfor",
trim_right=trim_right,
keep_trailing_newline=keep_trailing_newline,
)
def yaml_scalar_expression(var_name: str, raw_value: str | None = None) -> str:
"""Return a YAML-safe Jinja2 expression for a scalar value."""
raw = (raw_value or "").strip().lower()
if raw in {"true", "false"}:
return expression(f"'true' if {var_name} else 'false'")
if raw in {"null", "~"}:
return expression(f"'null' if {var_name} is none else {var_name}")
return variable(var_name)
def yaml_value_expression(value_expr: str, sample_value: Any | None = None) -> str:
"""Return a YAML-safe Jinja2 expression for a possibly typed sample value."""
if isinstance(sample_value, bool):
return expression(f"'true' if {value_expr} else 'false'")
if sample_value is None:
return expression(f"'null' if {value_expr} is none else {value_expr}")
return expression(value_expr)

71
src/jinjaturtle/multi.py
View file

@ -28,7 +28,6 @@ from pathlib import Path
from typing import Any, Iterable
import xml.etree.ElementTree as ET # nosec
from . import j2
from .core import dump_yaml, flatten_config, make_var_name, parse_config
from .handlers.xml import XmlHandler
@ -141,8 +140,8 @@ def _yaml_scalar_placeholder(
) -> str:
var = make_var_name(role_prefix, path)
if isinstance(sample, str):
return j2.quoted_variable(var)
return j2.variable(var)
return f'"{{{{ {var} }}}}"'
return f"{{{{ {var} }}}}"
def _yaml_render_union(
@ -169,13 +168,13 @@ def _yaml_render_union(
if _is_scalar(val) or val is None:
value = _yaml_scalar_placeholder(role_prefix, key_path, val)
if cond_var:
lines.append(f"{ind}{j2.if_defined(cond_var)}")
lines.append(f"{ind}{{% if {cond_var} is defined %}}")
lines.append(f"{ind}{key}: {value}")
if cond_var:
lines.append(f"{ind}{j2.endif()}")
lines.append(f"{ind}{{% endif %}}")
else:
if cond_var:
lines.append(f"{ind}{j2.if_defined(cond_var)}")
lines.append(f"{ind}{{% if {cond_var} is defined %}}")
lines.append(f"{ind}{key}:")
lines.extend(
_yaml_render_union(
@ -188,7 +187,7 @@ def _yaml_render_union(
)
)
if cond_var:
lines.append(f"{ind}{j2.endif()}")
lines.append(f"{ind}{{% endif %}}")
return lines
if isinstance(union_obj, list):
@ -203,13 +202,13 @@ def _yaml_render_union(
if _is_scalar(item) or item is None:
value = _yaml_scalar_placeholder(role_prefix, item_path, item)
if cond_var:
lines.append(f"{ind}{j2.if_defined(cond_var)}")
lines.append(f"{ind}{{% if {cond_var} is defined %}}")
lines.append(f"{ind}- {value}")
if cond_var:
lines.append(f"{ind}{j2.endif()}")
lines.append(f"{ind}{{% endif %}}")
elif isinstance(item, dict):
if cond_var:
lines.append(f"{ind}{j2.if_defined(cond_var)}")
lines.append(f"{ind}{{% if {cond_var} is defined %}}")
# First line: list marker with first key if possible
first = True
for k, v in item.items():
@ -223,22 +222,22 @@ def _yaml_render_union(
value = _yaml_scalar_placeholder(role_prefix, kp, v)
if first:
if k_cond:
lines.append(f"{ind}{j2.if_defined(k_cond)}")
lines.append(f"{ind}{{% if {k_cond} is defined %}}")
lines.append(f"{ind}- {k}: {value}")
if k_cond:
lines.append(f"{ind}{j2.endif()}")
lines.append(f"{ind}{{% endif %}}")
first = False
else:
if k_cond:
lines.append(f"{ind} {j2.if_defined(k_cond)}")
lines.append(f"{ind} {{% if {k_cond} is defined %}}")
lines.append(f"{ind} {k}: {value}")
if k_cond:
lines.append(f"{ind} {j2.endif()}")
lines.append(f"{ind} {{% endif %}}")
else:
# nested
if first:
if k_cond:
lines.append(f"{ind}{j2.if_defined(k_cond)}")
lines.append(f"{ind}{{% if {k_cond} is defined %}}")
lines.append(f"{ind}- {k}:")
lines.extend(
_yaml_render_union(
@ -250,11 +249,11 @@ def _yaml_render_union(
)
)
if k_cond:
lines.append(f"{ind}{j2.endif()}")
lines.append(f"{ind}{{% endif %}}")
first = False
else:
if k_cond:
lines.append(f"{ind} {j2.if_defined(k_cond)}")
lines.append(f"{ind} {{% if {k_cond} is defined %}}")
lines.append(f"{ind} {k}:")
lines.extend(
_yaml_render_union(
@ -266,20 +265,20 @@ def _yaml_render_union(
)
)
if k_cond:
lines.append(f"{ind} {j2.endif()}")
lines.append(f"{ind} {{% endif %}}")
if first:
# empty dict item
lines.append(f"{ind}- {{}}")
if cond_var:
lines.append(f"{ind}{j2.endif()}")
lines.append(f"{ind}{{% endif %}}")
else:
# list of lists - emit as scalar-ish fallback
value = j2.variable(make_var_name(role_prefix, item_path))
value = f"{{{{ {make_var_name(role_prefix, item_path)} }}}}"
if cond_var:
lines.append(f"{ind}{j2.if_defined(cond_var)}")
lines.append(f"{ind}{{% if {cond_var} is defined %}}")
lines.append(f"{ind}- {value}")
if cond_var:
lines.append(f"{ind}{j2.endif()}")
lines.append(f"{ind}{{% endif %}}")
return lines
# scalar at root
@ -307,15 +306,15 @@ def _toml_render_union(
else None
)
if cond:
lines.append(f"{j2.if_defined(cond)}")
lines.append(f"{{% if {cond} is defined %}}")
if isinstance(value, str):
lines.append(f"{key} = {j2.quoted_variable(var_name)}")
lines.append(f'{key} = "{{{{ {var_name} }}}}"')
elif isinstance(value, bool):
lines.append(f"{key} = {j2.lower(var_name)}")
lines.append(f"{key} = {{{{ {var_name} | lower }}}}")
else:
lines.append(f"{key} = {j2.variable(var_name)}")
lines.append(f"{key} = {{{{ {var_name} }}}}")
if cond:
lines.append(j2.endif())
lines.append("{% endif %}")
def walk(obj: dict[str, Any], path: tuple[str, ...]) -> None:
if path:
@ -325,7 +324,7 @@ def _toml_render_union(
else None
)
if cond:
lines.append(f"{j2.if_defined(cond)}")
lines.append(f"{{% if {cond} is defined %}}")
lines.append(f"[{'.'.join(path)}]")
scalar_items = {k: v for k, v in obj.items() if not isinstance(v, dict)}
@ -341,7 +340,7 @@ def _toml_render_union(
walk(v, path + (str(k),))
if path and (path in optional_containers):
lines.append(j2.endif())
lines.append("{% endif %}")
lines.append("")
# root scalars
@ -411,7 +410,7 @@ def _ini_render_union(
else None
)
if sec_cond:
lines.append(f"{j2.if_defined(sec_cond)}")
lines.append(f"{{% if {sec_cond} is defined %}}")
lines.append(f"[{section}]")
for key, raw_val in union.items(section, raw=True):
path = (section, key)
@ -422,16 +421,16 @@ def _ini_render_union(
v = (raw_val or "").strip()
quoted = len(v) >= 2 and v[0] == v[-1] and v[0] in {'"', "'"}
if key_cond:
lines.append(f"{j2.if_defined(key_cond)}")
lines.append(f"{{% if {key_cond} is defined %}}")
if quoted:
lines.append(f"{key} = {j2.quoted_variable(var)}")
lines.append(f'{key} = "{{{{ {var} }}}}"')
else:
lines.append(f"{key} = {j2.variable(var)}")
lines.append(f"{key} = {{{{ {var} }}}}")
if key_cond:
lines.append(j2.endif())
lines.append("{% endif %}")
lines.append("")
if sec_cond:
lines.append(j2.endif())
lines.append("{% endif %}")
lines.append("")
return "\n".join(lines).rstrip() + "\n"
@ -625,7 +624,7 @@ def process_directory(
if multiple_formats
else f"{role_prefix}_items"
)
template = f"{j2.to_json('data', indent=2)}\n"
template = "{{ data | to_json(indent=2, ensure_ascii=False) }}\n"
items: list[dict[str, Any]] = []
for rid, parsed in zip(rel_ids, parsed_list):
items.append({"id": rid, "data": parsed})

54
tests/test_cli.py
View file

@ -132,57 +132,3 @@ def test_cli_folder_single_output_file_when_one_format(tmp_path):
assert defaults_path.is_file()
assert template_path.is_file()
assert "to_json" in template_path.read_text(encoding="utf-8")
def test_cli_erb_outputs_puppet_hiera_and_erb_template(tmp_path):
cfg = tmp_path / "app.ini"
cfg.write_text("[main]\nport = 8080\n", encoding="utf-8")
data_out = tmp_path / "common.yaml"
template_out = tmp_path / "app.ini.erb"
exit_code = cli._main(
[
str(cfg),
"-r",
"php",
"--template-engine",
"erb",
"--defaults-output",
str(data_out),
"--template-output",
str(template_out),
]
)
assert exit_code == 0
assert "php::main_port: '8080'" in data_out.read_text(encoding="utf-8")
assert "port = <%= @main_port %>" in template_out.read_text(encoding="utf-8")
def test_cli_erb_can_use_separate_puppet_class_namespace(tmp_path):
cfg = tmp_path / "app.ini"
cfg.write_text("[main]\nport = 8080\n", encoding="utf-8")
data_out = tmp_path / "node.yaml"
template_out = tmp_path / "app.ini.erb"
exit_code = cli._main(
[
str(cfg),
"-r",
"php_etc_app_ini",
"--template-engine",
"erb",
"--puppet-class",
"php",
"--defaults-output",
str(data_out),
"--template-output",
str(template_out),
]
)
assert exit_code == 0
data = data_out.read_text(encoding="utf-8")
template = template_out.read_text(encoding="utf-8")
assert "php::php_etc_app_ini_main_port: '8080'" in data
assert "port = <%= @php_etc_app_ini_main_port %>" in template

58
tests/test_yaml_handler.py
View file

@ -10,7 +10,6 @@ from jinjaturtle.core import (
analyze_loops,
flatten_config,
generate_ansible_yaml,
generate_erb_template,
generate_jinja2_template,
)
from jinjaturtle.handlers.yaml import YamlHandler
@ -164,60 +163,12 @@ def test_yaml_indentless_sequence_loop_roundtrips_semantically(tmp_path: Path):
rendered = Template(template).render(**defaults)
assert yaml.safe_load(rendered) == yaml.safe_load(text)
assert "%} - {{ image }}" in template
assert "%} - {{ image }}" not in template
assert "%} - {{ image }}" not in template
assert " - waffleimage/ubuntu18.04" not in template
assert " vagrant:" not in template
def test_yaml_loop_preserves_blank_separator_after_list(tmp_path: Path):
from jinja2 import Template
cases = [
(
"blank",
"require:\n - rubocop-performance\n - rubocop-rspec\n\nAllCops:\n NewCops: enable\n",
"\n - rubocop-rspec\n\nAllCops:",
"<% end %>\nAllCops:",
),
(
"no_blank",
"require:\n - rubocop-performance\n - rubocop-rspec\nAllCops:\n NewCops: enable\n",
"\n - rubocop-rspec\nAllCops:",
"<% end %>AllCops:",
),
]
for label, text, rendered_expected, erb_expected in cases:
path = tmp_path / f"{label}.yml"
path.write_text(text, encoding="utf-8")
fmt, parsed = parse_config(path)
loop_candidates = analyze_loops(fmt, parsed)
flat_items = flatten_config(fmt, parsed, loop_candidates)
defaults = yaml.safe_load(
generate_ansible_yaml("role", flat_items, loop_candidates)
)
template = generate_jinja2_template(
fmt, parsed, "role", original_text=text, loop_candidates=loop_candidates
)
rendered = Template(template).render(**defaults)
assert rendered_expected in rendered
erb_template = generate_erb_template(
fmt,
parsed,
"role",
original_text=text,
loop_candidates=loop_candidates,
flat_items=flat_items,
)
assert erb_expected in erb_template
def test_yaml_loop_preserves_following_top_level_comments(tmp_path: Path):
from jinja2 import Environment, Template
from jinja2 import Template
from jinjaturtle.core import analyze_loops
@ -248,15 +199,10 @@ def test_yaml_loop_preserves_following_top_level_comments(tmp_path: Path):
fmt, parsed, "role", original_text=text, loop_candidates=loop_candidates
)
rendered = Template(template).render(**defaults)
ansible_rendered = (
Environment(trim_blocks=True).from_string(template).render(**defaults)
)
assert "# Offense count: 2" in rendered
assert "# This cop supports unsafe autocorrection" in rendered
assert "spec.rb\n\n# Offense count: 2" in ansible_rendered
assert yaml.safe_load(rendered) == yaml.safe_load(text)
assert yaml.safe_load(ansible_rendered) == yaml.safe_load(text)
def test_yaml_scalar_loop_preserves_quoted_list_items(tmp_path: Path):