diff --git a/README.md b/README.md index 26730fb..0e0ad48 100644 --- a/README.md +++ b/README.md @@ -4,230 +4,55 @@ JinjaTurtle logo -JinjaTurtle is a command-line tool that helps turn existing native -configuration files into reusable configuration-management templates. - -By default it generates: - -- a **Jinja2** template; and -- an **Ansible defaults YAML** file containing the variables used by that - template. - -It can also generate **ERB** templates and **Puppet Hiera-style YAML** data for -Puppet workflows. - -JinjaTurtle does not try to replace configuration-management tools. Its job is -to speed up the boring first pass: take a real config file, discover the values -inside it, replace those values with variables, and write the corresponding -variable data beside the template. +JinjaTurtle is a command-line tool to help you generate Jinja2 templates and +Ansible inventory from a native configuration file (or files) of a piece of +software. ## How it works -JinjaTurtle examines a source config file and keeps the original structure as -much as possible. + * The config file(s) is/are examined + * Parameter key names are generated based on the parameter names in the + config file. In keeping with Ansible best practices, you pass a prefix + for the key names, which should typically match the name of your Ansible + role. + * A Jinja2 file is generated from the file with those parameter key names + injected as the `{{ variable }}` names. + * An Ansible inventory YAML file is generated with those key names and the + *values* taken from the original config file as the default vars. -For the default Jinja2/Ansible mode: - -1. The config file is parsed. -2. Variable names are generated from the config keys and paths. -3. Those variable names are prefixed with `--role-name`, which should usually - match your Ansible role name. -4. A Jinja2 template is generated with values replaced by `{{ variable }}` - expressions. -5. An Ansible defaults YAML file is generated with those variables and the - original values. - -For ERB/Puppet mode: - -1. The same parse/flatten/loop analysis is used. -2. An ERB template is generated with Puppet-style instance variables such as - `<%= @memory_limit %>`. -3. The variables file is written as Puppet Hiera-style data, such as - `php::memory_limit: 256M`. -4. If `--puppet-class` is supplied, that class name is used as the Hiera - namespace while `--role-name` remains the local variable prefix. - -By default, the generated variable data and template are printed to stdout. Use -`--defaults-output` and `--template-output` to write them to files. - -## Jinja2 / Ansible example - -Say you have a `php.ini` file and you are inside an Ansible role with -`defaults/` and `templates/` directories: - -```shell -jinjaturtle php.ini \ - --role-name php \ - --defaults-output defaults/main.yml \ - --template-output templates/php.ini.j2 -``` - -Given a source value such as: - -```ini -memory_limit = 256M -``` - -JinjaTurtle will produce a template value like: - -```jinja2 -memory_limit = {{ php_memory_limit }} -``` - -and defaults data like: - -```yaml -php_memory_limit: 256M -``` - -## ERB / Puppet example - -Use `--template-engine erb` when you want Puppet ERB output: - -```shell -jinjaturtle php.ini \ - --role-name php \ - --template-engine erb \ - --defaults-output data/common.yaml \ - --template-output templates/php.ini.erb -``` - -Given the same source value: - -```ini -memory_limit = 256M -``` - -JinjaTurtle will produce an ERB template value like: - -```erb -memory_limit = <%= @memory_limit %> -``` - -and Hiera-style data like: - -```yaml -php::memory_limit: 256M -``` - -The `--defaults-output` option name is retained for CLI compatibility, but in -ERB mode the file is intended to be Puppet Hiera data rather than Ansible role -defaults. - -JinjaTurtle does **not** generate Puppet classes or `file` resources. A Puppet -module, or another tool such as Enroll, should still declare the class -parameters and call the template, for example with Puppet's `template()` -function. - -## Using `--puppet-class` - -Most direct usage can simply use the same value for the role name and Puppet -class name: - -```shell -jinjaturtle php.ini --role-name php --template-engine erb -``` - -This creates Hiera keys such as: - -```yaml -php::memory_limit: 256M -``` - -and local ERB variables such as: - -```erb -<%= @memory_limit %> -``` - -For generated systems, it can be useful to make `--role-name` more specific -while keeping the Hiera keys under the real Puppet class. For example: - -```shell -jinjaturtle php.ini \ - --role-name php_etc_php_ini \ - --puppet-class php \ - --template-engine erb -``` - -In that case the variable prefix can stay file-specific, while the Hiera data -is still written under `php::...`. +By default, the Jinja2 template and the Ansible inventory are printed to +stdout. However, it is possible to output the results to new files. ## What sort of config files can it handle? -JinjaTurtle supports common structured and semi-structured config formats: +TOML, YAML, INI, JSON and XML-style config files should be okay. There are always +going to be some edge cases in very complex files that are difficult to work +with, though, so you may still find that you need to tweak the results. -- TOML -- YAML -- INI-style files -- JSON -- XML -- Postfix `main.cf` -- systemd unit files, such as `*.service`, `*.socket`, `*.timer`, and related - unit types -- OpenSSH-style config files, including `ssh_config`, `sshd_config`, and common - `*.conf` snippets detected as SSH config +For XML and YAML files, JinjaTurtle will attempt to generate 'for' loops +and lists in the Ansible yaml if the config file looks homogenous enough to +support it. However, if it lacks the confidence in this, it will fall back to +using scalar-style flattened attributes. -For ambiguous extensions such as `*.conf`, JinjaTurtle uses lightweight content -sniffing. You can always force a handler with `--format`. +You may need or wish to tidy up the config to suit your needs. -For YAML, XML, TOML, INI-style, and other supported structured files, -JinjaTurtle will attempt to generate loops when a repeated structure looks -homogeneous enough. If it is not confident, it falls back to flattened scalar -variables. - -Some very complex files will still need manual cleanup. The goal is to speed up -conversion into Jinja2 or ERB templates, not to guarantee a perfect final module -without review. - -## JSON, quoting, and type preservation - -JinjaTurtle tries to preserve rendered config types. - -For JSON, it uses JSON-aware expressions rather than plain string substitution. -This avoids generating invalid JSON such as: - -```json -{"enabled": True} -``` - -when the correct rendered JSON should be: - -```json -{"enabled": true} -``` - -In Jinja2 mode this uses Ansible-style JSON filters. In ERB mode it emits Ruby -JSON generation where required, for example: - -```erb -<% require 'json' -%> -{ - "enabled": <%= JSON.generate(@enabled) %> -} -``` - -That is expected for JSON ERB templates. +The goal here is really to *speed up* converting files into Ansible/Jinja2, +but not necessarily to make it perfect. ## Can I convert multiple files at once? -Yes. Pass a directory instead of a single file and JinjaTurtle will convert the -files it understands in that directory. +Certainly! Pass the folder name instead of a specific file name, and JinjaTurtle +will convert any files it understands in that folder, storing all the various +vars in the destination defaults yaml file, and converting each file into a +Jinja2 template per file type. -```shell -jinjaturtle ./config-dir \ - --role-name myrole \ - --defaults-output defaults/main.yml \ - --template-output templates/ -``` +If all the files had the same 'type', there'll be one Jinja2 template. -Use `--recursive` to recurse into subdirectories. +You can also pass `--recursive` to recurse into subfolders. -In folder mode, variables for multiple files of the same type are grouped under -an `items`-style structure in the generated YAML so that the resulting templates -can be used with loops in Ansible. - -For example: +Note: when using 'folder' mode and multiple files of the same type, their vars +will be listed under an 'items' parent key in the yaml, each with an `id` key. +You'll then want to use a `loop` in Ansible later, e.g: ```yaml - name: Render configs @@ -268,9 +93,9 @@ sudo dnf upgrade --refresh sudo dnf install jinjaturtle ``` -### From PyPI +### From PyPi -```bash +``` pip install jinjaturtle ``` @@ -278,81 +103,61 @@ pip install jinjaturtle Clone the repo and then run inside the clone: -```bash +``` poetry install ``` ### AppImage -Download the AppImage from the Releases page, make it executable, and put it on -your `$PATH`. +Download the AppImage from the Releases and make it executable, and put it +on your `$PATH`. + +## How to run it + +Say you have a `php.ini` file and you are in a directory structure like an +Ansible role (with subfolders `defaults` and `templates`): + +```shell +jinjaturtle php.ini \ + --role-name php \ + --defaults-output defaults/main.yml \ + --template-output templates/php.ini.j2 +``` ## Full usage info -```text -usage: jinjaturtle [-h] [-r ROLE_NAME] [--recursive] - [-f {ini,json,toml,yaml,xml,postfix,systemd,ssh}] - [-d DEFAULTS_OUTPUT] [-t TEMPLATE_OUTPUT] - [--template-engine {jinja2,erb}] - [--puppet-class PUPPET_CLASS] - config +``` +usage: jinjaturtle [-h] -r ROLE_NAME [-f {json,ini,toml,yaml,xml,postfix,systemd}] [-d DEFAULTS_OUTPUT] [-t TEMPLATE_OUTPUT] config -Convert a config file into an Ansible defaults file and Jinja2 template. +Convert a config file into Ansible inventory and a Jinja2 template. positional arguments: - config Path to a config file OR a folder containing supported - config files. Supported: .toml, .yaml/.yml, .json, - .ini/.cfg/.conf, .xml, ssh_config/sshd_config + config Path to the source configuration file. options: -h, --help show this help message and exit -r, --role-name ROLE_NAME - Role name / variable prefix. In Jinja2 mode this is - usually the Ansible role name. In ERB mode it is used - as the local variable prefix. Defaults to jinjaturtle. - --recursive When CONFIG is a folder, recurse into subfolders. - -f, --format {ini,json,toml,yaml,xml,postfix,systemd,ssh} - Force config format instead of auto-detecting from - filename. + Ansible role name, used as variable prefix (e.g. cometbft). + -f, --format {ini,json,toml,xml} + Force config format instead of auto-detecting from filename. -d, --defaults-output DEFAULTS_OUTPUT - Path to write the generated variable YAML. If omitted, - it is printed to stdout. + Path to write defaults/main.yml. If omitted, default vars are printed to stdout. -t, --template-output TEMPLATE_OUTPUT - Path to write the generated config template. If omitted, - it is printed to stdout. - --template-engine {jinja2,erb} - Template syntax to generate. Defaults to jinja2. Use - erb for Puppet templates. - --puppet-class PUPPET_CLASS - Puppet class / Hiera namespace to use with - --template-engine erb. Defaults to --role-name. + Path to write the Jinja2 config template. If omitted, template is printed to stdout. ``` ## Additional supported formats -JinjaTurtle also templates some common bespoke config formats: +JinjaTurtle can also template some common "bespoke" config formats: - **Postfix main.cf** (`main.cf`) → `--format postfix` - **systemd unit files** (`*.service`, `*.socket`, etc.) → `--format systemd` -- **OpenSSH config** (`ssh_config`, `sshd_config`, and detected snippets) → - `--format ssh` -For ambiguous extensions like `*.conf`, JinjaTurtle uses lightweight content -sniffing. You can always force a specific handler with `--format`. +For ambiguous extensions like `*.conf`, JinjaTurtle uses lightweight content sniffing; you can always force a specific handler via `--format`. -## Relationship with Enroll - -JinjaTurtle can be used directly, but it is also useful as a helper for tools -that generate configuration-management code. - -Enroll can call JinjaTurtle when it is available on the `PATH`. In that setup, -Enroll decides where Puppet, Ansible, or Salt files belong, while JinjaTurtle -concentrates on converting harvested config files into templates plus variable -data. ## Found a bug, have a suggestion? -You can e-mail me; see `pyproject.toml` for details. You can also contact me on -the Fediverse: +You can e-mail me (see the pyproject.toml for details) or contact me on the Fediverse: https://goto.mig5.net/@mig5 diff --git a/debian/changelog b/debian/changelog index 1950637..44d4e28 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,9 +1,3 @@ -jinjaturtle (0.5.5) unstable; urgency=medium - - * erb support - - -- Miguel Jacq Sat, 20 Jun 2026 18:27:00 +1000 - jinjaturtle (0.5.4) unstable; urgency=medium * Make templates more faithful to the original file in terms of indentation, newlines, no deserialisation of things like < or >. diff --git a/pyproject.toml b/pyproject.toml index cbc30c8..1279f81 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [tool.poetry] name = "jinjaturtle" -version = "0.5.5" +version = "0.5.4" description = "Convert config files into Ansible defaults and Jinja2 templates." authors = ["Miguel Jacq "] license = "GPL-3.0-or-later" diff --git a/rpm/jinjaturtle.spec b/rpm/jinjaturtle.spec index f354c18..a303b15 100644 --- a/rpm/jinjaturtle.spec +++ b/rpm/jinjaturtle.spec @@ -1,4 +1,4 @@ -%global upstream_version 0.5.5 +%global upstream_version 0.5.4 Name: jinjaturtle Version: %{upstream_version} @@ -43,8 +43,6 @@ Convert config files into Ansible defaults and Jinja2 templates. %changelog * Sat Jun 20 2026 Miguel Jacq - %{version}-%{release} -- erb support -* Sat Jun 20 2026 Miguel Jacq - %{version}-%{release} - Make templates more faithful to the original file in terms of indentation, newlines, no deserialisation of things like < or >. - More test coverage * Fri Jun 19 2026 Miguel Jacq - %{version}-%{release} diff --git a/src/jinjaturtle/cli.py b/src/jinjaturtle/cli.py index 7658bfb..82b6775 100644 --- a/src/jinjaturtle/cli.py +++ b/src/jinjaturtle/cli.py @@ -5,15 +5,12 @@ import sys from defusedxml import defuse_stdlib from pathlib import Path -from . import j2 from .core import ( parse_config, analyze_loops, flatten_config, generate_ansible_yaml, generate_jinja2_template, - generate_puppet_hiera_yaml, - generate_erb_template, ) from .multi import process_directory @@ -56,21 +53,7 @@ def _build_arg_parser() -> argparse.ArgumentParser: ap.add_argument( "-t", "--template-output", - help="Path to write the generated config template. If omitted, template is printed to stdout.", - ) - ap.add_argument( - "--template-engine", - choices=[j2.NAME, "erb"], - default=j2.NAME, - help="Template syntax to generate (default: jinja2). Use erb for Puppet templates.", - ) - ap.add_argument( - "--puppet-class", - help=( - "Puppet class/Hiera namespace to use with --template-engine erb. " - "Defaults to --role-name. This lets tools use a file-specific " - "variable prefix while writing Hiera keys under the real Puppet class." - ), + help="Path to write the Jinja2 config template. If omitted, template is printed to stdout.", ) return ap @@ -95,21 +78,6 @@ def _main(argv: list[str] | None = None) -> int: print("# defaults/main.yml") print(defaults_yaml, end="") - # Optionally translate folder-mode templates to ERB. Folder mode keeps - # the existing data shape; single-file mode below is the preferred - # Puppet path because it can produce class-parameter Hiera keys. - if args.template_engine == "erb": - from .erb import translate_jinja2_to_erb - - for o in outputs: - o.template = translate_jinja2_to_erb( - o.template, - role_prefix=args.role_name, - puppet_class=args.puppet_class or args.role_name, - ) - - template_ext = "erb" if args.template_engine == "erb" else j2.TEMPLATE_EXTENSION - # Write templates if args.template_output: out_path = Path(args.template_output) @@ -118,16 +86,12 @@ def _main(argv: list[str] | None = None) -> int: else: out_path.mkdir(parents=True, exist_ok=True) for o in outputs: - (out_path / f"config.{o.fmt}.{template_ext}").write_text( + (out_path / f"config.{o.fmt}.j2").write_text( o.template, encoding="utf-8" ) else: for o in outputs: - name = ( - f"config.{template_ext}" - if len(outputs) == 1 - else f"config.{o.fmt}.{template_ext}" - ) + name = "config.j2" if len(outputs) == 1 else f"config.{o.fmt}.j2" print(f"# {name}") print(o.template, end="") @@ -145,36 +109,17 @@ def _main(argv: list[str] | None = None) -> int: # Flatten config (excluding loop paths if loops are detected) flat_items = flatten_config(fmt, parsed, loop_candidates) - if args.template_engine == "erb": - ansible_yaml = generate_puppet_hiera_yaml( - args.role_name, - flat_items, - loop_candidates, - puppet_class=args.puppet_class or args.role_name, - ) - template_str = generate_erb_template( - fmt, - parsed, - args.role_name, - original_text=config_text, - loop_candidates=loop_candidates, - flat_items=flat_items, - puppet_class=args.puppet_class or args.role_name, - ) - else: - # Generate defaults YAML (with loop collections if detected) - ansible_yaml = generate_ansible_yaml( - args.role_name, flat_items, loop_candidates - ) + # Generate defaults YAML (with loop collections if detected) + ansible_yaml = generate_ansible_yaml(args.role_name, flat_items, loop_candidates) - # Generate template (with loops if detected) - template_str = generate_jinja2_template( - fmt, - parsed, - args.role_name, - original_text=config_text, - loop_candidates=loop_candidates, - ) + # Generate template (with loops if detected) + template_str = generate_jinja2_template( + fmt, + parsed, + args.role_name, + original_text=config_text, + loop_candidates=loop_candidates, + ) if args.defaults_output: Path(args.defaults_output).write_text(ansible_yaml, encoding="utf-8") @@ -185,11 +130,7 @@ def _main(argv: list[str] | None = None) -> int: if args.template_output: Path(args.template_output).write_text(template_str, encoding="utf-8") else: - print( - "# config.erb" - if args.template_engine == "erb" - else f"# config.{j2.TEMPLATE_EXTENSION}" - ) + print("# config.j2") print(template_str, end="") return 0 diff --git a/src/jinjaturtle/core.py b/src/jinjaturtle/core.py index 673aad0..ff6a586 100644 --- a/src/jinjaturtle/core.py +++ b/src/jinjaturtle/core.py @@ -8,7 +8,6 @@ import re import yaml from .loop_analyzer import LoopAnalyzer, LoopCandidate -from .erb import puppet_class_name, puppet_local_var_name, translate_jinja2_to_erb from .handlers import ( BaseHandler, IniHandler, @@ -388,85 +387,6 @@ def generate_jinja2_template( ) -def _template_variable_names( - role_prefix: str, - flat_items: list[tuple[tuple[str, ...], Any]], - loop_candidates: list[LoopCandidate] | None = None, -) -> set[str]: - names = {make_var_name(role_prefix, path) for path, _value in flat_items} - if loop_candidates: - for candidate in loop_candidates: - names.add(make_var_name(role_prefix, candidate.path)) - return names - - -def generate_puppet_hiera_yaml( - role_prefix: str, - flat_items: list[tuple[tuple[str, ...], Any]], - loop_candidates: list[LoopCandidate] | None = None, - *, - puppet_class: str | None = None, -) -> str: - """Create Puppet Hiera data suitable for Automatic Parameter Lookup. - - ``role_prefix`` remains the source variable prefix used by JinjaTurtle while - ``puppet_class`` is the Puppet class/Hiera namespace. In the normal case - they are the same, so ``php_memory_limit`` becomes ``php::memory_limit``. - Enroll may pass a file-specific role prefix and a separate Puppet class to - avoid parameter-name collisions inside one generated Puppet module. - """ - - klass = puppet_class_name(puppet_class or role_prefix) - data: dict[str, Any] = {} - - for path, value in flat_items: - generated = make_var_name(role_prefix, path) - local = puppet_local_var_name(role_prefix, generated, puppet_class=klass) - data[f"{klass}::{local}"] = value - - if loop_candidates: - for candidate in loop_candidates: - generated = make_var_name(role_prefix, candidate.path) - local = puppet_local_var_name(role_prefix, generated, puppet_class=klass) - data[f"{klass}::{local}"] = candidate.items - - return dump_yaml(data, sort_keys=True) - - -def generate_erb_template( - fmt: str, - parsed: Any, - role_prefix: str, - *, - original_text: str | None = None, - loop_candidates: list[LoopCandidate] | None = None, - flat_items: list[tuple[tuple[str, ...], Any]] | None = None, - puppet_class: str | None = None, -) -> str: - """Generate a Puppet ERB template from JinjaTurtle's renderer-neutral data. - - The first implementation intentionally translates the Jinja2 subset emitted - by JinjaTurtle's existing format handlers. This keeps parsing, formatting - preservation, and loop detection identical between Jinja2 and ERB output - while still producing Puppet-native ``@parameter`` references. - """ - - jinja_template = generate_jinja2_template( - fmt, - parsed, - role_prefix, - original_text=original_text, - loop_candidates=loop_candidates, - ) - names = _template_variable_names(role_prefix, flat_items or [], loop_candidates) - return translate_jinja2_to_erb( - jinja_template, - role_prefix=role_prefix, - puppet_class=puppet_class or role_prefix, - variable_names=names, - ) - - def _stringify_timestamps(obj: Any) -> Any: """ Recursively walk a parsed config and turn any datetime/date/time objects diff --git a/src/jinjaturtle/erb.py b/src/jinjaturtle/erb.py deleted file mode 100644 index 445ec51..0000000 --- a/src/jinjaturtle/erb.py +++ /dev/null @@ -1,218 +0,0 @@ -from __future__ import annotations - -import re - - -def _safe_name(raw: str, *, fallback: str = "var") -> str: - text = re.sub(r"[^A-Za-z0-9_]+", "_", str(raw or fallback)).strip("_").lower() - text = re.sub(r"_+", "_", text) - if not text: - text = fallback - if not re.match(r"^[a-z_]", text): - text = f"{fallback}_{text}" - return text - - -def puppet_class_name(raw: str) -> str: - """Return a conservative Puppet class/Hiera namespace name.""" - - text = _safe_name(raw, fallback="jinjaturtle") - if not re.match(r"^[a-z]", text): - text = f"jinjaturtle_{text}" - return text - - -def _role_prefix_name(raw: str) -> str: - return _safe_name(raw, fallback="jinjaturtle") - - -def puppet_local_var_name( - role_prefix: str, - jinja_var_name: str, - *, - puppet_class: str | None = None, -) -> str: - """Map a generated JinjaTurtle variable to a Puppet class parameter. - - For the common case where ``--role-name php`` also means class ``php``, a - generated Jinja variable such as ``php_memory_limit`` becomes Puppet local - parameter ``memory_limit`` and Hiera key ``php::memory_limit``. - - Enroll sometimes needs a file-specific variable prefix to avoid collisions - inside a generated module. When ``puppet_class`` differs from - ``role_prefix`` we keep the full generated variable name as the local - parameter and only use ``puppet_class`` as the Hiera namespace. - """ - - var_name = _safe_name(jinja_var_name, fallback="value") - prefix = _role_prefix_name(role_prefix) - klass = puppet_class_name(puppet_class or role_prefix) - if klass == prefix and var_name.startswith(prefix + "_"): - stripped = var_name[len(prefix) + 1 :] - return stripped or var_name - return var_name - - -class ErbTranslator: - """Translate the Jinja2 subset emitted by JinjaTurtle into Puppet ERB.""" - - _TOKEN_RE = re.compile(r"({{.*?}}|{%.*?%})", re.S) - - def __init__( - self, - *, - role_prefix: str, - puppet_class: str | None = None, - variable_names: set[str] | None = None, - ) -> None: - self.role_prefix = role_prefix - self.puppet_class = puppet_class or role_prefix - self.variable_names = set(variable_names or set()) - self.loop_stack: list[tuple[str, str, str]] = [] - self.needs_json = False - - def translate(self, template_text: str) -> str: - parts = self._TOKEN_RE.split(template_text) - out: list[str] = [] - for token in parts: - if not token: - continue - if token.startswith("{{") and token.endswith("}}"): - expr = token[2:-2].strip() - out.append(f"<%= {self.expr_to_ruby(expr)} %>") - continue - if token.startswith("{%") and token.endswith("%}"): - stmt = token[2:-2].strip() - out.append(self.statement_to_erb(stmt)) - continue - out.append(token) - - rendered = "".join(out) - if self.needs_json and "require 'json'" not in rendered: - rendered = "<% require 'json' -%>\n" + rendered - return rendered - - def local_var(self, name: str) -> str: - return puppet_local_var_name( - self.role_prefix, name, puppet_class=self.puppet_class - ) - - def ruby_value(self, expr: str) -> str: - expr = expr.strip() - if expr in {"true", "True"}: - return "true" - if expr in {"false", "False"}: - return "false" - if expr in {"none", "None", "null"}: - return "nil" - if re.match(r"^[A-Za-z_][A-Za-z0-9_]*$", expr): - if any(expr == loop_var for loop_var, _idx, _coll in self.loop_stack): - return expr - return f"@{self.local_var(expr)}" - m = re.match(r"^([A-Za-z_][A-Za-z0-9_]*)\.([A-Za-z_][A-Za-z0-9_]*)$", expr) - if m: - base, key = m.groups() - if any(base == loop_var for loop_var, _idx, _coll in self.loop_stack): - return f"{base}[{key!r}]" - return f"@{self.local_var(base)}[{key!r}]" - return expr - - def expr_to_ruby(self, expr: str) -> str: - expr = expr.strip() - - # JinjaTurtle emits these YAML-preserving ternaries for booleans/nulls. - m = re.match( - r"^(['\"])(true|false)\1\s+if\s+([A-Za-z_][A-Za-z0-9_\.]*)\s+else\s+(['\"])(true|false)\4$", - expr, - ) - if m: - truthy = m.group(2) - cond = self.ruby_value(m.group(3)) - falsy = m.group(5) - return f"{cond} ? {truthy!r} : {falsy!r}" - - m = re.match( - r"^(['\"])(null)\1\s+if\s+([A-Za-z_][A-Za-z0-9_\.]*)\s+is\s+none\s+else\s+([A-Za-z_][A-Za-z0-9_\.]*)$", - expr, - ) - if m: - value = self.ruby_value(m.group(3)) - fallback = self.ruby_value(m.group(4)) - return f"{value}.nil? ? 'null' : {fallback}" - - if "|" in expr: - base, *filters = [part.strip() for part in expr.split("|")] - ruby = self.ruby_value(base) - for filt in filters: - if filt.startswith("lower"): - ruby = f"{ruby}.to_s.downcase" - elif filt.startswith("to_json") or filt.startswith("tojson"): - self.needs_json = True - if "indent" in filt: - ruby = f"JSON.pretty_generate({ruby})" - else: - ruby = f"JSON.generate({ruby})" - return ruby - - return self.ruby_value(expr) - - def statement_to_erb(self, stmt: str) -> str: - if stmt.endswith(("-", "+")): - stmt = stmt[:-1].rstrip() - - if stmt.startswith("for "): - m = re.match( - r"^for\s+([A-Za-z_][A-Za-z0-9_]*)\s+in\s+([A-Za-z_][A-Za-z0-9_]*)$", - stmt, - ) - if m: - loop_var, collection = m.groups() - idx_var = f"__jt_idx_{len(self.loop_stack)}" - collection_ruby = self.ruby_value(collection) - self.loop_stack.append((loop_var, idx_var, collection_ruby)) - return f"<% {collection_ruby}.each_with_index do |{loop_var}, {idx_var}| -%>" - - if stmt == "endfor": - if self.loop_stack: - self.loop_stack.pop() - return "<% end %>" - - if stmt.startswith("if "): - cond = stmt[3:].strip() - if cond == "not loop.last" and self.loop_stack: - _loop_var, idx_var, collection_ruby = self.loop_stack[-1] - return f"<% if {idx_var} < ({collection_ruby}.length - 1) -%>" - m = re.match(r"^([A-Za-z_][A-Za-z0-9_\.]*)\s+is\s+defined$", cond) - if m: - return f"<% unless {self.ruby_value(m.group(1))}.nil? -%>" - m = re.match(r"^([A-Za-z_][A-Za-z0-9_\.]*)\s+is\s+none$", cond) - if m: - return f"<% if {self.ruby_value(m.group(1))}.nil? -%>" - return f"<% if {self.expr_to_ruby(cond)} -%>" - - if stmt == "else": - return "<% else -%>" - - if stmt.startswith("elif "): - return f"<% elsif {self.expr_to_ruby(stmt[5:].strip())} -%>" - - if stmt == "endif": - return "<% end -%>" - - # Preserve unknown Jinja statements visibly as an ERB comment so the - # generated template does not contain invalid Jinja syntax. - return f"<%# Unsupported JinjaTurtle statement: {stmt} %>" - - -def translate_jinja2_to_erb( - template_text: str, - *, - role_prefix: str, - puppet_class: str | None = None, - variable_names: set[str] | None = None, -) -> str: - return ErbTranslator( - role_prefix=role_prefix, - puppet_class=puppet_class, - variable_names=variable_names, - ).translate(template_text) diff --git a/src/jinjaturtle/handlers/ini.py b/src/jinjaturtle/handlers/ini.py index 1aa5e22..ad92b72 100644 --- a/src/jinjaturtle/handlers/ini.py +++ b/src/jinjaturtle/handlers/ini.py @@ -5,7 +5,6 @@ from pathlib import Path from typing import Any from . import BaseHandler -from .. import j2 class IniHandler(BaseHandler): @@ -64,9 +63,9 @@ class IniHandler(BaseHandler): var_name = self.make_var_name(role_prefix, path) value = value.strip() if len(value) >= 2 and value[0] == value[-1] and value[0] in {'"', "'"}: - lines.append(f"{key} = {j2.quoted_variable(var_name)}") + lines.append(f'{key} = "{{{{ {var_name} }}}}"') else: - lines.append(f"{key} = {j2.variable(var_name)}") + lines.append(f"{key} = {{{{ {var_name} }}}}") lines.append("") return "\n".join(lines).rstrip() + "\n" @@ -142,9 +141,9 @@ class IniHandler(BaseHandler): if use_quotes: quote_char = raw_value[0] - replacement_value = j2.quoted_variable(var_name, quote_char) + replacement_value = f"{quote_char}{{{{ {var_name} }}}}{quote_char}" else: - replacement_value = j2.variable(var_name) + replacement_value = f"{{{{ {var_name} }}}}" new_content = ( before_eq + "=" + leading_ws + replacement_value + comment_part diff --git a/src/jinjaturtle/handlers/json.py b/src/jinjaturtle/handlers/json.py index 064535a..9d763a4 100644 --- a/src/jinjaturtle/handlers/json.py +++ b/src/jinjaturtle/handlers/json.py @@ -6,7 +6,6 @@ from pathlib import Path from typing import Any from . import DictLikeHandler -from .. import j2 from ..loop_analyzer import LoopCandidate @@ -32,7 +31,7 @@ class JsonHandler(DictLikeHandler): return self._generate_json_template(role_prefix, parsed) JSON_INDENT = 2 - JSON_VALUE_FILTER = j2.JSON_VALUE_FILTER + JSON_VALUE_FILTER = "to_json(ensure_ascii=False)" def _leading_indent(self, s: str, idx: int) -> int: """Return the number of leading spaces on the line containing idx.""" @@ -86,7 +85,7 @@ class JsonHandler(DictLikeHandler): useful in HTML, but noisy in configuration files. JinjaTurtle generates Ansible templates, so use Ansible's ``to_json`` filter instead. """ - return j2.filtered(var_name, self.JSON_VALUE_FILTER) + return f"{{{{ {var_name} | {self.JSON_VALUE_FILTER} }}}}" def _generate_json_template_from_text(self, role_prefix: str, text: str) -> str: """Replace JSON scalar values in-place, preserving original formatting. @@ -331,10 +330,10 @@ class JsonHandler(DictLikeHandler): # a blank line between iterations under default Jinja whitespace settings. return ( f"[\n" - f"{j2.for_start(item_var, collection_var)}" - f"{inner}{j2.to_json(item_var)}" - f"{j2.if_not_loop_last()},{j2.endif()}\n" - f"{j2.for_end()}{base}]" + f"{{% for {item_var} in {collection_var} %}}" + f"{inner}{{{{ {item_var} | to_json(ensure_ascii=False) }}}}" + f"{{% if not loop.last %}},{{% endif %}}\n" + f"{{% endfor %}}{base}]" ) def _generate_json_dict_loop( @@ -365,15 +364,16 @@ class JsonHandler(DictLikeHandler): for i, key in enumerate(keys): comma = "," if i < len(keys) - 1 else "" dict_lines.append( - f'{field}"{key}": ' f"{j2.to_json(f'{item_var}.{key}')}{comma}" + f'{field}"{key}": ' + f"{{{{ {item_var}.{key} | to_json(ensure_ascii=False) }}}}{comma}" ) # Comma between *items* goes after the closing brace. - dict_lines.append(f"{inner}}}{j2.if_not_loop_last()},{j2.endif()}") + dict_lines.append(f"{inner}}}{{% if not loop.last %}},{{% endif %}}") dict_body = "\n".join(dict_lines) # Put the `{% for %}` at the start of the first item line to avoid blank lines. return ( f"[\n" - f"{j2.for_start(item_var, collection_var)}{inner}{dict_body}\n" - f"{j2.for_end()}{base}]" + f"{{% for {item_var} in {collection_var} %}}{inner}{dict_body}\n" + f"{{% endfor %}}{base}]" ) diff --git a/src/jinjaturtle/handlers/postfix.py b/src/jinjaturtle/handlers/postfix.py index e2e442a..65f6be9 100644 --- a/src/jinjaturtle/handlers/postfix.py +++ b/src/jinjaturtle/handlers/postfix.py @@ -4,7 +4,6 @@ from pathlib import Path from typing import Any from . import BaseHandler -from .. import j2 class PostfixMainHandler(BaseHandler): @@ -93,7 +92,7 @@ class PostfixMainHandler(BaseHandler): lines: list[str] = [] for k, v in parsed.items(): var = self.make_var_name(role_prefix, (k,)) - lines.append(f"{k} = {j2.variable(var)}") + lines.append(f"{k} = {{{{ {var} }}}}") return "\n".join(lines).rstrip() + "\n" return self._generate_from_text(role_prefix, original_text) @@ -165,13 +164,11 @@ class PostfixMainHandler(BaseHandler): quoted = len(v) >= 2 and v[0] == v[-1] and v[0] in {'"', "'"} if quoted: replacement = ( - f"{before_eq}={leading_ws}{j2.quoted_variable(var)}" - f"{comment_part}{newline}" + f'{before_eq}={leading_ws}"{{{{ {var} }}}}"{comment_part}{newline}' ) else: replacement = ( - f"{before_eq}={leading_ws}{j2.variable(var)}" - f"{comment_part}{newline}" + f"{before_eq}={leading_ws}{{{{ {var} }}}}{comment_part}{newline}" ) out_lines.append(replacement) diff --git a/src/jinjaturtle/handlers/ssh.py b/src/jinjaturtle/handlers/ssh.py index a6c0d2f..bedaa2a 100644 --- a/src/jinjaturtle/handlers/ssh.py +++ b/src/jinjaturtle/handlers/ssh.py @@ -6,7 +6,6 @@ from pathlib import Path from typing import Any from . import BaseHandler -from .. import j2 _SECTION_KEYWORDS = {"host", "match"} @@ -266,9 +265,9 @@ class SshConfigHandler(BaseHandler): var = self.make_var_name(role_prefix, path) if ln.quoted and ln.value: quote_char = ln.value[0] - replacement_value = j2.quoted_variable(var, quote_char) + replacement_value = f"{quote_char}{{{{ {var} }}}}{quote_char}" else: - replacement_value = j2.variable(var) + replacement_value = f"{{{{ {var} }}}}" rendered = ( f"{ln.before_value}{replacement_value}" diff --git a/src/jinjaturtle/handlers/systemd.py b/src/jinjaturtle/handlers/systemd.py index 8118808..044fd86 100644 --- a/src/jinjaturtle/handlers/systemd.py +++ b/src/jinjaturtle/handlers/systemd.py @@ -5,7 +5,6 @@ from pathlib import Path from typing import Any from . import BaseHandler -from .. import j2 @dataclass @@ -168,15 +167,9 @@ class SystemdUnitHandler(BaseHandler): v = (ln.value or "").strip() quoted = len(v) >= 2 and v[0] == v[-1] and v[0] in {'"', "'"} if quoted: - repl = ( - f"{ln.before_eq}={ln.leading_ws_after_eq}" - f"{j2.quoted_variable(var)}{ln.comment}" - ) + repl = f'{ln.before_eq}={ln.leading_ws_after_eq}"{{{{ {var} }}}}"{ln.comment}' else: - repl = ( - f"{ln.before_eq}={ln.leading_ws_after_eq}" - f"{j2.variable(var)}{ln.comment}" - ) + repl = f"{ln.before_eq}={ln.leading_ws_after_eq}{{{{ {var} }}}}{ln.comment}" newline = "\n" if ln.raw.endswith("\n") else "" out_lines.append(repl + newline) diff --git a/src/jinjaturtle/handlers/toml.py b/src/jinjaturtle/handlers/toml.py index 15d5a2a..ba4bbb4 100644 --- a/src/jinjaturtle/handlers/toml.py +++ b/src/jinjaturtle/handlers/toml.py @@ -4,7 +4,6 @@ from pathlib import Path from typing import Any from . import DictLikeHandler -from .. import j2 from ..loop_analyzer import LoopCandidate try: @@ -17,14 +16,6 @@ class TomlHandler(DictLikeHandler): fmt = "toml" flatten_lists = False # keep lists as scalars - def _toml_value_expr(self, var_name: str, value: Any | None = None) -> str: - if isinstance(value, bool): - return j2.lower(var_name) - return j2.variable(var_name) - - def _toml_quoted_expr(self, var_name: str, quote: str = '"') -> str: - return j2.quoted_variable(var_name, quote) - def parse(self, path: Path) -> Any: if tomllib is None: raise RuntimeError( @@ -77,12 +68,12 @@ class TomlHandler(DictLikeHandler): def emit_kv(path: tuple[str, ...], key: str, value: Any) -> None: var_name = self.make_var_name(role_prefix, path + (key,)) if isinstance(value, str): - lines.append(f"{key} = {self._toml_quoted_expr(var_name)}") + lines.append(f'{key} = "{{{{ {var_name} }}}}"') elif isinstance(value, bool): # Booleans need | lower filter (Python True/False → TOML true/false) - lines.append(f"{key} = {self._toml_value_expr(var_name, value)}") + lines.append(f"{key} = {{{{ {var_name} | lower }}}}") else: - lines.append(f"{key} = {self._toml_value_expr(var_name, value)}") + lines.append(f"{key} = {{{{ {var_name} }}}}") def walk(obj: dict[str, Any], path: tuple[str, ...] = ()) -> None: scalar_items = {k: v for k, v in obj.items() if not isinstance(v, dict)} @@ -130,10 +121,10 @@ class TomlHandler(DictLikeHandler): def emit_kv(path: tuple[str, ...], key: str, value: Any) -> None: var_name = self.make_var_name(role_prefix, path + (key,)) if isinstance(value, str): - lines.append(f"{key} = {self._toml_quoted_expr(var_name)}") + lines.append(f'{key} = "{{{{ {var_name} }}}}"') elif isinstance(value, bool): # Booleans need | lower filter (Python True/False → TOML true/false) - lines.append(f"{key} = {self._toml_value_expr(var_name, value)}") + lines.append(f"{key} = {{{{ {var_name} | lower }}}}") elif isinstance(value, list): # Check if this list is a loop candidate if path + (key,) in loop_paths: @@ -148,21 +139,24 @@ class TomlHandler(DictLikeHandler): # Scalar list loop lines.append( f"{key} = [" - f"{j2.for_start(item_var, collection_var)}" - f"{j2.variable(item_var)}" - f"{j2.if_not_loop_last()}, {j2.endif()}" - f"{j2.for_end()}" + f"{{% for {item_var} in {collection_var} %}}" + f"{{{{ {item_var} }}}}" + f"{{% if not loop.last %}}, {{% endif %}}" + f"{{% endfor %}}" f"]" ) elif candidate.item_schema in ("simple_dict", "nested"): # Dict list loop - TOML array of tables # This is complex for TOML, using simplified approach - lines.append(f"{key} = " f"{j2.to_json(var_name)}") + lines.append( + f"{key} = " + f"{{{{ {var_name} | to_json(ensure_ascii=False) }}}}" + ) else: # Not a loop, treat as regular variable - lines.append(f"{key} = {self._toml_value_expr(var_name, value)}") + lines.append(f"{key} = {{{{ {var_name} }}}}") else: - lines.append(f"{key} = {self._toml_value_expr(var_name, value)}") + lines.append(f"{key} = {{{{ {var_name} }}}}") def walk(obj: dict[str, Any], path: tuple[str, ...] = ()) -> None: scalar_items = {k: v for k, v in obj.items() if not isinstance(v, dict)} @@ -288,17 +282,13 @@ class TomlHandler(DictLikeHandler): nested_path = path + (sub_key,) nested_var = self.make_var_name(role_prefix, nested_path) if isinstance(sub_val, str): - inner_bits.append( - f"{sub_key} = {self._toml_quoted_expr(nested_var)}" - ) + inner_bits.append(f'{sub_key} = "{{{{ {nested_var} }}}}"') elif isinstance(sub_val, bool): inner_bits.append( - f"{sub_key} = {self._toml_value_expr(nested_var, sub_val)}" + f"{sub_key} = {{{{ {nested_var} | lower }}}}" ) else: - inner_bits.append( - f"{sub_key} = {self._toml_value_expr(nested_var, sub_val)}" - ) + inner_bits.append(f"{sub_key} = {{{ {nested_var} }}}") replacement_value = "{ " + ", ".join(inner_bits) + " }" new_content = ( before_eq + "=" + leading_ws + replacement_value + comment_part @@ -320,11 +310,11 @@ class TomlHandler(DictLikeHandler): if use_quotes: quote_char = raw_value[0] - replacement_value = self._toml_quoted_expr(var_name, quote_char) + replacement_value = f"{quote_char}{{{{ {var_name} }}}}{quote_char}" elif is_bool: - replacement_value = j2.lower(var_name) + replacement_value = f"{{{{ {var_name} | lower }}}}" else: - replacement_value = j2.variable(var_name) + replacement_value = f"{{{{ {var_name} }}}}" new_content = ( before_eq + "=" + leading_ws + replacement_value + comment_part @@ -402,7 +392,7 @@ class TomlHandler(DictLikeHandler): # Build loop out_lines.append( - f"{j2.for_start(item_var, collection_var)}\n" + f"{{% for {item_var} in {collection_var} %}}\n" ) out_lines.append(f"[[{'.'.join(table_path)}]]\n") @@ -412,21 +402,14 @@ class TomlHandler(DictLikeHandler): continue if isinstance(value, str): out_lines.append( - f"{key} = " - f"{self._toml_quoted_expr(f'{item_var}.{key}')}\n" - ) - elif isinstance(value, bool): - out_lines.append( - f"{key} = " - f"{self._toml_value_expr(f'{item_var}.{key}', value)}\n" + f'{key} = "{{{{ {item_var}.{key} }}}}"\n' ) else: out_lines.append( - f"{key} = " - f"{self._toml_value_expr(f'{item_var}.{key}', value)}\n" + f"{key} = {{{{ {item_var}.{key} }}}}\n" ) - out_lines.append(f"{j2.for_end()}\n") + out_lines.append("{% endfor %}\n") # Skip all content until the next different table skip_until_next_table = True @@ -490,15 +473,17 @@ class TomlHandler(DictLikeHandler): # Scalar list loop replacement_value = ( f"[" - f"{j2.for_start(item_var, collection_var)}" - f"{j2.variable(item_var)}" - f"{j2.if_not_loop_last()}, {j2.endif()}" - f"{j2.for_end()}" + f"{{% for {item_var} in {collection_var} %}}" + f"{{{{ {item_var} }}}}" + f"{{% if not loop.last %}}, {{% endif %}}" + f"{{% endfor %}}" f"]" ) else: # Dict/nested loop - use to_json filter for complex arrays - replacement_value = j2.to_json(collection_var) + replacement_value = ( + f"{{{{ {collection_var} | to_json(ensure_ascii=False) }}}}" + ) new_content = ( before_eq + "=" + leading_ws + replacement_value + comment_part @@ -525,17 +510,13 @@ class TomlHandler(DictLikeHandler): nested_path = path + (sub_key,) nested_var = self.make_var_name(role_prefix, nested_path) if isinstance(sub_val, str): - inner_bits.append( - f"{sub_key} = {self._toml_quoted_expr(nested_var)}" - ) + inner_bits.append(f'{sub_key} = "{{{{ {nested_var} }}}}"') elif isinstance(sub_val, bool): inner_bits.append( - f"{sub_key} = {self._toml_value_expr(nested_var, sub_val)}" + f"{sub_key} = {{{{ {nested_var} | lower }}}}" ) else: - inner_bits.append( - f"{sub_key} = {self._toml_value_expr(nested_var, sub_val)}" - ) + inner_bits.append(f"{sub_key} = {{{{ {nested_var} }}}}") replacement_value = "{ " + ", ".join(inner_bits) + " }" new_content = ( before_eq + "=" + leading_ws + replacement_value + comment_part @@ -557,11 +538,11 @@ class TomlHandler(DictLikeHandler): if use_quotes: quote_char = raw_value[0] - replacement_value = self._toml_quoted_expr(var_name, quote_char) + replacement_value = f"{quote_char}{{{{ {var_name} }}}}{quote_char}" elif is_bool: - replacement_value = j2.lower(var_name) + replacement_value = f"{{{{ {var_name} | lower }}}}" else: - replacement_value = j2.variable(var_name) + replacement_value = f"{{{{ {var_name} }}}}" new_content = ( before_eq + "=" + leading_ws + replacement_value + comment_part diff --git a/src/jinjaturtle/handlers/xml.py b/src/jinjaturtle/handlers/xml.py index 51c7a73..fed6aba 100644 --- a/src/jinjaturtle/handlers/xml.py +++ b/src/jinjaturtle/handlers/xml.py @@ -6,7 +6,6 @@ from typing import Any import xml.etree.ElementTree as ET # nosec from .base import BaseHandler -from .. import j2 from ..loop_analyzer import LoopCandidate @@ -173,7 +172,7 @@ class XmlHandler(BaseHandler): for attr_name in list(elem.attrib.keys()): attr_path = path + (f"@{attr_name}",) var_name = self.make_var_name(role_prefix, attr_path) - elem.set(attr_name, j2.variable(var_name)) + elem.set(attr_name, f"{{{{ {var_name} }}}}") # Children children = [c for c in list(elem) if isinstance(c.tag, str)] @@ -186,7 +185,7 @@ class XmlHandler(BaseHandler): else: text_path = path + ("value",) var_name = self.make_var_name(role_prefix, text_path) - elem.text = j2.variable(var_name) + elem.text = f"{{{{ {var_name} }}}}" # Handle children - check for loops first counts = Counter(child.tag for child in children) @@ -340,12 +339,12 @@ class XmlHandler(BaseHandler): # Build loop result_lines.append( - f"{indent_str}{j2.for_start(item_var, collection_var)}" + f"{indent_str}{{% for {item_var} in {collection_var} %}}" ) # Add each line of the sample with proper indentation for sample_line in sample_lines: result_lines.append(f"{indent_str} {sample_line}") - result_lines.append(f"{indent_str}{j2.for_end()}") + result_lines.append(f"{indent_str}{{% endfor %}}") else: # Keep the marker if we can't find the candidate result_lines.append(line) @@ -361,11 +360,11 @@ class XmlHandler(BaseHandler): end = line.find("-->", start) condition = line[start:end] indent = len(line) - len(line.lstrip()) - final_lines.append(f"{' ' * indent}{j2.if_defined(condition)}") + final_lines.append(f"{' ' * indent}{{% if {condition} is defined %}}") # Replace with {% endif %} elif "