ae.managed_files

managed files

this portion of the ae namespace is creating or refreshing files from templates.

the update strategy can be individually configured for every template folder and file.

the engine to convert templates into destination files is designed to be easily extendable (see the templates module).

variations in the file content, like the name and version of a concrete project, can be dynamically replaced with actual values from individual context variables, either by embedded f-string-expressions or by replacers/placeholders. replacers are useful especially for Python code file templates because they are get syntactically interpreted as Python comments in the template file, and could get replaced by Python code statements in the resulting destination file.

use the function deploy_template() to convert a single template into a destination file. for bulk destination file deployments from multiple templates, use the TemplateMngr class.

Module Attributes

DEPLOY_LOCK_EXT	additional file ext; blocking the deployment of a template
PATH_PREFIXES_ARGS_SEP	seperator/suffix of template file/path prefixes arguments
F_STRINGS_PATH_PFX	file name prefix if template contains f-strings
OVERWRITABLE_BIN_TEMPLATE_PATH_PFX	template file name prefix of a binary file that will get overwritten if it exists at their destination.
OVERWRITABLE_TEMPLATE_PATH_PFX	template file name prefix of a text file that will get overwritten if it exists at their destination.
PUTTABLE_TEMPLATE_PATH_PFX	template file name prefix of a text file that will get created if it not exists at their destination and replaced/updated if it contains a REFRESHABLE_TEMPLATE_MARKER.
UPDATABLE_TEMPLATE_PATH_PFX	template file name prefix of an update-only text file, that get updated only if it already exists at their destination and contains a REFRESHABLE_TEMPLATE_MARKER.
STOP_PARSING_PATH_PFX	file name prefix to support template of template
REFRESHABLE_TEMPLATE_MARKER	to mark the content (header) of a refreshable/overwritable text file that gets updated/replaced from a template.
MANAGED_FILE_ERROR_COMMENT	managed file error comment marker
MANAGED_FILE_SKIP_COMMENT	managed file skip-reason comment marker
MANAGED_FILE_WARNING_COMMENT	managed file warning comment marker
TEMPLATE_PLACEHOLDER_ID_PREFIX	template replacers id prefix marker
TEMPLATE_PLACEHOLDER_ID_SUFFIX	template replacers id suffix marker
TEMPLATE_PLACEHOLDER_ARGS_SUFFIX	template replacers args suffix marker
TEMPLATE_INCLUDE_FILE_PLACEHOLDER_ID	replacers id of replace_with_file_content_or_default()
TEMPLATE_REPLACE_WITH_PLACEHOLDER_ID	replacers id of replace_with_template_args()
ContentTransformer	file content transformer function
ContentType	content type of managed file (None==file-not-exists)
ContextVars	template placeholder variables to be replaced by its value
OutdatedFilesPathsContents	type of the value returned by the property outdated_files of TemplateMngr
PathPrefixesParsers	registered path prefixes parsers
PathPrefixesArgCounts	path prefixes with its arg counts
PathPrefixesArgs	path prefixes with its arg values
Replacer	template content replacers function type
TemplateFiles	(patcher id, template file path, destination path prefixes)
TplVars	template placeholder variables to be replaced by its value
DEFAULT_PATH_PREFIXES_PARSERS	mapping of the default path prefixes parsers with to a tuple of the prefix args count and the parser callee.
DEFAULT_REPLACERS	map of default replacers callables used by patch_string().

Functions

deploy_template(template_file_path[, ...])	create/update a file from a template.
extend_content_with_marker(file_name, ...)	create/update the content of an overwritable text file with placeholders (compiled from a template file).
patch_string(content, tpl_vars, **replacers)	replace f-string / dynamic placeholders in content with variable values / return values of replacers callables.
path_pfx_overwritable_binary_content(mf, *_args)	path prefix callee for the OVERWRITABLE_BIN_TEMPLATE_PATH_PFX prefix.
path_pfx_overwritable_content(mf, *_args)	path prefix callee for the OVERWRITABLE_TEMPLATE_PATH_PFX prefix.
path_pfx_parametrize_with_context(mf, *_args)	path prefix callee for the F_STRINGS_PATH_PFX prefix.
path_pfx_puttable_content(mf, *_args)	path prefix callee for the PUTTABLE_TEMPLATE_PATH_PFX prefix.
path_pfx_updatable_content(mf, *_args)	path prefix callee for the UPDATABLE_TEMPLATE_PATH_PFX prefix.
prefix_parser(dst_path, prefixes_arg_counts)	detect path/file name prefixes including their prefix args returning stripped path/file name and prefixes args.
replace_with_file_content_or_default(args_str)	return file content if the file specified in first string arg exists, else return empty string or 2nd arg str.
replace_with_template_args(args_str)	template placeholder replacer function to hide uncompleted code from code-inspections/editor-warnings.
transform_parametrize_content(mf)	content transformer callee added via the F_STRINGS_PATH_PFX path prefix.
transform_puttable_content(mf)	content transformer callee added via the PUTTABLE_TEMPLATE_PATH_PFX path prefix.
transform_updatable_content(mf)	content transformer callee added via the UPDATABLE_TEMPLATE_PATH_PFX path prefix.

Classes

ManagedFile(manager, patcher, template_path)	represents a template/managed file
PathPrefixesFunc(*args, **kwargs)
TemplateMngr(template_files, prefix_parsers, ...)	checks/deploys/renews managed files from templates and context variables.

DEPLOY_LOCK_EXT = '.locked'

additional file ext; blocking the deployment of a template

PATH_PREFIXES_ARGS_SEP = '_'

seperator/suffix of template file/path prefixes arguments

F_STRINGS_PATH_PFX = 'fSt-'

file name prefix if template contains f-strings

OVERWRITABLE_BIN_TEMPLATE_PATH_PFX = 'Obi-'

template file name prefix of a binary file that will get overwritten if it exists at their destination.

OVERWRITABLE_TEMPLATE_PATH_PFX = 'Ovw-'

template file name prefix of a text file that will get overwritten if it exists at their destination.

PUTTABLE_TEMPLATE_PATH_PFX = 'PutMar-'

template file name prefix of a text file that will get created if it not exists at their destination and replaced/updated if it contains a REFRESHABLE_TEMPLATE_MARKER.

UPDATABLE_TEMPLATE_PATH_PFX = 'UpdMar-'

template file name prefix of an update-only text file, that get updated only if it already exists at their destination and contains a REFRESHABLE_TEMPLATE_MARKER.

STOP_PARSING_PATH_PFX = '_z_'

file name prefix to support template of template

REFRESHABLE_TEMPLATE_MARKER = 'THIS FILE IS EXCLUSIVELY MAINTAINED'

to mark the content (header) of a refreshable/overwritable text file that gets updated/replaced from a template.

MANAGED_FILE_ERROR_COMMENT = '* error: '

managed file error comment marker

MANAGED_FILE_SKIP_COMMENT = '- skip reason: '

managed file skip-reason comment marker

MANAGED_FILE_WARNING_COMMENT = '# '

managed file warning comment marker

TEMPLATE_PLACEHOLDER_ID_PREFIX = '# '

template replacers id prefix marker

TEMPLATE_PLACEHOLDER_ID_SUFFIX = '#('

template replacers id suffix marker

TEMPLATE_PLACEHOLDER_ARGS_SUFFIX = ')#'

template replacers args suffix marker

TEMPLATE_INCLUDE_FILE_PLACEHOLDER_ID = 'IncludeFile'

replacers id of replace_with_file_content_or_default()

TEMPLATE_REPLACE_WITH_PLACEHOLDER_ID = 'ReplaceWith'

replacers id of replace_with_template_args()

type ContentTransformer = Callable[[ManagedFile], str | bytes]

file content transformer function

type ContentType = str | bytes | None

content type of managed file (None==file-not-exists)

type ContextVars = dict[str, Any]

template placeholder variables to be replaced by its value

type OutdatedFilesPathsContents = list[tuple[str, ContentType, ContentType]]

type of the value returned by the property outdated_files of TemplateMngr

class PathPrefixesFunc(*args, **kwargs)

Bases: Protocol

__call__(managed_file, /, *path_prefix_args)

Call self as a function.

Return type:

None

__init__(*args, **kwargs)

__non_callable_proto_members__ = {}

__parameters__ = ()

__protocol_attrs__ = {'__call__'}

classmethod __subclasshook__(other)

Abstract classes can override this to customize issubclass().

This is invoked early on by abc.ABCMeta.__subclasscheck__(). It should return True, False or NotImplemented. If it returns NotImplemented, the normal algorithm is used. Otherwise, it overrides the normal algorithm (and the outcome is cached).

_is_protocol = True

_is_runtime_protocol = True

PathPrefixesParsers

registered path prefixes parsers

alias of dict[str, tuple[int, PathPrefixesFunc]]

PathPrefixesArgCounts

path prefixes with its arg counts

alias of Iterable[tuple[str, int]]

PathPrefixesArgs

path prefixes with its arg values

alias of list[tuple[str, tuple[str, …]]]

Replacer

template content replacers function type

alias of Callable[[str], str]

TemplateFiles

(patcher id, template file path, destination path prefixes)

alias of list[tuple[str, str, str]]

TplVars

template placeholder variables to be replaced by its value

alias of dict[str, Any]

class ManagedFile(manager, patcher, template_path, dst_path='.')

Bases: object

represents a template/managed file

__init__(manager, patcher, template_path, dst_path='.')

create new managed file instance

Parameters:

• manager (TemplateMngr) – TemplateMngr instance, to reference path prefixes, context vars and replacers.

• patcher (str) – templates collection (project) name and version (to be added into the destination file).

• template_path (str) – template/source file path.

• dst_path (str) – destination file path with optional path prefixes in its file and/or folder names.

_content_transformers: list[ContentTransformer]

comments: list[str]

to collect comments, errors and skip-reasons of this managed file

file_encoding: str | None

encoding of this managed file set via ManagedFile.add_content_transformer(); default=’bin-bytes’

file_content: ContentType

old_content: ContentType

old dst file content loaded if unskipped in path prefixes

is_refreshable

set to True in path prefix parser to allow to overwrite destination file

is_up_to_date

set to True if destination file is up-to-date

__repr__()

show destination path, patcher and attributes of this managed file.

add_content_transformer(tf, encoding=None)

add a content transformer callable to this managed file.

Parameters:

• tf (Callable[[ManagedFile], str | bytes]) – content transformer callable, to be called with this instance as argument and returning the new/transformed content.

• encoding (str | None) – text file content encoding or None (like passed to the built-in open() function); or specify ‘bin-bytes’ to add a binary transformer (bytes file content).

content_transformations()

load the file contents of the source and destination file and run all the collected content transformers

property dst_file_path: str

return relative destination path of this managed file (cleaned from path/file name prefixes).

extend_dst_file_path(ext_path)

extend destination file path with the specified folder path.

Parameters:

ext_path (str) – path to extend the destination path with, e.g. move from project root to package path.

error(message)

add an error comment to this managed file.

Parameters:

message (str) – error comment text to add.

process_path_prefixes()

parse, reduce and call back the template file name/path prefixes to check for early deploy skip or errors.

Return type:

bool

Returns:

False if one of the path prefixes parsers errored or skipped this managed file.

skip(message)

add a skip reason comment to this managed file.

Parameters:

message (str) – skip reason comment text to add.

property skip_or_error: bool

return True if this managed file got skipped or added an error comment.

warning(message)

add a warning comment to this managed file.

Parameters:

message (str) – warning comment text to add.

write_file_content()

deploy file content of this managed file to its dst_file_path, creating not-existing folders.

class TemplateMngr(template_files, prefix_parsers, context_vars, *, prefix_args_sep='_', **replacers)

Bases: object

checks/deploys/renews managed files from templates and context variables.

Hint

example usages of this class can be found in the helper functions deploy_template() (in this module) and check_templates() (of the pjm project-manager tool).

__init__(template_files, prefix_parsers, context_vars, *, prefix_args_sep='_', **replacers)

Parameters:

• template_files (list[tuple[str, str, str]]) – list of template description tuples with [0]: patcher, e.g. the template project & version, [1]: path to the template file, [2]: destination file path with optional f-string-placeholders and path prefixes. order this list by priority, because if there is more than one template results in the same destination file path, then only the first one will be deployed.

• prefix_parsers (dict[str, tuple[int, PathPrefixesFunc]]) – template file/path prefixes as mapping of a path prefix string to a tuple of a path prefix arg count and a processing/parsing callee.

• context_vars (dict[str, Any]) – context variables dict to replace template placeholders. to get more globals (by try_eval()) extend this argument with a ‘_add_base_globals’ key.

• prefix_args_sep (str) – path prefixes arguments separator/suffix; defaults to PATH_PREFIXES_ARGS_SEP.

• replacers (Callable[[str], str]) – optional replacers: key=placeholder-id and value=replacer callable.

managed_files: list[ManagedFile]

deploy_files: dict[str, ManagedFile]

_compile()

__repr__()

show deployed, deployable and managed file counts.

property checked_files: set[str]

return a set of destination file paths of all the managed/checked template files.

deploy()

deploy all the missing/outdated managed files.

log_lines(verbose=False)

return a list of the log lines of all the managed/checked template files.

Parameters:

verbose (bool) – pass True to get more verbose log entries and include also not deployed files.

Return type:

list[str]

Returns:

list of log entry lines (preformatted with indent to direct console printout).

property missing_files: set[str]

return a set of destination file paths of the missing files created from templates.

property outdated_files: OutdatedFilesPathsContents

list of tuples of destination file path, new, and old file contents for each outdated file.

property path_prefixes_arg_counts: Iterable[tuple[str, int]]

iterable of tuples wit the prefix id/string and its args count for all registered path prefixes.

property skipped_files: set[str]

return a set of destination file paths of the skipped or erroneous managed files.

deploy_template(template_file_path, dst_path='.', patcher='deploy_template_default_patcher', prefixes_parsers=None, tpl_vars=None)

create/update a file from a template.

Parameters:

• template_file_path (str) – template/source file path.

• dst_path (str) – destination file name and path with optional path prefixes/args (will be removed). defaults to the current working directory if not specified.

• patcher (str) – patcher id string, e.g. the template project & version.

• prefixes_parsers (Optional[dict[str, tuple[int, PathPrefixesFunc]]]) – mapping of a prefix to a tuple of the prefix args count and the prefix callable/parser.

• tpl_vars (Optional[dict[str, Any]]) – template/project env/dev variables dict of the destination project to patch/refresh. providing values for the file/path names, the templates f-string placeholders, and the path prefix parser callees (e.g. to specify the project_type, project_path or package_path values).

Return type:

str

Returns:

absolute and stripped destination file path if template got deployed, or an empty string if any error or skip reason occurred.

extend_content_with_marker(file_name, content, patcher)

create/update the content of an overwritable text file with placeholders (compiled from a template file).

Parameters:

• file_name (str) – the name (and path) of the file to create/update/patch.

• content (str) – the content of the file (without the placeholder template marker).

• patcher (str) – patching entity/project name/version, to be placed with the placeholder template marker.

Return type:

str

Returns:

the patched content of the text file (with updated outsource marker).

patch_string(content, tpl_vars, **replacers)

replace f-string / dynamic placeholders in content with variable values / return values of replacers callables.

Parameters:

• content (str) – f-string to patch (e.g., a template file’s content).

• tpl_vars (dict[str, Any]) – dict with variables used as globals for f-string replacements. extend this argument with a ‘_add_base_globals’ key to add useful globals (see try_eval()).

• replacers (Callable[[str], str]) – optional kwargs dict with key/name=placeholder-id and value=replacer-callable. to specify additional replacers and also to overwrite or to deactivate the default template placeholder replacers specified in DEFAULT_TEMPLATE_PLACEHOLDERS

Return type:

str

Returns:

string resulting from the evaluation of the specified content f-string and from the default and additionally specified template :paramref;`~patch_string.replacers`.

Raises:

• AssertionError – if TEMPLATE_PLACEHOLDER_ARGS_SUFFIX is a replacers comment.

• Exception – if evaluation of the :paramref;`~patch_string.content` f-string failed (because of missing-globals-NameError/SyntaxError/ValueError/…).

_path_pfx_check_single_refreshable(mf)

Return type:

None

path_pfx_parametrize_with_context(mf, *_args)

path prefix callee for the F_STRINGS_PATH_PFX prefix.

Parameters:

mf (ManagedFile) – ManagedFile instance.

path_pfx_overwritable_binary_content(mf, *_args)

path prefix callee for the OVERWRITABLE_BIN_TEMPLATE_PATH_PFX prefix.

Parameters:

mf (ManagedFile) – ManagedFile instance.

path_pfx_overwritable_content(mf, *_args)

path prefix callee for the OVERWRITABLE_TEMPLATE_PATH_PFX prefix.

Parameters:

mf (ManagedFile) – ManagedFile instance.

path_pfx_puttable_content(mf, *_args)

path prefix callee for the PUTTABLE_TEMPLATE_PATH_PFX prefix.

Parameters:

mf (ManagedFile) – ManagedFile instance.

path_pfx_updatable_content(mf, *_args)

path prefix callee for the UPDATABLE_TEMPLATE_PATH_PFX prefix.

Parameters:

mf (ManagedFile) – ManagedFile instance.

DEFAULT_PATH_PREFIXES_PARSERS: dict[str, tuple[int, PathPrefixesFunc]] = {'Obi-': (0, <function path_pfx_overwritable_binary_content>), 'Ovw-': (0, <function path_pfx_overwritable_content>), 'PutMar-': (0, <function path_pfx_puttable_content>), 'UpdMar-': (0, <function path_pfx_updatable_content>), 'fSt-': (0, <function path_pfx_parametrize_with_context>)}

mapping of the default path prefixes parsers with to a tuple of the prefix args count and the parser callee.

prefix_parser(dst_path, prefixes_arg_counts, args_sep='_')

detect path/file name prefixes including their prefix args returning stripped path/file name and prefixes args.

Parameters:

• dst_path (str) – destination file path to parse for path/file name prefixes and args.

• prefixes_arg_counts (Iterable[tuple[str, int]]) – iterable of tuples with a prefix and the number of args of each prefix.

• args_sep (str) – string/char used as path prefixes args seperator/suffix

Return type:

tuple[str, list[tuple[str, tuple[str, ...]]]]

Returns:

tuple with stripped path/file name and a list of tuples with the removed prefix and its prefix args.

replace_with_file_content_or_default(args_str)

return file content if the file specified in first string arg exists, else return empty string or 2nd arg str.

Parameters:

args_str (str) – pass either file name, or file name and default literal separated by a comma character. whitespace (spaces, tabs, newline, cr) get removed from the start/end of the file name. a default literal gets parsed like a config variable, the literal value gets return.

Return type:

str

Returns:

file content or default literal value or empty string (if the file does not exist and there is no comma char in args_str).

replace_with_template_args(args_str)

template placeholder replacer function to hide uncompleted code from code-inspections/editor-warnings.

Parameters:

args_str (str) – args string to return, replacing the template placeholder (interpreted as comment in Python code).

Return type:

str

Returns:

args string specified as argument of the TEMPLATE_REPLACE_WITH_PLACEHOLDER_ID.

DEFAULT_REPLACERS = {'IncludeFile': <function replace_with_file_content_or_default>, 'ReplaceWith': <function replace_with_template_args>}

map of default replacers callables used by patch_string().

transform_parametrize_content(mf)

content transformer callee added via the F_STRINGS_PATH_PFX path prefix.

Parameters:

mf (ManagedFile) – ManagedFile instance.

Return type:

str

Returns:

transformed file content.

transform_puttable_content(mf)

content transformer callee added via the PUTTABLE_TEMPLATE_PATH_PFX path prefix.

Parameters:

mf (ManagedFile) – ManagedFile instance.

Return type:

str

Returns:

transformed file content.

transform_updatable_content(mf)

content transformer callee added via the UPDATABLE_TEMPLATE_PATH_PFX path prefix.

Parameters:

mf (ManagedFile) – ManagedFile instance.

Return type:

str

Returns:

transformed file content.