ae.dynamicod

dynamic execution of code blocks and expressions

this ae namespace portion provides useful helper functions to evaluate Python expressions and execute Python code dynamically at application run-time. these functions are e.g. used by the Literal class to dynamically determine literal values.

dynamically executed code block or expression string offers convenience for powerful system and application configuration and for data-driven architectures.

for the dynamic execution of functions and code blocks the helper functions try_call(), try_exec() and exec_with_return() are provided. the helper function try_eval() evaluates dynamic expressions.

Note

security considerations

make sure that any dynamically executed code is from a secure source to prevent code injections of malware. treat configuration files from untrusted sources with extreme caution and only execute them after a complete check and/or within a sandbox/controlled-environment.

Hint

most of the helper functions provided by this module have the two arguments glo_vars and loc_vars to pass the global and local variable names and values that are accessible from the dynamically evaluated/executed expression/code. therefore, code injections can be controlled by passing dictionaries to these arguments which are either empty or contain only secure symbols.

Module Attributes

base_globals

default if no global variables get passed in dynamic code/expression evaluations

Functions

exec_with_return(code_block[, ...])

execute python code block and return the resulting value of its last code line.

try_call(callee, *args[, ignored_exceptions])

execute callable while ignoring specified exceptions and return callable return value.

try_eval(expr[, ignored_exceptions, ...])

evaluate expression string ignoring specified exceptions and return evaluated value.

try_exec(code_block[, ignored_exceptions, ...])

execute python code block string ignoring specified exceptions and return value of last code line in block.
_merge_vars(add)[source]
Return type:

dict[str, Any]

exec_with_return(code_block, ignored_exceptions=(), glo_vars=None, loc_vars=None)[source]

execute python code block and return the resulting value of its last code line.

Parameters:
Return type:

Optional[Any]

Returns:

value of the expression at the last code line or UNSET if either code block is empty, only contains comment lines, or one of the ignorable exceptions raised or if last code line is no expression.

inspired by this SO answer https://stackoverflow.com/questions/33409207/how-to-return-value-from-exec-in-function/52361938#52361938.

try_call(callee, *args, ignored_exceptions=(), **kwargs)[source]

execute callable while ignoring specified exceptions and return callable return value.

Parameters:

  • callee (Callable) – pointer to callable (either function pointer, lambda expression, a class, …).

  • args – function arguments tuple.

  • ignored_exceptions (tuple[Type[Exception], ...]) – tuple of ignored exceptions.

  • kwargs – function keyword arguments dict.

Return type:

Optional[Any]

Returns:

function return value or UNSET if an ignored exception got thrown.

try_eval(expr, ignored_exceptions=(), glo_vars=None, loc_vars=None)[source]

evaluate expression string ignoring specified exceptions and return evaluated value.

Parameters:
Return type:

Optional[Any]

Returns:

function return value or UNSET if an ignored exception got thrown.

try_exec(code_block, ignored_exceptions=(), glo_vars=None, loc_vars=None)[source]

execute python code block string ignoring specified exceptions and return value of last code line in block.

Parameters:
Return type:

Optional[Any]

Returns:

function return value or UNSET if an ignored exception got thrown.

base_globals = {'ASCII_TO_UNICODE': {0: '␀', 1: '␁', 2: '␂', 3: '␃', 4: '␄', 5: '␅', 6: '␆', 7: '␇', 8: '␈', 9: '␉', 10: '␊', 11: '␋', 12: '␌', 13: '␍', 14: '␎', 15: '␏', 16: '␐', 17: '␑', 18: '␒', 19: '␓', 20: '␔', 21: '␕', 22: '␖', 23: '␗', 24: '␘', 25: '␙', 26: '␚', 27: '␛', 28: '␜', 29: '␝', 30: '␞', 31: '␟', 32: '␣', 34: '＂', 35: '﹟', 36: '﹩', 37: '﹪', 38: '﹠', 39: '‘', 40: '⟮', 41: '⟯', 42: '﹡', 43: '﹢', 44: '﹐', 47: '⁄', 58: '﹕', 59: '﹔', 60: '⟨', 61: '﹦', 62: '⟩', 63: '﹖', 64: '﹫', 91: '⟦', 92: '﹨', 93: '⟧', 94: '＾', 123: '﹛', 124: '।', 125: '﹜', 127: '␡'}, 'ASCII_UNICODE': (('/', '⁄'), ('|', '।'), ('\\', '﹨'), (':', '﹕'), ('*', '﹡'), ('?', '﹖'), ('"', '＂'), ("'", '‘'), ('<', '⟨'), ('>', '⟩'), ('(', '⟮'), (')', '⟯'), ('[', '⟦'), (']', '⟧'), ('{', '﹛'), ('}', '﹜'), ('#', '﹟'), (';', '﹔'), ('@', '﹫'), ('&', '﹠'), ('=', '﹦'), ('+', '﹢'), ('$', '﹩'), ('%', '﹪'), ('^', '＾'), (',', '﹐'), (' ', '␣'), ('\x7f', '␡'), ('\x00', '␀'), ('\x01', '␁'), ('\x02', '␂'), ('\x03', '␃'), ('\x04', '␄'), ('\x05', '␅'), ('\x06', '␆'), ('\x07', '␇'), ('\x08', '␈'), ('\t', '␉'), ('\n', '␊'), ('\x0b', '␋'), ('\x0c', '␌'), ('\r', '␍'), ('\x0e', '␎'), ('\x0f', '␏'), ('\x10', '␐'), ('\x11', '␑'), ('\x12', '␒'), ('\x13', '␓'), ('\x14', '␔'), ('\x15', '␕'), ('\x16', '␖'), ('\x17', '␗'), ('\x18', '␘'), ('\x19', '␙'), ('\x1a', '␚'), ('\x1b', '␛'), ('\x1c', '␜'), ('\x1d', '␝'), ('\x1e', '␞'), ('\x1f', '␟')), 'Any': typing.Any, 'CFG_EXT': '.cfg', 'Callable': typing.Callable, 'DATE_ISO': '%Y-%m-%d', 'DATE_TIME_ISO': '%Y-%m-%d %H:%M:%S.%f', 'DEF_ENCODE_ERRORS': 'backslashreplace', 'DEF_ENCODING': 'ascii', 'DEF_PROJECT_PARENT_FOLDER': 'src', 'DOCS_FOLDER': 'docs', 'Final': typing.Final, 'Generator': typing.Generator, 'GivenFormatter': <class 'ae.base.GivenFormatter'>, 'HTTPError': <class 'urllib.error.HTTPError'>, 'INI_EXT': '.ini', 'Iterable': typing.Iterable, 'NAME_PARTS_SEP': '_', 'NOW_STR_FORMAT': '{sep}%Y%m%d{sep}%H%M%S{sep}%f', 'Optional': typing.Optional, 'PACKAGE_INCLUDE_FILES_PREFIX': 'ae_', 'PY_CACHE_FOLDER': '__pycache__', 'PY_EXT': '.py', 'PY_INIT': '__init__.py', 'PY_MAIN': '__main__.py', 'Request': <class 'urllib.request.Request'>, 'TEMPLATES_FOLDER': 'templates', 'TESTS_FOLDER': 'tests', 'Type': typing.Type, 'UNICODE_TO_ASCII': {2404: '|', 8216: "'", 8260: '/', 9216: '\x00', 9217: '\x01', 9218: '\x02', 9219: '\x03', 9220: '\x04', 9221: '\x05', 9222: '\x06', 9223: '\x07', 9224: '\x08', 9225: '\t', 9226: '\n', 9227: '\x0b', 9228: '\x0c', 9229: '\r', 9230: '\x0e', 9231: '\x0f', 9232: '\x10', 9233: '\x11', 9234: '\x12', 9235: '\x13', 9236: '\x14', 9237: '\x15', 9238: '\x16', 9239: '\x17', 9240: '\x18', 9241: '\x19', 9242: '\x1a', 9243: '\x1b', 9244: '\x1c', 9245: '\x1d', 9246: '\x1e', 9247: '\x1f', 9249: '\x7f', 9251: ' ', 10214: '[', 10215: ']', 10216: '<', 10217: '>', 10222: '(', 10223: ')', 11003: '://', 65104: ',', 65108: ';', 65109: ':', 65110: '?', 65115: '{', 65116: '}', 65119: '#', 65120: '&', 65121: '*', 65122: '+', 65126: '=', 65128: '\\', 65129: '$', 65130: '%', 65131: '@', 65282: '"', 65342: '^'}, 'UNSET': <ae.base.UnsetType object>, 'URI_SEP_STR': '://', 'URI_SEP_UNICODE_CHAR': '⫻', 'URLError': <class 'urllib.error.URLError'>, 'UnformattedValue': <class 'ae.base.UnformattedValue'>, 'UnsetType': <class 'ae.base.UnsetType'>, 'VariableMap': dict[str, typing.Any], '__annotations__': {}, '__builtins__': {'ArithmeticError': <class 'ArithmeticError'>, 'AssertionError': <class 'AssertionError'>, 'AttributeError': <class 'AttributeError'>, 'BaseException': <class 'BaseException'>, 'BaseExceptionGroup': <class 'BaseExceptionGroup'>, 'BlockingIOError': <class 'BlockingIOError'>, 'BrokenPipeError': <class 'BrokenPipeError'>, 'BufferError': <class 'BufferError'>, 'BytesWarning': <class 'BytesWarning'>, 'ChildProcessError': <class 'ChildProcessError'>, 'ConnectionAbortedError': <class 'ConnectionAbortedError'>, 'ConnectionError': <class 'ConnectionError'>, 'ConnectionRefusedError': <class 'ConnectionRefusedError'>, 'ConnectionResetError': <class 'ConnectionResetError'>, 'DeprecationWarning': <class 'DeprecationWarning'>, 'EOFError': <class 'EOFError'>, 'Ellipsis': Ellipsis, 'EncodingWarning': <class 'EncodingWarning'>, 'EnvironmentError': <class 'OSError'>, 'Exception': <class 'Exception'>, 'ExceptionGroup': <class 'ExceptionGroup'>, 'False': False, 'FileExistsError': <class 'FileExistsError'>, 'FileNotFoundError': <class 'FileNotFoundError'>, 'FloatingPointError': <class 'FloatingPointError'>, 'FutureWarning': <class 'FutureWarning'>, 'GeneratorExit': <class 'GeneratorExit'>, 'IOError': <class 'OSError'>, 'ImportError': <class 'ImportError'>, 'ImportWarning': <class 'ImportWarning'>, 'IndentationError': <class 'IndentationError'>, 'IndexError': <class 'IndexError'>, 'InterruptedError': <class 'InterruptedError'>, 'IsADirectoryError': <class 'IsADirectoryError'>, 'KeyError': <class 'KeyError'>, 'KeyboardInterrupt': <class 'KeyboardInterrupt'>, 'LookupError': <class 'LookupError'>, 'MemoryError': <class 'MemoryError'>, 'ModuleNotFoundError': <class 'ModuleNotFoundError'>, 'NameError': <class 'NameError'>, 'None': None, 'NotADirectoryError': <class 'NotADirectoryError'>, 'NotImplemented': NotImplemented, 'NotImplementedError': <class 'NotImplementedError'>, 'OSError': <class 'OSError'>, 'OverflowError': <class 'OverflowError'>, 'PendingDeprecationWarning': <class 'PendingDeprecationWarning'>, 'PermissionError': <class 'PermissionError'>, 'ProcessLookupError': <class 'ProcessLookupError'>, 'RecursionError': <class 'RecursionError'>, 'ReferenceError': <class 'ReferenceError'>, 'ResourceWarning': <class 'ResourceWarning'>, 'RuntimeError': <class 'RuntimeError'>, 'RuntimeWarning': <class 'RuntimeWarning'>, 'StopAsyncIteration': <class 'StopAsyncIteration'>, 'StopIteration': <class 'StopIteration'>, 'SyntaxError': <class 'SyntaxError'>, 'SyntaxWarning': <class 'SyntaxWarning'>, 'SystemError': <class 'SystemError'>, 'SystemExit': <class 'SystemExit'>, 'TabError': <class 'TabError'>, 'TimeoutError': <class 'TimeoutError'>, 'True': True, 'TypeError': <class 'TypeError'>, 'UnboundLocalError': <class 'UnboundLocalError'>, 'UnicodeDecodeError': <class 'UnicodeDecodeError'>, 'UnicodeEncodeError': <class 'UnicodeEncodeError'>, 'UnicodeError': <class 'UnicodeError'>, 'UnicodeTranslateError': <class 'UnicodeTranslateError'>, 'UnicodeWarning': <class 'UnicodeWarning'>, 'UserWarning': <class 'UserWarning'>, 'ValueError': <class 'ValueError'>, 'Warning': <class 'Warning'>, 'ZeroDivisionError': <class 'ZeroDivisionError'>, '__annotations__': {}, '__build_class__': <built-in function __build_class__>, '__debug__': True, '__doc__': "Built-in functions, types, exceptions, and other objects.\n\nThis module provides direct access to all 'built-in'\nidentifiers of Python; for example, builtins.len is\nthe full name for the built-in function len().\n\nThis module is not normally accessed explicitly by most\napplications, but can be useful in modules that provide\nobjects with the same name as a built-in value, but in\nwhich the built-in of that name is also needed.", '__import__': <built-in function __import__>, '__loader__': <class '_frozen_importlib.BuiltinImporter'>, '__name__': 'builtins', '__package__': '', '__spec__': ModuleSpec(name='builtins', loader=<class '_frozen_importlib.BuiltinImporter'>, origin='built-in'), 'abs': <built-in function abs>, 'aiter': <built-in function aiter>, 'all': <built-in function all>, 'anext': <built-in function anext>, 'any': <built-in function any>, 'ascii': <built-in function ascii>, 'bin': <built-in function bin>, 'bool': <class 'bool'>, 'breakpoint': <built-in function breakpoint>, 'bytearray': <class 'bytearray'>, 'bytes': <class 'bytes'>, 'callable': <built-in function callable>, 'chr': <built-in function chr>, 'classmethod': <class 'classmethod'>, 'compile': <built-in function compile>, 'complex': <class 'complex'>, 'copyright': Copyright (c) 2001-2023 Python Software Foundation. All Rights Reserved.  Copyright (c) 2000 BeOpen.com. All Rights Reserved.  Copyright (c) 1995-2001 Corporation for National Research Initiatives. All Rights Reserved.  Copyright (c) 1991-1995 Stichting Mathematisch Centrum, Amsterdam. All Rights Reserved., 'credits':     Thanks to CWI, CNRI, BeOpen, Zope Corporation, the Python Software     Foundation, and a cast of thousands for supporting Python     development.  See www.python.org for more information., 'delattr': <built-in function delattr>, 'dict': <class 'dict'>, 'dir': <built-in function dir>, 'divmod': <built-in function divmod>, 'enumerate': <class 'enumerate'>, 'eval': <built-in function eval>, 'exec': <built-in function exec>, 'exit': Use exit() or Ctrl-D (i.e. EOF) to exit, 'filter': <class 'filter'>, 'float': <class 'float'>, 'format': <built-in function format>, 'frozenset': <class 'frozenset'>, 'getattr': <built-in function getattr>, 'globals': <built-in function globals>, 'hasattr': <built-in function hasattr>, 'hash': <built-in function hash>, 'help': Type help() for interactive help, or help(object) for help about object., 'hex': <built-in function hex>, 'id': <built-in function id>, 'input': <built-in function input>, 'int': <class 'int'>, 'isinstance': <built-in function isinstance>, 'issubclass': <built-in function issubclass>, 'iter': <built-in function iter>, 'len': <built-in function len>, 'license': Type license() to see the full license text, 'list': <class 'list'>, 'locals': <built-in function locals>, 'map': <class 'map'>, 'max': <built-in function max>, 'memoryview': <class 'memoryview'>, 'min': <built-in function min>, 'next': <built-in function next>, 'object': <class 'object'>, 'oct': <built-in function oct>, 'open': <built-in function open>, 'ord': <built-in function ord>, 'pow': <built-in function pow>, 'print': <built-in function print>, 'property': <class 'property'>, 'quit': Use quit() or Ctrl-D (i.e. EOF) to exit, 'range': <class 'range'>, 'repr': <built-in function repr>, 'reversed': <class 'reversed'>, 'round': <built-in function round>, 'set': <class 'set'>, 'setattr': <built-in function setattr>, 'slice': <class 'slice'>, 'sorted': <built-in function sorted>, 'staticmethod': <class 'staticmethod'>, 'str': <class 'str'>, 'sum': <built-in function sum>, 'super': <class 'super'>, 'tuple': <class 'tuple'>, 'type': <class 'type'>, 'vars': <built-in function vars>, 'zip': <class 'zip'>}, '__cached__': '/home/docs/checkouts/readthedocs.org/user_builds/ae/envs/latest/lib/python3.12/site-packages/ae/__pycache__/dynamicod.cpython-312.pyc', '__doc__': '\ndynamic execution of code blocks and expressions\n================================================\n\nthis ae namespace portion provides useful helper functions to evaluate Python expressions and execute\nPython code dynamically at application run-time. these functions are e.g. used by the :class:`~ae.literal.Literal`\nclass to dynamically determine literal values.\n\ndynamically executed code block or expression string offers convenience for powerful system and application\nconfiguration and for data-driven architectures.\n\nfor the dynamic execution of functions and code blocks the helper functions :func:`try_call`, :func:`try_exec`\nand :func:`exec_with_return` are provided. the helper function :func:`try_eval` evaluates dynamic expressions.\n\n.. note::\n    **security considerations**\n\n    make sure that any dynamically executed code is from a secure source to prevent code injections of malware.\n    treat configuration files from untrusted sources with extreme caution and\n    only execute them after a complete check and/or within a sandbox/controlled-environment.\n\n.. hint::\n    most of the helper functions provided by this module have the two arguments `glo_vars` and `loc_vars` to\n    pass the global and local variable names and values that are accessible from the dynamically evaluated/executed\n    expression/code. therefore, code injections can be controlled by passing dictionaries to these arguments which are\n    either empty or contain only secure symbols.\n', '__file__': '/home/docs/checkouts/readthedocs.org/user_builds/ae/envs/latest/lib/python3.12/site-packages/ae/dynamicod.py', '__loader__': <_frozen_importlib_external.SourceFileLoader object>, '__name__': 'ae.dynamicod', '__package__': 'ae', '__spec__': ModuleSpec(name='ae.dynamicod', loader=<_frozen_importlib_external.SourceFileLoader object>, origin='/home/docs/checkouts/readthedocs.org/user_builds/ae/envs/latest/lib/python3.12/site-packages/ae/dynamicod.py'), '__version__': '0.3.15', '_dc': ('abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ', '0123456789', <module 'datetime' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/datetime.py'>, <module 'logging' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/logging/__init__.py'>, <module 'logging.config' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/logging/config.py'>, <module 'os' (frozen)>, <module 'threading' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/threading.py'>, <module 'unicodedata' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/lib-dynload/unicodedata.cpython-312-x86_64-linux-gnu.so'>, <module 'weakref' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/weakref.py'>), '_merge_vars': <function _merge_vars>, 'ascii_dec_str': <function ascii_dec_str>, 'ascii_enc_lit': <function ascii_enc_lit>, 'ascii_letters': 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ', 'ast': <module 'ast' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/ast.py'>, 'base64': <module 'base64' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/base64.py'>, 'base_globals': dict(...), 'camel_to_snake': <function camel_to_snake>, 'contextmanager': <function contextmanager>, 'datetime': <module 'datetime' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/datetime.py'>, 'dedefuse': <function dedefuse>, 'deep_dict_update': <function deep_dict_update>, 'defuse': <function defuse>, 'digits': '0123456789', 'dummy_function': <function dummy_function>, 'duplicates': <function duplicates>, 'env_str': <function env_str>, 'evaluate_literal': <function evaluate_literal>, 'exec_with_return': <function exec_with_return>, 'extend_file': <function extend_file>, 'force_encoding': <function force_encoding>, 'format_given': <function format_given>, 'in_wd': <function in_wd>, 'literal_eval': <function literal_eval>, 'logging': <module 'logging' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/logging/__init__.py'>, 'logging_config': <module 'logging.config' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/logging/config.py'>, 'mask_secrets': <function mask_secrets>, 'mask_url': <function mask_url>, 'norm_line_sep': <function norm_line_sep>, 'norm_name': <function norm_name>, 'norm_path': <function norm_path>, 'now_str': <function now_str>, 'on_ci_host': <function on_ci_host>, 'os': <module 'os' (frozen)>, 'os_path_abspath': <function abspath>, 'os_path_basename': <function basename>, 'os_path_dirname': <function dirname>, 'os_path_expanduser': <function expanduser>, 'os_path_isdir': <function isdir>, 'os_path_isfile': <function isfile>, 'os_path_join': <function join>, 'os_path_normpath': <built-in function _path_normpath>, 'os_path_realpath': <function realpath>, 'os_path_relpath': <function relpath>, 'os_path_sep': '/', 'os_path_splitext': <function splitext>, 'pep8_format': <function pep8_format>, 'read_bin_file': <function read_bin_file>, 'read_file': <function read_file>, 'round_traditional': <function round_traditional>, 'sign': <function sign>, 'snake_to_camel': <function snake_to_camel>, 'socket': <module 'socket' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/socket.py'>, 'ssl': <module 'ssl' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/ssl.py'>, 'stmt': <class 'ast.stmt'>, 'string': <module 'string' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/string.py'>, 'threading': <module 'threading' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/threading.py'>, 'to_ascii': <function to_ascii>, 'try_call': <function try_call>, 'try_eval': <function try_eval>, 'try_exec': <function try_exec>, 'unicodedata': <module 'unicodedata' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/lib-dynload/unicodedata.cpython-312-x86_64-linux-gnu.so'>, 'url_failure': <function url_failure>, 'urlopen': <function urlopen>, 'urlparse': <function urlparse>, 'urlunparse': <function urlunparse>, 'utc_datetime': <function utc_datetime>, 'weakref': <module 'weakref' from '/home/docs/.asdf/installs/python/3.12.12/lib/python3.12/weakref.py'>, 'write_bin_file': <function write_bin_file>, 'write_file': <function write_file>}

default if no global variables get passed in dynamic code/expression evaluations