ae.base

basic constants, helper functions, classes and context managers

this module is pure python, has no external dependencies, and provides a comprehensive toolkit of base constants, common helper functions, useful classes, and context managers for a wide variety of programming tasks.

string manipulation

functions for converting, cleaning, normalizing, and formatting strings.

ascii_dec_str(): decodes an ascii string literal converted by ascii_enc_lit() back to its Unicode form.
ascii_enc_lit(): encodes a Unicode string into a reversible 7-bit ASCII representation, useful for transport protocol/HTTP headers.
camel_to_snake(): converts a string from CamelCase to snake_case.
snake_to_camel(): converts a string from snake_case to CamelCase.
norm_name(): normalizes a string to be a valid identifier (e.g., for variable-, method-, or file-names).
norm_line_sep(): converts all line separator combinations (CRLF, CR) in a string to a single newline (LF).
defuse(): converts special characters in string to Unicode alternatives, making it safe for use as a URL slug, path or filename.
dedefuse(): reverses the operation of defuse(), restoring the original string.
force_encoding(): ensures text is in a specific encoding without raising errors, replacing characters as needed.
to_ascii(): converts a Unicode string into its closest ASCII representation by removing accents and diacritics.
format_given(): a replacement for str.format_map that formats a string but leaves placeholders intact if they are not found in the provided mapping.

data structure utilities

helpers for working with lists, dictionaries, and other data structures.

evaluate_literal(): replacement for ast.literal_eval() that also interprets/recognizes unquoted strings as str type.
duplicates(): returns a list of all duplicate items found in any type of iterable.
deep_dict_update(): recursively updates a dictionary in-place with values from another dictionary.
mask_secrets(): hides sensitive string values (e.g., passwords, API keys) in deeply nested data structures, useful for logging.

file, path & I/O operations

simplify file system interactions with wrappers and context managers.

extend_file(): append string to a file or create it if file not exists.
in_wd(): a context manager to temporarily switch/change the current working directory.
norm_path(): normalizes a path by expanding user home directories (~), resolving ., .., symbolic links, and converting between absolute and relative paths.
read_bin_file(): reads the entire content of a binary file into a bytes object.
read_file(): reads the entire content of a text file into a string.
write_bin_file(): writes a bytes object to a file, overwriting existing content.
write_file(): writes a string into a file, overwriting existing content.

networking utilities

mask_url(): hides or replaces the password/token portion of a URL for safe logging.
url_failure(): determines if and why an HTTP|FTP target is unavailable.

general utilities & helpers

a collection of miscellaneous mathematical, date/time, and other standalone helper functions.

mathematical

sign(): returns the sign of a number (-1 for negative, 0 for zero, 1 for positive).
round_traditional(): rounds a float value using traditional rounding rules (e.g., 0.5 rounds up).

date & time

utc_datetime(): Returns the current date and time as a timezone-naive datetime object in UTC.
now_str(): creates a compact, sortable timestamp string from the current UTC time.

miscellaneous

dummy_function(): a null function that accepts any arguments and returns None.
env_str(): retrieves the string value of an OS environment variable, with an option to automatically convert the variable name to the conventional format.
on_ci_host(): detects if it is running on the CI of a Git repository server (GitHub or GitLab).

base types and classes

UnsetType: the class for the UNSET singleton object, useful as a sentinel value when None is a valid input.
UnformattedValue: a helper class for format_given() to represent a placeholder that was not found in the formatting map.
GivenFormatter: a helper class for format_given() that overrides default formatting behavior to keep missing placeholders.

base constants

predefined constants for defaults, project structure, file conventions, to decrease redundancy and increase performance.

project & file structure

CFG_EXT: file extension for CFG/INI configuration files (‘.cfg’).
DEF_PROJECT_PARENT_FOLDER: default directory name for grouping source code projects (‘src’).
DOCS_FOLDER: default name for a project’s documentation folder (‘docs’).
INI_EXT: file extension for INI configuration files (‘.ini’).
PACKAGE_INCLUDE_FILES_PREFIX: prefix for files/folders to be included in setup package data (used by ae.updater and aedev.project_manager)
PY_CACHE_FOLDER: default name for Python’s cache folder (‘__pycache__’).
PY_EXT: file extension for Python modules (‘.py’).
PY_INIT: the filename for a Python package initializer (‘__init__.py’).
PY_MAIN: the filename for a Python executable’s main module (‘__main__.py’).
TESTS_FOLDER: default name for a project’s tests folder (‘tests’).
TEMPLATES_FOLDER: default name for a folder containing file templates (‘templates’).

formats & default settings

DATE_ISO: ISO format string for dates (“%Y-%m-%d”).
DATE_TIME_ISO: ISO format string for datetime.datetime dates (“%Y-%m-%d %H:%M:%S.%f”).
DEF_ENCODE_ERRORS: the default error handling strategy for encoding (‘backslashreplace’).
DEF_ENCODING: the default encoding used for string operations (‘ascii’).
NAME_PARTS_SEP: the character used as a separator in name conversions (‘_’).
NOW_STR_FORMAT: the datetime format string, used e.g. by now_str() for creating timestamps.
UNSET: a singleton instance of UnsetType, used where None is a valid data value.

os.path shortcuts

the following are direct references to functions in the os.path module for convenient and quicker access:

os_path_abspath: os.path.abspath()
os_path_basename: os.path.basename()
os_path_dirname: os.path.dirname()
os_path_expanduser: os.path.expanduser()
os_path_isdir: os.path.isdir()
os_path_isfile: os.path.isfile()
os_path_join: os.path.join()
os_path_normpath: os.path.normpath()
os_path_realpath: os.path.realpath()
os_path_relpath: os.path.relpath()
os_path_sep: os.path.sep
os_path_splitext: os.path.splitext()

Module Attributes

`DOCS_FOLDER`	project documentation root folder name
`TESTS_FOLDER`	name of project folder to store unit/integration tests
`TEMPLATES_FOLDER`	template folder name, used in template and namespace root projects to maintain and provide common file templates
`PACKAGE_INCLUDE_FILES_PREFIX`	file/folder names prefix included in setup package_data/ae_updater
`PY_CACHE_FOLDER`	python cache folder name
`PY_EXT`	file extension for modules and hooks
`PY_INIT`	init-module file name of a python package
`PY_MAIN`	main-module file name of a python executable
`CFG_EXT`	CFG config file extension
`INI_EXT`	INI config file extension
`DATE_ISO`	ISO string format for date values (e.g. in config files/variables).
`DATE_TIME_ISO`	ISO string format for datetime values
`DEF_PROJECT_PARENT_FOLDER`	default directory name to put code project roots underneath of it
`DEF_ENCODE_ERRORS`	default encode error handling for UnicodeEncodeErrors
`DEF_ENCODING`	encoding for `force_encoding()` that will always work independent from destination (console, file sys, ...).
`NAME_PARTS_SEP`	name parts separator character, e.g. for `norm_name()`.
`NOW_STR_FORMAT`	timestamp format of `now_str()`
`ASCII_UNICODE`	transformation table of special ASCII characters to a similar/alternative non-functional/-escaping Unicode char, see https://www.compart.com/en/unicode/category/Po and https://xahlee.info/comp/unicode_naming_slash.html (http!)
`URI_SEP_STR`	separator between service and address(host/path) in URIs
`URI_SEP_UNICODE_CHAR`	single Unicode char for `URI_SEP_STR` U+2AFB: TRIPLE SOLIDUS BINARY RELATION
`ASCII_TO_UNICODE`	`str.translate()` map to convert ASCII to an alternative defused Unicode character - used by `defuse()`
`UNICODE_TO_ASCII`	`str.translate()` Unicode to ASCII map - used by `dedefuse()`
`UNSET`	pseudo value used for attributes/arguments if `None` is needed as a valid value

Functions

`ascii_dec_str`(encoded_str)	convert non-ASCII chars in a string literal encoded with `ascii_enc_lit()` to Unicode chars.
`ascii_enc_lit`(unicode_str)	convert a Unicode string with non-ASCII chars to a revertible 7-bit/ASCII literal/representation.
`camel_to_snake`(name)	convert a name from CamelCase to snake_case.
`dedefuse`(value)	convert a string that got defused with `defuse()` back to its original form.
`deep_dict_update`(data, update[, overwrite])	update the optionally nested data dict in-place with the items and subitems from the update dict.
`defuse`(value)	convert a file path or a URI into a defused/presentational form to be usable as URL slug or file/folder name.
`dummy_function`(_args, *_kwargs)	null function accepting any arguments and returning None.
`duplicates`(values)	determine all duplicates in the iterable specified in the `values` argument.
`env_str`(name[, convert_name])	determine the string value of an OS environment variable, optionally preventing invalid variable name.
`evaluate_literal`(literal_string)	evaluates a Python expression while accepting unquoted strings as str type.
`extend_file`(file_path, content[, encoding, ...])	create/extend the text file specified by `file_path` with the specified content string.
`force_encoding`(text[, encoding, errors])	force/ensure the encoding of text (str or bytes) without any UnicodeDecodeError/UnicodeEncodeError.
`format_given`(text, placeholder_map[, strict])	replacement for Python's str.format_map(), keeping intact placeholders that are not in the specified mapping.
`in_wd`(new_cwd)	context manager to temporarily switch the current working directory / cwd.
`mask_secrets`(data[, fragments])	partially-hide secret string values like passwords/credit-card-numbers in deeply nestable data structures.
`mask_url`(url[, replacement])	hide\|replace the password/token in a URL.
`norm_line_sep`(text)	convert any combination of line separators in the `text` arg to new-line characters.
`norm_name`(name[, allow_num_prefix])	normalize name to start with a letter/alphabetic/underscore and to contain only alphanumeric/underscore chars.
`norm_path`(path[, make_absolute, ...])	normalize a path, replacing ../. parts or the tilde character (home folder) and transform to relative/abs.
`now_str`([sep])	return the current UTC timestamp as string (to use as suffix for file and variable/attribute names).
`on_ci_host`()	check and return True if it is running on the GitLab/GitHub CI host/server.
`pep8_format`(value[, indent_level])	PEP-8-conform representation code string of deep dict/list structures, superseding `pprint.pformat()`.
`read_bin_file`(file_path)	returning the binary content of the specified by the `file_path` argument.
`read_file`(file_path[, encoding, error_handling])	returning the string content of the text file specified by `file_path` argument.
`round_traditional`(num_value[, num_digits])	round numeric value traditional.
`sign`(number)	return ths sign (-1, 0, 1) of a number.
`snake_to_camel`(name[, back_convertible])	convert name from snake_case to CamelCase.
`to_ascii`(unicode_str)	converts Unicode string into ascii representation.
`url_failure`(url[, token, username, ...])	determine if and why an FTP or HTTP[S] target is not available via a GET request.
`utc_datetime`()	return the current UTC timestamp as string (to use as suffix for file and variable/attribute names).
`write_bin_file`(file_path, content[, make_dirs])	(over)write the file specified by `file_path` with the specified binary/bytes content.
`write_file`(file_path, content[, encoding, ...])	(over)write the file specified by `file_path` with the specified string content.

Classes

`GivenFormatter`()	helper class for `format_given()` to keep placeholder with format unchanged if not found.
`UnformattedValue`(key)	helper class for `format_given()` to keep placeholder with format unchanged if not found.
`UnsetType`()	(singleton) UNSET (type) object class.

DOCS_FOLDER = 'docs': project documentation root folder name

TESTS_FOLDER = 'tests': name of project folder to store unit/integration tests

TEMPLATES_FOLDER = 'templates': template folder name, used in template and namespace root projects to maintain and provide common file templates

PACKAGE_INCLUDE_FILES_PREFIX = 'ae_': file/folder names prefix included in setup package_data/ae_updater

PY_CACHE_FOLDER = '__pycache__': python cache folder name

PY_EXT = '.py': file extension for modules and hooks

PY_INIT = '__init__.py': init-module file name of a python package

PY_MAIN = '__main__.py': main-module file name of a python executable

CFG_EXT = '.cfg': CFG config file extension

INI_EXT = '.ini': INI config file extension

DATE_ISO = '%Y-%m-%d': ISO string format for date values (e.g. in config files/variables)

DATE_TIME_ISO = '%Y-%m-%d %H:%M:%S.%f': ISO string format for datetime values

DEF_PROJECT_PARENT_FOLDER = 'src': default directory name to put code project roots underneath of it

DEF_ENCODE_ERRORS = 'backslashreplace': default encode error handling for UnicodeEncodeErrors

DEF_ENCODING = 'ascii': encoding for force_encoding() that will always work independent from destination (console, file sys, …).

NAME_PARTS_SEP = '_': name parts separator character, e.g. for norm_name()

NOW_STR_FORMAT = '{sep}%Y%m%d{sep}%H%M%S{sep}%f': timestamp format of now_str()

ascii_dec_str(encoded_str)[source]

convert non-ASCII chars in a string literal encoded with ascii_enc_lit() to Unicode chars.

Parameters:: encoded_str¶ (str) – string literal to decode (covert contained ASCII-encoded characters back Unicode chars).
Return type:: str
Returns:: decoded Unicode string.
Raises:: SyntaxError if invalid string literal got specified in encoded_str.

ascii_enc_lit(unicode_str)[source]

convert a Unicode string with non-ASCII chars to a revertible 7-bit/ASCII literal/representation.

Parameters:: unicode_str¶ (str) – string to encode/convert.
Return type:: str
Returns:: revertible representation of the specified string, using only ASCII characters, e.g., to put in an http header.

camel_to_snake(name)[source]

convert a name from CamelCase to snake_case.

Parameters:: name¶ (str) – name string in CamelCaseFormat.
Return type:: str
Returns:: name in snake_case_format.

deep_dict_update(data, update, overwrite=True)[source]

update the optionally nested data dict in-place with the items and subitems from the update dict.

Parameters:

data¶ (dict) – dict to be updated/extended. non-existing keys of dict-subitems will be added.
update¶ (dict) – dict with the [sub-]items to update in the data dict.
overwrite¶ (bool) – pass False to not overwrite an already existing value.

Hint

see the module/portion ae.deep for more deep update helper functions.

ASCII_UNICODE = (('/', '⁄'), ('|', '।'), ('\\', '﹨'), (':', '﹕'), ('*', '﹡'), ('?', '﹖'), ('"', '＂'), ("'", '‘'), ('<', '⟨'), ('>', '⟩'), ('(', '⟮'), (')', '⟯'), ('[', '⟦'), (']', '⟧'), ('{', '﹛'), ('}', '﹜'), ('#', '﹟'), (';', '﹔'), ('@', '﹫'), ('&', '﹠'), ('=', '﹦'), ('+', '﹢'), ('$', '﹩'), ('%', '﹪'), ('^', '＾'), (',', '﹐'), (' ', '␣'), ('\x7f', '␡'), ('\x00', '␀'), ('\x01', '␁'), ('\x02', '␂'), ('\x03', '␃'), ('\x04', '␄'), ('\x05', '␅'), ('\x06', '␆'), ('\x07', '␇'), ('\x08', '␈'), ('\t', '␉'), ('\n', '␊'), ('\x0b', '␋'), ('\x0c', '␌'), ('\r', '␍'), ('\x0e', '␎'), ('\x0f', '␏'), ('\x10', '␐'), ('\x11', '␑'), ('\x12', '␒'), ('\x13', '␓'), ('\x14', '␔'), ('\x15', '␕'), ('\x16', '␖'), ('\x17', '␗'), ('\x18', '␘'), ('\x19', '␙'), ('\x1a', '␚'), ('\x1b', '␛'), ('\x1c', '␜'), ('\x1d', '␝'), ('\x1e', '␞'), ('\x1f', '␟')): transformation table of special ASCII characters to a similar/alternative non-functional/-escaping Unicode char, see https://www.compart.com/en/unicode/category/Po and https://xahlee.info/comp/unicode_naming_slash.html (http!)

URI_SEP_STR = '://': separator between service and address(host/path) in URIs

URI_SEP_UNICODE_CHAR = '⫻': single Unicode char for URI_SEP_STR U+2AFB: TRIPLE SOLIDUS BINARY RELATION

ASCII_TO_UNICODE = {0: '␀', 1: '␁', 2: '␂', 3: '␃', 4: '␄', 5: '␅', 6: '␆', 7: '␇', 8: '␈', 9: '␉', 10: '␊', 11: '␋', 12: '␌', 13: '␍', 14: '␎', 15: '␏', 16: '␐', 17: '␑', 18: '␒', 19: '␓', 20: '␔', 21: '␕', 22: '␖', 23: '␗', 24: '␘', 25: '␙', 26: '␚', 27: '␛', 28: '␜', 29: '␝', 30: '␞', 31: '␟', 32: '␣', 34: '＂', 35: '﹟', 36: '﹩', 37: '﹪', 38: '﹠', 39: '‘', 40: '⟮', 41: '⟯', 42: '﹡', 43: '﹢', 44: '﹐', 47: '⁄', 58: '﹕', 59: '﹔', 60: '⟨', 61: '﹦', 62: '⟩', 63: '﹖', 64: '﹫', 91: '⟦', 92: '﹨', 93: '⟧', 94: '＾', 123: '﹛', 124: '।', 125: '﹜', 127: '␡'}: str.translate() map to convert ASCII to an alternative defused Unicode character - used by defuse()

UNICODE_TO_ASCII = {2404: '|', 8216: "'", 8260: '/', 9216: '\x00', 9217: '\x01', 9218: '\x02', 9219: '\x03', 9220: '\x04', 9221: '\x05', 9222: '\x06', 9223: '\x07', 9224: '\x08', 9225: '\t', 9226: '\n', 9227: '\x0b', 9228: '\x0c', 9229: '\r', 9230: '\x0e', 9231: '\x0f', 9232: '\x10', 9233: '\x11', 9234: '\x12', 9235: '\x13', 9236: '\x14', 9237: '\x15', 9238: '\x16', 9239: '\x17', 9240: '\x18', 9241: '\x19', 9242: '\x1a', 9243: '\x1b', 9244: '\x1c', 9245: '\x1d', 9246: '\x1e', 9247: '\x1f', 9249: '\x7f', 9251: ' ', 10214: '[', 10215: ']', 10216: '<', 10217: '>', 10222: '(', 10223: ')', 11003: '://', 65104: ',', 65108: ';', 65109: ':', 65110: '?', 65115: '{', 65116: '}', 65119: '#', 65120: '&', 65121: '*', 65122: '+', 65126: '=', 65128: '\\', 65129: '$', 65130: '%', 65131: '@', 65282: '"', 65342: '^'}: str.translate() Unicode to ASCII map - used by dedefuse()

dedefuse(value)[source]

convert a string that got defused with defuse() back to its original form.

Parameters:: value¶ (str) – string defused with the function defuse().
Return type:: str
Returns:: re-activated form of the string (with all ASCII special characters recovered).

defuse(value)[source]

convert a file path or a URI into a defused/presentational form to be usable as URL slug or file/folder name.

Parameters:: value¶ (str) – any string to defuse (replace special chars with Unicode alternatives).
Return type:: str
Returns:: string with its special characters replaced by its pure presentational alternatives.

the ASCII character range 0..31 gets converted to the Unicode range U+2400 + ord(char): 0==U+2400 … 31==U+241F.

in most unix variants only the slash and the ASCII 0 characters are not allowed in file names.

in MS Windows are not allowed: ASCII 0..31 / | : * ? ” % < > ( ). some blogs recommend also not allowing (convert) the characters # and ‘.

only old POSIX seems to be even more restricted (only allowing alphanumeric characters plus . - and _).

more on allowed characters in file names in the answers of RedGrittyBrick on https://superuser.com/questions/358855 and of Christopher Oezbek on https://stackoverflow.com/questions/1976007.

file name length is not restricted/shortened by this function, although the maximum is 255 characters on most OSs.

Hint

use the dedefuse() function to convert the defused string back to the corresponding URI/file-path.

dummy_function(*_args, **_kwargs)[source]

null function accepting any arguments and returning None.

Parameters:

_args¶ – ignored positional arguments.
_kwargs¶ – ignored keyword arguments.

Returns:

always None.

duplicates(values)[source]

determine all duplicates in the iterable specified in the values argument.

inspired by Ritesh Kumars answer to https://stackoverflow.com/questions/9835762.

Parameters:: values¶ (Iterable) – iterable (list, tuple, str, …) to search for duplicate items.
Return type:: list
Returns:: list of the duplicate items found (can contain the same duplicate multiple times).

env_str(name, convert_name=False)[source]

determine the string value of an OS environment variable, optionally preventing invalid variable name.

Parameters:

name¶ (str) – name of an OS environment variable.
convert_name¶ (bool) – pass True to prevent invalid variable names by converting CamelCase names into SNAKE_CASE, lower-case into upper-case and all non-alpha-numeric characters into underscore characters.

Return type:

str | None

Returns:

string value of OS environment variable if found, else None.

evaluate_literal(literal_string)[source]

evaluates a Python expression while accepting unquoted strings as str type.

Parameters:: literal_string¶ (str) – any literal of the base types (like dict, list, set, tuple) which are recognized by ast.literal_eval().
Return type:: bool | bytes | dict | complex | float | int | list | set | str | tuple
Returns:: an instance of the data type or the specified string, even if it is not quoted with high comma characters. None will be returned if the specified literal is the string “None”.

extend_file(file_path, content, encoding=None, make_dirs=False)[source]

create/extend the text file specified by file_path with the specified content string.

Parameters:

file_path¶ (str) – file path/name to write the passed content into (overwriting any previous content!).
content¶ (str) – new file content passed either as string or as bytes. if a byte array gets passed, then this method will automatically write the content as binary.
encoding¶ (str | None) – encoding used to convert/interpret the string content to write.
make_dirs¶ (bool) – pass True to create not existing parent folders of the specified file path.

Raises:

IsADirectoryError – file_path points to a directory instead of a file.
LookupError – unknown encoding name.
NotADirectoryError – part of the path expected to be a directory is actually a file.
OSError – disk full, filename too long, too many open files, network or device disconnected, file_path is misspelled or contains invalid characters.
PermissionError – if the current OS user account lacks permissions to write the file content.
TypeError – content is not of type str.
UnicodeEncodeError – content cannot be encoded using the selected encoding.
ValueError – other encoding errors, invalid mode or incompatible arguments.

force_encoding(text, encoding='ascii', errors='backslashreplace')[source]

force/ensure the encoding of text (str or bytes) without any UnicodeDecodeError/UnicodeEncodeError.

Parameters:

text¶ (str | bytes) – text as str/bytes.
encoding¶ (str) – encoding (def= DEF_ENCODING).
errors¶ (str) – encode error handling (def= DEF_ENCODE_ERRORS).

Return type:

str

Returns:

text as str (with all characters checked/converted/replaced to be encode-able).

class UnformattedValue(key)[source]

Bases: object

helper class for format_given() to keep placeholder with format unchanged if not found.

__init__(key)[source]

__format__(format_spec)[source]: overriding Python object class method to return placeholder unchanged, including the curly brackets.

class GivenFormatter[source]

Bases: Formatter

helper class for format_given() to keep placeholder with format unchanged if not found.

get_value(key, args, kwargs)[source]: overriding to keep placeholder unchanged if not found

format_given(text, placeholder_map, strict=False)[source]

replacement for Python’s str.format_map(), keeping intact placeholders that are not in the specified mapping.

Parameters:

text¶ (str) – text/template in which the given/specified placeholders will get replaced. in contrary to str.format_map(), no KeyError will be raised for placeholders not specified in placeholder_map.
placeholder_map¶ (dict[str, Any]) – dict with placeholder keys to be replaced in text argument.
strict¶ (bool) – pass True to raise an error for text templates containing unpaired curly brackets.

Returns:

the specified text with only the placeholders specified in placeholder_map replaced with their respective map value.

inspired by the answer of CodeManX in `https://stackoverflow.com/questions/3536303`__

in_wd(new_cwd)[source]

context manager to temporarily switch the current working directory / cwd.

Parameters:: new_cwd¶ (str) – path to the directory to switch to (within the context/with block). an empty string gets interpreted as the current working directory.
Return type:: Generator[None, None, None]

the following example demonstrates a typical usage, together with a temporary path, created with the help of Pythons TemporaryDirectory class:

with tempfile.TemporaryDirectory() as tmp_dir, in_wd(tmp_dir):
    # within the context the tmp_dir is set as the current working directory
    assert os.getcwd() == tmp_dir
# here the current working directory got set back to the original path and the temporary directory got removed

mask_secrets(data, fragments=('password', 'pwd'))[source]

partially-hide secret string values like passwords/credit-card-numbers in deeply nestable data structures.

Parameters:

data¶ (Union[dict, Iterable]) – iterable deep data structure wherein its item values get masked if their related dict item key contains one of the fragments specified in fragments.
fragments¶ (Iterable[str]) – dict key string fragments of which the related value will be masked. each fragment has to be specified with lower case chars! defaults to (‘password’, ‘pwd’) if not passed.

Return type:

Union[dict, Iterable]

Returns:

specified data structure with the secrets masked (¡in-place!).

mask_url(url, replacement='¿¿¿')[source]

hide|replace the password/token in a URL.

Parameters:

url¶ (str) – URL in which an optional password|token will be searched and replaced.
replacement¶ (str) – optional replacement string, if not specified then the default value will be used.

Return type:

str

Returns:

URL with the credentials masked/replaced.

norm_line_sep(text)[source]

convert any combination of line separators in the text arg to new-line characters.

Parameters:: text¶ (str) – string containing any combination of line separators (’\r\n’ or ‘\r’).
Return type:: str
Returns:: normalized/converted string with only new-line (’\n’) line separator characters.

norm_name(name, allow_num_prefix=False)[source]

normalize name to start with a letter/alphabetic/underscore and to contain only alphanumeric/underscore chars.

Parameters:

name¶ (str) – any string to be converted into a valid variable/method/file/… name.
allow_num_prefix¶ (bool) – pass True to allow leading digits in the returned normalized name.

Return type:

str

Returns:

cleaned/normalized/converted name string (e.g., for a variable-/method-/file-name).

norm_path(path, make_absolute=True, remove_base_path='', remove_dots=True, resolve_sym_links=True)[source]

normalize a path, replacing ../. parts or the tilde character (home folder) and transform to relative/abs.

Parameters:

path¶ (str) – path string to normalize/transform.
make_absolute¶ (bool) – pass False to not convert the returned path to an absolute path.
remove_base_path¶ (str) – pass a valid base path to return a relative path, even if the argument values of make_absolute or resolve_sym_links are True.
remove_dots¶ (bool) – pass False to not replace/remove the . and .. placeholders.
resolve_sym_links¶ (bool) – pass False to not resolve symbolic links, passing True implies a True value also for the make_absolute argument.

Return type:

str

Returns:

normalized path string: absolute if remove_base_path is empty and either make_absolute or resolve_sym_links is True; relative if remove_base_path is a base path of path or if path got specified as a relative path and neither make_absolute nor resolve_sym_links is True.

Hint

the normalize() function additionally replaces PATH_PLACEHOLDERS.

now_str(sep='')[source]

return the current UTC timestamp as string (to use as suffix for file and variable/attribute names).

Parameters:: sep¶ (str) – optional prefix and separator character (separating date from time and in time part the seconds from the microseconds).
Return type:: str
Returns:: naive UTC timestamp (without timezone info) as string (length=20 + 3 * len(sep)).

on_ci_host()[source]

check and return True if it is running on the GitLab/GitHub CI host/server.

Return type:: bool
Returns:: True if running on CI host, else False.

Note

env vars always available: ‘CI’ on GitHub (Pre-pipeline); ‘CI_PROJECT_ID’ (internal ProjectId) on GitLab

pep8_format(value, indent_level=0)[source]

PEP-8-conform representation code string of deep dict/list structures, superseding pprint.pformat().

Parameters:

value¶ (Any) – value to format PEP-8-conform (hanging indent always with 4 spaces)..
indent_level¶ (int) – level of indentation. pass e.g. 1 to indent the output with 4 spaces.

Returns:

representation string of the specified value.

read_bin_file(file_path)[source]

returning the binary content of the specified by the file_path argument.

Parameters:

file_path¶ (str) – path/name of the file to load the content from.

Return type:

bytes

Returns:

file content bytes array.

Raises:

FileNotFoundError – if the file to read from does not exist.
OSError – if file_path is misspelled or contains invalid characters.
PermissionError – if the current OS user account lacks permissions to read the file content.

read_file(file_path, encoding=None, error_handling='ignore')[source]

returning the string content of the text file specified by file_path argument.

Parameters:

file_path¶ (str) – path/name of the file to load the content from.
encoding¶ (str | None) – encoding used to load and convert/interpret the file content (passed onto the encoding parameter of the built-in open function).
error_handling¶ (str | None) – pass ‘strict’ or None to raise a ValueError exception on encoding errors. the default value ‘ignore’ will ignore any decoding errors (resulting in missing characters in the return value). passed onto the errors parameter of the built-in open function.

Return type:

str

Returns:

the content of the file as a string.

Raises:

FileNotFoundError – if the file to read from does not exist.
IsADirectoryError – file_path points to a directory instead of a file.
LookupError – unknown encoding name.
NotADirectoryError – part of the path expected to be a directory is actually a file.
OSError – filename too long, too many open files, device/network error, file_path misspelled or contains invalid characters.
PermissionError – if the current OS user account lacks permissions to read the file content.
UnicodeDecodeError – file content cannot be decoded with the specified encoding or error_handling.
ValueError – invalid error_handling argument.

round_traditional(num_value, num_digits=0)[source]

round numeric value traditional.

needed because python round() is working differently, e.g., round(0.075, 2) == 0.07 instead of 0.08 inspired by https://stackoverflow.com/questions/31818050/python-2-7-round-number-to-nearest-integer.

Parameters:

num_value¶ (float) – float value to be round.
num_digits¶ (int) – number of digits to be round (def=0 - rounds to an integer value).

Return type:

float

Returns:

rounded value.

sign(number)[source]

return ths sign (-1, 0, 1) of a number.

Parameters:: number¶ (float) – any number of type float or int.
Return type:: int
Returns:: -1 if the number is negative, 0 if it is zero, or 1 if it is positive.

snake_to_camel(name, back_convertible=False)[source]

convert name from snake_case to CamelCase.

Parameters:

name¶ (str) – name string composed of parts separated by an underscore character (NAME_PARTS_SEP).
back_convertible¶ (bool) – pass True to get the first character of the returned name in lower-case if the snake name has no leading underscore character (and to allow the conversion between snake and camel case without information loss).

Return type:

str

Returns:

name in camel case.

to_ascii(unicode_str)[source]

converts Unicode string into ascii representation.

useful for fuzzy string compare; inspired by MiniQuark’s answer in: https://stackoverflow.com/questions/517923/what-is-the-best-way-to-remove-accents-in-a-python-unicode-string

Parameters:: unicode_str¶ (str) – string to convert.
Return type:: str
Returns:: converted string (replaced accents, diacritics, … into normal ascii characters).

url_failure(url, token='', username='', password='', git_repo=False, timeout=None)[source]

determine if and why an FTP or HTTP[S] target is not available via a GET request.

Parameters:

url¶ (str) – URL of a target|page|file to check (not downloaded, fetching only the header).
token¶ (str) – optional bearer token to authenticate (only for HTTPS protocol).
username¶ (str) – optional username to authenticate (for HTTPS, together with the password argument).
password¶ (str) – optional password to authenticate (for HTTPS, together with the username argument).
git_repo¶ (bool) – optimized check for Git repository HTTP servers/sites (like GitHub, GitLab, Bitbucket, Gitea, SourceHut, Mercury, etc. as long as they implement Smart HTTP). if specified then the url has to point to a repository.
timeout¶ (float | None) – connection timeout in seconds (see urllib.request.urlopen()).

Return type:

str

Returns:

empty string if target header is available, else an error description. if an FTP|HTTP response error occurred then the error/status code will be returned in the first 3 characters.

Note

credentials for server authentication can be specified either (1) embedded into the specified url argument, (2) as bearer token in the token argument or (3) via the username/password arguments. in all cases the functino will remove these secrets from the returned error description string.

utc_datetime()[source]

return the current UTC timestamp as string (to use as suffix for file and variable/attribute names).

Return type:: datetime
Returns:: timestamp string of the actual UTC date and time.

write_bin_file(file_path, content, make_dirs=False)[source]

(over)write the file specified by file_path with the specified binary/bytes content.

Parameters:

file_path¶ (str) – file path/name to write the passed content into (overwriting any previous content!).
content¶ (bytes) – new file content specified as bytes.
make_dirs¶ (bool) – pass True to automatically create not existing folders of the file path.

Raises:

FileExistsError – if the file to write to exists already and is write-protected.
FileNotFoundError – if parts of the file path do not exist.
IsADirectoryError – file_path points to a directory instead of a file.
NotADirectoryError – part of the path expected to be a directory is actually a file.
OSError – disk full, filename too long, too many open files, network or device disconnected, file_path is misspelled or contains invalid characters.
PermissionError – if the current OS user account lacks permissions to write the file content.
TypeError – content is not of type bytes.

write_file(file_path, content, encoding=None, make_dirs=False)[source]

(over)write the file specified by file_path with the specified string content.

Parameters:

file_path¶ (str) – file path/name to write the passed content into (overwriting any previous content!).
content¶ (str) – new file content passed as string.
encoding¶ (str | None) – encoding used to write/convert/interpret the file content to write (defaults to utf-8).
make_dirs¶ (bool) – pass True to automatically create not existing folders of the file path (specified in file_path).

Raises:

FileExistsError – if the file to write to exists already and is write-protected.
FileNotFoundError – if parts of the file path do not exist.
IsADirectoryError – file_path points to a directory instead of a file.
LookupError – unknown encoding name.
NotADirectoryError – part of the path expected to be a directory is actually a file.
OSError – disk full, filename too long, too many open files, network or device disconnected, file_path is misspelled or contains invalid characters.
PermissionError – if the current OS user account lacks permissions to write the file content.
TypeError – content is not of type str.
UnicodeEncodeError – content cannot be encoded using the selected encoding.
ValueError – other encoding errors, invalid mode or incompatible arguments.

to extend this function for Android 14+, see https://github.com/beeware/toga/pull/1158#issuecomment-2254564657 and https://gist.github.com/neonankiti/05922cf0a44108a2e2732671ed9ef386 Yes, to use ACTION_CREATE_DOCUMENT, you don’t supply a URI in the intent. You wait for the intent result, and that will contain a URI which you can write to. See #1158 (comment - https://github.com/beeware/toga/pull/1158#issuecomment-2254564657) for a link to a Java example, and #1158 (comment - https://github.com/beeware/toga/pull/1158#issuecomment-1446196973) for how to wait for an intent result. Related german docs: https://developer.android.com/training/data-storage/shared/media?hl=de

class UnsetType[source]

Bases: object

(singleton) UNSET (type) object class.

__bool__()[source]: ensure to be evaluated as False, like None.

__len__()[source]: ensure to be evaluated as empty.

UNSET: Final = <ae.base.UnsetType object>: pseudo value used for attributes/arguments if None is needed as a valid value