ae.system

Python system helpers

the ae.system portion provides base helpers for the Python and OS systems environments.

Note

on import, this module checks if it is running on the Android OS. if so, it will monkey patch the shutil module to ensure functions like copy and move work correctly. to prevent permission-related errors, this module should be one of the first imports in your Android app’s main module.

operating system info helpers

useful helper functions to inspect the operating system:

os_platform: a string identifying the operating system (e.g., ‘linux’, ‘win32’, ‘android’, ‘ios’).
os_device_id: a string with the ID/name of the device.
os_host_name(): determines the operating system’s host/machine name.
os_local_ip(): determines the local IP address of the machine.
os_user_name(): determines the current logged-in user’s name.

manage environment variables & .env files

parse_dotenv(): parses a .env file and returns its key-value pairs as a dictionary.
late_env_var_resolver(): substitutes environment variables within the value of other environment variables.
load_dotenvs(): detects and loads all relevant .env files from the current working directory and optional also from the main module’s path.
load_env_var_defaults(): recursively searches parent directories for .env files and loads any undeclared variables.
sys_env_dict(): returns a dictionary containing the most important Python runtime OS environment values.
sys_env_text(): compiles a formatted text block with system environment information, useful for logging.

application & project helpers

functions to aid in application setup, configuration, and build introspection.

app_name_guess(): attempts to determine the name of the currently running application from its environment.
build_config_variable_values(): reads variable values from an APP_BUILD_CFG_FILENAME/buildozer.spec file.
instantiate_config_parser(): returns a ConfigParser instance pre-configured for case-sensitive keys and extended interpolation.
project_main_file(): determines the absolute path to the main module file of a project package (where the __version__ of the app|package is defined).
main_file_paths_parts(): returns a tuple of possible main/version file path names combinations of any project.

modules and call stack inspection

dynamically inspect modules, execution frames, and variables on the call stack.

module_attr(): dynamically gets a reference to a module or any attribute (variable, function, class) within it.
module_file_path(): determines the absolute file path of the module from which it is called.
module_find(): determine the file path of a Python module.
module_load(): search, import and execute a Python module dynamically without adding it to sys.modules.
stack_frames(): a generator that yields frames from the call stack, starting at a specified depth.
stack_var(): finds the value of a specific variable by searching up the call stack.
stack_vars(): returns the global and local variables from a specific frame in the call stack.
full_stack_trace(): generates a complete, detailed stack trace including local variables from an exception.

Hint

the ae.core.AppBase class uses these helper functions to determine the version and title of an application, if these values are not specified in the instance initializer.

system constants and classes

APP_BUILD_CFG_FILENAME: default name for a build configuration file (buildozer.spec).
DOTENV_FILENAME: default name for environment variable files (.env).
DOTENV_LINE_MATCHER: compiled regular expression to parse a line of an OS environment file.
DOTENV_VAR_IN_VAL_MATCHER: compiled regular expression to parse the value of an OS env variable.
SKIPPED_MODULES: a tuple of module names to be ignored by stack inspection functions.

an instance of the class PyMo is representing a Python module/package, and provides:

project, namespace, module, package and portion names
project, module, package and portion paths
dynamic load of a code module or package

Module Attributes

`APP_BUILD_CFG_FILENAME`	gui app build config file
`DOTENV_FILENAME`	name of the file containing console/shell environment variables
`MODULE_NAME_SEPS`	separators considered equivalent for comparison (PEP 503)
`SKIPPED_MODULES`	skipped modules used as default by `stack_var()` and `stack_vars()`
`EnvVarsType`	environment variables dict/mapping
`EnvVarsLateResolvedType`	mapping of DOTENV_VAR_IN_VAL_MATCHER results
`os_platform`	operating system / platform string (see `_os_platform()`).
`os_device_id`	user-definable id/name of the device, defaults to os_host_name() on most platforms, alternatives are:

Functions

`app_name_guess`()	guess/try to determine the name of the currently running app (w/o assessing not yet initialized app instance).
`build_config_variable_values`(*names_defaults)	determine build config variable values from the `buildozer.spec` file in the current directory.
`full_stack_trace`(ex[, frames_with_locals])	generates a complete, detailed stack trace including local variables from an exception.
`instantiate_config_parser`()	instantiate and prepare config file parser.
`late_env_var_resolver`(env_vars, loaded_vars, ...)	late resolve/expand/substitute of env variables in env var values.
`load_dotenvs`([from_module_path])	detect and load not defined OS environment variables from `.env` files.
`load_env_var_defaults`(start_dir, env_vars)	load undeclared env var defaults from a chain of `.env` files starting in the specified folder or its parent.
`main_file_paths_parts`(portion_name)	determine possible/supported main/version file name and path parts, relative to the project root folder.
`module_attr`(import_name, attr_name)	determine dynamically a reference to any attribute (variable/func/class) declared in a module.
`module_file_path`([local_object])	determine the absolute path of the module from which this function got called.
`module_find`(import_name)	determine the file path of a Python module.
`module_load`(import_name[, path])	search, import and execute a Python module dynamically without adding it to sys.modules.
`os_host_name`()	determine the operating system host/machine name.
`os_local_ip`()	determine ip address of this system/machine in the local network (LAN or WLAN).
`os_user_name`()	determine the operating system username.
`parse_dotenv`(file_path, late_resolved[, ...])	parse `.env` file content and return environment variable names as dict keys and values as dict values.
`project_main_file`(import_name[, project_path])	determine the main module file path of a project package containing the project __version__ module variable.
`stack_frames`([depth])	generator returning the call stack frame from the level given in `depth`.
`stack_var`(name, *skip_modules[, scope, depth])	determine variable value in calling stack/frames.
`stack_vars`(*skip_modules[, find_name, ...])	determine all global and local variables in a calling stack/frames.
`sys_env_dict`()	returns dict with python system run-time environment values.
`sys_env_text`([ind_ch, ind_len, key_ch, ...])	compile a formatted text block with system environment info.

Classes

PyMo(import_name[, project_path])

helper class to create, represent, inspect, load or import a Python module or package.

APP_BUILD_CFG_FILENAME = 'buildozer.spec': gui app build config file

DOTENV_FILENAME = '.env': name of the file containing console/shell environment variables

MODULE_NAME_SEPS = ('_', '.', '-'): separators considered equivalent for comparison (PEP 503)

SKIPPED_MODULES = ('ae.base', 'ae.system', 'ae.files', 'ae.paths', 'ae.dynamicod', 'ae.core', 'ae.console', 'ae.snell', 'ae.managed_files', 'ae.gui', 'ae.gui.app', 'ae.gui.tours', 'ae.gui.utils', 'ae.kivy', 'ae.kivy.apps', 'ae.kivy.behaviors', 'ae.kivy.i18n', 'ae.kivy.tours', 'ae.kivy.widgets', 'ae.enaml_app', 'ae.toga_app', 'ae.pyglet_app', 'ae.pygobject_app', 'ae.dabo_app', 'ae.qpython_app', 'ae.appjar_app', 'importlib._bootstrap', 'importlib._bootstrap_external'): skipped modules used as default by stack_var() and stack_vars()

EnvVarsType

environment variables dict/mapping

alias of MutableMapping[str, str]

EnvVarsLateResolvedType

mapping of DOTENV_VAR_IN_VAL_MATCHER results

alias of dict[str, list[tuple[str, str, str, str]]]

app_name_guess()[source]

guess/try to determine the name of the currently running app (w/o assessing not yet initialized app instance).

Return type:: str
Returns:: application name/id or “unguessable” if not guessable.

build_config_variable_values(*names_defaults, section='app')[source]

determine build config variable values from the buildozer.spec file in the current directory.

Parameters:

names_defaults¶ (tuple[str, Any]) – tuple of tuples of build config variable names and default values.
section¶ (str) – name of the spec file section, using ‘app’ as default.

Return type:

tuple[Any, ...]

Returns:

tuple of build config variable values (using the passed default value if not specified in the APP_BUILD_CFG_FILENAME spec file or if this file does not exist in cwd).

full_stack_trace(ex, frames_with_locals=3)[source]

generates a complete, detailed stack trace including local variables from an exception.

Parameters:

ex¶ (Exception) – exception instance.
frames_with_locals¶ (int) – number of the deepest frames to show also the local variables for.

Return type:

str

Returns:

text block string (formatted by os.linesep) with stack trace info.

instantiate_config_parser()[source]

instantiate and prepare config file parser.

ensures that the ConfigParser instance is correctly configured, e.g., to support case-sensitive config variable names and to use ExtendedInterpolation as the interpolation argument.

Return type:: ConfigParser

late_env_var_resolver(env_vars, loaded_vars, late_resolved)[source]

late resolve/expand/substitute of env variables in env var values.

Parameters:

env_vars¶ (MutableMapping[str, str]) – all cached environment variables (preferred to os.environ), will get substituted. also used to search&resolve env var values (if not found then searched in os.environ).
loaded_vars¶ (MutableMapping[str, str]) – recently loaded environment variables, will get substituted.
late_resolved¶ (dict[str, list[tuple[str, str, str, str]]]) – matches of loaded env vars to be resolved late (after all env vars got detected and loaded). the key of this dict is the name of the env variable which has other env vars in its values to be resolved/substituted. the item value of this dict is a list of matcher group tuples for each found env variable. the group/tuple items are (0) escape character, (1) the dollar character, (2) the env var name literal (optionally in curly brackets) and (3/-1) the env var name.

load_dotenvs(from_module_path=False)[source]

detect and load not defined OS environment variables from .env files.

Parameters:

from_module_path¶ (bool) –

pass True to load OS environment variables (that are not already loaded from .env files situated in or above the current working directory) also from/above the folder of the first module in the call stack that gets not excluded/skipped by stack_var().

in order to also load .env files in/above the project folder. call this function from the main module of project/app.

Note

only variables that are not already defined in the OS environment variables mapping os.environ will be loaded/added. variables will be loaded first from the first .env file found in or above the current working directory, while the variable values in the deeper situated files are overwriting the values defined in the .env files situated in the above folders.

load_env_var_defaults(start_dir, env_vars)[source]

load undeclared env var defaults from a chain of .env files starting in the specified folder or its parent.

Parameters:

start_dir¶ (str) – folder to start search of an .env file, if not found, then also checks the parent folder. if an .env `` file got found, then put their shell environment variable values into the specified :paramref:`~load_env_var_defaults.env_vars` mapping if they are not already there. after processing the first ``.env file, it repeats to check for further .env files in the parent folder to load them too, until either detecting a folder without an .env file or until an .env got loaded from the root folder.
env_vars¶ (MutableMapping[str, str]) – environment variables mapping to be amended with env variable values from any found .env file. pass Python’s os.environ to amend this mapping directly with all the already not declared environment variables.

Return type:

MutableMapping[str, str]

Returns:

env var names (keys) and values added to env_vars.

main_file_paths_parts(portion_name)[source]

determine possible/supported main/version file name and path parts, relative to the project root folder.

Parameters:: portion_name¶ (str) – portion or package name.
Return type:: tuple[tuple[str, ...], ...]
Returns:: tuple of tuples of main/version file name path parts.

module_attr(import_name, attr_name)[source]

determine dynamically a reference to any attribute (variable/func/class) declared in a module.

Parameters:

import_name¶ (str) – import-/dot-name of the distribution/module/package to load/import.
attr_name¶ (str) – name of the attribute declared within the module.

Return type:

Any | UnsetType | None

Returns:

module attribute value, or None if the module got not found or UNSET if the module attribute doesn’t exist.

Note

a previously not imported module will not be added to sys.modules by this function.

module_file_path(local_object=None)[source]

determine the absolute path of the module from which this function got called.

Parameters:: local_object¶ (Optional[Callable]) – optional local module, class, method, function, traceback, frame, or code object of the calling module (passing lambda: 0 also works). omit this argument in order to use the __file__ module variable (which will not work if the module is frozen by py2exe or PyInstaller).
Return type:: str
Returns:: module path (inclusive module file name) or empty string if not found/determinable.

module_find(import_name)[source]

determine the file path of a Python module.

Parameters:: import_name¶ (str) – dot-name of the module to find.
Return type:: Union[str, list[str]]
Returns:: absolute file path of the found module, else a list of error strings.

module_load(import_name, path=<ae.base.UnsetType object>)[source]

search, import and execute a Python module dynamically without adding it to sys.modules.

Parameters:

import_name¶ (str) – dot-name of the module to import.
path¶ (str | UnsetType | None) – optional file path of the module to import. if this arg is not specified or has the default value (UNSET), then the path will be determined from the import name. specify None to prevent the module search.

Return type:

ModuleType | list[str]

Returns:

a reference to the module if the module could be loaded, else a list of error strings.

os_host_name()[source]

determine the operating system host/machine name.

Return type:: str
Returns:: machine name string.

os_local_ip()[source]

determine ip address of this system/machine in the local network (LAN or WLAN).

inspired by answers of SO users @dml and @fatal_error to the question: https://stackoverflow.com/questions/166506.

Return type:: str
Returns:: ip address of this machine in the local network (WLAN or LAN/ethernet) or empty string if this machine is not connected to any network.

_os_platform()[source]

determine the operating system where this code is running (used to initialize the os_platform variable).

Return type:

str

Returns:

operating system (extension) as string. extending Python’s sys.platform() for mobile platforms like Android and iOS:

’android’ for all Android systems.
’cygwin’ for MS Windows with an installed Cygwin extension.
’darwin’ for all Apple Mac OS X systems.
’freebsd’ for all other BSD-based unix systems.
’ios’ for all Apple iOS systems.
’linux’ for all other unix systems (like Arch, Debian/Ubuntu, Suse, …).
’win32’ for MS Windows systems (w/o the Cygwin extension).

os_platform = 'linux'

operating system / platform string (see _os_platform()).

this string value gets determined for most of the operating systems with the help of Python’s sys.platform() function and additionally detects the operating systems iOS and Android (currently not fully supported by Python).

os_device_id = 'build-32916323-project-510766-ae'

user-definable id/name of the device, defaults to os_host_name() on most platforms, alternatives are:

on Android (check with adb shell ‘settings get global device_name’ and adb shell ‘settings list global’):

Settings.Global.DEVICE_NAME (Settings.Global.getString(context.getContentResolver(), “device_name”))
android.os.Build.DEVICE/.MANUFACTURER/.BRAND/.HOST
DeviceName.getDeviceName()

on MS Windows:

os.environ[‘COMPUTERNAME’]

on all other platforms:

socket.gethostname()

os_user_name()[source]

determine the operating system username.

Return type:: str
Returns:: username string.

parse_dotenv(file_path, late_resolved, exclude_vars=())[source]

parse .env file content and return environment variable names as dict keys and values as dict values.

Parameters:

file_path¶ (str) – string with the name/path of an existing .env/DOTENV_FILENAME file.
late_resolved¶ (dict[str, list[tuple[str, str, str, str]]]) – mapping extended with matches of env vars found in the returned env var values.
exclude_vars¶ (Container) – names of env vars to preserve their value (do not return).

Return type:

MutableMapping[str, str]

Returns:

mapping with parsed environment variable names and values.

project_main_file(import_name, project_path='')[source]

determine the main module file path of a project package containing the project __version__ module variable.

Parameters:

import_name¶ (str) – name of the module/package (including namespace prefixes, separated with dots).
project_path¶ (str) – optional path where the project of the package/module is situated. not needed if the current working directory is the root folder of either the import_name project or of a sister project (under the same project parent folder).

Return type:

str

Returns:

absolute file path of the main module or empty string if no main/version file is found.

stack_frames(depth=1)[source]

generator returning the call stack frame from the level given in depth.

Parameters:: depth¶ (int) – the stack level to start; the first returned frame by this generator. the default value (1) refers to the next deeper stack frame, respectively the one of the caller of this function. pass 2 or a higher value if you want to start with an even deeper frame/level.
Return type:: Generator
Returns:: generated frames of the call stack.

stack_var(name, *skip_modules, scope='', depth=1)[source]

determine variable value in calling stack/frames.

Parameters:

name¶ (str) – variable name to search in the calling stack frames.
skip_modules¶ (str) – module names to skip (def=see SKIPPED_MODULES module constant).
scope¶ (str) – pass ‘locals’ to only check for local variables (ignoring globals) or ‘globals’ to only check for global variables (ignoring locals). the default value (an empty string) will not restrict the scope, returning either a local or global value.
depth¶ (int) – the calling level from which on to search. the default value (1) refers to the next deeper stack frame, which is the caller of the function. pass 2 or an even higher value if you want to start the variable search from a deeper level in the call stack.

Return type:

Optional[Any]

Returns:

the variable value of a deeper level within the call stack or UNSET if the variable was not found.

stack_vars(*skip_modules, find_name='', min_depth=1, max_depth=0, scope='')[source]

determine all global and local variables in a calling stack/frames.

Parameters:

skip_modules¶ (str) – module names to skip (def=see SKIPPED_MODULES module constant).
find_name¶ (str) – if passed, then the returned stack frame must contain a variable with the passed name.
scope¶ (str) – scope to search the variable name passed via find_name. pass ‘locals’ to only search for local variables (ignoring globals) or ‘globals’ to only check for global variables (ignoring locals). passing an empty string will find the variable within either locals or globals.
min_depth¶ (int) – the call stack level from which on to search. the default value (1) refers the next deeper stack frame, respectively, to the caller of this function. pass 2 or a higher value if you want to get the variables from a deeper level in the call stack.
max_depth¶ (int) – the maximum depth in the call stack from which to return the variables. if the specified argument is not zero and no skip_modules are specified, then the first deeper stack frame that is not within the default SKIPPED_MODULES will be returned. if this argument and find_name get not passed, then the variables of the top stack frame will be returned.

Return type:

tuple[dict[str, Any], dict[str, Any], int]

Returns:

tuple of the global and local variable dicts and the depth in the call stack.

sys_env_dict()[source]

returns dict with python system run-time environment values.

Return type:: dict[str, Any]
Returns:: python system run-time environment values like python_ver, argv, cwd, executable, frozen and bundle_dir (if bundled with pyinstaller).

Hint

sys_env_text(ind_ch=' ', ind_len=12, key_ch='=', key_len=15, extra_sys_env_dict=None)[source]

compile a formatted text block with system environment info.

Parameters:

ind_ch¶ (str) – indent character (defaults to “ “).
ind_len¶ (int) – indent depths (default=12 characters).
key_ch¶ (str) – key-value separator character (default=”=”).
key_len¶ (int) – key-name minimum length (default=15 characters).
extra_sys_env_dict¶ (Optional[MutableMapping[str, str]]) – dict with additional system info items.

Return type:

str

Returns:

text block with system environment info.

class PyMo(import_name, project_path='')[source]

Bases: ErrorMsgMixin

helper class to create, represent, inspect, load or import a Python module or package.

__init__(import_name, project_path='')[source]

create a new instance of the class PyMp.

Parameters:

import_name¶ (str) – import name of the module/package/portion.
project_path¶ (str) – optional project root folder path.

classmethod from_name(name, namespace_name='', project_path='')[source]

create a PyMp instance from the name of a project/package/portion/module.

Parameters:

name¶ (str) – import/pip/package/project name of the module/package/portion.
namespace_name¶ (str) – namespace name/path w/ or w/o the module/portion name (at the end).
project_path¶ (str) – root folder path of the source project.

Return type:

Self

Returns:

PyMo instance of the Python module/package/portion. specifying invalid arguments results in an instance with an import_name+ of ‘import.name.error’.

classmethod from_path(project_path, namespace_name='')[source]

create a PyMp instance from the specified project-root folder (source-code) path.

Parameters:

project_path¶ (str) – root folder path of the project source code.
namespace_name¶ (str) – namespace name/path with or without the module/portion name.

Return type:

Self

Returns:

PyMo instance of the Python module/package/portion.

__repr__()[source]: Return repr(self).

_warn(msg)[source]

property imported_module: ModuleType | None: try to import module, add it to sys.modules and return the spec of it (or None if not found/imported).

property loaded_module: ModuleType | None: try to load module (without adding it to sys.modules). return the spec of it or None if not found.

property module_file_path: str: module file path (absolute if project_path got specified, else relative to the project root folder).

property namespace_name: str: namespace name/path of the Python module/package represented by this instance.

property name_parts: list[str]: module-/portion-name parts of the Python module, including namespace name prefix(es).

property package_name: str: name of the package/project of the Python module.

property package_dir_path: str: package path w/o PY_INIT, w/ project_path if it got specified, else relative to the project root folder.

property package_file_path: str: package path w/ PY_INIT (and project_path if specified, else relative to the project root folder).

property pip_name: str: pip package name of the Python module.

property portion_name: str: name of the Python module

property project_name: str: alias of the package_name property

property project_root_path: str: normalized absolute project root path of the Python module.