ae.console

console application environment

an instance of the ConsoleApp class is representing a python application with dynamically configurable logging, debugging features (inherited from AppBase), command line arguments and config files and options.

auto-collected app name, title and version

if one of the kwargs app_title or app_version is not specified in the init call of the ConsoleApp class instance, then they will automatically get determined from your app main module: the app title from the docstring title, and the application version string from the __version__ variable:

""" module docstring title """
from ae.console import ConsoleApp

__version__ = '1.2.3'

ca = ConsoleApp()

assert ca.app_title == "module docstring title"
assert ca.app_version == '1.2.3'

ConsoleApp also determines on instantiation the name/id of your application, if not explicitly specified in app_name. other application environment vars/options (like e.g., the application startup folder path and the current working directory path) will be automatically initialized and provided via the app instance.

define command line arguments and options

the methods add_argument() and add_option() are defining command line arguments and config options, finally parsed/loaded by calling run_app():

ca = ConsoleApp(app_title="command line arguments demo", app_version="3.6.9")
ca.add_argument('argument_name_or_id', help="Help text for this command line argument")
ca.add_option('option_name_or_id', "help text for this command line option", "default_value")
...
ca.run_app()

the values of the commend line arguments and options are determined via the methods get_argument() and get_option(). additional configuration values, stored in INI/CFG files, are accessible via the get_variable() method.

configuration files, sections, variables and options

a config file consists of config sections, each section provides config variables and config options to parametrize your application at run-time.

config files

configuration files can be shared between apps or used exclusively by one app. the following file names are recognized and loaded automatically on app initialization:

config file	used for …. config variables and options
<any_path_name_and_ext>	application/domain specific
<app_name>.ini	application specific (read-/write-able)
<app_name>.cfg	application specific (read-only)
.app_env.cfg	application/suite specific (read-only)
.sys_env.cfg	general system (read-only)
.sys_env<SYS_ENV_ID>.cfg	the system with SYS_ID (read-only)

the above table is ordered by the preference to search (priority to determine/get) the value of a config variable/option. so the values stored in the domain/app-specific config file will always precede/overwrite the values of the system-specific config files. a more detailed documentation of the exact search order you find in the doc-string of the add_cfg_files() method.

app/domain-specific config files can be specified explicitly, either on initialization of the ConsoleApp instance via the kwarg additional_cfg_file, or by calling the method add_cfg_files(). they can have any file extension and can be placed into any accessible folder.

all other config files have defined names with a .ini or .cfg file extension, and get automatically recognized in the current working directory, in the user data directory (see ae.paths.user_data_path()) and in the application installation directory.

config sections

this module is supporting the config file format of Pythons built-in ConfigParser class, extended by complex config value types. the following examples show a config file with two config sections containing one config option (named log_file) and two config variables (configVar1 and configVar2):

[aeOptions] log_file = ‘./logs/your_log_file.log’ configVar1 = [‘list-element1’, (‘list-element2-1’, ‘list-element2-2’,), {}]

[YourSectionName] configVar2 = {‘key1’: ‘value 1’, ‘key2’: 2222, ‘key3’: datetime.datetime.now()}

the config section aeOptions (defined by MAIN_SECTION_NAME) is the default or main section, storing the fallback values of any pre-defined command line config option and of some config variables.

config variables

to read/fetch the value of a config variable, call the method get_variable() with their variable and section names.

config variables can store complex data types. in the above example config file, the config variable configVar1 holds a list with 3 items: the first item is a string, the second a tuple, and the third an empty dict. the type of the config variable gets automatically detected when you store a config variable value.

a config variable value can be stored into a config file, by calling the set_variable() method.

Note

the value of a config variable stored in a config file can be overwritten by defining an OS environment variable with a name that is equal to the snake+upper-case converted names of the config-section and -variable.

e.g., declare an OS environment variable with the name AE_OPTIONS_LOG_FILE to overwrite the value of the pre-defined config option/variable log_file.

config options

config options are actually config variables which can be specified also as command line option. most config options are expecting a value, specified after the option name and a space or an equal character, like shown for the log_file config option in the following example command line:

$ your_application –log_file=’your_new_log_file.log’

config options and their default value are getting defined via the add_option() method of the main ConsoleApp instance. if the fallback/default value you specify in this call is either None or UNSET then the resulting default value will be searched first in the OS environment variables and then in the config files, similar like for any other config variable (but with the section name fixed to aeOptions).

Hint

the value type of a config option can be fixed on their definition via the value argument.

the method get_option() determines the value of a config option:

my_log_file_name = ca.get_option('log_file')

to change the fallback value of a configuration option call the set_option(). specify False to the save_to_config argument for a temporary change of the option fallback value.

pre-defined configuration options

the following config options/variables are pre-defined in the main config section and recognized by this module, some of them also by the ae.core module/portion:

debug_level: debug logging verbosity level config option
force: config option to force the app to not quit on skip-able/ignorable errors.
log_file: ae logging file name (this is also a config option - set-able as command line arg)
logging_params: general ae logging configuration parameters (py and ae logging)
py_logging_params: python logging configuration
registered_users: list of registered user names/ids (extended by calls of ConsoleApp.register_user() method)
user_id: id of the app user (default is the operating system user name <ae.system.os_user_name>)
user_specific_cfg_vars: list of config variables storing an individual value for each registered user (see section user specific config variables)

for more verbose logging, specify either on the command line or in a config file, the config option debug_level (or as short option -D) with a value of 2 (for verbose). the supported config option values are documented here.

the force command line option (short option -f) can be specified multiple times as command line argument to skip/ignore multiple errors, that are explicitly checked by calls to the ConsoleApp.chk() method.

the value of the pre-defined config option log_file specifies the log file path/file_name. also, this option can be abbreviated on the command line with the short -L option id.

Note

after an explicit definition of the optional config option user_id via add_option() it will be automatically used to initialize the user_id attribute.

user specific config variables

config variables specified in the set user_specific_cfg_vars get automatically recognized as user-specific. override the method :meth`~ConsoleApp._init_default_user_cfg_vars` in your main app instance to define or revoke which config variables the app is storing individually for each user.

Hint

to permit individual sets of user-specific config variables for a user (or group), add the config variable user_specific_cfg_vars in the user-specific config file section(s). don’t forget in this special case to also add there also this config variable, e.g., as (‘aeOptions’, ‘user_specific_cfg_vars’).

Module Attributes

`MAIN_SECTION_NAME`	default name of the main config section
`USER_NAME_MAX_LEN`	maximum length of a username/id <ae.console.ConsoleApp.user_id>

Functions

config_value_string(value)

convert passed value to a string to store them in a config/ini file.

Classes

ConsoleApp([app_title, app_name, ...])

provides command line arguments and options, config options, logging and debugging for your application.

MAIN_SECTION_NAME: str = 'aeOptions': default name of the main config section

USER_NAME_MAX_LEN = 12: maximum length of a username/id <ae.console.ConsoleApp.user_id>

config_value_string(value)[source]

convert passed value to a string to store them in a config/ini file.

Parameters:: value¶ (Any) – value to convert to ini variable string/literal.
Return type:: str
Returns:: ini variable literal string.

Note

Literal converts the returned string format back into the representing value.

class ConsoleApp(app_title='', app_name='', app_version='', sys_env_id='', debug_level=2, multi_threading=False, suppress_stdout=False, cfg_opt_eval_vars=None, additional_cfg_files=(), cfg_opt_val_stripper=None, **logging_params)[source]

Bases: AppBase

provides command line arguments and options, config options, logging and debugging for your application.

most applications only need a single instance of this class. each instance is encapsulating a ConfigParser and an ArgumentParser instance. so only apps with threads and different sets of config options for each thread could create a separate instance of this class.

instance attributes (ordered alphabetically - ignoring underscore characters):

_arg_parser ArgumentParser instance.
cfg_opt_choices valid choices for pre-/user-defined options.
cfg_opt_eval_vars additional dynamic variable values that are getting set via the cfg_opt_eval_vars argument of the method ConsoleApp.__init__() and get then used in the evaluation of evaluable config option values.
_cfg_files iterable of config file names that are getting loaded and parsed (specify additional configuration/INI files via the additional_cfg_files argument).
cfg_options pre-/user-defined options (dict of Literal instances defined via add_option()).
_cfg_opt_val_stripper callable to strip option values.
_cfg_parser ConfigParser instance.
_main_cfg_fnam main config file name.
_main_cfg_mod_time last modification datetime of the main config file.
_parsed_arguments ArgumentParser.parse_args() return.

__init__(app_title='', app_name='', app_version='', sys_env_id='', debug_level=2, multi_threading=False, suppress_stdout=False, cfg_opt_eval_vars=None, additional_cfg_files=(), cfg_opt_val_stripper=None, **logging_params)[source]

initialize a new ConsoleApp instance.

Parameters:

app_title¶ (str) –
application title/description to set the instance attribute app_title.

if not specified, then the docstring of your app’s main module will be used (see example).
app_name¶ (str) –
application instance name to set the instance attribute app_name.

if not specified, then the base name of the main module file name will be used.
app_version¶ (str) –
application version string to set the instance attribute app_version.

if not specified, then the value of a global variable with the name __version__` will be used (if declared in the actual call stack).
sys_env_id¶ (str) –
system environment id to set the instance attribute sys_env_id.

this value is also used as a file name suffix to load all the system config variables in sys_env<suffix>.cfg. pass e.g. ‘LIVE’ to init this ConsoleApp instance with config values from sys_envLIVE.cfg.

the default value of this argument is an empty string.

Note

if the argument value results as empty string, then the value of the optionally defined OS environment variable AE_OPTIONS_SYS_ENV_ID will be used as default.
debug_level¶ (int) –
default debug level to set the instance property debug_level.

the default value of this argument is DEBUG_LEVEL_DISABLED.
multi_threading¶ (bool) – pass True if this instance is used in a multi-threading app.
suppress_stdout¶ (bool) – pass True (for wsgi apps) to prevent any python print outputs to stdout.
cfg_opt_eval_vars¶ (Optional[dict]) – dict of additional application-specific data values that are used in eval expressions (e.g., AcuSihotMonitor.ini).
additional_cfg_files¶ (Iterable) – iterable of additional CFG/INI file names (opt. incl. abs/rel. path).
cfg_opt_val_stripper¶ (Optional[Callable]) – callable to strip/reformat/normalize the option choices.
logging_params¶ – all other kwargs are interpreted as logging configuration values - the supported kwargs are all the method kwargs of init_logging().

_cfg_parser: ConfigParser instance

cfg_options: dict[str, Literal]: all config options

cfg_opt_choices: dict[str, Iterable]: all valid config option choices

cfg_opt_eval_vars: dict: app-specific vars for init of cfg options

_cfg_files: list: specified/added INI/CFG file paths

_main_cfg_fnam: str: default main config file <app_name>.INI in the cwd (possibly overwritten by :meth:`.load_cfg_files)

_main_cfg_mod_time: float: main config file modification datetime

_cfg_opt_val_stripper: Callable | None: callable to strip or normalize config option choice values

_parsed_arguments: Namespace | None: storing returned namespace of ArgumentParser.parse_args() call, used to retrieve command line args

registered_users: list[str]

user_specific_cfg_vars: set[tuple[str, str]]

_arg_parser: ArgumentParser

_init_default_user_cfg_vars()[source]

init user default config variables.

override this method to add module-/app-specific config vars that can be set individually per user.

_init_logging(logging_params)[source]

determine and init logging config.

Parameters:: logging_params¶ (dict[str, Any]) – logging config dict that will be amended with cfg values.
Return type:: Optional[str]
Returns:: None if py logging is active, log file name if ae logging is set in cfg or args or empty string if no logging got configured in cfg/args.

the logging configuration can be specified in several alternative places. the precedence on various existing configurations is (the highest precedence first):

log_file configuration option specifies the name of the used ae log file (will be read after initialisation of this app instance)
logging_params configuration variable dict with a py_logging_params key to activate python logging
logging_params configuration variable dict with the ae log file name in the key log_file_name
py_logging_params configuration variable to use the python logging module
log_file configuration variable specifying ae log file
logging_params dict passing the python logging configuration in the key py_logging_params to this method
logging_params dict passing the ae log file in the logging key log_file_name to this method

property debug_level: int

debug level property:

Getter:: return the current debug level of this app instance.
Setter:: change the debug level of this app instance.

add_cfg_files(*additional_cfg_files)[source]

extend the list of available and additional config files (in _cfg_files).

Parameters:: additional_cfg_files¶ (str) – domain/app-specific config file names to be defined/registered additionally.
Return type:: str
Returns:: empty string on success else line-separated list of the error message texts.

underneath the search order of the config files variable value - the first found one will be returned:

the domain/app-specific config files added in your app code by this method. these files will be searched for the config option value in reversed order - so the last added config file will be the first one where the config value will be searched.
config files added via additional_cfg_files argument of ConsoleApp.__init__() (searched in the reversed order).
<app_name>.INI file in the <app_dir>
<app_name>.CFG file in the <app_dir>
<app_name>.INI file in the <usr_dir>
<app_name>.CFG file in the <usr_dir>
<app_name>.INI file in the <cwd>
<app_name>.CFG file in the <cwd>
.sys_env.cfg in the <app_dir>
.sys_env<sys_env_id>.cfg in the <app_dir>
.app_env.cfg in the <app_dir>
.sys_env.cfg in the <usr_dir>
.sys_env<sys_env_id>.cfg in the <usr_dir>
.app_env.cfg in the <usr_dir>
.sys_env.cfg in the <cwd>
.sys_env<sys_env_id>.cfg in the <cwd>
.app_env.cfg in the <cwd>
.sys_env.cfg in the parent folder of the <cwd>
.sys_env<sys_env_id>.cfg in the parent folder of the <cwd>
.app_env.cfg in the parent folder of the <cwd>
.sys_env.cfg in the parent folder of the parent folder of the <cwd>
.sys_env<sys_env_id>.cfg in the parent folder of the parent folder of the <cwd>
.app_env.cfg in the parent folder of the parent folder of the <cwd>
value argument passed into the add_option() method call (defining the option)
default_value argument passed into this method (only if the method add_option didn’t get called)

legend of the placeholders in the above search order lists (see also ae.paths.PATH_PLACEHOLDERS):

<cwd> is the current working directory of your application (determined with os.getcwd())
<app_name> is the base app name without extension of your main Python code file.
<app_dir> is the application data directory (APPDATA/<app_name> in Windows, ~/.config/<app_name> in Linux).
<usr_dir> is the user data directory (APPDATA in Windows, ~/.config in Linux).
<sys_env_id> is the specified argument of ConsoleApp.__init__().

cfg_section_variable_names(section, cfg_parser=None)[source]

determine current config variable names/keys of the passed config file section.

Parameters:

section¶ (str) – config file section name.
cfg_parser¶ (Optional[ConfigParser]) – ConfigParser instance to use (def=self._cfg_parser).

Return type:

tuple[str, ...]

Returns:

tuple of all config variable names.

_get_cfg_parser_val(name, section, default_value=None, cfg_parser=None)[source]

determine thread-safe the value of a config variable from the config file.

Parameters:

name¶ (str) – name/option_id of the config variable.
section¶ (str) – name of the config section.
default_value¶ (Optional[Any]) – default value to return if this config value is not specified in any config file.
cfg_parser¶ (Optional[ConfigParser]) – ConfigParser instance to use (def=self._cfg_parser).

Return type:

Any

Returns:

config var value. str values enclosed in single high commas will be returned without high commas. code blocks and multiline-strings enclosed in triple high commas will be returned with the high commas.

load_cfg_files(config_modified=True)[source]

(re)load and parse all config files.

Parameters:: config_modified¶ (bool) – pass False to prevent the refresh/overwrite the initial config file modified date.

is_main_cfg_file_modified()[source]

determine if the main config file got modified.

Return type:: bool
Returns:: True if the content of the main config file got modified/changed.

get_variable(name, section=None, default_value=None, cfg_parser=None, value_type=None)[source]

determine value of a config option, an OS environ variable or a config variable.

Parameters:

name¶ (str) – name of a config option or of an existing/declared config variable.
section¶ (Optional[str]) – name of the config section. defaulting to the app options section (MAIN_SECTION_NAME) if not specified or if None or empty string passed. if name specifies a user-specific config option, then its value will get retrieved from the user-specific section (section name gets then extended with the help of the user_section() method).
default_value¶ (Optional[Any]) – default value to return if not defined as OS environ variable nor specified in any config file. this argument gets ignored for variables defined as config option; for these the default value specified in the add_option() will be used.
cfg_parser¶ (Optional[ConfigParser]) – optional ConfigParser instance to use (def= _cfg_parser).
value_type¶ (Optional[Type]) – optional type of the config value. only used for config variables and ignored for config options.

Return type:

Any

Returns:

variable value which will be searched in the config options, the OS environment and in the config variables in the following order and manner:

config option with a name equal to the name argument (only if the passed section value is either empty, None or equal to MAIN_SECTION_NAME). if the default value of a config option (specified in the add_option() call to define it) is either None, UNSET or an empty string then the value will be searched in the OS environment and the config files.
user-specific OS environment variable with a matching name, compiled from (1) the section argument, (2) the string ‘usr_id’, (3) the user_id and (4) name argument, and all four parts concatenated with an underscore character, and normalized by the env_str() function called with a True value for their convert_name argument.
OS environment variable matching a env_str()-normalized name, compiled from the arguments (1) the section and (2) the name,
config file variable with a name and section equal to the values passed into the name and section arguments.

if no variable value could be found, then a None value will be returned.

set_variable(name, value, cfg_fnam=None, section=None, old_name='')[source]

change the value of a config variable and if exists, the related config option.

if the passed string in name is the id of a defined config option and section is either empty or equal to the value of MAIN_SECTION_NAME then the value of this config option will be changed too.

if the section does not exist, it will be created (in a contrary to Pythons ConfigParser).

Parameters:

name¶ (str) – name/option_id of the config value to set.
value¶ (Any) – value to assign to the config value, specified by the name argument.
cfg_fnam¶ (Optional[str]) – file name (def= _main_cfg_fnam) to save the new option value to.
section¶ (Optional[str]) – name of the config section. defaulting to the app options section (MAIN_SECTION_NAME) if not specified or if None or empty string passed.
old_name¶ (str) – old name/option_id that has to be removed (used to rename config option name/key).

Return type:

str

Returns:

empty string on success else error message text.

del_section(section, cfg_fnam=None)[source]

delete a section from the main or the specified config file.

Parameters:

section¶ (str) – name of the config section to delete/remove.
cfg_fnam¶ (Optional[str]) – optional path/name of the config file (def= _main_cfg_fnam).

Returns:

empty string on success else error message text.

add_argument(*args, **kwargs)[source]

define a new command line argument.

original/underlying args/kwargs of argparse.ArgumentParser are used - please see the description/definition of add_argument().

get_argument(name)[source]

determine the command line parameter value.

Parameters:: name¶ (str) – argument id of the parameter.
Return type:: Any
Returns:: value of the parameter.

add_option(name, desc, value, short_opt='', choices=None, multiple=False, **add_kwargs)[source]

defining and adding a new config option for this app.

Parameters:

name¶ (str) – string specifying the option id and the short description of this new option. the name value will also be available as the long command line argument option (case-sens.).
desc¶ (str) – description and command line help string of this new option.
value¶ (Any) – last fallback/default value to be returned by get_option() and get_variable() if this option is not specified as a command line argument and in the config files of the main app. pass UNSET to define a boolean flag option, which then can be specified without a value on the command line in order to result as True, and else as False. pass ‘++’ to add a ‘count’ action option, which can be specified multiple times, and its option value is the number of times it get specified. specifying a value on the command line for an option added with an UNSET or ‘++’ argument results in an argument parsing error and SystemExit.
short_opt¶ (Union[str, UnsetType]) – short option character. if not passed or passed as ‘’ then the first character of the name will be used. passing UNSET or None prevents the declaration of a short option. please note that the short options ‘h’, ‘D’ and ‘L’ are already used internally by the classes ArgumentParser and ConsoleApp.
choices¶ (Optional[Iterable]) – optional list of valid option values (default=allow all values).
multiple¶ (bool) – pass True if the option can be added multiple times to the command line (default=False).
add_kwargs¶ – additional kwargs to be passed onto ArgsParser.add_argument().

Hint

the value of a config option can be of any type and gets represented by an instance of the Literal class. supported value types and literals are documented here.

_change_option(name, value)[source]: change the specified config option and the related instance shortcut|property to the specified value.

get_option(name, default_value=None)[source]

determine the value of a config option specified by its name (option id).

Parameters:

name¶ (str) – name/id of the config option.
default_value¶ (Optional[Any]) – default value of the option (if not defined with add_option).

Return type:

Any

Returns:

the first found value of the specified option (name). the returned value has the same type as the value specified in the add_option() call. if not given on the command line, then it gets search next in the default config section (MAIN_SECTION_NAME) of the collected config files (the exact search order is documented in the doc-string of the method add_cfg_files()). if not found in the config file, then the default value specified of the option definition (the add_option() call) will be used. the other default value, specified in the default_value kwarg of this method, will be returned only if the option name/id never got defined.

set_option(name, value, cfg_fnam=None, save_to_config=True)[source]

set or change the value of a config option.

Parameters:

name¶ (str) – id of the config option to set.
value¶ (Any) – new value to assign to the option, identified by name.
cfg_fnam¶ (Optional[str]) – the config file name to save the new option value. if not specified, then the default file name of set_variable() will be used.
save_to_config¶ (bool) – pass False to prevent saving the new option value to the main/specified config file. the value of the config option will be changed in any case.

Return type:

str

Returns:

‘’/empty string on success else error message text.

parse_arguments()[source]

parse all command line args.

this method gets normally only called once, and after all, the options have been added with add_option(). add_option() will then set the determined config file value as the default value, and then the following call of this method will overwrite it with command line argument value, if given.

property user_id: id of the user of this app.

load_user_cfg()[source]: load users configuration.

register_user(new_user_id='', reset_cfg_vars=False, set_as_default=True)[source]

register/reset the specified or current user, creating/copying a new set of user-specific config vars.

Parameters:

new_user_id¶ (str) – user id to register. if not specified, then register the current os user (self.user_id).
reset_cfg_vars¶ (bool) – pass True to reset the user-specific-variables to the default values.
set_as_default¶ (bool) – pass False to not set the specified user id as default for the next app run/start.

Raises:

AssertionError – if the specified/current user id/name is empty, too long or contains invalid characters.

Return type:

bool

Returns:

True if the specified or the current os user id/name was not registered, else False.

user_section(section, name='')[source]

return the user section name if the passed (section, name) setting id is user-specific.

Parameters:

section¶ (str) – section name.
name¶ (str) – config variable name. if specified, then this variable has to be a user-specific one. if not specified and the user is registered, then return always the user section.

Return type:

str

Returns:

passed section name or user-specific section name.

app_env_dict()[source]

collect run-time app environment data and settings - for app logging and debugging.

Return type:: dict[str, Any]
Returns:: dict with app environment data/settings.

run_app()[source]: prepare app run. call after definition of command line arguments/options and before run of app code.

show_help()[source]: print usage/help message to console output.

Hint

the console output includes the usage, the options defined via add_option(), and the command line arguments defined via add_argument() (respective the same-named ArgumentParser method). see also the description/definition of print_help().

show_parse_error_and_exit(message)[source]

show help and arg parse error message and shutdown/exit/quit this app instance with exit code 255.

Parameters:: message¶ (str) – args parse error message to display at the end of the help printout on the console.

Hint

this method gets also used to patch the argparse.ArgumentParser.error() method.

chk(error_code, check_result, error_message)[source]

exit/quit this console app if the check_result argument is False and the force app option is zero/False.

Parameters:

error_code¶ (int) – used OS app exit error code on app exit/quit/shutdown.
check_result¶ (bool) – result of the app run check/assertion.
error_message¶ (str) – error message to print to the console/shell on app exit/quit.

shutdown(exit_code=0, error_message='', timeout=None)[source]

shutdown this ConsoleApp instance, and also any created sub-app-instances.

Parameters:

exit_code¶ (Optional[int]) – optional OS exit code (def=0). pass None to prevent call of sys.exit(exit_code). if exit code is between 1 and 9 then the help message is printed to the console/shell.
error_message¶ (str) – optional shutdown error message.
timeout¶ (Optional[float]) – optional timeout float value in seconds used for the thread termination/joining, for the shutdowns of the app/sub-app instances and for the acquisition of the threading locks of the ae log file and the app instances.