ae.sys_data

external system data structures

this module allows your application to easily manage data structures to interface data flows from and onto external systems.

a external system can be nearly anything: like e.g.: a database or web server or an application or software suite with an interface for data exchanges; even data files can be used as an external system.

the same data can be represented differently on different systems. this module allows you to specify individual data structures for each system and provides hooks to easily integrate system-specific data conversion functions.

data structures

use the classes of this module to define any kind of simple data structures - like lists, mappings and trees:

  • a list can get represented by a instance of one of the classes Records or Values. each Records instance is a sequence of 0..n Record instances. a Values instance is a sequence of 1..n Value instances.

  • a mapping get represented by an instance of the class Record, whereas each mapping item gets represented by an instance of the private class _Field.

  • a node of a tree structure can be represented by an instance of the classes Values, Records or Record.

by combining these simple data structures you can build any complex data structures. the root of such a complex data structure can be defined by an instance of either Records or Record.

the leaves of any simple or complex data structure are represented by instances of the Value class.

the following diagram is showing a complex data structure with all the possible combinations (of a single system):

digraph {
    node [shape=record]
    rec1 [label="{<rec1>Record (root) | { <A>Field A | <B>Field B | <C>Field C | <D>Field D } }"]
    "Records (root)" -> rec1 [arrowhead=crow style=tapered penwidth=3]
    rec1:A -> "Value (of Field A)" [minlen=3]
    rec1:B -> "Values"
    "Values" -> "Value (of one Values item)" [minlen=2 arrowhead=crow style=tapered penwidth=3]
    rec2 [label="{<rec2>Record (sub-record) | { <CA>Field CA | <CB>Field CB | <CN>... } }"]
    rec1:C -> rec2
    rec2:CA -> "Value (of Field CA)" [minlen=2]
    rec3 [label="{<rec3>Record (sub-records-sub-record) | { <DA>Field DA | <DB>Field DB | <DN>... } }"]
    rec1:D -> "Records (sub-records)"
    "Records (sub-records)" -> rec3 [arrowhead=crow style=tapered penwidth=3]
    rec3:DA -> "Value (of Field DA)"
}

data reference index paths

a index path is is a tuple of indexes/keys that is referencing one part or a value of a data structure.

the items of an index path are either of type int (specifying a list index in a Values or Records instance) or of type str (specifying a field name of a Record instance).

e.g. the index path to reference in the above graph example the field DA from the record Record (root) would be ('D', 0, 'DA'). the first item is referencing the field D in the root record. the second item 0 is referencing the first item of the Records instance/value of field D, which is a Record instance. and finally the last item DA specifies the field name in this Record instance.

the hierarchically highest placed Record instance in a data structure is called the root record, and the Value instances at the lowest level in a data structure are the leaves of a data structure.

each _Field is providing apart from its field value also a reference to its root record as well as a root index. the root index is an index path to reference the _Field instance from the root record.

if the format/representation of the data value or of any references differs at one of the used systems then a _Field instance allows you to store a separate system-specific value. to access a system-specific value or reference in your data structure you have to specify additionally to the index path also a system identifier and optionally a direction identifier.

system and direction identifiers

a system id is an user-defined string that is uniquely identifying a system and should consist of at least two characters (see _ASP_SYS_MIN_LEN).

a direction id is specifying the data flow direction. the two pre-defined direction ids are:

  • FAD_FROM for pulling data from a system or

  • FAD_ONTO for pushing data onto a system.

the methods pull() and push() of the Record class are used to pull/push and convert system-specific data from/onto a system.

Module Attributes

ACTION_INSERT

insert action

ACTION_UPDATE

update action

ACTION_UPSERT

insert or update (if already exists) action

ACTION_DELETE

delete action

ACTION_SEARCH

search action

ACTION_PARSE

parse action

ACTION_BUILD

build action

ACTION_PULL

pull-from-system action

ACTION_PUSH

push-to-system action

ACTION_COMPARE

compare action

FAT_IDX

main/system field name within parent Record or list index within Records/Values

FAT_VAL

main/system field value - storing one of the VALUE_TYPES instance

FAT_CLEAR_VAL

field default/clear value (init by _Field.set_clear_val(), used by clear_leaves())

FAT_REC

root Record instance

FAT_RCX

field index path (idx_path) from the root Record instance

FAT_CAL

calculator callable

FAT_CHK

validator callable

FAT_CNV

system value converter callable

FAT_FLT

field filter callable

FAT_SQE

SQL expression to fetch field value from db

ALL_FATS

tuple of all pre-defined field aspect types/prefixes

FAD_FROM

FROM field aspect direction

FAD_ONTO

ONTO field aspect direction

IDX_PATH_SEP

separator character used for idx_path values (especially if field has a Record value).

ALL_FIELDS

special key of fields_patches argument of Record.copy() to allow aspect value patching for all fields.

CALLABLE_SUFFIX

suffix for aspect keys - used by _Field.set_aspects()

IDX_TYPES

tuple of classes/types used for system data index path items

PATH_TYPES

path items, plus field name path, plus index path

VALUE_TYPES

tuple of classes/types used for system data values

LIST_TYPES

tuple of classes/types used for system data lists

NODE_TYPES

tuple of classes/types used for system data nodes

NODE_CHILD_TYPES

tuple of classes/types used for system data node children

PARENT_TYPES

value types that can have children (Value excluded)

Functions

aspect_key(type_or_key[, system, direction])

compiles an aspect key from the given args

aspect_key_direction(key)

determines the direction id string from an aspect key.

aspect_key_system(key)

determines the system id string from an aspect key.

compose_current_index(node, idx_path, …)

determine tuple with the current indexes.

deeper(deepness, instance)

check and calculate resulting/remaining deepness for Record/_Field/Records.copy() when going one level deeper.

field_name_idx_path(field_name[, …])

converts a field name path string into an index path tuple.

field_names_idx_paths(field_names)

return list of the full idx paths names for all the fields specified in the field_names argument.

get_current_index(node)

get current index of passed node.

idx_path_field_name(idx_path[, add_sep])

convert index path tuple/list into field name string.

init_current_index(node, idx_path, use_curr_idx)

determine current index of node and if not set the initialize to the first index path item.

set_current_index(node[, idx, add])

set current index of node.

string_to_records(str_val, field_names[, …])

convert formatted string into a Records instance containing several Record instances.

template_idx_path(idx_path[, is_sub_rec])

check/determine if idx_path is referring to template item.

use_current_index(node, idx_path, use_curr_idx)

determine index path of node by using current index of node if exists and is enabled by use_curr_idx arg.

use_rec_default_root_rec_idx(rec, root_rec)

helper function to determine resulting root record and root index.

use_rec_default_sys_dir(rec, system, direction)

helper function to determine resulting system/direction.

Classes

Record([template, fields, system, …])

instances of this mapping class are used to represent record-like data structures.

Records([seq])

Records class.

Value([iterable])

represents a value.

Values([seq])

ordered/mutable sequence/list, which contains 0..n instances of the class Value.

ACTION_INSERT = 'INSERT'

insert action

ACTION_UPDATE = 'UPDATE'

update action

ACTION_UPSERT = 'UPSERT'

insert or update (if already exists) action

ACTION_DELETE = 'DELETE'

delete action

search action

ACTION_PARSE = 'PARSE'

parse action

ACTION_BUILD = 'BUILD'

build action

ACTION_PULL = 'PULL'

pull-from-system action

ACTION_PUSH = 'PUSH'

push-to-system action

ACTION_COMPARE = 'COMPARE'

compare action

FAT_IDX = 'idx'

main/system field name within parent Record or list index within Records/Values

FAT_VAL = 'vle'

main/system field value - storing one of the VALUE_TYPES instance

FAT_CLEAR_VAL = 'vwc'

field default/clear value (init by _Field.set_clear_val(), used by clear_leaves())

FAT_REC = 'rrd'

root Record instance

FAT_RCX = 'rrx'

field index path (idx_path) from the root Record instance

FAT_CAL = 'clc'

calculator callable

FAT_CHK = 'chk'

validator callable

FAT_CNV = 'cnv'

system value converter callable

FAT_FLT = 'flt'

field filter callable

FAT_SQE = 'sqc'

SQL expression to fetch field value from db

ALL_FATS = ('idx', 'vle', 'vwc', 'rrd', 'rrx', 'clc', 'chk', 'cnv', 'flt', 'sqc')

tuple of all pre-defined field aspect types/prefixes

FAD_FROM = 'From'

FROM field aspect direction

FAD_ONTO = 'Onto'

ONTO field aspect direction

IDX_PATH_SEP = '/'

separator character used for idx_path values (especially if field has a Record value).

don’t use dot char because this is used e.g. to separate system field names in xml element name paths.

ALL_FIELDS = '**'

special key of fields_patches argument of Record.copy() to allow aspect value patching for all fields.

CALLABLE_SUFFIX = '()'

suffix for aspect keys - used by _Field.set_aspects()

_ASP_TYPE_LEN = 3

aspect key type string length

_ASP_DIR_LEN = 4

aspect key direction string length

_ASP_SYS_MIN_LEN = 2

aspect key system id string length

IDX_TYPES: Tuple[Type, ...] = (<class 'int'>, <class 'str'>)

tuple of classes/types used for system data index path items

PATH_TYPES = (<class 'int'>, <class 'str'>, <class 'tuple'>)

path items, plus field name path, plus index path

AspectKeyType

alias of str

IdxItemType

types of idx_path items

alias of Union[int, str]

IdxPathType

idx_path type

alias of Tuple[Union[int, str], …]

IdxTypes

types of either idx_path or idx_path items

alias of Union[int, str, Tuple[Union[int, str], …]]

NodeType

node types (NODE_TYPES)

alias of Union[Record, Records]

ListType

list/sequence types (LIST_TYPES)

alias of Union[Values, Records]

ValueType

value types (VALUE_TYPES)

alias of Union[Value, Values, Record, Records]

NodeChildType

node child types

alias of Union[_Field, Record]

FieldCallable

callable with _Field argument

alias of Callable[[_Field], Any]

FieldValCallable

callable with _Field and additional argument

alias of Callable[[_Field, Any], Any]

aspect_key(type_or_key, system='', direction='')[source]

compiles an aspect key from the given args

Parameters
  • type_or_key (str) – either FAT_* type or full key (including already the system and direction)-

  • system (str) – system id string (if type_or_key is a pure FAT_* constant).

  • direction (str) – direction string FAD_* constant (if type_or_key is a pure FAT_* constant).

Return type

str

Returns

compiled aspect key as string.

aspect_key_system(key)[source]

determines the system id string from an aspect key.

Parameters

key (str) – aspect key string.

Return type

str

Returns

system id (SDI_* constant).

aspect_key_direction(key)[source]

determines the direction id string from an aspect key.

Parameters

key (str) – aspect key string.

Return type

str

Returns

direction id (FAD_* constant).

deeper(deepness, instance)[source]

check and calculate resulting/remaining deepness for Record/_Field/Records.copy() when going one level deeper.

Parameters
  • deepness (int) – <0 will be returned unchanged until last level is reached (-1==full deep copy, -2==deep copy until deepest Value, -3==deep copy until deepest _Field.

  • instance (Any) – instance to be processed/copied (if this method is returning != 0/zero).

Return type

int

Returns

if deepness == 0 then return 0, if deepness < 0 then return 0 if the deepest level is reached, else (deepness > 0) return deepness - 1.

field_name_idx_path(field_name, return_root_fields=False)[source]

converts a field name path string into an index path tuple.

Parameters
  • field_name (Union[int, str, Tuple[Union[int, str], …]]) – field name str or field name index/path string or field index tuple or int (for Records index).

  • return_root_fields (bool) – pass True to also return len()==1-tuple for fields with no deeper path (def=False).

Return type

Tuple[Union[int, str], …]

Returns

index path tuple (idx_path) or empty tuple if the field has no deeper path and return_root_fields==False.

field_names_idx_paths(field_names)[source]

return list of the full idx paths names for all the fields specified in the field_names argument.

Parameters

field_names (Sequence[Tuple[Union[int, str], …]]) – sequence/list/tuple of field (main or system) names.

Return type

List[Tuple[Union[int, str], …]]

Returns

list of their idx paths names.

idx_path_field_name(idx_path, add_sep=False)[source]

convert index path tuple/list into field name string.

Parameters
  • idx_path (Tuple[Union[int, str], …]) – index path to convert.

  • add_sep (bool) – pass True to always separate index with IDX_PATH_SEP. the default value False will only put a separator char if the field value is a Record (to separate the root field name from the sub field name).

Return type

str

Returns

field name string.

compose_current_index(node, idx_path, use_curr_idx)[source]

determine tuple with the current indexes.

Parameters
  • node (Union[Values, Records, Record]) – root node/list (Record or Records/Values instance) to process.

  • idx_path (Tuple[Union[int, str], …]) – index path relative to root node passed in node arg.

  • use_curr_idx (List) – list of index counters within idx_path where the current index has to be used.

Return type

Tuple[Union[int, str], …]

Returns

tuple of current indexes.

get_current_index(node)[source]

get current index of passed node.

Parameters

node (Union[Values, Records, Record]) – instance of Record or Records (real node) or Values (simple list).

Return type

Union[int, str, None]

Returns

current index value.

init_current_index(node, idx_path, use_curr_idx)[source]

determine current index of node and if not set the initialize to the first index path item.

Parameters
  • node (Union[Values, Records, Record]) – root node/list (Record or Records/Values instance) to process.

  • idx_path (Tuple[Union[int, str], …]) – index path relative to root node passed in node arg.

  • use_curr_idx (Optional[List]) – list of index counters within idx_path where the current index has to be used.

Return type

Tuple[Union[int, str], …]

Returns

tuple of current indexes.

set_current_index(node, idx=None, add=None)[source]

set current index of node.

Parameters
  • node (Union[Values, Records, Record]) – root node/list (Record or Records/Values instance) to process.

  • idx (Union[int, str, None]) – index value to set (str for field name; int for list index); if given add will be ignored.

  • add (Optional[int]) – value to add to list index; will be ignored if idx arg get passed.

Return type

Union[int, str]

Returns

the finally set/new index value.

use_current_index(node, idx_path, use_curr_idx, check_idx_type=False, delta=1)[source]

determine index path of node by using current index of node if exists and is enabled by use_curr_idx arg.

Parameters
  • node (Union[Values, Records, Record]) – root node/list (Record or Records/Values instance) to process.

  • idx_path (Tuple[Union[int, str], …]) – index path relative to root node passed in node arg.

  • use_curr_idx (Optional[List]) – list of index counters within idx_path where the current index has to be used.

  • check_idx_type (bool) – pass True to additionally check if the index type is correct (def=False).

  • delta (int) – value to decrease the list index counters within use_curr_idx (def=1).

Return type

Tuple[Union[int, str], …]

Returns

tuple of current indexes.

string_to_records(str_val, field_names, rec_sep=',', fld_sep='=', root_rec=None, root_idx=())[source]

convert formatted string into a Records instance containing several Record instances.

Parameters
  • str_val (str) – formatted string to convert.

  • field_names (Sequence) – list/tuple of field names of each record

  • rec_sep (str) – character(s) used in str_val to separate the records.

  • fld_sep (str) – character(s) used in str_val to separate the field values of each record.

  • root_rec (Optional[Record]) – root to which the returned records will be added.

  • root_idx (Tuple[Union[int, str], …]) – index path from root where the returned records will be added.

Return type

Records

Returns

converted Records instance.

template_idx_path(idx_path, is_sub_rec=False)[source]

check/determine if idx_path is referring to template item.

Parameters
  • idx_path (Tuple[Union[int, str], …]) – index path to check.

  • is_sub_rec (bool) – pass True to only check sub-record-fields (will then return always False for root-fields).

Return type

bool

Returns

True if idx_path is referencing a template item (with index zero/0), else False.

use_rec_default_root_rec_idx(rec, root_rec, idx=(), root_idx=(), met='')[source]

helper function to determine resulting root record and root index.

Parameters
Return type

Tuple[Record, Tuple[Union[int, str], …]]

Returns

resulting root record and root index (as tuple).

use_rec_default_sys_dir(rec, system, direction)[source]

helper function to determine resulting system/direction.

Parameters
Return type

Tuple[str, str]

Returns

resulting system and direction (as tuple).

class Value(iterable=(), /)[source]

Bases: list

represents a value.

this class inherits directly from the Python list class. each instance can hold either a (single/atomic) value (which can be anything: numeric, char/string or any object) or a list of these single/atomic values.

__getitem__(key)[source]

determine atomic value.

Parameters

key (Union[slice, int]) – list index if value is a list.

Return type

Union[Any, MutableSequence[Any]]

Returns

list item value.

__setitem__(key, value)[source]

set/initialize list item identified by key to the value passed in value.

Parameters
  • key (Union[slice, int]) – list index if value is a list.

  • value (Any) – the new value of the list item.

Return type

None

__repr__()[source]

representation which can be used to serialize and re-create Value instance.

Return type

str

Returns

Value representation string.

__str__()[source]

string representation of this Value instance.

Return type

str

Returns

Value string.

property initialized

flag if this Value instance got already initialized.

Returns

True if already set to a value, else False.

node_child(idx_path, moan=False, **__)[source]

check if idx_path is correct (has to be empty) and if yes then return self.

this method is to simplify the data structure hierarchy implementation.

Parameters
  • idx_path (Tuple[Union[int, str], …]) – this argument has to be an empty tuple/list.

  • moan (bool) – pass True to raise AssertionError if idx_path is not empty.

Return type

Optional[Value]

Returns

self or None (if idx_path is not empty and moan == False).

value(*idx_path, **__)[source]

check if idx_path is correct (has to be empty) and if yes then return self.

this method is to simplify the data structure hierarchy implementation.

Parameters

idx_path (Union[int, str]) – this argument has to be an empty tuple.

Return type

Optional[Value]

Returns

self or None (if idx_path is not empty and moan == False).

val(*idx_path, **__)[source]

check if idx_path is correct (either empty or contain one int) and if yes then return list item.

this method is to simplify the data structure hierarchy implementation.

Parameters

idx_path – this argument is either empty or contains a list index.

Returns

atomic/single value or list item value or empty string.

set_val(val, *idx_path, **__)[source]

set a/the value of this instance.

Parameters
  • val (Any) – simple/atomic value to be set.

  • idx_path (Union[int, str]) – this argument is either empty or contains a list index.

Returns

self.

copy(*_, **__)[source]

copy the value of this Value instance into a new one.

Returns

new Value instance containing the same immutable value.

clear_leaves(**__)[source]

clear/reset the value of this instance.

use self[-1] = ‘’ to clear only the newest/top val.

Returns

self.

class Values(seq=())[source]

Bases: list

ordered/mutable sequence/list, which contains 0..n instances of the class Value.

__init__(seq=())[source]

create new Values instance.

Parameters

seq (Iterable) – Iterable used to initialize the new instance (pass list, tuple or other iterable).

__repr__()[source]

Return repr(self).

Return type

str

__str__()[source]

Return str(self).

Return type

str

node_child(idx_path, use_curr_idx=None, moan=False, selected_sys_dir=None)[source]

determine and return node instance specified by idx_path if exists in this instance or underneath.

Parameters
  • idx_path (Union[int, str, Tuple[Union[int, str], …]]) – index path to the node, relative to this instance.

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

  • moan (bool) – flag to check data integrity; pass True to raise AssertionError if not.

  • selected_sys_dir (Optional[dict]) – optional dict to return the currently selected system/direction.

Return type

Union[Value, Values, Record, Records, None]

Returns

found node instance or None if not found.

value(*idx_path, system='', direction='', **kwargs)[source]

determine the ValueType instance referenced by idx_path of this Values/Records instance.

Parameters
  • idx_path (Union[int, str]) – index path items.

  • system (str) – system id.

  • direction (str) – direction id.

  • kwargs – extra args (will be passed to underlying data structure).

Return type

Union[Value, Values, Record, Records, None]

Returns

found Value or Values instance, or None if not found.

set_value(value, *idx_path, system='', direction='', protect=False, root_rec=None, root_idx=(), use_curr_idx=None)[source]

set the ValueType instance referenced by idx_path of this Values/Records instance.

Parameters
Return type

Union[Values, Records]

Returns

self (this instance of Values or Records).

val(*idx_path, system='', direction='', flex_sys_dir=True, use_curr_idx=None, **kwargs)[source]

determine the user/system value referenced by idx_path of this Values/Records instance.

Parameters
  • idx_path (Union[int, str]) – index path items.

  • system (str) – system id.

  • direction (str) – direction id.

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

  • kwargs – extra args (will be passed to underlying data structure).

Return type

Any

Returns

found user/system value, or None if not found or empty string if value was not set yet.

set_val(val, *idx_path, protect=False, extend=True, use_curr_idx=None)[source]

set the user/system value referenced by idx_path of this Values instance.

Parameters
  • val (Any) – user/system value to set/change.

  • idx_path (Union[int, str]) – index path items.

  • protect (bool) – pass True to prevent replacement of already existing ValueType.

  • extend (bool) – pass True to allow extension of data structure.

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

Return type

Values

Returns

self (this instance of Values).

copy(deepness=0, root_rec=None, root_idx=(), **kwargs)[source]

copy the values/records of this Values/Records instance.

Parameters
  • deepness (int) – deep copy levels: <0==see deeper(), 0==only copy current instance, >0==deep copy to deepness value - _Field occupies two deepness: 1st=_Field, 2nd=Value).

  • root_rec (Optional[Record]) – destination root record.

  • root_idx (Tuple[Union[int, str], …]) – destination index path (tuple of field names and/or list/Records indexes).

  • kwargs – additional arguments (will be passed on - most of them used by Record.copy).

Return type

Union[Values, Records]

Returns

new instance of self (which is an instance of Values or Records).

clear_leaves(system='', direction='', flex_sys_dir=True, reset_lists=True)[source]

clear/reset the user/system values of all the leaves of this Values/Records instance.

Parameters
  • system (str) – system id.

  • direction (str) – direction id (FAD_* constants).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

  • reset_lists (bool) – pass False to prevent the reset of all the underlying lists.

Return type

Union[Values, Records]

Returns

class Record(template=None, fields=None, system='', direction='', action='', root_rec=None, root_idx=(), field_items=False)[source]

Bases: collections.OrderedDict

instances of this mapping class are used to represent record-like data structures.

isinstance(…, dict) not working if using MutableMapping instead of OrderedDict as super class. and dict cannot be used as super class because instance as kwarg will then not work: see the Groxx’s answer in stackoverflow question at https://stackoverflow.com/questions/3387691/how-to-perfectly-override-a-dict/47361653#47361653.

__init__(template=None, fields=None, system='', direction='', action='', root_rec=None, root_idx=(), field_items=False)[source]

create new Record instance, which is an ordered collection of _Field items.

Parameters
  • template (Optional[Records]) – pass Records instance to use first item/[0] as template (after deep copy / values cleared).

  • fields (Optional[Iterable]) – OrderedDict/dict of _Field instances (field order is not preserved when using dict) or Record instance (fields will be referenced, not copied!) or list of (field_name, fld_or_val) tuples.

  • system (str) – main/current system of this record,

  • direction (str) – interface direction of this record.

  • action (str) – current action (see ACTION_INSERT, ACTION_SEARCH, ACTION_DELETE, …)

  • root_rec (Optional[Record]) – root record of this record (def=self will be a root record).

  • root_idx (Optional[Tuple[Union[int, str], …]]) – root index of this record (def=()).

  • field_items (bool) – pass True to get Record items - using __getitem__() - as of type _Field (not as val()).

__repr__()[source]

Return repr(self).

Return type

str

__str__()[source]

Return str(self).

Return type

str

__contains__(idx_path)[source]

True if the dictionary has the specified key, else False.

Return type

bool

__getitem__(key)[source]

x.__getitem__(y) <==> x[y]

Return type

Any

__setitem__(key, value)[source]

Set self[key] to value.

node_child(idx_path, use_curr_idx=None, moan=False, selected_sys_dir=None)[source]

get the node child specified by idx_path relative to this Record instance.

Parameters
  • idx_path (Union[int, str, Tuple[Union[int, str], …]]) – index path or field name index string.

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

  • moan (bool) – flag to check data integrity; pass True to raise AssertionError if not.

  • selected_sys_dir (Optional[dict]) – optional dict to return the currently selected system/direction.

Return type

Union[Value, Values, Record, Records, _Field, None]

Returns

found node instance or None if not found.

set_node_child(fld_or_val, *idx_path, system=None, direction=None, protect=False, root_rec=None, root_idx=(), use_curr_idx=None, to_value_type=False)[source]

set/replace the child of the node specified by idx_path with the value of fld_or_val.

Parameters
  • fld_or_val (Any) – field or value - to be set.

  • idx_path (Union[int, str]) – index path of the node child to set/replace.

  • system (Optional[str]) – system id (pass None to use default system id of this Record instance).

  • direction (Optional[str]) – direction id (pass None to use default direction id of this Record instance).

  • protect (bool) – pass True to prevent the replacement of a node child.

  • root_rec (Optional[Record]) – root record of this data structure.

  • root_idx (Optional[Tuple[Union[int, str], …]]) – root index path of this node.

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

  • to_value_type (bool) – pass True ensure conversion of fld_or_val to a Value instance.

Return type

Record

Returns

self (this Record instance).

value(*idx_path, system=None, direction=None, **kwargs)[source]

search the Value specified by idx_path and return it if found.

Parameters
  • idx_path (Union[int, str]) – index path of Value.

  • system (Optional[str]) – system id (pass None to use default system id of this Record instance).

  • direction (Optional[str]) – direction id (pass None to use default direction id of this Record instance).

  • kwargs – additional keyword args (to be passed onto underlying data structure).

Return type

Union[Value, Values, Record, Records, None]

Returns

found Value instance of None if not found.

set_value(value, *idx_path, system=None, direction=None, protect=False, root_rec=None, root_idx=(), use_curr_idx=None)[source]

set/replace the Value instance of the node specified by idx_path.

Parameters
  • value (Union[Value, Values, Record, Records]) – Value instance to be set/replaced.

  • idx_path (Union[int, str]) – index path of the Value to be set.

  • system (Optional[str]) – system id (pass None to use default system id of this Record instance).

  • direction (Optional[str]) – direction id (pass None to use default direction id of this Record instance).

  • protect (bool) – pass True to protect existing node value from to be changed/replaced.

  • root_rec (Optional[Record]) – root Record instance of this data structure.

  • root_idx (Tuple[Union[int, str], …]) – root index to this node/Record instance.

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

Return type

Record

Returns

self (this Record instance).

val(*idx_path, system=None, direction=None, flex_sys_dir=True, use_curr_idx=None, **kwargs)[source]

determine the user/system value referenced by idx_path of this Record instance.

Parameters
  • idx_path (Union[int, str]) – index path items.

  • system (Optional[str]) – system id (pass None to use default system id of this Record instance).

  • direction (Optional[str]) – direction id (pass None to use default direction id of this Record instance).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

  • kwargs – extra args (will be passed to underlying data structure).

Return type

Any

Returns

found user/system value, or None if not found or empty string if value was not set yet.

set_val(val, *idx_path, system=None, direction=None, flex_sys_dir=True, protect=False, extend=True, converter=None, root_rec=None, root_idx=(), use_curr_idx=None, to_value_type=False)[source]

set the user/system value referenced by idx_path of this Record instance.

Parameters
  • val (Any) – user/system value to be set/replaced.

  • idx_path (Union[int, str]) – index path of the Value to be set.

  • system (Optional[str]) – system id (pass None to use default system id of this Record instance).

  • direction (Optional[str]) – direction id (pass None to use default direction id of this Record instance).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

  • protect (bool) – pass True to protect existing node value from to be changed/replaced.

  • extend (bool) – pass False to prevent extension of data structure.

  • converter (Optional[Callable[[_Field, Any], Any]]) – converter callable to convert user values between systems.

  • root_rec (Optional[Record]) – root Record instance of this data structure.

  • root_idx (Optional[Tuple[Union[int, str], …]]) – root index to this node/Record instance.

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

  • to_value_type (bool) – pass True ensure conversion of val to a Value instance.

Return type

Record

Returns

self (this Record instance).

_add_field(field, idx='')[source]

add _Field instance to this Record instance.

Parameters
  • field (_Field) – _Field instance to add.

  • idx (str) – name/key/idx string to map and identify this field (mostly identical to field.name()).

Return type

Record

Returns

self.

add_fields(fields, root_rec=None, root_idx=())[source]

adding fields to this Record instance.

Parameters
  • fields (Iterable) – either a dict, a Record or a list with (key/field_name, val/_Field) tuples. key strings that are containing digits/numbers are interpreted as name/idx paths (then also the specified sub-Records/-Fields will be created). values can be either _Field instances or field values.

  • root_rec (Optional[Record]) – root record of this record (def=self will be a root record).

  • root_idx (Tuple[Union[int, str], …]) – root index of this record (def=()).

Return type

Record

Returns

self

add_system_fields(system_fields, sys_fld_indexes=None, system=None, direction=None, extend=True)[source]

add/set fields to this Record instance from the field definition passed into system_fields.

make sure before you call this method that this Record instance has the system and direction attributes specified/set.

Parameters
  • system_fields (Iterable[Iterable[Any]]) – tuple/list of tuples/lists with system and main field names and optional field aspects. the index of the field names and aspects within the inner tuples/lists get specified by sys_fld_indexes.

  • sys_fld_indexes (Optional[Dict]) – mapping/map-item-indexes of the inner tuples of system_fields. keys are field aspect types (FAT_* constants), optionally extended with a direction (FAD_* constant) and a system (SDI_* constant). if the value aspect key (FAT_VAL) contains a callable then it will be set as the calculator (FAT_CAL) aspect; if contains a field value then also the clear_val of this field will be set to the specified value.

  • system (Optional[str]) – system of the fields to be added - if not passed self.system will be used.

  • direction (Optional[str]) – direction (FAD constants) of the fields to be added - if not passed used self.direction.

  • extend (bool) – True=add not existing fields, False=apply new system aspects only on existing fields.

Return type

Record

Returns

self

collect_system_fields(sys_fld_name_path, path_sep)[source]

compile list of system _Field instances of this Record instance.

Parameters
  • sys_fld_name_path (Sequence) – sequence/tuple/list of system field names/keys.

  • path_sep (str) – system field name/key separator character(s).

Return type

List[_Field]

Returns

list of :class:’_Field` instances which are having system field name/keys set.

compare_leaves(rec, field_names=(), exclude_fields=())[source]

compare the leaf ‘compare’ values of this Record instance with the one passed into rec.

‘Compare values’ are simplified user/system values generated by Record.compare_val().

Parameters
  • rec (Record) – other Record instance to compare to.

  • field_names (Iterable) – field names to include in compare; pass empty tuple (==default) to include all fields.

  • exclude_fields (Iterable) – field names to exclude from compare.

Return type

List[str]

Returns

list of str with the differences between self and rec.

compare_val(*idx_path)[source]

determine normalized/simplified user/system value of the node specified by idx_path.

Parameters

idx_path (Union[int, str]) – index path to the node.

Return type

Any

Returns

normalized/simplified compare value.

copy(deepness=0, root_rec=None, root_idx=(), onto_rec=None, filter_fields=None, fields_patches=None)[source]

copy the fields of this record.

Parameters
  • deepness (int) – deep copy level: <0==see deeper(), 0==only copy this record instance, >0==deep copy to deepness value - _Field occupies two deepness: 1st=_Field, 2nd=Value).

  • root_rec (Optional[Record]) – destination root record - using onto_rec/new record if not specified.

  • root_idx (Tuple[Union[int, str], …]) – destination root index (tuple/list with index path items: field names, list indexes).

  • onto_rec (Optional[Record]) – destination record; pass None to create new Record instance.

  • filter_fields (Optional[Callable[[_Field], Any]]) – method called for each copied field (return True to filter/hide/not-include into copy).

  • fields_patches (Optional[Dict[str, Dict[str, Union[str, Value, Values, Record, Records, Callable[[_Field], Any]]]]]) – dict[field_name_or_ALL_FIELDS:dict[aspect_key:val_or_callable]] to set/overwrite aspect values in each copied _Field instance). the keys of the outer dict are either field names or the ALL_FIELDS value; aspect keys ending with the CALLABLE_SUFFIX have a callable in the dict item that will be called for each field with the field instance as argument; the return value of the callable will then be used as the (new) aspect value. set the aspect value that stores the field value (aspect key == FAT_VAL) by passing a data structure instance (of type ValueType).

Return type

Record

Returns

new/extended record instance.

clear_leaves(system='', direction='', flex_sys_dir=True, reset_lists=True)[source]

clear the leaf values including this Record instance and all deeper data structures.

Parameters
  • system (str) – system id (pass None to use default system id of this Record instance).

  • direction (str) – direction id (pass None to use default direction id of this Record instance).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

  • reset_lists (bool) – pass False to prevent reset of sub-lists.

Return type

Record

Returns

leaves(system=None, direction=None, flex_sys_dir=True)[source]

generate leaves/_Field-instances of this Record instance.

Parameters
  • system (Optional[str]) – system id (pass None to use default system id of this Record instance).

  • direction (Optional[str]) – direction id (pass None to use default direction id of this Record instance).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

Return type

Generator[_Field, None, None]

Returns

leaf/_Field-instance generator.

leaf_indexes(*idx_path, system=None, direction=None, flex_sys_dir=True)[source]

generate leaf-/_Field-index paths for all fields of this Record instance.

Parameters
  • idx_path (Union[int, str]) – index path to be added as base index path (index path to this Record instance).

  • system (Optional[str]) – system id (pass None to use default system id of this Record instance).

  • direction (Optional[str]) – direction id (pass None to use default direction id of this Record instance).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

Return type

Generator[Tuple[Union[int, str], …], None, None]

Returns

leaf/_Field-instance generator.

leaf_names(system='', direction='', col_names=(), field_names=(), exclude_fields=(), name_type=None)[source]

compile a tuple of name types (specified by name_type) for this Record instance.

Parameters
  • system (str) – system id (pass None to use default system id of this Record instance).

  • direction (str) – direction id (pass None to use default direction id of this Record instance).

  • col_names (Iterable) – system field/column names to include; pass empty tuple (==default) to include all.

  • field_names (Iterable) – field names to include; pass empty tuple (==default) to include all fields.

  • exclude_fields (Iterable) – field names to exclude.

  • name_type (Optional[str]) – type of name to be included/returned - see available name types underneath.

Return type

Tuple[Union[int, str, Tuple[Union[int, str], …]], …]

Returns

tuple of field names/indexes of the included/found leaves.

possible values for the name_type argument are:

  • ‘s’: user/system field/column name.

  • ‘f’: field name.

  • ‘r’: root name (index path converted into string by idx_path_field_name()).

  • ‘S’: index path with user/system field/column name of leaf.

  • ‘F’: index path tuple.

  • None: names depends on each leaf:
    • root name if leaf is not a root field.

    • user/system name if system is not empty/None.

    • field name.

merge_leaves(rec, system='', direction='', flex_sys_dir=True, extend=True)[source]

merge the leaves of the other record in rec into this Record instance.

Parameters
  • rec (Record) – other Record to merge into this one.

  • system (str) – system id (pass None to use default system id of this Record instance).

  • direction (str) – direction id (pass None to use default direction id of this Record instance).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

  • extend (bool) – pass False to prevent extension of this data structure.

Return type

Record

Returns

self (this Record instance).

match_key(match_fields)[source]

make tuple of user/system values for all the fields in match_fields.

Parameters

match_fields (Iterable) – Iterable with field names/index paths.

Return type

tuple

Returns

tuple of user/system values.

merge_values(rec, system='', direction='', flex_sys_dir=True, extend=True)[source]

merge user/system values of the other record in rec into this Record instance.

Parameters
  • rec (Record) – other Record to merge into this one.

  • system (str) – system id (pass None to use default system id of this Record instance).

  • direction (str) – direction id (pass None to use default direction id of this Record instance).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

  • extend (bool) – pass False to prevent extension of this data structure.

Return type

Record

Returns

self (this Record instance).

missing_fields(required_fields=())[source]

check field-names/index-paths specified in required_fields.

Parameters

required_fields (Iterable) – list/tuple/iterable of field names or index paths of required fields.

Return type

List[Tuple[Union[int, str], …]]

Returns

list of index paths for the ones that are missing or having an empty/None value.

pop(idx, default=None)[source]

check if field name exists and if yes then remove the _Field instance from this Record instance.

Parameters
  • idx (str) – field name.

  • default (Optional[_Field]) – return value if a field with the name specified by idx does not exist in this Record.

Return type

Optional[_Field]

Returns

removed _Field instance if found, else None.

this method got added because the OrderedDict.pop() method does not call __delitem__() (see also __contains__())

pull(from_system)[source]

pull all user/system values and convert them into field values.

Parameters

from_system (str) – system id of the system to pull from.

Return type

Record

Returns

self.

push(onto_system)[source]

push field values of this Record instance to the related user/system values (converted).

Parameters

onto_system (str) – system id of the system to push to.

Return type

Record

Returns

self.

set_current_system_index(sys_fld_name_prefix, path_sep, idx_val=None, idx_add=1)[source]

check and if possible set the current system index of this Record instance.

Parameters
  • sys_fld_name_prefix (str) – user/system field name prefix.

  • path_sep (str) – user/system name path separator.

  • idx_val (Optional[int]) – new current system index value to set (if passed then idx_add will be ignored).

  • idx_add (Optional[int]) – increment current system index value with the passed int value.

Return type

Optional[Record]

Returns

self (this Record instance) if current index got changed, else None.

set_env(system=None, direction=None, action=None, root_rec=None, root_idx=())[source]

set the environment (system/direction/action) of this Record instance.

Parameters
Return type

Record

Returns

self.

sql_columns(from_system, col_names=())[source]

return list of sql column names for given system.

Parameters
  • from_system (str) – system from which the data will be selected/fetched.

  • col_names (Iterable) – optionally restrict to select columns to names given in this list.

Return type

List[str]

Returns

list of sql column names.

sql_select(from_system, col_names=())[source]

return list of sql column names/expressions for given system.

Parameters
  • from_system (str) – system from which the data will be selected/fetched.

  • col_names (Iterable) – optionally restrict to select columns to names given in this list.

Return type

List[str]

Returns

list of sql column names/expressions.

to_dict(filter_fields=None, key_type=<class 'str'>, push_onto=True, use_system_key=True, put_system_val=True, put_empty_val=False, system=None, direction=None)[source]

copy Record leaf values into a dict.

Parameters
  • filter_fields (Optional[Callable[[_Field], Any]]) – callable returning True for each field that need to be excluded in returned dict, pass None to include all fields (if put_empty_val == True).

  • key_type (Union[Type[str], Type[tuple], None]) – type of dict keys: None=field name, tuple=index path tuple, str=index path string (def).

  • push_onto (bool) – pass False to prevent self.push(system).

  • use_system_key (bool) – pass False to put leaf field name/index; def=True to use system field name/keys, specified by the system/direction args.

  • put_system_val (bool) – pass False to include/use main field val; def=True to include system val specified by the system/direction args.

  • put_empty_val (bool) – pass True to also include fields with an empty value (None/’’).

  • system (Optional[str]) – system id to determine included leaf and field val (if put_system_val == True).

  • direction (Optional[str]) – direction id to determine included leaf and field val (if put_system_val == True).

Return type

Dict[Union[int, str, Tuple[Union[int, str], …]], Any]

Returns

dict of filtered leaf user/system values with field names/idx_path-tuples as their key.

update(mapping=(), **kwargs)[source]

update this Record instance - overwriting/extending OrderedDict/super().update() to return self.

Parameters
  • mapping – mapping to use to update this Record instance.

  • kwargs – optional keyword arguments.

Returns

self (this Record instance).

class Records(seq=())[source]

Bases: ae.sys_data.Values

Records class.

each instance of this Records class is a list of 0..n Record instances.

the not overwritten methods of the inherited Values class are also available - like e.g. Values.node_child() or Values.val().

__init__(seq=())[source]

create new Records instance.

Parameters

seq (Iterable) – Iterable used to initialize the new instance (pass list, tuple or other iterable).

__getitem__(key)[source]

x.__getitem__(y) <==> x[y]

Return type

Union[Value, Values, Record, Records, List[Union[Value, Values, Record, Records]]]

__setitem__(key, value)[source]

Set self[key] to value.

set_node_child(rec_or_fld_or_val, *idx_path, system='', direction='', protect=False, root_rec=None, root_idx=(), use_curr_idx=None)[source]

set/replace the child of the node specified by idx_path with the value of rec_or_fld_or_val.

Parameters
  • rec_or_fld_or_val – record, field or value - to be set.

  • idx_path (Union[int, str]) – index path of the node child to set/replace.

  • system (str) – system id (pass None to use default system id of this Records instance).

  • direction (str) – direction id (pass None to use default direction id of this Records instance).

  • protect (bool) – pass True to prevent the replacement of a node child.

  • root_rec (Optional[Record]) – root record of this data structure.

  • root_idx (Optional[Tuple[Union[int, str], …]]) – root index path of this node.

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

Return type

Records

Returns

self (this Records instance).

set_val(val, *idx_path, system='', direction='', flex_sys_dir=True, protect=False, extend=True, converter=None, root_rec=None, root_idx=(), use_curr_idx=None)[source]

set the user/system value referenced by idx_path of this Records instance.

Parameters
  • val (Any) – user/system value to be set/replaced.

  • idx_path (Union[int, str]) – index path of the Value to be set.

  • system (str) – system id (pass None to use default system id of this Records instance).

  • direction (str) – direction id (pass None to use default direction id of this Records instance).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

  • protect (bool) – pass True to protect existing node value from to be changed/replaced.

  • extend (bool) – pass False to prevent extension of data structure.

  • converter (Optional[Callable[[_Field, Any], Any]]) – converter callable to convert user values between systems.

  • root_rec (Optional[Record]) – root Record instance of this data structure.

  • root_idx (Optional[Tuple[Union[int, str], …]]) – root index to this node/Records instance.

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

Return type

Records

Returns

self (this Records instance).

append_record(root_rec, root_idx=(), from_rec=None, clear_leaves=True)[source]

add/append Record to this Records instance.

Parameters
  • root_rec (Record) – root Record instance of this data structure.

  • root_idx (Tuple[Union[int, str], …]) – root index to this node/Records instance.

  • from_rec (Optional[Record]) – Record instance to append, if not passed then use a copy of the first/template Record of this Records instance, else create new Record instance.

  • clear_leaves (bool) – pass False to not clear the leaf values.

Return type

Record

Returns

added/appended Record instance.

compare_records(records, match_fields, field_names=(), exclude_fields=(), record_comparator=None)[source]

compare the records of this instance with the ones passed in the records argument.

Parameters
  • records (Records) – other instance of Records to be compared against self.

  • match_fields (Iterable) – iterable with field names/index paths to determine each Record id/pkey.

  • field_names (Iterable) – iterable with field names/index paths that get compared.

  • exclude_fields (Iterable) – iterable with field names/index-paths that get excluded from to be compared.

  • record_comparator (Optional[Callable[[Record, Record], List[str]]]) – optional callable for additional compare (called for each Record).

Return type

List[str]

Returns

list of differences.

index_match_fields(match_fields)[source]

create/initialize match index for this Records instance (stored in match_index attribute).

Parameters

match_fields (Iterable) – iterable with field names/index paths to determine each Record id/pkey.

Return type

Records

Returns

self (this Records instance).

leaves(system='', direction='', flex_sys_dir=True)[source]

generate leaves/_Field-instances of this Records instance.

Parameters
  • system (str) – system id (pass None to use default system id of the underlying Record instances).

  • direction (str) – direction id (pass None to use default direction of the underlying Record instances).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

Return type

Generator[_Field, None, None]

Returns

leaf/_Field-instance generator.

leaf_indexes(*idx_path, system='', direction='', flex_sys_dir=True)[source]

generate leaf-/_Field-index paths for all fields of this Records instance.

Parameters
  • idx_path (Union[int, str]) – index path to be added as base index path (index path to this Records instance).

  • system (str) – system id (pass None to use default system id of the underlying Record instances).

  • direction (str) – direction id (pass None to use default direction of the underlying Record instances).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

Return type

Generator[Tuple[Union[int, str], …], None, None]

Returns

leaf/_Field-instance generator.

merge_records(records, match_fields=())[source]

merge the records passed in records into this Records instance.

Parameters
  • records (Records) – records to be merged in.

  • match_fields (Iterable) – match fields used to identify already existing records (merge values in this cases).

Return type

Records

Returns

self.

set_env(system='', direction='', root_rec=None, root_idx=())[source]

set the environment (system/direction/action) of each record underneath this Records instance.

Parameters
Return type

Records

Returns

self.

class _Field(root_rec=None, root_idx=(), allow_values=False, **aspects)[source]

Bases: object

Internal/Private class used by Record to create record field instances.

an instance of _Field is representing one record field. the field properties are internally stored within a private dict (_Field._aspects) and are called the ‘aspects’ of a field.

field aspects get used by a field instance e.g. to: * store field value(s) * define callable(s) to convert, filter or validate field values * associate the root record and root index * store any other user-defined field properties (like sql column expressions, comments, …)

the FAT_ALL constant contains all pre-defined aspects (see the other FAT_* constants defined at the top of this module). these values are called ‘aspect keys’ and are used as dict keys in the private dict.

each aspect can additionally have a separate property for each systems/directions - in this case the aspect key gets extended with direction/system ids. the two available direction ids are pre-defined by the constants FAD_FROM and FAD_ONTO. the system ids are not defined in this module, they have to be defined by the application. aspect keys can be compiled with the function aspect_key().

__init__(root_rec=None, root_idx=(), allow_values=False, **aspects)[source]
__repr__()[source]

Return repr(self).

__str__()[source]

Return str(self).

__getitem__(key)[source]
node_child(idx_path, use_curr_idx=None, moan=False, selected_sys_dir=None)[source]

get the node child specified by idx_path relative to this _Field instance.

Parameters
  • idx_path (Union[int, str, Tuple[Union[int, str], …]]) – index path or field name index string.

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

  • moan (bool) – flag to check data integrity; pass True to raise AssertionError if not.

  • selected_sys_dir (Optional[dict]) – optional dict to return the currently selected system/direction.

Return type

Union[Value, Values, Record, Records, _Field, None]

Returns

found node instance or None if not found.

value(*idx_path, system='', direction='', flex_sys_dir=False)[source]

search the Value specified by idx_path and return it if found.

Parameters
  • idx_path (Union[int, str]) – index path of Value.

  • system (str) – system id (‘’ stands for the main/system-independent value).

  • direction (str) – direction id (‘’ stands for the main/system-independent value).

  • flex_sys_dir (bool) – pass True to allow fallback to main (non-user/-system) value.

Return type

Union[Value, Values, Record, Records, None]

Returns

found Value instance of None if not found.

set_value(value, *idx_path, system='', direction='', protect=False, root_rec=None, root_idx=(), use_curr_idx=None)[source]

set/replace the Value instance of the node specified by idx_path.

Parameters
  • value (Union[Value, Values, Record, Records]) – Value instance to be set/replaced.

  • idx_path (Union[int, str]) – index path of the Value to be set.

  • system (str) – system id (‘’ stands for the main/system-independent value).

  • direction (str) – direction id (‘’ stands for the main/system-independent value).

  • protect (bool) – pass True to protect existing node value from to be changed/replaced.

  • root_rec (Optional[Record]) – root Record instance of this data structure.

  • root_idx (Tuple[Union[int, str], …]) – root index to this node/Record instance.

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

Return type

_Field

Returns

self (this _Field instance).

val(*idx_path, system='', direction='', flex_sys_dir=True, use_curr_idx=None, **kwargs)[source]

determine the user/system value referenced by idx_path.

Parameters
  • idx_path (Union[int, str]) – index path items.

  • system (str) – system id (‘’ stands for the main/system-independent value).

  • direction (str) – direction id (‘’ stands for the main/system-independent value).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

  • kwargs – extra args (will be passed to underlying data structure).

Return type

Any

Returns

found user/system value, or None if not found or empty string if value was not set yet.

set_val(val, *idx_path, system='', direction='', flex_sys_dir=True, protect=False, extend=True, converter=None, root_rec=None, root_idx=(), use_curr_idx=None, to_value_type=False)[source]

set the user/system value referenced by idx_path.

Parameters
  • val (Any) – user/system value to be set/replaced.

  • idx_path (Union[int, str]) – index path of the Value to be set.

  • system (str) – system id (‘’ stands for the main/system-independent value).

  • direction (str) – direction id (‘’ stands for the main/system-independent value).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

  • protect (bool) – pass True to protect existing node value from to be changed/replaced.

  • extend (bool) – pass False to prevent extension of data structure.

  • converter (Optional[Callable[[_Field, Any], Any]]) – converter callable to convert user values between systems.

  • root_rec (Optional[Record]) – root Record instance of this data structure.

  • root_idx (Tuple[Union[int, str], …]) – root index to this node/Record instance.

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

  • to_value_type (bool) – pass True ensure conversion of val to a Value instance.

Return type

_Field

Returns

self (this _Field instance).

leaf_value(system='', direction='', flex_sys_dir=False)[source]

determine the leaf value of this field (and optionally system/direction).

Parameters
  • system (str) – system id (‘’ stands for the main/system-independent value).

  • direction (str) – direction id (‘’ stands for the main/system-independent value).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

Return type

Union[Value, Values, Record, Records, None]

Returns

the main or user/system value of this leaf/field or None if deeper value exists and flex_sys_dir is False.

used also to check if a deeper located sys field exists in the current data structure.

leaves(system='', direction='', flex_sys_dir=True)[source]

generate all sub-leaves/_Field-instances underneath this _Field instance.

Parameters
  • system (str) – system id (‘’ stands for the main/system-independent value).

  • direction (str) – direction id (‘’ stands for the main/system-independent value).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

Return type

Generator[_Field, None, None]

Returns

leaf/_Field-instance generator.

leaf_indexes(*idx_path, system='', direction='', flex_sys_dir=True)[source]

generate leaf-/_Field-index paths for all sub-fields underneath (if exist) or this _Field instance.

Parameters
  • idx_path (Union[int, str]) – index path to this _Field instance.

  • system (str) – system id (‘’ stands for the main/system-independent value).

  • direction (str) – direction id (‘’ stands for the main/system-independent value).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

Return type

Generator[Tuple[Union[int, str], …], None, None]

Returns

leaf/_Field-instance generator.

find_aspect_key(*aspect_types, system='', direction='')[source]

search for the passed aspect_types in this _Field instance.

Parameters
  • aspect_types (str) – aspect types (dict key prefixes) to search for.

  • system (Optional[str]) – system id (‘’ stands for the main/system-independent value).

  • direction (Optional[str]) – direction id (‘’ stands for the main/system-independent value).

Return type

Optional[str]

Returns

the full aspect key of the first found aspect that is matching the passed aspect type and optionally also the passed system and direction ids.

the search will done in the following order: * all passed aspect types including the passed system/direction ids. * all passed aspect types including the passed system and both directions (if direction id get not passed). * all passed aspect types including the passed system and without direction id. * all passed aspect types without system and direction ids.

set_env(system='', direction='', root_rec=None, root_idx=())[source]

set the environment (system/direction/action) of each record underneath this _Field instance.

Parameters
Return type

_Field

Returns

self (this _Field instance).

set_system_root_rec_idx(system=None, direction=None, root_rec=None, root_idx=())[source]

set the root record and index of the data structure where this _Field instance is included.

Parameters
  • system (Optional[str]) – system id (don’t pass or pass None to leave unchanged).

  • direction (Optional[str]) – direction id (don’t pass or pass None to leave unchanged).

  • root_rec (Optional[Record]) – new root Record instance of this data structure (pass None to leave unchanged).

  • root_idx (Optional[Tuple[Union[int, str], …]]) – root index to this node/Record instance (pass None to determine).

Return type

_Field

Returns

self (this _Field instance).

aspect_exists(*aspect_types, system='', direction='', flex_sys_dir=False)[source]

check if aspect exists and return full aspect key (including system/direction) if yes.

Parameters
  • aspect_types (str) – aspect types (dict key prefixes) to search for.

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

  • flex_sys_dir (Optional[bool]) – pass False to prevent fallback to system-independent value.

Returns

the full aspect key of the first found aspect that is matching the passed aspect type and optionally also the passed system and direction ids.

Return type

Optional[str]

Returns

aspect_value(*aspect_types, system='', direction='', flex_sys_dir=False)[source]
Parameters
  • aspect_types (str) – aspect types (dict key prefixes) to search for.

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

  • flex_sys_dir (Optional[bool]) – pass False to prevent fallback to system-independent value.

Return type

Any

Returns

the value of the first found aspect that is matching the passed aspect type and optionally also the passed system and direction ids or None if not found.

del_aspect(type_or_key, system='', direction='')[source]

remove aspect from this field.

Parameters
  • type_or_key (str) – either FAT_* type or full key (including already the system and direction)-

  • system (str) – system id string (if type_or_key is a pure FAT_* constant).

  • direction (str) – direction string FAD_* constant (if type_or_key is a pure FAT_* constant).

Return type

Any

Returns

the aspect value of the removed aspect.

set_aspect(aspect_value, type_or_key, system='', direction='', protect=False, allow_values=False)[source]

set/change the value of an aspect identified by type_or_key, system and direction.

Parameters
  • aspect_value (Any) – the value to set on the aspect.

  • type_or_key (str) – either FAT_* type or full key (including already the system and direction)-

  • system (str) – system id string (if type_or_key is a pure FAT_* constant).

  • direction (str) – direction string FAD_* constant (if type_or_key is a pure FAT_* constant).

  • protect (bool) – pass True to prevent overwrite of already existing/set aspect value.

  • allow_values (bool) – pass True to allow change of field value aspect (FAT_VAL aspect key).

Return type

_Field

Returns

self (this _Field instance).

set_aspects(allow_values=False, **aspects)[source]

set multiple aspects provided in aspects.

Parameters
  • allow_values (bool) – pass True to allow change of field value aspect (FAT_VAL aspect key).

  • aspects (Any) – dict of aspects where the dict key is the full/complete aspect key.

Return type

_Field

Returns

self (this _Field instance).

add_aspects(allow_values=False, **aspects)[source]

add multiple aspects provided in aspects.

Parameters
  • allow_values (bool) – pass True to allow change of field value aspect (FAT_VAL aspect key).

  • aspects (Any) – dict of aspects where the dict key is the full/complete aspect key.

Return type

_Field

Returns

self (this _Field instance).

name(system='', direction='', flex_sys_dir=True)[source]

determine one of the names of this field.

Parameters
  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

Return type

str

Returns

main or system-specific name of this field.

del_name(system, direction='')[source]

remove system-specific name from this field.

Parameters
  • system (str) – system id (has to be non-empty system id).

  • direction (str) – direction id (def=’’ stands for both directions).

Return type

_Field

Returns

self (this _Field instance).

has_name(name, selected_sys_dir=None)[source]

check if this field has a name identical to the one passed in name.

Parameters
  • name (str) – name to search for.

  • selected_sys_dir (Optional[dict]) – pass dict to get back the system and direction ids of the found name.

Return type

Optional[str]

Returns

full aspect key of the found name (including system/direction) or None if not found.

set_name(name, system='', direction='', protect=False)[source]

set/change one of the names of this field.

Parameters
  • name (str) – the new name of this field.

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

  • protect (bool) – pass True to prevent overwrite of already set/existing name.

Return type

_Field

Returns

self (this _Field instance).

root_rec(system='', direction='')[source]

determine and return root record of this field with given system and direction ids.

Parameters
  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

Optional[Record]

Returns

root record instance or None if not set.

set_root_rec(rec, system='', direction='')[source]

set/change the root record of this field, system and direction.

Parameters
  • rec (Record) – root record instance.

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

_Field

Returns

self (this _Field instance).

root_idx(system='', direction='')[source]

return the root index of this field for the specified system and direction ids.

Parameters
  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

Tuple[Union[int, str], …]

Returns

root index of this field.

set_root_idx(idx_path, system='', direction='')[source]

set/change the root record of this field, system and direction.

Parameters
  • idx_path (Tuple[Union[int, str], …]) – root index for this field/system/direction.

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

_Field

Returns

self (this _Field instance).

calculator(system='', direction='')[source]

return the calculation callable for this field, system and direction.

Parameters
  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

Optional[Callable[[_Field], Any]]

Returns

callable used to calculate the value of this field or None if not set/exists/found.

set_calculator(calculator, system='', direction='', protect=False)[source]

set/change the field value calculator of this field, system and direction.

Parameters
  • calculator (Callable[[_Field], Any]) – new callable used to calculate the value of this field.

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

  • protect (bool) – pass True to prevent overwrite of already set/existing calculator callable.

Return type

_Field

Returns

self (this _Field instance).

clear_val(system='', direction='')[source]

return the initial field value (the clear value) of this field, system and direction.

Parameters
  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

Any

Returns

the found clear/init value or None if not set/found.

set_clear_val(clr_val, system='', direction='')[source]

set/change the clear/init value of this field, system and direction.

Parameters
  • clr_val – new clear/init value of this field.

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

_Field

Returns

self (this _Field instance).

_ensure_system_value(system, direction='', root_rec=None, root_idx=())[source]

check if a field value for the specified system/direction exists and if not then create it.

Parameters
  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

  • root_rec (Optional[Record]) – root Record instance of this field, system and direction.

  • root_idx (Tuple[Union[int, str], …]) – root index to this node/_Field instance.

converter(system, direction='')[source]

return the converter callable for this field, system and direction.

Parameters
  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

Optional[Callable[[_Field, Any], Any]]

Returns

callable used to convert the field value between systems or None if not set.

separate system-specific representations of the field value can be (automatically) converted by specifying a converter callable aspect.

set_converter(converter, system, direction='', protect=True, root_rec=None, root_idx=())[source]

set/change the field value converter of this field, system and direction.

Parameters
  • converter (Callable[[_Field, Any], Any]) – new callable used to convert the value of this field between systems.

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

  • protect (bool) – pass False to allow overwrite of already set converter callable.

  • root_rec (Optional[Record]) – root Record instance of this field, system and direction.

  • root_idx (Tuple[Union[int, str], …]) – root index to this node/_Field instance.

Return type

_Field

Returns

self (this _Field instance).

convert(val, system, direction)[source]

convert field value from/onto system.

Parameters
  • val (Any) – field value to convert.

  • system (str) – system to convert from/onto.

  • direction (str) – conversion direction (from or onto - see FAD_FROM and FAD_ONTO).

Return type

Any

Returns

converted field value.

filterer(system='', direction='')[source]

return the filter callable for this field, system and direction.

Parameters
  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

Optional[Callable[[_Field], Any]]

Returns

callable used to filter this field/parent-record or None if not set.

set_filterer(filterer, system='', direction='', protect=False)[source]

set/change the filterer callable of this field, system and direction.

Parameters
  • filterer (Callable[[_Field], Any]) – new callable used to filter this field or the parent record.

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

  • protect (bool) – pass True to prevent overwrite of already set/existing filter callable.

Return type

_Field

Returns

self (this _Field instance).

sql_expression(system='', direction='')[source]

return the sql column expression for this field, system and direction.

Parameters
  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

Optional[str]

Returns

sql column expression string if set or None if not set.

set_sql_expression(sql_expression, system='', direction='', protect=False)[source]

set/change sql column expression of this field, system and direction.

Parameters
  • sql_expression (str) – new sql column expression used to fetch associated db column of this field.

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

  • protect (bool) – pass True to prevent overwrite of already set/existing filter callable.

Return type

_Field

Returns

self (this _Field instance).

validator(system='', direction='')[source]

return the validation callable for this field, system and direction.

Parameters
  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

Optional[Callable[[_Field, Any], Any]]

Returns

validation callable if set or None if not set.

set_validator(validator, system='', direction='', protect=False, root_rec=None, root_idx=())[source]

set/change the field value validator of this field, system and direction.

Parameters
  • validator (Callable[[_Field, Any], Any]) – new callable used to validate the value of this field, systems and direction.

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

  • protect (bool) – pass False to allow overwrite of already set converter callable.

  • root_rec (Optional[Record]) – root Record instance of this field, system and direction.

  • root_idx (Tuple[Union[int, str], …]) – root index to this node/_Field instance.

Return type

_Field

Returns

self (this _Field instance).

validate(val, system='', direction='')[source]

validate field value for specified system and direction.

Parameters
  • val (Any) – field value to validate (if ok to be set as new field value).

  • system (str) – system id of new field value.

  • direction (str) – direction id of new field value.

Return type

bool

Returns

True if val is ok to be set as new field value else False.

append_record(system='', direction='', flex_sys_dir=True, root_rec=None, root_idx=())[source]

append new record to the Records value of this field/system/direction.

Parameters
  • system (str) – system id of the field value to extend.

  • direction (str) – direction id of the field value to extend.

  • flex_sys_dir (bool) – pass False to prevent fallback to system-independent value.

  • root_rec (Optional[Record]) – root Record instance of this field, system and direction.

  • root_idx (Tuple[Union[int, str], …]) – root index to this node/_Field instance.

Return type

Record

Returns

added/appended Record instance.

clear_leaves(system='', direction='', flex_sys_dir=True, reset_lists=True)[source]

clear/reset field values and if reset_lists == True also Records/Values lists to one item.

Parameters
  • system (str) – system of the field value to clear, pass None to clear all field values.

  • direction (str) – direction of the field value to clear.

  • flex_sys_dir (bool) – if True then also clear field value if system is given and field has no system value.

  • reset_lists (bool) – if True/def then also clear Records/lists to one item.

Return type

_Field

Returns

self (this _Field instance).

copy(deepness=0, root_rec=None, root_idx=(), **kwargs)[source]

copy the aspects (names, indexes, values, …) of this field.

Parameters
  • deepness – deep copy level: <0==see deeper(), 0==only copy current instance, >0==deep copy to deepness value - _Field occupies two deepness: 1st=_Field, 2nd=Value).

  • root_rec (Optional[Record]) – destination root record.

  • root_idx (Tuple[Union[int, str], …]) – destination index path (tuple of field names and/or list/Records/Values indexes).

  • kwargs – additional arguments (will be passed on - most of them used by Record.copy).

Return type

_Field

Returns

new/copied _Field instance.

parent(system='', direction='', value_types=None)[source]

determine one of the parent ValueType instances in this data structure above of this field.

Parameters
  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

  • value_types (Optional[Tuple[Type[Union[Value, Values, Record, Records]], …]]) – pass tuple of ValueType to restrict search to one of the passed types.

Return type

Union[Value, Values, Record, Records, None]

Returns

found parent instance or None if not set or if type is not of passed value_types.

pull(from_system, root_rec, root_idx)[source]

pull the system-specific value (specified by from_system) into the main value of this field.

Parameters
  • from_system (str) – system id of the system to pull from.

  • root_rec (Record) – root Record instance of this field, system and direction.

  • root_idx (Tuple[Union[int, str], …]) – root index to this node/_Field instance.

Return type

_Field

Returns

self (this _Field instance).

push(onto_system, root_rec, root_idx)[source]

push the main value of this field onto the system-specific value (specified by onto_system).

Parameters
  • onto_system (str) – system id of the system to pull from.

  • root_rec (Record) – root Record instance of this field, system and direction.

  • root_idx (Tuple[Union[int, str], …]) – root index to this node/_Field instance.

Return type

_Field

Returns

self (this _Field instance).

string_to_records(str_val, field_names, rec_sep=',', fld_sep='=', system='', direction='')[source]

convert formatted string into a Records instance containing several Record instances.

Parameters
  • str_val (str) – formatted string to convert.

  • field_names (Sequence) – list/tuple of field names of each record

  • rec_sep (str) – character(s) used in str_val to separate the records.

  • fld_sep (str) – character(s) used in str_val to separate the field values of each record.

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

Records

Returns

converted Records instance.

record_field_val(*idx_path, system='', direction='')[source]

get/determine the value of any field specified via idx_path within this data structure.

Parameters
  • idx_path (Union[int, str]) – index path of the field.

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

Any

Returns

the field value if found else None.

this method has an alias named rfv().

rfv(*idx_path, system='', direction='')

alias of method record_field_val()

Return type

Any

system_record_val(*idx_path, system='', direction='', use_curr_idx=None)[source]

get/determine the current value of a/this field within this data structure.

Parameters
  • idx_path (Union[int, str]) – index path of the field.

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

  • use_curr_idx (Optional[list]) – list of counters to specify if and which current indexes have to be used.

Return type

Any

Returns

the currently selected field value if found else None.

this method has an alias named srv().

srv(*idx_path, system='', direction='', use_curr_idx=None)

alias of method system_record_val()

Return type

Any

in_actions(*actions, system='', direction='')[source]

determine if current data structure is in one of the passed actions.

Parameters
  • actions (str) – tuple of actions (see ACTION_ constants).

  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

bool

Returns

True if the data structure has set one of the passed actions else False.

this method has an alias named ina().

ina(*actions, system='', direction='')

alias of method in_actions()

Return type

bool

current_records_idx(system='', direction='')[source]

determine current index of Records instance, situated above of this field in this data structure.

Parameters
  • system (str) – system id (def=’’ stands for the main/system-independent value).

  • direction (str) – direction id (def=’’ stands for the main/system-independent value).

Return type

Union[int, str, None]

Returns

full index path or None if no current index exists above this field.

this method has an alias named crx().

crx(system='', direction='')

alias of method current_records_idx()

Return type

Union[int, str, None]

VALUE_TYPES = (<class 'ae.sys_data.Value'>, <class 'ae.sys_data.Values'>, <class 'ae.sys_data.Record'>, <class 'ae.sys_data.Records'>)

tuple of classes/types used for system data values

LIST_TYPES = (<class 'ae.sys_data.Values'>, <class 'ae.sys_data.Records'>)

tuple of classes/types used for system data lists

NODE_TYPES = (<class 'ae.sys_data.Record'>, <class 'ae.sys_data.Records'>)

tuple of classes/types used for system data nodes

NODE_CHILD_TYPES = (<class 'ae.sys_data._Field'>, <class 'ae.sys_data.Record'>)

tuple of classes/types used for system data node children

PARENT_TYPES = (<class 'ae.sys_data.Values'>, <class 'ae.sys_data.Record'>, <class 'ae.sys_data.Records'>)

value types that can have children (Value excluded)