ae.deep

deep data structure search and replace

this portion is pure python and has no external dependencies, apart from the ae.base namespace portion.

any parts of deeply nested data/object structure can be easily changed by the functions deep_assignment(), deep_object() and deep_replace(), without the need to specify large complex data paths.

Note

although these 3 functions are demonstrating the beauty and flexibility of Python, they can create a lot of side-effects and (wrongly used) even crash your interpreter process.

we make no guarantees or warranties, either express or implied - so please use them with care!

deep_search() can be very useful for discovering internals of the Python language/libraries or to debug and test deep and complex data structures.

Functions

deep_assignment(obj_keys, new_value)

set sub-attribute/item with a mutable parent object to a new value within a deeply nested object/data structure.

deep_object(obj, key_path[, new_value])

determine object in a deep nested object structure and optionally assign a new value to it.

deep_replace(data, replace_with[, …])

replace values within the passed (nested) data structure.

deep_search(obj, found[, skip_types, …])

search key and/or value within the passed (nested) object structure.

DeepDataType

deep data structure root types

alias of Union[dict, list]

DeepDataPath

deep data path list of tuples with object and key/attribute

alias of List[Tuple[Any, Any]]

deep_assignment(obj_keys, new_value)[source]

set sub-attribute/item with a mutable parent object to a new value within a deeply nested object/data structure.

Parameters
  • obj_keys (List[Tuple[Any, Any]]) –

    list of (object, key) tuples identifying an element within a deeply nested data structure or object hierarchy. the root of the data/object structure is the object at list index 0 and the element to be changed is identified by the object and key in the last list item of this argument.

    the referenced data structure can contain immutable objects (like tuples and str) which will be accordingly changed/replaced if affected/needed.

    at least one object/element within the data structure has to be mutable, else a ValueError will be raised.

    if the last list item references a single character in a string then also the type of new_value has to be string (in this case the single character will be replaced with the string in new_value). if the types are not matching then a TypeError will be raised.

  • new_value (Any) – value to be assigned to the element referenced by the last list item of the argument in obj_key_path.

deep_object(obj, key_path, new_value=<ae.base._UNSET object>)[source]

determine object in a deep nested object structure and optionally assign a new value to it.

Parameters
  • obj (Any) – start object to search in (and its sub-objects).

  • key_path (str) –

    composed key string containing dict keys, tuple/list/str indexes and attribute names. the dot (.) character is identifying attribute names. [ and ] are enclosing index values, like shown in the following examples:

    class AClass:
        str_attr_name = 'a_attr_val'
        dict_attr = dict(a=3)
    
    class BClass:
        str_attr_name = 'b_b_b_b_b'
        a_obj = AClass()
    
    b = BClass()
    assert deep_object(b, 'str_attr_name') == 'b_b_b_b_b'
    assert deep_object(b, 'a_obj.str_attr_name') == 'a_attr_val'
    assert deep_object(b, 'a_obj.dict_attr["a"]') == 3
    

    key path strings for dicts can alternatively be specified without the high commas (enclosing the key string), like e.g.:

    d = dict(a_str_key=1)
    assert deep_object(d, '["a_str_key"]') == 1  # with high commas returns 1
    assert deep_object(d, '[a_str_key]') == 1    # same result/return value
    

    when the first part of the key path string is specifying an index you can also leave away the opening square bracket:

    assert deep_object(d, 'a_str_key]') == 1     # again - the same return 1
    

  • new_value (Optional[Any]) –

    optional new value - replacing the found object. the old value will be returned.

    Note

    tuples and strings that are embedding the found object will be automatically updated/replaced up in the data tree structure until a mutable object (list, dict or object) get found.

Return type

Any

Returns

specified object/value (the old value if new_value got passed) or UNSET if not found/exists (key path string is invalid).

deep_replace(data, replace_with, immutable_types=(<class 'tuple'>, ), obj_keys=None)[source]

replace values within the passed (nested) data structure.

Parameters
  • data (Union[dict, list]) – list or dict data structure to be deep searched and replaced. can contain any combination of deep nested list/dict objects. the sub-structure-types dict and list as well as the immutable types specified by immutable_types will be recursively deep searched (top down) by passing their items one by one to the function specified by replace_with.

  • replace_with (Callable[[List[Tuple[Any, Any]], Any, Any], Any]) – called for each item with 3 arguments (data-struct-path, key in data-structure, value), and if the return value is not equal to UNSET then it will be used to overwrite the value in the data-structure.

  • immutable_types (Tuple[Type, …]) – tuple of immutable iterable types, treated as replaceable items. each of the immutable types passed in this tuple has to be convertible to a list object. by default only the items of a tuple are replaceable. to also allow the replacement of single characters in a string pass the argument value (tuple, str) into this parameter.

  • obj_keys (Optional[List[Tuple[Any, Any]]]) – used (internally only) to pass the parent data-struct path in recursive calls.

search key and/or value within the passed (nested) object structure.

Parameters
  • obj (Any) –

    root object to start the top-down deep search from, which can contain any combination of deep nested elements/objects. for each sub-element the callable passed into found will be executed. if the callable returns True then the data path, the key and the value will stored in a tuple and added to the search result list (finally returned to the caller of this function).

    for iterable objects of type dict/tuple/list, the sub-items will be searched, as well as the attributes determined via the Python dir function. to reduce the number of items/ attributes to be searched use the parameters skip_types and/or skip_attr_prefix.

  • found (Callable[[List[Tuple[Any, Any]], Any, Any], Any]) – called for each item with 3 arguments (data-struct-path, key in data-structure, value), and if the return value is True then the data/object path, the last key and value will be added as a new item to the returned list.

  • skip_types (Tuple[Type, …]) – tuple of types to skip from to be searched deeper (see specified default tuple in the parameter declaration).

  • skip_attr_prefix (str) – name prefix string for attribute(s) that has/have to be skipped from deeper search. by default the attributes which names starting with double-underscore characters will not deeper searched.

  • obj_keys (Optional[List[Tuple[Any, Any]]]) – used (internally only) to pass the parent data-struct path in recursive calls.

Return type

List[Tuple[List[Tuple[Any, Any]], Any, Any]]

Returns

list of tuples (data-struct-path, key, value); one tuple for each found item within the passed obj argument.