ae.paths

generic file path helpers

this pure python namespace portion is providing useful path helper functions as well as generic system paths for most platforms, like e.g.:

  • android OS

  • iOS

  • linux

  • macOS

  • Windows

the only external hard dependency of this module are the ae namespace portions ae.base and ae.files. optional dependencies are:

path helper functions

the generator functions coll_files(), coll_folders() and coll_items() are building the fundament for most of the file and folder collection functionality, provided by this module. for example the function path_files() is using these generators to determine the files within a folder structure that are matching the specified wildcards and path part placeholders. similarly the function path_folders() for folders, and path_items() to collect both, file and folder names.

use the functions copy_files() and move_files() to duplicate and move multiple files or entire file path trees. these two functions are based on copy_file() and move_file(). the functions copy_tree() and move_tree() providing an alternative way to copy or move entire directory trees.

the helper function normalize() converts path strings containing path placeholders into regular path strings, resolving symbolic links, or is converting paths string from absolute paths to relative paths and vice versa.

to determine if the path of a file or folder is matching a glob-like path pattern/mask with wildcards, the functions path_match() and paths_match() can be used. useful specially for cases where you don’t have direct access to the file system.

file paths for series of files, e.g. for logging, can be determined via the series_file_name() function.

the function skip_py_cache_files() can be used in path file collections to skip the files situated in the Python cache folder (PY_CACHE_FOLDER respectively __pycache__).

generic system paths

generic system paths are determined by the following helper functions:

these system paths together with additional generic paths like e.g. the current working directory, storage paths provided by the plyer package, or user and application paths, are provided as path placeholders, which get stored within the PATH_PLACEHOLDERS dict by calling the function add_common_storage_paths().

path_name() and placeholder_path() are converting regular path strings or parts of it into path placeholders.

file/folder collection and classification

more complex collections of files and folder paths, and the grouping of them, can be done with the classes Collector, described in the underneath section collecting files, and FilesRegister, described in the section files register.

use the Collector class for temporary quick file path searches on your local file systems as well as on remote servers/hosts. one implementation example is e.g. the method deployed_code_files() of the aedev.pythonanywhere module.

the class FilesRegister helps you to create and cache file path registers permanently, to quickly find at any time the best fitting match for a requested purpose. for example the gui_app module is using it to dynamically select image/font/audio/… resource files depending on the current user preferences, hardware and/or software environment.

collecting files

to collect file names in the current working directory create an instance of the Collector class and call its collect() method with a file or folder path, which can contain wildcards:

.. code-block:: python

from ae.paths import Collector coll = Collector() coll.collect(’*.png’) image_files_list = coll.files

after that a list containing the found file names can then be retrieved from the files attribute.

collect() can be called multiple times to accumulate and extend the files list:

.. code-block:: python

coll = Collector() coll.collect(’.png’) coll.collect(’.jpg’) image_files_list = coll.files

multiple calls of Collector.collect() can be joined into one code line, because it is returning its instance. the following statement is equivalent to the last example:

.. code-block:: python

image_files_list = Collector().collect(’.png’).collect(’.jpg’).files

by specifying the ** wildcard entire folder trees can be scanned. the following example is collection all the files, including the hidden ones, in the folder tree under the current working directory:

.. code-block:: python

test_files = Collector().collect(’/*’).collect(’/.*’).files

Hint

the second call of the Collector.collect() method in this example has to be done only if you are using Python’s glob.glob() as the searcher callback, which excludes hidden files (with a leading dot) from to match the * wildcard.

Collector is by default only returning files. to also collect folder paths in a deep folder tree, you have to pass the collecting generator function to the optional item_collector parameter of the Collector class. the accumulated files and folders can then be retrieved from their respective instance attributes files, paths and selected:

.. code-block:: python

coll = Collector(item_collector=coll_items) coll.collect(…) … files = coll.files folders = coll.paths file_and_folder_items = coll.selected

the found files are provided by the Collector.files instance attribute. found folders will be separately collected within the Collector instance attribute paths. the selected attribute contains all found files and folders in a single list.

collect from multiple locations

in a single call of collect(), providing the method parameters append or select, you can scan multiple combinations of path prefixes and suffixes, which can both contain wildcards and folder names, whereas the suffixes containing also parts of the file names to search for.

Hint

the wildcards *, ** and ? are allowed in the prefixes as well as in suffixes.

the resulting file paths are relative or absolute, depending on if the specified prefix(es) containing absolute or relative paths.

in the following example determines the relative paths of all folders directly underneath the current working directory with a name that contains the string ‘xxx’ or is starting with ‘yyy’ or is ending with ‘zzz’:

.. code-block:: python

coll = Collector(item_collector=coll_folders) coll.collect(‘’, append=(’xxx’, ‘yyy*’, ‘*zzz’)) folders = coll.paths

Hint

replace empty string in the first argument of collect() with ‘{cwd}’ to get absolute paths.

the following example is collecting the absolute paths of files with the name xxx.cfg from the first found location/folder, starting to search in the current working directory, then in the folder above the application data folder, and finally in a folder with the name of the main application underneath the user data folder:

.. code-block:: python

coll = Collector() coll.collect(“{cwd}”, “{app}/..”, “{usr}/{main_app_name}”, append=”xxx.cfg”) found_files = coll.files

to set or change the generic path placeholder parts values, e.g. of the main application name ({main_app_name}) and the application data path ({app}), you simply specify their corresponding values as kwargs in the construction of the Collector instance:

.. code-block:: python

coll = Collector(main_app_name=…, app=…)

additionally you can specify any other path placeholders that will be automatically used and replaced by the Collector instance:

.. code-block:: python

coll = Collector(any_other_placeholder=…)

by default only the found file(s)/folder(s) of the first combination will be collected. to collect all files instead, pass an empty tuple to the method argument only_first_of of :meth:`~Collector.collect’:

.. code-block:: python

coll.collect(…, append=…, [select=…, ] only_first_of=())

add one of the strings ‘prefix’, ‘append’ or ‘select’ to the only_first_of tuple argument to collect only the files/folders of the first combination of the specified prefixes, append-suffixes and select-suffixes.

by using the select argument the found files and folders will additionally be collected in the Collector instance attribute selected.

combinations collected via the select argument that are not existing will be logged. the results are provided by the instance attributes failed, prefix_failed and suffix_failed.

files register

a files register is an instance of the FilesRegister, providing property based file collection and selection, which is e.g. used by the ae.gui_app ae namespace portion to find and select resource files like icon/image or sound files.

files can be collected from various places by a single instance of the class FilesRegister:

.. code-block:: python

from ae.paths import FilesRegister

file_reg = FilesRegister(‘first/path/to/collect’) file_reg.add_paths(‘second/path/to/collect/files/from’)

registered_file = file_reg.find_file(‘file_name’)

in this example the FilesRegister instance collects all files that are existing in any sub-folders underneath the two provided paths. then the find_file() method will return a file object of type RegisteredFile of the last collected file with the stem (base name w/o extension) ‘file_name’.

multiple files with the same stem can be collected and registered e.g. with different formats, to be selected by the app by their different properties, which are specified in the folder names underneath the collected paths. assuming your application is providing an icon image in two sizes, provided within the following folder structure, situated in the current working directory:

resources/
    size_72/
        app_icon.jpg
    size_150/
        app_icon.png

when you then create an instance of FilesRegister both image files from the resources folder will get registered, interpreting the sub-folder names (size_*) as properties or attributes for the registered files, where size will result as the property name and the string after the underscore as the property value.

to retrieve the paths of the application image file with the size 72, call the find_file() method:

.. code-block:: python

file_reg = FilesRegister(‘resources’) app_icon_image_path = file_reg.find_file(‘app_icon’, dict(size=72))

as a shortcut you can alternatively call the object directly (leaving .find_file away):

.. code-block:: python

app_icon_image_path = file_reg(‘app_icon’, dict(size=150))

the resulting file path in app_icon_image_path will be “resources/size_72/app_icon.jpg” in the penultimate example and “resources/size_150/app_icon.png” in the last example.

an instance of FilesRegister (file_reg) behaves like a dict object, where the item key is the file stem (‘app_icon’) and the item value is a list of instances of RegisteredFile. both files in the resources folder are provided as one dict item:

.. code-block:: python

file_reg = FilesRegister(‘resources’) assert ‘app_icon’ in file_reg assert len(file_reg) == 1 assert len(file_reg[‘app_icon’]) == 2 assert isinstance(file_reg[‘app_icon’][0], RegisteredFile)

for more complex selections you can use callables passed into the property_matcher and file_sorter arguments of find_file().

Module Attributes

APPEND_TO_END_OF_FILE_LIST

special flag default value for the first_index argument of the add_* methods of FilesRegister to append new file objects to the end of the name's register file object list.

INSERT_AT_BEGIN_OF_FILE_LIST

special flag default value for the first_index argument of the add_* methods of FilesRegister to insert new file objects always at the begin of the name's register file object list.

COLLECTED_FOLDER

item type value for a folder|directory|node

CollArgType

argument type passed to most callable arguments of coll_items() except of searcher

CollCreatorReturnType

type of the return value of the creator callable, which will be returned to the caller

CollYieldType

type of collected item, which is either COLLECTED_FOLDER for a folder or the file extension for a file

CollYieldItems

type of collected item iterator yielding tuples of (CollYieldType, item[_path])

SearcherRetType

type of the return value of the callable searcher argument of coll_items()

SearcherType

type of the callable searcher argument of coll_items()

copy_file(src, dst, *[, follow_symlinks])

Copy data and metadata.

copy_tree(src, dst[, symlinks, ignore, ...])

Recursively copy a directory tree and return the destination directory.

move_file(src, dst[, copy_function])

Recursively move a file or directory to another location.

move_tree(src, dst[, copy_function])

Recursively move a file or directory to another location.

PATH_PLACEHOLDERS

placeholders of user-, os- and app-specific system paths and file name parts

Functions

add_common_storage_paths()

add common storage paths to PATH_PLACEHOLDERS depending on the operating system (OS).

app_data_path()

determine the os-specific absolute path of the {app} directory where user app data can be stored.

app_docs_path()

determine the os-specific absolute path of the {ado} directory where user documents app are stored.

coll_files(file_mask[, file_class])

determine existing file(s) underneath the folder specified by file_mask.

coll_folders(folder_mask[, folder_class])

determine existing folder(s) underneath the folder specified by folder_mask.

coll_item_type(item_path)

classify path item to be either file, folder or non-existent/excluded/skipped.

coll_items(item_mask[, searcher, selector, ...])

determine path-/file-like item(s) specified with optional wildcards by item_mask.

copy_files(src_folder, dst_folder[, ...])

copy files from src_folder into optionally created dst_folder, optionally overwriting destination files.

move_files(src_folder, dst_folder[, overwrite])

move files from src_folder into optionally created dst_folder, optionally overwriting destination files.

normalize(path[, make_absolute, ...])

normalize/transform path replacing PATH_PLACEHOLDERS and the tilde character (for home folder).

path_files(file_mask[, file_class])

determine existing file(s) underneath the folder specified by file_mask.

path_folders(folder_mask[, folder_class])

determine existing folder(s) underneath the folder specified by folder_mask.

path_items(item_mask[, selector, creator])

determine existing file/folder item(s) underneath the folder specified by item_mask.

path_join(*parts)

join path parts preventing trailing path separator if last part is empty string.

path_match(path, mask)

return True if the specified path matches the specified path mask/pattern.

path_name(path)

determine placeholder key name of the specified path.

paths_match(paths, masks)

filter the paths matching at least one of the specified glob-like wildcard masks.

placeholder_key(path)

determine PATH_PLACEHOLDERS key of specified path.

placeholder_path(path)

replace begin of path string with the longest prefix found in PATH_PLACEHOLDERS.

series_file_name(file_path[, digits, ...])

determine non-existent series file name with a unique series index.

skip_py_cache_files(file_path)

file exclude callback for the files under Python's cache folders.

user_data_path()

determine the os-specific absolute path of the {usr} directory where user data can be stored.

user_docs_path()

determine the os-specific absolute path of the {doc} directory where the user is storing the personal documents.

Classes

Collector([item_collector])

file/folder collector class

FilesRegister(*add_path_args[, ...])

file register catalog - see also files register examples.

APPEND_TO_END_OF_FILE_LIST = 9223372036854775807

special flag default value for the first_index argument of the add_* methods of FilesRegister to append new file objects to the end of the name’s register file object list.

INSERT_AT_BEGIN_OF_FILE_LIST = -9223372036854775807

special flag default value for the first_index argument of the add_* methods of FilesRegister to insert new file objects always at the begin of the name’s register file object list.

COLLECTED_FOLDER = None

item type value for a folder|directory|node

CollArgType = typing.Any

argument type passed to most callable arguments of coll_items() except of searcher

CollCreatorReturnType = typing.Any

type of the return value of the creator callable, which will be returned to the caller

CollYieldType

type of collected item, which is either COLLECTED_FOLDER for a folder or the file extension for a file

alias of Optional[str]

CollYieldItems

type of collected item iterator yielding tuples of (CollYieldType, item[_path])

alias of Iterable[tuple[Optional[str], Any]]

SearcherRetType

type of the return value of the callable searcher argument of coll_items()

alias of Iterable[Any]

SearcherType

type of the callable searcher argument of coll_items()

alias of Callable[[str], Iterable[Any]]

coll_item_type(item_path)[source]

classify path item to be either file, folder or non-existent/excluded/skipped.

Parameters:

item_path (str) – file/folder path string.

Return type:

Optional[str]

Returns:

COLLECTED_FOLDER for folders, the file extension for files or None if not found.

coll_items(item_mask, searcher=functools.partial(<function glob>, recursive=True), selector=<class 'str'>, type_detector=<function coll_item_type>, creator=<class 'str'>, **creator_kwargs)[source]

determine path-/file-like item(s) specified with optional wildcards by item_mask.

Parameters:
  • item_mask (str) – file path mask with optional glob() wildcards, the ‘~’ shortcut for the home folder path and any path placeholders, which is specifying the files/folders to collect. use the ‘**’ glob as path to include also items from sub-folders deeper than one level.

  • searcher (Callable[[str], Iterable[Any]]) – callable to convert|resolve path with optional wildcards in item_mask into multiple item file/folder path strings.

  • type_detector (Callable[[Any], Optional[str]]) – callable to typify/classify a found item to be stored in the first tuple items of the returned list. if not passed then path_item_type() will be used.

  • selector (Callable[[Any], Union[bool, Any]]) – called with each found file/folder name to check if it has to be added to the returned list. the default argument (str) results in returning every file/folder found by glob().

  • creator (Callable[[Any], Any]) – each found file/folder will be passed as argument to this class/callable and the instance/return-value will be appended as an item to the returned item list. if not passed then the str class will be used, which means that the items of the returned list will be strings of the file/folder path and name. passing a class, like e.g. ae.files.CachedFile, ae.files.CachedFile or pathlib.Path, will create instances of this class. alternatively you can pass a callable which will be called on each found file/folder. in this case the return value of the callable will be inserted in the related item of the returned list. silly mypy does not support Union[Type[Any], Callable[[str, KwArg()], Any]].

  • creator_kwargs – additional/optional kwargs passed onto the used item_class apart from the item name.

Return type:

Iterable[tuple[Optional[str], Any]]

Returns:

iterator/generator yielding found and selected file system items as instances of the item creator class (passed as creator argument, defaulting to the str class).

coll_files(file_mask, file_class=<class 'str'>, **file_kwargs)[source]

determine existing file(s) underneath the folder specified by file_mask.

Parameters:
  • file_mask (str) – glob file mask (with optional glob wildcards and PATH_PLACEHOLDERS) specifying the files to collect (by default including the sub-folders).

  • file_class (Union[Type[Any], Callable]) – factory used for the returned list items (see coll_items.creator). silly mypy does not support Union[Type[Any], Callable[[str, KwArg()], Any]].

  • file_kwargs – additional/optional kwargs apart from file name passed onto the used item_class.

Return type:

Iterable[tuple[Optional[str], Any]]

Returns:

list of files of the class specified by file_mask.

coll_folders(folder_mask, folder_class=<class 'str'>, **folder_kwargs)[source]

determine existing folder(s) underneath the folder specified by folder_mask.

Parameters:
  • folder_mask (str) – glob folder mask (with optional glob wildcards and PATH_PLACEHOLDERS) specifying the folders to collect (by default including the sub-folders).

  • folder_class (Union[Type[Any], Callable]) – class or factory used for the returned list items (see creator). silly mypy does not support Union[Type[Any], Callable[[str, KwArg()], Any]].

  • folder_kwargs – additional/optional kwargs apart from file name passed onto the used item_class.

Return type:

Iterable[tuple[Optional[str], Any]]

Returns:

list of folders of the class specified by folder_mask.

add_common_storage_paths()[source]

add common storage paths to PATH_PLACEHOLDERS depending on the operating system (OS).

the following storage paths are provided by the plyer PyPi package (not all of them are available in each OS):

  • application: user application directory.

  • documents: user documents directory.

  • downloads: user downloads directory.

  • external_storage: external storage root directory.

  • home: user home directory.

  • music: user music directory.

  • pictures: user pictures directory.

  • root: root directory of the operating system partition.

  • sdcard: SD card root directory (only available in Android if sdcard is inserted).

  • videos: user videos directory.

additionally storage paths that are only available on certain OS (inspired by the method get_drives, implemented in https://github.com/kivy-garden/filebrowser/blob/master/kivy_garden/filebrowser/__init__.py):

  • Linux: external storage devices/media mounted underneath the system partition root in /mnt or /media.

  • Apple Mac OsX or iOS: external storage devices/media mounted underneath the system partition root in /Volume.

  • MS Windows: additional drives mapped as the drive partition name.

app_data_path()[source]

determine the os-specific absolute path of the {app} directory where user app data can be stored.

Hint

use app_docs_path() instead to get a more public path to the user.

Return type:

str

Returns:

path string of the user app data folder.

app_docs_path()[source]

determine the os-specific absolute path of the {ado} directory where user documents app are stored.

Hint

use app_data_path() instead to get a more hidden path to the user.

Return type:

str

Returns:

path string of the user documents app folder.

copy_file(src, dst, *, follow_symlinks=True)

alias for shutil.copy2() (compatible to shutil.copy(), shutil.copyfile() and ae.files.copy_bytes().

copy_tree(src, dst, symlinks=False, ignore=None, copy_function=<function copy2>, ignore_dangling_symlinks=False, dirs_exist_ok=False)

alias for shutil.copytree().

move_file(src, dst, copy_function=<function copy2>)

alias for shutil.move() (see also move_tree()).

move_tree(src, dst, copy_function=<function copy2>)

another alias for shutil.move() (see also move_file()).

copy_files(src_folder, dst_folder, overwrite=False, copier=<function copy2>)[source]

copy files from src_folder into optionally created dst_folder, optionally overwriting destination files.

Parameters:
  • src_folder (str) – path to source folder/directory where the files get copied from. placeholders in PATH_PLACEHOLDERS will be recognized and substituted.

  • dst_folder (str) – path to destination folder/directory where the files get copied to. all placeholders in PATH_PLACEHOLDERS are recognized and will be substituted.

  • overwrite (bool) – pass True to overwrite existing files in the destination folder/directory. on False the files will only get copied if they not exist in the destination.

  • copier (Callable) – copy/move function with src_file and dst_file parameters, returning file path/name.

Return type:

List[str]

Returns:

list of copied files, with their destination path.

move_files(src_folder, dst_folder, overwrite=False)[source]

move files from src_folder into optionally created dst_folder, optionally overwriting destination files.

Parameters:
  • src_folder (str) – path to source folder/directory where the files get moved from. placeholders in PATH_PLACEHOLDERS will be recognized and substituted. please note that the source folders itself will neither be moved nor removed (but will be empty after the operation finished).

  • dst_folder (str) – path to destination folder/directory where the files get moved to. all placeholders in PATH_PLACEHOLDERS are recognized and will be substituted.

  • overwrite (bool) – pass True to overwrite existing files in the destination folder/directory. on False the files will only get moved if they not exist in the destination.

Return type:

List[str]

Returns:

list of moved files, with their destination path.

normalize(path, make_absolute=True, remove_base_path='', remove_dots=True, resolve_sym_links=True)[source]

normalize/transform path replacing PATH_PLACEHOLDERS and the tilde character (for home folder).

Parameters:
  • path (str) – path string to normalize/transform.

  • make_absolute (bool) – pass False to not convert path to an absolute path.

  • remove_base_path (str) – pass a valid base path to return a relative path, even if the argument values of make_absolute or resolve_sym_links are True.

  • remove_dots (bool) – pass False to not replace/remove the . and .. placeholders.

  • resolve_sym_links (bool) – pass False to not resolve symbolic links, passing True implies a True value also for the make_absolute argument.

Return type:

str

Returns:

normalized path string: absolute if remove_base_path is empty and either make_absolute or resolve_sym_links is True; relative if remove_base_path is a base path of path or if path got passed as relative path and neither make_absolute nor resolve_sym_links is True.

path_files(file_mask, file_class=<class 'str'>, **file_kwargs)[source]

determine existing file(s) underneath the folder specified by file_mask.

Parameters:
  • file_mask (str) – glob file mask (with optional glob wildcards and PATH_PLACEHOLDERS) specifying the files to collect (by default including the sub-folders).

  • file_class (Union[Type[Any], Callable]) – factory used for the returned list items (see path_items.creator). silly mypy does not support Union[Type[Any], Callable[[str, KwArg()], Any]].

  • file_kwargs – additional/optional kwargs apart from file name passed onto the used item_class.

Return type:

List[Any]

Returns:

list of files of the class specified by file_mask.

path_folders(folder_mask, folder_class=<class 'str'>, **folder_kwargs)[source]

determine existing folder(s) underneath the folder specified by folder_mask.

Parameters:
  • folder_mask (str) – glob folder mask (with optional glob wildcards and PATH_PLACEHOLDERS) specifying the folders to collect (by default including the sub-folders).

  • folder_class (Union[Type[Any], Callable]) – class or factory used for the returned list items (see creator). silly mypy does not support Union[Type[Any], Callable[[str, KwArg()], Any]].

  • folder_kwargs – additional/optional kwargs apart from file name passed onto the used item_class.

Return type:

List[Any]

Returns:

list of folders of the class specified by folder_mask.

path_items(item_mask, selector=<class 'str'>, creator=<class 'str'>, **creator_kwargs)[source]

determine existing file/folder item(s) underneath the folder specified by item_mask.

Parameters:
  • item_mask (str) – file path mask (with optional glob wildcards and PATH_PLACEHOLDERS) specifying the files/folders to collect.

  • selector (Callable[[str], Any]) – called with each found file/folder name to check if it has to be added to the returned list. the default argument (str) results in returning every file/folder found by glob().

  • creator (Union[Type[Any], Callable]) – each found file/folder will be passed as argument to this class/callable and the instance/return-value will be appended as an item to the returned item list. if not passed then the str class will be used, which means that the items of the returned list will be strings of the file/folder path and name. passing a class, like e.g. ae.files.CachedFile, ae.files.CachedFile or pathlib.Path, will create instances of this class. alternatively you can pass a callable which will be called on each found file/folder. in this case the return value of the callable will be inserted in the related item of the returned list. silly mypy does not support Union[Type[Any], Callable[[str, KwArg()], Any]].

  • creator_kwargs – additional/optional kwargs passed onto the used item_class apart from the item name.

Return type:

List[Any]

Returns:

list of found and selected items of the item class (item_mask).

path_join(*parts)[source]

join path parts preventing trailing path separator if last part is empty string.

Parameters:

parts (str) – path parts to join.

Return type:

str

Returns:

joined path string.

Hint

although os.path.join() is implemented in C, this function is faster:: .. code-block:: python

import os import timeit from ae.paths import path_join paths_secs = timeit.timeit(‘path_join(“test”, “sub_test”, “sub_sub_test”)’, globals=globals()) os_secs = timeit.timeit(‘os.path.join(“test”, “sub_test”, “sub_sub_test”)’, globals=globals()) assert paths_secs < os_secs

even if you import os.path.join() without the namespace prefixes, like this:: .. code-block:: python

from os.path import join as os_join os_secs = timeit.timeit(‘os_join(“test”, “sub_test”, “sub_sub_test”)’, globals=globals()) assert paths_secs < os_secs

_path_match_replacement = re.compile('(/\\\\\\*\\\\\\*|\\\\\\*\\\\\\*/|\\\\\\*|\\\\\\?|\\\\\\[\\\\\\*\\\\\\]|\\\\\\[\\\\\\?\\\\\\]|\\\\\\[!|\\\\\\[|\\\\\\])')

pre-compiled regular expression for path_match(), inspired by the great SO answer of Pugsley (see https://stackoverflow.com/questions/27726545/63212852#63212852)

path_match(path, mask)[source]

return True if the specified path matches the specified path mask/pattern.

Parameters:
  • path (str) – path string to match.

  • mask (str) – path mask/pattern including glob-like wildcards.

Return type:

bool

Returns:

True if the path specified by path matches the mask/pattern specified by the mask argument.

path_name(path)[source]

determine placeholder key name of the specified path.

Parameters:

path (str) – path string to determine name of (can contain placeholders).

Return type:

str

Returns:

name (respectively dict key in PATH_PLACEHOLDERS) of the found path or empty string if not found.

paths_match(paths, masks)[source]

filter the paths matching at least one of the specified glob-like wildcard masks.

Parameters:
  • paths (Sequence[str]) – sequence of path strings to be checked if they match at least one pattern/mask, specified by the masks argument.

  • masks (Sequence[str]) – sequence of path masks/pattern with glob-like wildcards.

Return type:

Iterable[str]

Returns:

list of the paths specified by paths that match at least one mask, specified by the masks argument.

placeholder_key(path)[source]

determine PATH_PLACEHOLDERS key of specified path.

Parameters:

path (str) – path string starting with a PATH_PLACEHOLDERS path prefix.

Return type:

str

Returns:

placeholder key (if found as path prefix), else empty string.

placeholder_path(path)[source]

replace begin of path string with the longest prefix found in PATH_PLACEHOLDERS.

Parameters:

path (str) – path string (optionally including sub-folders and file name).

Return type:

str

Returns:

path string with replaced placeholder prefix (if found).

series_file_name(file_path, digits=2, marker=' ', create=False)[source]

determine non-existent series file name with a unique series index.

Parameters:
  • file_path (str) – file path and name (optional with extension).

  • digits (int) – number of digits used for the series index.

  • marker (str) – marker that will be put at the end of the file name and before the series index.

  • create (bool) – pass True to create the file (to reserve the series index).

Return type:

str

Returns:

file path extended with unique/new series index.

skip_py_cache_files(file_path)[source]

file exclude callback for the files under Python’s cache folders.

Parameters:

file_path (str) – path to file to check for exclusion, relative to the project root folder.

Return type:

bool

Returns:

True if the file specified in file_path has to excluded, else False.

user_data_path()[source]

determine the os-specific absolute path of the {usr} directory where user data can be stored.

Hint

this path is not accessible on Android devices, use user_docs_path() instead to get a more public path to the user.

Return type:

str

Returns:

path string of the user data folder.

user_docs_path()[source]

determine the os-specific absolute path of the {doc} directory where the user is storing the personal documents.

Hint

use user_data_path() instead to get a more hidden user data.

Return type:

str

Returns:

path string of the user documents folder.

PATH_PLACEHOLDERS = {'ado': '/home/docs/Documents/docs', 'app': '/home/docs/.config/docs', 'app_name': 'docs', 'cwd': '/home/docs/checkouts/readthedocs.org/user_builds/ae/checkouts/latest/docs', 'doc': '/home/docs/Documents', 'usr': '/home/docs/.config'}

placeholders of user-, os- and app-specific system paths and file name parts

class Collector(item_collector=<function coll_files>, **placeholders)[source]

Bases: object

file/folder collector class

__init__(item_collector=<function coll_files>, **placeholders)[source]

create new file/folder/item collector instance with individual (extended or overriding) placeholders.

Parameters:
  • item_collector (Callable[[str], Iterable[tuple[Optional[str], Any]]]) – callable to determine the item type to collect. the default is the coll_files() function. pass e.g. coll_folders() to collect only folders or coll_items() to collect both (files and folders). overload the arguments of these functions (with partial) to adapt/change their default arguments.

  • placeholders – all other kwargs are placeholders with their names:replacements as keys:values. the placeholders provided by PATH_PLACEHOLDERS are available too (but will be overwritten by these arguments).

paths: List[Any]

list of found/collected folders

files: List[Any]

list of found/collected files

selected: List[Any]

list of found/collected file/folder item instances

failed

number of not found select-combinations

prefix_failed: Dict[str, int]

not found select-combinations count for each prefix

suffix_failed: Dict[str, int]

not found select-combinations count for each suffix

placeholders

path part placeholders of this Collector instance

check_add(item_mask, select=False)[source]

check if item mask match file/folder(s) and if yes append accordingly to collecting instance lists.

Parameters:
  • item_mask (str) – file/folder mask, optionally including wildcards in the glob.glob format.

  • select (bool) – pass True to additionally add found files/folders into selected.

Return type:

bool

Returns:

True if at least one file/folder got found/added, else False.

_collect_appends(prefix, appends, only_first_of)[source]
_collect_selects(prefix, selects, only_first_of)[source]
collect(*prefixes, append=(), select=(), only_first_of=('append', 'prefix', 'select'))[source]

collect additional files/folders by combining the given prefixes with all the given append/select suffixes.

Note

all arguments of this method can either be passed either as tuples, or for a single value as string.

Parameters:
  • prefixes (str) – tuple of file/folder paths to be used as prefix.

  • append (Union[str, Tuple[str, ...]]) – tuple of file/folder names to be used as append suffix.

  • select (Union[str, Tuple[str, ...]]) – tuple of file/folder names to be used as select suffix. in contrary to append is logging not found prefixes combinations in the instance attributes failed, prefix_failed and suffix_failed.

  • only_first_of (Union[str, Tuple[str, ...]]) – tuple with the strings ‘prefix’, ‘append’ or ‘select’ or one of these strings. if it contains the string ‘prefix’ then only the files/folders of the first combination will be collected. if it contains ‘append’ then only the files/folders of the first combination done with the suffixes passed into the append argument will be collected. if it contains ‘select’ then only the files/folders of the first combination done with the suffixes passed into the select argument will be collected. pass empty tuple to collect all combinations.

Return type:

Collector

each of the passed prefixes will be combined with the suffixes specified in append and in select. the resulting file/folder paths that are exist, will then be added to the appropriate instance attribute, either files for a file or paths for a folder.

additionally the existing file/folder paths from the combinations of prefixes and select will be added in the selected list attribute.

Hint

more details and some examples are available in the doc string of this module.

property error_message: str

returns an error message if an error occurred.

Returns:

error message string if collection failure/error occurred, else an empty string.

class FilesRegister(*add_path_args, property_matcher=None, file_sorter=None, **add_path_kwargs)[source]

Bases: dict

file register catalog - see also files register examples.

__init__(*add_path_args, property_matcher=None, file_sorter=None, **add_path_kwargs)[source]

create files register instance.

this method gets redirected with add_path_args and add_path_kwargs arguments to add_paths().

Parameters:
__call__(*find_args, **find_kwargs)[source]

add_path_args and kwargs will be completely redirected to find_file().

Return type:

Union[str, RegisteredFile, CachedFile, Path, PurePath, Any, None]

add_file(file_obj, first_index=9223372036854775807)[source]

add a single file to the list of this dict mapped by the file-name/stem as dict key.

Parameters:
  • file_obj (Union[str, RegisteredFile, CachedFile, Path, PurePath, Any]) – either file path string or any object with a stem attribute.

  • first_index (int) – pass list index -n-1..n-1 to insert file_obj in the name’s list. values greater than n (==len(file_list)) will append the file_obj to the end of the file object list and values less than n-1 will insert the file_obj to the start of the file.

add_files(files, first_index=9223372036854775807)[source]

add files from another FilesRegister instance.

Parameters:
  • files (Iterable[Union[str, RegisteredFile, CachedFile, Path, PurePath, Any]]) – iterable with file objects to be added.

  • first_index (int) – pass list index -n-1..n-1 to insert the first file_obj in each name’s register list. values greater than n (==len(file_list)) will append the file_obj to the end of the file object list. the order of the added items will be unchanged if this value is greater or equal to zero. negative values will add the items from files in reversed order and after the item specified by this index value (so passing -1 will append the items to the end in reversed order, while passing -(n+1) will insert them at the beginning in reversed order).

Return type:

List[str]

Returns:

list of paths of the added files.

add_paths(*file_path_masks, first_index=9223372036854775807, file_class=<class 'ae.files.RegisteredFile'>, **init_kwargs)[source]

add files found in the folder(s) specified by the file_path_masks args.

Parameters:
  • file_path_masks (str) – file path masks (with optional wildcards and PATH_PLACEHOLDERS) specifying the files to collect (by default including the sub-folders).

  • first_index (int) – pass list index -n-1..n-1 to insert the first file_obj in each name’s register list. values greater than n (==len(file_list)) will append the file_obj to the end of the file object list. the order of the added items will be unchanged if this value is greater or equal to zero. negative values will add the found items in reversed order and after the item specified by this index value (so passing -1 will append the items to the end in reversed order, while passing -(n+1) will insert them at the beginning in reversed order).

  • file_class (Type[Union[str, RegisteredFile, CachedFile, Path, PurePath, Any]]) – the used file object class (see FileObject). each found file object will be passed to the class constructor (callable) and added to the list which is an item of this dict.

  • init_kwargs – additional/optional kwargs passed onto the used file_class. pass e.g. the object_loader to use, if file_class is CachedFile (instead of the default: RegisteredFile).

Return type:

List[str]

Returns:

list of paths of the added files.

add_register(files_register, first_index=9223372036854775807)[source]

add files from another FilesRegister instance.

Parameters:
  • files_register (FilesRegister) – files register instance containing the file_obj to be added.

  • first_index (int) – pass list index -n-1..n-1 to insert the first file_obj in each name’s register list. values greater than n (==len(file_list)) will append the file_obj to the end of the file object list. the order of the added items will be unchanged if this value is greater or equal to zero. negative values will add the found items in reversed order and after the item specified by this index value (so passing -1 will append the items to the end in reversed order, while passing -(n+1) will insert them at the beginning in reversed order).

Return type:

List[str]

Returns:

list of paths of the added files.

find_file(name, properties=None, property_matcher=None, file_sorter=None)[source]

find file_obj in this register via properties, property matcher callables and/or file sorter.

Parameters:
Return type:

Union[str, RegisteredFile, CachedFile, Path, PurePath, Any, None]

Returns:

registered/cached file object of the first found/correct file.

reclassify(file_class=<class 'ae.files.CachedFile'>, **init_kwargs)[source]

re-instantiate all name’s file registers items to instances of the class file_class.

Parameters:
  • file_class (Type[Union[str, RegisteredFile, CachedFile, Path, PurePath, Any]]) – the new file object class (see FileObject). each found file object will be passed to the class constructor (callable) and the return value will then replace the file object in the file list.

  • init_kwargs – additional/optional kwargs passed onto the used file_class. pass e.g. the object_loader to use, if file_class is CachedFile (the default file object class).