ae.paths
generic file path helpers
this pure python namespace portion is providing useful path helper functions as well as generic system paths for most platforms, like e.g.:
android OS
iOS
linux
macOS
Windows
the only external hard dependency of this module are the ae namespace portions ae.base
and ae.files
.
optional dependencies are:
on android OS the PyPi package jnius, needed by the functions
user_data_path()
anduser_docs_path()
.the plyer PyPi package, needed by the function
add_common_storage_paths()
.
path helper functions
the generator functions coll_files()
, coll_folders()
and coll_items()
are building the fundament
for most of the file and folder collection functionality, provided by this module. for example the function
path_files()
is using these generators to determine the files within a folder structure that are matching
the specified wildcards and path part placeholders. similarly the
function path_folders()
for folders, and path_items()
to collect both, file and folder names.
use the functions copy_files()
and move_files()
to duplicate and move multiple files or entire
file path trees. these two functions are based on copy_file()
and move_file()
.
the functions copy_tree()
and move_tree()
providing an alternative way to copy or move
entire directory trees.
the helper function normalize()
converts path strings containing path placeholders into regular path strings,
resolving symbolic links, or is converting paths string from absolute paths to relative paths and vice versa.
to determine if the path of a file or folder is matching a glob-like path pattern/mask with wildcards, the
functions path_match()
and paths_match()
can be used. useful specially for cases where you don’t
have direct access to the file system.
file paths for series of files, e.g. for logging, can be determined via the series_file_name()
function.
the function skip_py_cache_files()
can be used in path file collections to skip the files situated in the
Python cache folder (PY_CACHE_FOLDER
respectively __pycache__
).
generic system paths
generic system paths are determined by the following helper functions:
app_data_path()
: application data path.app_docs_path()
: application documents path.user_data_path()
: user data path.user_docs_path()
: user documents path.
these system paths together with additional generic paths like e.g. the current working directory, storage paths
provided by the plyer package, or user and application paths, are provided as path placeholders, which get
stored within the PATH_PLACEHOLDERS
dict by calling the function add_common_storage_paths()
.
path_name()
and placeholder_path()
are converting regular path strings or parts of it into path
placeholders.
file/folder collection and classification
more complex collections of files and folder paths, and the grouping of them, can be done
with the classes Collector
, described in the underneath section collecting files,
and FilesRegister
, described in the section files register.
use the Collector class for temporary quick file path searches on your
local file systems as well as on remote servers/hosts. one implementation example is e.g. the method
deployed_code_files()
of the aedev.pythonanywhere
module.
the class FilesRegister helps you to create and cache file path
registers permanently, to quickly find at any time the best fitting match for a requested purpose.
for example the gui_app
module is using it to dynamically select
image/font/audio/… resource files depending on the current user preferences, hardware
and/or software environment.
collecting files
to collect file names in the current working directory create an instance of the Collector
class and call
its collect()
method with a file or folder path, which can contain wildcards:
.. code-block:: python
from ae.paths import Collector coll = Collector() coll.collect(’*.png’) image_files_list = coll.files
after that a list containing the found file names can then be retrieved from the files
attribute.
collect()
can be called multiple times to accumulate and extend the files
list:
.. code-block:: python
coll = Collector() coll.collect(’.png’) coll.collect(’.jpg’) image_files_list = coll.files
multiple calls of Collector.collect()
can be joined into one code line, because it is returning its instance.
the following statement is equivalent to the last example:
.. code-block:: python
image_files_list = Collector().collect(’.png’).collect(’.jpg’).files
by specifying the **
wildcard entire folder trees can be scanned. the following example is collection all
the files, including the hidden ones, in the folder tree under the current working directory:
.. code-block:: python
test_files = Collector().collect(’/*’).collect(’/.*’).files
Hint
the second call of the Collector.collect()
method in this example has to be done only if you are
using Python’s glob.glob()
as the searcher callback, which excludes hidden files (with a leading dot)
from to match the *
wildcard.
Collector
is by default only returning files. to also collect folder paths in a deep folder tree, you have
to pass the collecting generator function to the optional item_collector
parameter of the
Collector
class. the accumulated files and folders can then be retrieved from their respective instance
attributes files
, paths
and selected
:
.. code-block:: python
coll = Collector(item_collector=coll_items) coll.collect(…) … files = coll.files folders = coll.paths file_and_folder_items = coll.selected
the found files are provided by the Collector.files
instance attribute. found folders will be separately
collected within the Collector
instance attribute paths
. the selected
attribute contains all found files and folders in a single list.
collect from multiple locations
in a single call of collect()
, providing the method parameters append
or select
, you can scan multiple combinations of path prefixes and suffixes, which can
both contain wildcards and folder names, whereas the suffixes containing also parts of the file names to search for.
Hint
the wildcards *, ** and ? are allowed in the prefixes as well as in suffixes.
the resulting file paths are relative or absolute, depending on if the specified prefix(es) containing absolute or relative paths.
in the following example determines the relative paths of all folders directly underneath the current working directory with a name that contains the string ‘xxx’ or is starting with ‘yyy’ or is ending with ‘zzz’:
.. code-block:: python
coll = Collector(item_collector=coll_folders) coll.collect(‘’, append=(’xxx’, ‘yyy*’, ‘*zzz’)) folders = coll.paths
Hint
replace empty string in the first argument of collect()
with ‘{cwd}’ to get absolute paths.
the following example is collecting the absolute paths of files with the name xxx.cfg from the first found location/folder, starting to search in the current working directory, then in the folder above the application data folder, and finally in a folder with the name of the main application underneath the user data folder:
.. code-block:: python
coll = Collector() coll.collect(“{cwd}”, “{app}/..”, “{usr}/{main_app_name}”, append=”xxx.cfg”) found_files = coll.files
to set or change the generic path placeholder parts values, e.g. of the main application name ({main_app_name}) and
the application data path ({app}), you simply specify their corresponding values as kwargs in the construction
of the Collector
instance:
.. code-block:: python
coll = Collector(main_app_name=…, app=…)
additionally you can specify any other path placeholders that will be automatically used and replaced by the
Collector
instance:
.. code-block:: python
coll = Collector(any_other_placeholder=…)
by default only the found file(s)/folder(s) of the first combination will be collected. to collect all files instead,
pass an empty tuple to the method argument only_first_of
of :meth:`~Collector.collect’:
.. code-block:: python
coll.collect(…, append=…, [select=…, ] only_first_of=())
add one of the strings ‘prefix’, ‘append’ or ‘select’ to the only_first_of
tuple
argument to collect only the files/folders of the first combination of the specified prefixes, append-suffixes
and select-suffixes.
by using the select
argument the found files and folders will additionally be collected
in the Collector
instance attribute selected
.
combinations collected via the select
argument that are not existing will be logged.
the results are provided by the instance attributes failed
, prefix_failed
and
suffix_failed
.
files register
a files register is an instance of the FilesRegister
, providing property based file collection and selection,
which is e.g. used by the ae.gui_app
ae namespace portion to find and select resource files like icon/image or
sound files.
files can be collected from various places by a single instance of the class FilesRegister
:
.. code-block:: python
from ae.paths import FilesRegister
file_reg = FilesRegister(‘first/path/to/collect’) file_reg.add_paths(‘second/path/to/collect/files/from’)
registered_file = file_reg.find_file(‘file_name’)
in this example the FilesRegister
instance collects all files that are existing in any sub-folders underneath
the two provided paths. then the find_file()
method will return a file object of type
RegisteredFile
of the last collected file with the stem (base name w/o extension) ‘file_name’.
multiple files with the same stem can be collected and registered e.g. with different formats, to be selected by the app by their different properties, which are specified in the folder names underneath the collected paths. assuming your application is providing an icon image in two sizes, provided within the following folder structure, situated in the current working directory:
resources/
size_72/
app_icon.jpg
size_150/
app_icon.png
when you then create an instance of FilesRegister
both image files from the resources folder will get
registered, interpreting the sub-folder names (size_*) as properties or attributes for the registered files,
where size will result as the property name and the string after the underscore as the property value.
to retrieve the paths of the application image file with the size 72
, call the
find_file()
method:
.. code-block:: python
file_reg = FilesRegister(‘resources’) app_icon_image_path = file_reg.find_file(‘app_icon’, dict(size=72))
as a shortcut you can alternatively call the object directly (leaving .find_file away):
.. code-block:: python
app_icon_image_path = file_reg(‘app_icon’, dict(size=150))
the resulting file path in app_icon_image_path will be “resources/size_72/app_icon.jpg” in the penultimate example and “resources/size_150/app_icon.png” in the last example.
an instance of FilesRegister
(file_reg) behaves like a dict object, where the item key is the file stem
(‘app_icon’) and the item value is a list of instances of RegisteredFile
. both files in the
resources folder are provided as one dict item:
.. code-block:: python
file_reg = FilesRegister(‘resources’) assert ‘app_icon’ in file_reg assert len(file_reg) == 1 assert len(file_reg[‘app_icon’]) == 2 assert isinstance(file_reg[‘app_icon’][0], RegisteredFile)
for more complex selections you can use callables passed into the property_matcher
and file_sorter
arguments of find_file()
.
Module Attributes
special flag default value for the first_index argument of the add_* methods of |
|
special flag default value for the first_index argument of the add_* methods of |
|
item type value for a folder|directory|node |
|
argument type passed to most callable arguments of |
|
type of the return value of the |
|
type of collected item, which is either |
|
type of collected item iterator yielding tuples of (CollYieldType, item[_path]) |
|
type of the return value of the callable |
|
type of the callable |
|
|
Copy data and metadata. |
|
Recursively copy a directory tree and return the destination directory. |
|
Recursively move a file or directory to another location. |
|
Recursively move a file or directory to another location. |
placeholders of user-, os- and app-specific system paths and file name parts |
Functions
add common storage paths to |
|
determine the os-specific absolute path of the {app} directory where user app data can be stored. |
|
determine the os-specific absolute path of the {ado} directory where user documents app are stored. |
|
|
determine existing file(s) underneath the folder specified by |
|
determine existing folder(s) underneath the folder specified by |
|
classify path item to be either file, folder or non-existent/excluded/skipped. |
|
determine path-/file-like item(s) specified with optional wildcards by |
|
copy files from src_folder into optionally created dst_folder, optionally overwriting destination files. |
|
move files from src_folder into optionally created dst_folder, optionally overwriting destination files. |
|
normalize/transform path replacing PATH_PLACEHOLDERS and the tilde character (for home folder). |
|
determine existing file(s) underneath the folder specified by |
|
determine existing folder(s) underneath the folder specified by |
|
determine existing file/folder item(s) underneath the folder specified by |
|
join path parts preventing trailing path separator if last part is empty string. |
|
return True if the specified path matches the specified path mask/pattern. |
|
determine placeholder key name of the specified path. |
|
filter the paths matching at least one of the specified glob-like wildcard masks. |
|
determine |
|
replace begin of path string with the longest prefix found in |
|
determine non-existent series file name with a unique series index. |
|
file exclude callback for the files under Python's cache folders. |
determine the os-specific absolute path of the {usr} directory where user data can be stored. |
|
determine the os-specific absolute path of the {doc} directory where the user is storing the personal documents. |
Classes
|
file/folder collector class |
|
file register catalog - see also files register examples. |
- APPEND_TO_END_OF_FILE_LIST = 9223372036854775807
special flag default value for the first_index argument of the add_* methods of
FilesRegister
to append new file objects to the end of the name’s register file object list.
- INSERT_AT_BEGIN_OF_FILE_LIST = -9223372036854775807
special flag default value for the first_index argument of the add_* methods of
FilesRegister
to insert new file objects always at the begin of the name’s register file object list.
- COLLECTED_FOLDER = None
item type value for a folder|directory|node
- CollArgType = typing.Any
argument type passed to most callable arguments of
coll_items()
except ofsearcher
- CollCreatorReturnType = typing.Any
type of the return value of the
creator
callable, which will be returned to the caller
- CollYieldType
type of collected item, which is either
COLLECTED_FOLDER
for a folder or the file extension for a file
- CollYieldItems
type of collected item iterator yielding tuples of (CollYieldType, item[_path])
- SearcherRetType
type of the return value of the callable
searcher
argument ofcoll_items()
- SearcherType
type of the callable
searcher
argument ofcoll_items()
- coll_item_type(item_path)[source]
classify path item to be either file, folder or non-existent/excluded/skipped.
- coll_items(item_mask, searcher=functools.partial(<function glob>, recursive=True), selector=<class 'str'>, type_detector=<function coll_item_type>, creator=<class 'str'>, **creator_kwargs)[source]
determine path-/file-like item(s) specified with optional wildcards by
item_mask
.- Parameters:
item_mask¶ (
str
) – file path mask with optionalglob()
wildcards, the ‘~’ shortcut for the home folder path and anypath placeholders
, which is specifying the files/folders to collect. use the ‘**’ glob as path to include also items from sub-folders deeper than one level.searcher¶ (
Callable
[[str
],Iterable
[Any
]]) – callable to convert|resolve path with optional wildcards initem_mask
into multiple item file/folder path strings.type_detector¶ (
Callable
[[Any
],Optional
[str
]]) – callable to typify/classify a found item to be stored in the first tuple items of the returned list. if not passed thenpath_item_type()
will be used.selector¶ (
Callable
[[Any
],Union
[bool
,Any
]]) – called with each found file/folder name to check if it has to be added to the returned list. the default argument (str) results in returning every file/folder found by glob().creator¶ (
Callable
[[Any
],Any
]) – each found file/folder will be passed as argument to this class/callable and the instance/return-value will be appended as an item to the returned item list. if not passed then the str class will be used, which means that the items of the returned list will be strings of the file/folder path and name. passing a class, like e.g.ae.files.CachedFile
,ae.files.CachedFile
orpathlib.Path
, will create instances of this class. alternatively you can pass a callable which will be called on each found file/folder. in this case the return value of the callable will be inserted in the related item of the returned list. silly mypy does not support Union[Type[Any], Callable[[str, KwArg()], Any]].creator_kwargs¶ – additional/optional kwargs passed onto the used item_class apart from the item name.
- Return type:
- Returns:
iterator/generator yielding found and selected file system items as instances of the item creator class (passed as
creator
argument, defaulting to the str class).
- coll_files(file_mask, file_class=<class 'str'>, **file_kwargs)[source]
determine existing file(s) underneath the folder specified by
file_mask
.- Parameters:
file_mask¶ (
str
) – glob file mask (with optional glob wildcards andPATH_PLACEHOLDERS
) specifying the files to collect (by default including the sub-folders).file_class¶ (
Union
[Type
[Any
],Callable
]) – factory used for the returned list items (seecoll_items.creator
). silly mypy does not support Union[Type[Any], Callable[[str, KwArg()], Any]].file_kwargs¶ – additional/optional kwargs apart from file name passed onto the used item_class.
- Return type:
- Returns:
list of files of the class specified by
file_mask
.
- coll_folders(folder_mask, folder_class=<class 'str'>, **folder_kwargs)[source]
determine existing folder(s) underneath the folder specified by
folder_mask
.- Parameters:
folder_mask¶ (
str
) – glob folder mask (with optional glob wildcards andPATH_PLACEHOLDERS
) specifying the folders to collect (by default including the sub-folders).folder_class¶ (
Union
[Type
[Any
],Callable
]) – class or factory used for the returned list items (seecreator
). silly mypy does not support Union[Type[Any], Callable[[str, KwArg()], Any]].folder_kwargs¶ – additional/optional kwargs apart from file name passed onto the used item_class.
- Return type:
- Returns:
list of folders of the class specified by
folder_mask
.
- add_common_storage_paths()[source]
add common storage paths to
PATH_PLACEHOLDERS
depending on the operating system (OS).the following storage paths are provided by the plyer PyPi package (not all of them are available in each OS):
application: user application directory.
documents: user documents directory.
downloads: user downloads directory.
external_storage: external storage root directory.
home: user home directory.
music: user music directory.
pictures: user pictures directory.
root: root directory of the operating system partition.
sdcard: SD card root directory (only available in Android if sdcard is inserted).
videos: user videos directory.
additionally storage paths that are only available on certain OS (inspired by the method get_drives, implemented in https://github.com/kivy-garden/filebrowser/blob/master/kivy_garden/filebrowser/__init__.py):
Linux: external storage devices/media mounted underneath the system partition root in /mnt or /media.
Apple Mac OsX or iOS: external storage devices/media mounted underneath the system partition root in /Volume.
MS Windows: additional drives mapped as the drive partition name.
- app_data_path()[source]
determine the os-specific absolute path of the {app} directory where user app data can be stored.
Hint
use
app_docs_path()
instead to get a more public path to the user.- Return type:
- Returns:
path string of the user app data folder.
- app_docs_path()[source]
determine the os-specific absolute path of the {ado} directory where user documents app are stored.
Hint
use
app_data_path()
instead to get a more hidden path to the user.- Return type:
- Returns:
path string of the user documents app folder.
- copy_file(src, dst, *, follow_symlinks=True)
alias for
shutil.copy2()
(compatible toshutil.copy()
,shutil.copyfile()
andae.files.copy_bytes()
.
- copy_tree(src, dst, symlinks=False, ignore=None, copy_function=<function copy2>, ignore_dangling_symlinks=False, dirs_exist_ok=False)
alias for
shutil.copytree()
.
- move_file(src, dst, copy_function=<function copy2>)
alias for
shutil.move()
(see alsomove_tree()
).
- move_tree(src, dst, copy_function=<function copy2>)
another alias for
shutil.move()
(see alsomove_file()
).
- copy_files(src_folder, dst_folder, overwrite=False, copier=<function copy2>)[source]
copy files from src_folder into optionally created dst_folder, optionally overwriting destination files.
- Parameters:
src_folder¶ (
str
) – path to source folder/directory where the files get copied from. placeholders inPATH_PLACEHOLDERS
will be recognized and substituted.dst_folder¶ (
str
) – path to destination folder/directory where the files get copied to. all placeholders inPATH_PLACEHOLDERS
are recognized and will be substituted.overwrite¶ (
bool
) – pass True to overwrite existing files in the destination folder/directory. on False the files will only get copied if they not exist in the destination.copier¶ (
Callable
) – copy/move function with src_file and dst_file parameters, returning file path/name.
- Return type:
- Returns:
list of copied files, with their destination path.
- move_files(src_folder, dst_folder, overwrite=False)[source]
move files from src_folder into optionally created dst_folder, optionally overwriting destination files.
- Parameters:
src_folder¶ (
str
) – path to source folder/directory where the files get moved from. placeholders inPATH_PLACEHOLDERS
will be recognized and substituted. please note that the source folders itself will neither be moved nor removed (but will be empty after the operation finished).dst_folder¶ (
str
) – path to destination folder/directory where the files get moved to. all placeholders inPATH_PLACEHOLDERS
are recognized and will be substituted.overwrite¶ (
bool
) – pass True to overwrite existing files in the destination folder/directory. on False the files will only get moved if they not exist in the destination.
- Return type:
- Returns:
list of moved files, with their destination path.
- normalize(path, make_absolute=True, remove_base_path='', remove_dots=True, resolve_sym_links=True)[source]
normalize/transform path replacing PATH_PLACEHOLDERS and the tilde character (for home folder).
- Parameters:
make_absolute¶ (
bool
) – pass False to not convert path to an absolute path.remove_base_path¶ (
str
) – pass a valid base path to return a relative path, even if the argument values ofmake_absolute
orresolve_sym_links
are True.remove_dots¶ (
bool
) – pass False to not replace/remove the . and .. placeholders.resolve_sym_links¶ (
bool
) – pass False to not resolve symbolic links, passing True implies a True value also for themake_absolute
argument.
- Return type:
- Returns:
normalized path string: absolute if
remove_base_path
is empty and eithermake_absolute
orresolve_sym_links
is True; relative ifremove_base_path
is a base path ofpath
or ifpath
got passed as relative path and neithermake_absolute
norresolve_sym_links
is True.
- path_files(file_mask, file_class=<class 'str'>, **file_kwargs)[source]
determine existing file(s) underneath the folder specified by
file_mask
.- Parameters:
file_mask¶ (
str
) – glob file mask (with optional glob wildcards andPATH_PLACEHOLDERS
) specifying the files to collect (by default including the sub-folders).file_class¶ (
Union
[Type
[Any
],Callable
]) – factory used for the returned list items (seepath_items.creator
). silly mypy does not support Union[Type[Any], Callable[[str, KwArg()], Any]].file_kwargs¶ – additional/optional kwargs apart from file name passed onto the used item_class.
- Return type:
- Returns:
list of files of the class specified by
file_mask
.
- path_folders(folder_mask, folder_class=<class 'str'>, **folder_kwargs)[source]
determine existing folder(s) underneath the folder specified by
folder_mask
.- Parameters:
folder_mask¶ (
str
) – glob folder mask (with optional glob wildcards andPATH_PLACEHOLDERS
) specifying the folders to collect (by default including the sub-folders).folder_class¶ (
Union
[Type
[Any
],Callable
]) – class or factory used for the returned list items (seecreator
). silly mypy does not support Union[Type[Any], Callable[[str, KwArg()], Any]].folder_kwargs¶ – additional/optional kwargs apart from file name passed onto the used item_class.
- Return type:
- Returns:
list of folders of the class specified by
folder_mask
.
- path_items(item_mask, selector=<class 'str'>, creator=<class 'str'>, **creator_kwargs)[source]
determine existing file/folder item(s) underneath the folder specified by
item_mask
.- Parameters:
item_mask¶ (
str
) – file path mask (with optional glob wildcards andPATH_PLACEHOLDERS
) specifying the files/folders to collect.selector¶ (
Callable
[[str
],Any
]) – called with each found file/folder name to check if it has to be added to the returned list. the default argument (str) results in returning every file/folder found by glob().creator¶ (
Union
[Type
[Any
],Callable
]) – each found file/folder will be passed as argument to this class/callable and the instance/return-value will be appended as an item to the returned item list. if not passed then the str class will be used, which means that the items of the returned list will be strings of the file/folder path and name. passing a class, like e.g.ae.files.CachedFile
,ae.files.CachedFile
orpathlib.Path
, will create instances of this class. alternatively you can pass a callable which will be called on each found file/folder. in this case the return value of the callable will be inserted in the related item of the returned list. silly mypy does not support Union[Type[Any], Callable[[str, KwArg()], Any]].creator_kwargs¶ – additional/optional kwargs passed onto the used item_class apart from the item name.
- Return type:
- Returns:
list of found and selected items of the item class (
item_mask
).
- path_join(*parts)[source]
join path parts preventing trailing path separator if last part is empty string.
Hint
although
os.path.join()
is implemented in C, this function is faster:: .. code-block:: pythonimport os import timeit from ae.paths import path_join paths_secs = timeit.timeit(‘path_join(“test”, “sub_test”, “sub_sub_test”)’, globals=globals()) os_secs = timeit.timeit(‘os.path.join(“test”, “sub_test”, “sub_sub_test”)’, globals=globals()) assert paths_secs < os_secs
even if you import
os.path.join()
without the namespace prefixes, like this:: .. code-block:: pythonfrom os.path import join as os_join os_secs = timeit.timeit(‘os_join(“test”, “sub_test”, “sub_sub_test”)’, globals=globals()) assert paths_secs < os_secs
- _path_match_replacement = re.compile('(/\\\\\\*\\\\\\*|\\\\\\*\\\\\\*/|\\\\\\*|\\\\\\?|\\\\\\[\\\\\\*\\\\\\]|\\\\\\[\\\\\\?\\\\\\]|\\\\\\[!|\\\\\\[|\\\\\\])')
pre-compiled regular expression for
path_match()
, inspired by the great SO answer of Pugsley (see https://stackoverflow.com/questions/27726545/63212852#63212852)
- path_match(path, mask)[source]
return True if the specified path matches the specified path mask/pattern.
- path_name(path)[source]
determine placeholder key name of the specified path.
- Parameters:
path¶ (
str
) – path string to determine name of (can contain placeholders).- Return type:
- Returns:
name (respectively dict key in
PATH_PLACEHOLDERS
) of the found path or empty string if not found.
- paths_match(paths, masks)[source]
filter the paths matching at least one of the specified glob-like wildcard masks.
- Parameters:
- Return type:
- Returns:
list of the paths specified by
paths
that match at least one mask, specified by themasks
argument.
- placeholder_key(path)[source]
determine
PATH_PLACEHOLDERS
key of specified path.- Parameters:
path¶ (
str
) – path string starting with aPATH_PLACEHOLDERS
path prefix.- Return type:
- Returns:
placeholder key (if found as path prefix), else empty string.
- placeholder_path(path)[source]
replace begin of path string with the longest prefix found in
PATH_PLACEHOLDERS
.
- series_file_name(file_path, digits=2, marker=' ', create=False)[source]
determine non-existent series file name with a unique series index.
- Parameters:
- Return type:
- Returns:
file path extended with unique/new series index.
- skip_py_cache_files(file_path)[source]
file exclude callback for the files under Python’s cache folders.
- user_data_path()[source]
determine the os-specific absolute path of the {usr} directory where user data can be stored.
Hint
this path is not accessible on Android devices, use
user_docs_path()
instead to get a more public path to the user.- Return type:
- Returns:
path string of the user data folder.
- user_docs_path()[source]
determine the os-specific absolute path of the {doc} directory where the user is storing the personal documents.
Hint
use
user_data_path()
instead to get a more hidden user data.- Return type:
- Returns:
path string of the user documents folder.
- PATH_PLACEHOLDERS = {'ado': '/home/docs/Documents/docs', 'app': '/home/docs/.config/docs', 'app_name': 'docs', 'cwd': '/home/docs/checkouts/readthedocs.org/user_builds/ae/checkouts/latest/docs', 'doc': '/home/docs/Documents', 'usr': '/home/docs/.config'}
placeholders of user-, os- and app-specific system paths and file name parts
- class Collector(item_collector=<function coll_files>, **placeholders)[source]
Bases:
object
file/folder collector class
- __init__(item_collector=<function coll_files>, **placeholders)[source]
create new file/folder/item collector instance with individual (extended or overriding) placeholders.
- Parameters:
item_collector¶ (
Callable
[[str
],Iterable
[tuple
[Optional
[str
],Any
]]]) – callable to determine the item type to collect. the default is thecoll_files()
function. pass e.g.coll_folders()
to collect only folders orcoll_items()
to collect both (files and folders). overload the arguments of these functions (with partial) to adapt/change their default arguments.placeholders¶ – all other kwargs are placeholders with their names:replacements as keys:values. the placeholders provided by
PATH_PLACEHOLDERS
are available too (but will be overwritten by these arguments).
- failed
number of not found select-combinations
- placeholders
path part placeholders of this Collector instance
- check_add(item_mask, select=False)[source]
check if item mask match file/folder(s) and if yes append accordingly to collecting instance lists.
- collect(*prefixes, append=(), select=(), only_first_of=('append', 'prefix', 'select'))[source]
collect additional files/folders by combining the given prefixes with all the given append/select suffixes.
Note
all arguments of this method can either be passed either as tuples, or for a single value as string.
- Parameters:
prefixes¶ (
str
) – tuple of file/folder paths to be used as prefix.append¶ (
Union
[str
,Tuple
[str
,...
]]) – tuple of file/folder names to be used as append suffix.select¶ (
Union
[str
,Tuple
[str
,...
]]) – tuple of file/folder names to be used as select suffix. in contrary toappend
is logging not foundprefixes
combinations in the instance attributesfailed
,prefix_failed
andsuffix_failed
.only_first_of¶ (
Union
[str
,Tuple
[str
,...
]]) – tuple with the strings ‘prefix’, ‘append’ or ‘select’ or one of these strings. if it contains the string ‘prefix’ then only the files/folders of the first combination will be collected. if it contains ‘append’ then only the files/folders of the first combination done with the suffixes passed into theappend
argument will be collected. if it contains ‘select’ then only the files/folders of the first combination done with the suffixes passed into theselect
argument will be collected. pass empty tuple to collect all combinations.
- Return type:
each of the passed
prefixes
will be combined with the suffixes specified inappend
and inselect
. the resulting file/folder paths that are exist, will then be added to the appropriate instance attribute, eitherfiles
for a file orpaths
for a folder.additionally the existing file/folder paths from the combinations of
prefixes
andselect
will be added in theselected
list attribute.Hint
more details and some examples are available in the doc string of this
module
.
- class FilesRegister(*add_path_args, property_matcher=None, file_sorter=None, **add_path_kwargs)[source]
Bases:
dict
file register catalog - see also files register examples.
- __init__(*add_path_args, property_matcher=None, file_sorter=None, **add_path_kwargs)[source]
create files register instance.
this method gets redirected with
add_path_args
andadd_path_kwargs
arguments toadd_paths()
.- Parameters:
add_path_args¶ – if passed then
add_paths()
will be called with this args tuple.property_matcher¶ (
Optional
[Callable
[[Union
[str
,RegisteredFile
,CachedFile
,Path
,PurePath
,Any
]],bool
]]) – property matcher callable, used as default value byfind_file()
if not passed there.file_sorter¶ (
Optional
[Callable
[[Union
[str
,RegisteredFile
,CachedFile
,Path
,PurePath
,Any
]],Any
]]) – file sorter callable, used as default value byfind_file()
if not passed there.add_path_kwargs¶ – passed onto call of
add_paths()
if theadd_path_args
got provided by the caller.
- __call__(*find_args, **find_kwargs)[source]
add_path_args and kwargs will be completely redirected to
find_file()
.- Return type:
Union
[str
,RegisteredFile
,CachedFile
,Path
,PurePath
,Any
,None
]
- add_file(file_obj, first_index=9223372036854775807)[source]
add a single file to the list of this dict mapped by the file-name/stem as dict key.
- Parameters:
file_obj¶ (
Union
[str
,RegisteredFile
,CachedFile
,Path
,PurePath
,Any
]) – either file path string or any object with a stem attribute.first_index¶ (
int
) – pass list index -n-1..n-1 to insertfile_obj
in the name’s list. values greater than n (==len(file_list)) will append the file_obj to the end of the file object list and values less than n-1 will insert the file_obj to the start of the file.
- add_files(files, first_index=9223372036854775807)[source]
add files from another
FilesRegister
instance.- Parameters:
files¶ (
Iterable
[Union
[str
,RegisteredFile
,CachedFile
,Path
,PurePath
,Any
]]) – iterable with file objects to be added.first_index¶ (
int
) – pass list index -n-1..n-1 to insert the first file_obj in each name’s register list. values greater than n (==len(file_list)) will append the file_obj to the end of the file object list. the order of the added items will be unchanged if this value is greater or equal to zero. negative values will add the items fromfiles
in reversed order and after the item specified by this index value (so passing -1 will append the items to the end in reversed order, while passing -(n+1) will insert them at the beginning in reversed order).
- Return type:
- Returns:
list of paths of the added files.
- add_paths(*file_path_masks, first_index=9223372036854775807, file_class=<class 'ae.files.RegisteredFile'>, **init_kwargs)[source]
add files found in the folder(s) specified by the
file_path_masks
args.- Parameters:
file_path_masks¶ (
str
) – file path masks (with optional wildcards andPATH_PLACEHOLDERS
) specifying the files to collect (by default including the sub-folders).first_index¶ (
int
) – pass list index -n-1..n-1 to insert the first file_obj in each name’s register list. values greater than n (==len(file_list)) will append the file_obj to the end of the file object list. the order of the added items will be unchanged if this value is greater or equal to zero. negative values will add the found items in reversed order and after the item specified by this index value (so passing -1 will append the items to the end in reversed order, while passing -(n+1) will insert them at the beginning in reversed order).file_class¶ (
Type
[Union
[str
,RegisteredFile
,CachedFile
,Path
,PurePath
,Any
]]) – the used file object class (seeFileObject
). each found file object will be passed to the class constructor (callable) and added to the list which is an item of this dict.init_kwargs¶ – additional/optional kwargs passed onto the used
file_class
. pass e.g. the object_loader to use, iffile_class
isCachedFile
(instead of the default:RegisteredFile
).
- Return type:
- Returns:
list of paths of the added files.
- add_register(files_register, first_index=9223372036854775807)[source]
add files from another
FilesRegister
instance.- Parameters:
files_register¶ (
FilesRegister
) – files register instance containing the file_obj to be added.first_index¶ (
int
) – pass list index -n-1..n-1 to insert the first file_obj in each name’s register list. values greater than n (==len(file_list)) will append the file_obj to the end of the file object list. the order of the added items will be unchanged if this value is greater or equal to zero. negative values will add the found items in reversed order and after the item specified by this index value (so passing -1 will append the items to the end in reversed order, while passing -(n+1) will insert them at the beginning in reversed order).
- Return type:
- Returns:
list of paths of the added files.
- find_file(name, properties=None, property_matcher=None, file_sorter=None)[source]
find file_obj in this register via properties, property matcher callables and/or file sorter.
- Parameters:
properties¶ (
Optional
[Dict
[str
,Union
[int
,float
,str
]]]) – properties to select the correct file.property_matcher¶ (
Optional
[Callable
[[Union
[str
,RegisteredFile
,CachedFile
,Path
,PurePath
,Any
]],bool
]]) – callable to match the correct file.file_sorter¶ (
Optional
[Callable
[[Union
[str
,RegisteredFile
,CachedFile
,Path
,PurePath
,Any
]],Any
]]) – callable to sort resulting match results.
- Return type:
Union
[str
,RegisteredFile
,CachedFile
,Path
,PurePath
,Any
,None
]- Returns:
registered/cached file object of the first found/correct file.
- reclassify(file_class=<class 'ae.files.CachedFile'>, **init_kwargs)[source]
re-instantiate all name’s file registers items to instances of the class
file_class
.- Parameters:
file_class¶ (
Type
[Union
[str
,RegisteredFile
,CachedFile
,Path
,PurePath
,Any
]]) – the new file object class (seeFileObject
). each found file object will be passed to the class constructor (callable) and the return value will then replace the file object in the file list.init_kwargs¶ – additional/optional kwargs passed onto the used file_class. pass e.g. the object_loader to use, if
file_class
isCachedFile
(the default file object class).