namespace portions documentation

welcome to the documentation of the portions (modules and packages) of this freely extendable ae namespace (PEP 420).

features and use-cases

the portions of this namespace are simplifying your Python application, libraries and services in the areas/domains:

• cloud storage

• console app configuration and management

• context help

• database access

• data processing/validation

• file creation from templates

• file handling

• git commands execution, debugging and logging

• GUI app environment and resource management

• GUI app tours

• i18n (localization)

• library configuration and namespace management

• logging

• multi-platform/-OS

• networking

• pip commands execution

• QR codes

• sideloading

• system configuration settings

• user preferences (font sizes, colors, theming, …)

• virtual environment management

• web server deployment

there are some screenshots of app screens done with the help of ae portions.

code maintenance guidelines

portions code features

• open source

• pure python

• fully typed (PEP 526)

• fully documented

• 100 % test coverage

• multi thread save

• code checks (using pylint and flake8)

design pattern and software principles

• DRY - don’t repeat yourself

• KIS - keep it simple

contributing

we want to make it as easy and fun as possible for you to contribute to this project.

reporting bugs

before you create a new issue, please check to see if you are using the latest version of this project; the bug may already be resolved.

also search for similar problems in the issue tracker; it may already be an identified problem.

include as much information as possible into the issue description, at least:

1. version numbers (of Python and any involved packages).

2. small self-contained code example that reproduces the bug.

3. steps to reproduce the error.

4. any traceback/error/log messages shown.

requesting new features

1. on the git repository host server create new issue, providing a clear and detailed explanation of the feature you want and why it’s important to add.

2. if you are able to implement the feature yourself (refer to the contribution steps section below).

contribution steps

thanks for your contribution – we’ll get your merge request reviewed. you could also review other merge requests, just like other developers will review yours and comment on them. based on the comments, you should address them. once the reviewers approve, the maintainers will merge.

before you start make sure you have a GitLab account.

contribution can be done either with the project-manager tool or directly by using the git command and the Gitlab server.

using the project manager tool pjm

1. fork and clone the repository of this project to your computer

   in your console change the working directory to your project’s parent folder. then run the following command:

   pjm fork ae-group/ae_ae
   

   Note

   the pjm fork action will also add the forked repository as the upstream remote to your local repository.

   now change your current working directory to the new working-tree|project root folder, created by the pjm fork action, and execute the pjm renew action with the new_feature_or_fix part replaced by an appropriate branch name, describing shortly the new feature or the bug-fix of your contribution:

   pjm -b new_feature_or_fix renew
   

   this will prepare a new release version of the project and upgrade the project files created from templates to its latest version.

2. code and check

   now use your favorite IDE/Editor to implement the new feature or code the bug fix. don’t forget to amend the project with new unit and integrity tests, and ensure they pass, by executing from time to time the pjm check action.

3. publish your changes

   before you initiate a push/merge request against the Gitlab server, execute the pjm prepare action, which will create, with the help of the git diff command, a .commit_msg.txt file in the working tree root of your project, containing a short summary in the first line followed with a blank line and a list of the project files that got added, changed or deleted.

   Hint

   the .commit_msg.txt file can be amended by any text editor before you run the pjm commit action. for changes initiated by an issue please include the issue number (in the format fixes #<issue-number>) into this file. you may use Markdown syntax in this file for simple styling.

   to finally commit and upload your changes run the following three pjm actions in the root folder of your project:

   pjm commit
   pjm push
   pjm request
   

   the pjm commit command is first executing a pjm check action to do a finally check of the project resources and to run the unit and integrity tests. if all these checks pass then a new git commit will be created, including your changes to the project. pjm push``will then push the commit to your ``origin remote repository (your fork) and pjm request will finally create a bew merge/pull request against the upstream remote repository (the forked one).

   Hint

   to complete the workflow a maintainer of the project has to execute the pjm release action. this will merge your changes into the main branch develop of the upstream repository and then release a new version of the project onto PyPI.

more detailed information of the features of the pjm tool are available within the pjm user manual.

using git and Gitlab

alternatively to the pjm tool you could directly use the git command suite and the Gitlab website to achieve the same (with a lot more of typing and fiddling ;-):

1. fork the upstream repository into your user account.

2. clone your forked repo as origin remote to your computer, and add an upstream remote for the destination repo by running the following commands in the console of your local machine:

   git clone https://gitlab.com/<YourGitLabUserName>/ae_ae.git
   git remote add upstream https://gitlab.com/ae-group/ae_ae.git
   

3. checkout out a new local feature branch and update it to the latest version of the develop branch:

   git checkout -b <new_feature_or_fix_branch_name> develop
   git pull --rebase upstream develop
   

   please keep your code clean by staying current with the develop branch, where code will be merged. if you find another bug, please fix it in a separated branch instead.

4. push the branch to your fork. treat it as a backup:

   git push origin <new_feature_or_fix_branch_name>
   

5. code

   implement the new feature or the bug fix; include tests, and ensure they pass.

6. check

   run the basic code style and typing checks locally (pylint, mypy and flake8) before you commit.

7. commit

   for every commit please write a short summary in the first line followed with a blank line and then more detailed descriptions of the change. for bug fixes please include any issue number (in the format #nnn) in your summary:

   git commit -m "issue #123: put change summary here (can be a issue title)"
   

   Note

   never leave the commit message blank! provide a detailed, clear, and complete description of your changes!

8. publish your changes (prepare a Merge Request)

   before submitting a merge request, update your branch to the latest code:

   git pull --rebase upstream develop
   

   if you have made many commits, we ask you to squash them into atomic units of work. most issues should have one commit only, especially bug fixes, which makes them easier to back port:

   git checkout develop
   git pull --rebase upstream develop
   git checkout <new_feature_or_fix_branch_name>
   git rebase -i develop
   

   push changes to your fork:

   git push -f
   

9. issue/make a GitLab Merge Request:

   • navigate to your fork where you just pushed to

   • click Merge Request

   • in the branch field write your feature branch name (this is filled with your default branch name)

   • click Update Commit Range

   • ensure the changes you implemented are included in the Commits tab

   • ensure that the Files Changed tab incorporate all of your changes

   • fill in some details about your potential patch including a meaningful title

   • click New merge request.

release to PyPI

the release of a new/changed project will automatically be initiated by the GitLab CI, using the two protected vars PYPI_USERNAME and PYPI_PASSWORD (marked as masked) from the users group of this namespace, in order to provide the user name and password of the maintainers PyPI account (on Gitlab.com at Settings/CI_CD/Variables).

useful links and resources

• General GitLab documentation

• GitLab workflow documentation

• pjm (project-manager) tool project project repository and user manual

create new namespace

a PEP 420 namespace splits the codebase of a library or framework into multiple project repositories, called portions (of the namespace).

Hint

the aedev namespace is providing the project-manager (pjm) tool to create and maintain namespace root and its portion projects.

the id of a new namespace has to be available on PyPI.

the owner name of your namespace (group-name) has to be available on your git repository server. it defaults to the namespace name plus the suffix '-group'.

register a new namespace portion

follow the steps underneath to add and register a new module portion onto the ae namespace:

1. open a console window and change the current directory to the parent directory of your projects root folders.

2. choose a not-existing/unique name for the new portion (referred as <portion-name> in the next steps).

3. run pjm --namespace_name=ae --project_name=ae_<portion_name> new_module to create a new project folder ae_<portion-name>, and to register the portion name within the namespace.

4. run cd ae_<portion-name> to change the current to the working tree root of the new portion project. within the project folder you will find the initial project files created from templates and a pre-configured git repository (with the remote already set and the initial files unstaged, to be extended, staged and finally committed).

5. optionally run pyenv local venv_name (or any other similar tool) to create/prepare a local virtual environment.

6. fans of TDD are then coding unit tests in the prepared test module test_ae_<portion-name>.py, situated within the tests sub-folder of your new code project folder.

7. extend the file <portion_name>.py situated in the ae sub-folder to implement the new portion.

8. run pjm check-integrity to run the linting and unit tests (if they fail go one or two steps back).

9. run pjm prepare, then amend the commit message within the file .commit_msg.txt and run pjm commit.

the registration of a new portion to the ae namespace has to be done by a namespace maintainer. if you have a maintainer role in the namespace owner group ae-group (at https://gitlab.com/ae-group) then you can push and merge the new portion directly (running pjm push and pjm request). otherwise contact one of the maintainers to add it for you.

registered portions will automatically be included into the ae namespace documentation, available at ReadTheDocs.

registered namespace package portions

the following list contains all registered portions of the ae namespace, plus additional modules of each portion.

Hint

a not on the ordering: portions with no dependencies are at the begin of the following list. the portions that are depending on other portions of the ae namespace are listed more to the end.

ae.base	basic constants, helper functions, classes and context managers
ae.app_log	runtime logging helpers
ae.system	Python system helpers
ae.deep	easy handling of deeply nested data structures
ae.django_utils	helpers for django projects
ae.notify	send notifications via email, Telegram or WhatsApp
ae.valid	data validation helper functions
ae.files	generic file object helpers
ae.paths	generic file path helpers
ae.cloud_storage	distribute files to and retrieve them from cloud storage hosts.
ae.oaio_model	Our Asynchronously Interchangeable Objects Data Structures
ae.oaio_client	Our Asynchronously Interchangeable Objects Client
ae.updater	application environment updater
ae.core	application core constants, helper functions and base classes
ae.dynamicod	dynamic execution of code blocks and expressions
ae.i18n	internationalization / localization helpers
ae.parse_date	parse date strings more flexible and less strict
ae.literal	literal type detection and evaluation
ae.console	console application environment
ae.shell	shell execution and environment helpers
ae.managed_files	managed files
ae.pythonanywhere	PythonAnywhere Web API Client
ae.lockname	named threading locks
ae.progress	display progress of long-running processes
ae.sys_core	dynamic system configuration, initialization and connection
ae.sys_data	external system data structures
ae.sys_core_sh	SiHOT PMS system core xml interface
ae.sys_data_sh	Sihot system data interface
ae.db_core	database connection and data manipulation base classes
ae.db_ora	database system core layer to access Oracle databases
ae.db_pg	postgres database layer
ae.transfer_service	transfer client and server services
ae.sideloading_server	sideloading server
ae.gui	helper functions and base application classes for GUI applications
ae.gui.utils	GUI app constants and helper functions
ae.gui.tours	app tour base classes
ae.gui.app	abstract Framework-independent base class for python applications with a graphical user interface
ae.kivy_glsl	add glsl shaders to your kivy widget
ae.kivy_dyn_chi	dynamic children mix-in for kivy container widgets
ae.kivy_relief_canvas	inner/outer elliptic/square reliefs for any kivy widget
ae.kivy	core application classes and widgets for GUIApp-conform Kivy apps
ae.kivy.widgets	ae.kivy.widgets module
ae.kivy.tours	ae.kivy.tours module
ae.kivy.apps	ae.kivy.apps module
ae.kivy.behaviors	ae.kivy.behaviors module
ae.kivy.i18n	ae.kivy.i18n module
ae.kivy_auto_width	automatic width mix-in classes for kivy widgets
ae.kivy_file_chooser	extended kivy file chooser widget
ae.kivy_iterable_displayer	iterable displayer widget
ae.kivy_qr_displayer	qr code displayer widget
ae.kivy_sideloading	kivy mixin and widgets to integrate a sideloading server in your app
ae.kivy_user_prefs	user preferences widgets for your kivy app
ae.lisz_app_data	lisz demo app data handling
ae.enaml_app	enaml application widgets, helper functions and classes
ae.enaml_app.functions	enaml application helper functions.

indices and tables

• portion repositories at gitlab.com

• Index

• Module Index

• ae namespace projects and documentation

• aedev namespace projects and documentation