namespace portions documentation

welcome to the documentation of the portions (app/service modules and sub-packages) of this freely extendable ae namespace (PEP 420).

features and use-cases

the portions of this namespace are simplifying your Python application or service in the areas/domains:

* data processing/validation
* file handling
* i18n (localization)
* configuration settings
* console
* logging
* database access
* networking
* multi-platform/-OS
* context help
* app tours
* user preferences (font size, color, theming, ...)
* QR codes
* sideloading

screenshots of apps developed with ae portions

_images/glsl_tester_demo.gif

Fig. 1 GlslTest app demo

root list of a dark themed kivy lisz app

Fig. 2 kivy lisz app root list

fruits sub-list of a dark themed kivy lisz app

Fig. 3 fruits sub-list

fruits sub-list of a light themed kivy lisz app

Fig. 4 using light theme

user preferences drop down

Fig. 5 lisz user preferences

kivy lisz color editor

Fig. 6 kivy lisz color editor

lisz app using bigger font size

Fig. 7 bigger font size

fruits sub-list of dark themed enaml lisz app

Fig. 8 enaml/qt lisz app

fruits sub-list of a light themed enaml lisz app in landscape

Fig. 9 light themed in landscape

code maintenance guidelines

portions code requirements

  • pure python

  • fully typed (PEP 526)

  • fully documented

  • 100 % test coverage

  • multi thread save

  • code checks (using pylint and flake8)

design pattern and software principles

contributing

we want to make it as easy and fun as possible for you to contribute to this project.

reporting bugs

before you create a new issue, please check to see if you are using the latest version of this project; the bug may already be resolved.

also search for similar problems in the issue tracker; it may already be an identified problem.

include as much information as possible into the issue description, at least:

  1. version numbers (of Python and any involved packages).

  2. small self-contained code example that reproduces the bug.

  3. steps to reproduce the error.

  4. any traceback/error/log messages shown.

requesting new features

  1. on the git repository host server create new issue, providing a clear and detailed explanation of the feature you want and why it’s important to add.

  2. if you are able to implement the feature yourself (refer to the contribution steps section below).

contribution steps

thanks for your contribution – we’ll get your merge request reviewed. you could also review other merge requests, just like other developers will review yours and comment on them. based on the comments, you should address them. once the reviewers approve, the maintainers will merge.

before you start make sure you have a GitLab account.

contribution can be done either with the git-repo-manager tool or directly by using the git command and the Gitlab server.

using the git repository manager grm
  1. fork and clone the repository of this project to your computer

    in your console change the working directory to your project’s parent folder. then run the following command with the new_feature_or_fix part replaced by an appropriate branch name, describing shortly your contribution:

    grm -b new_feature_or_fix fork ae-group/ae_ae
    

    Note

    the grm fork action will also add the forked repository as the remote upstream to your local repository.

    after the repository fork you change your current working directory to the new working tree root folder, created by the grm fork action, and execute the grm renew action. this will prepare a new package version of the project and upgrade the project files created from templates to its latest version.

  2. code and check

    now use your favorite IDE/Editor to implement the new feature or code the bug fix. don’t forget to amend the project with new unit and integrity tests, and ensure they pass, by executin from time to time the grm check action.

  3. publish your changes

    before you initiate a push/merge request against the Gitlab server, execute the grm prepare action, which will create, with the help of the git diff command, a .commit_msg.txt file in the working tree root of your project, containing a short summary in the first line followed with a blank line and a list of the project files that got added, changed or deleted.

    Hint

    the .commit_msg.txt file can be amended by any text editor before you run the grm commit action. for changes initiated by an issue please include the issue number (in the format fixes #<issue-number>) into this file. you may use Markdown syntax in this file for simple styling.

    to finally commit and upload your changes run the following three grm actions in the root folder of your project:

    grm commit
    grm push
    grm request
    

    the grm commit command is first executing a grm check action to do a finally check of the project resources and to run the unit and integrity tests. if all these checks pass then a new git commit will be created, including your changes to the project. grm push``will then push the commit to your ``origin remote repository (your fork) and grm request will finally create a bew merge/pull request against the upstream remote repository (the forked one).

    Hint

    to complete the workflow a maintainer of the project has to execute the grm release action. this will merge your changes into the main branch develop of the upstream repository and then release a new version of the project onto PyPI.

more detailed information of the features of the grm tool are available within the grm user manual.

using git and Gitlab

alternatively to the grm tool you could directly use the git command suite and the Gitlab website to achieve the same (with a lot more of typing and fiddling ;-):

  1. fork the upstream repository into your user account.

  2. clone your forked repo as origin remote to your computer, and add an upstream remote for the destination repo by running the following commands in the console of your local machine:

    git clone https://gitlab.com/<YourGitLabUserName>/ae_ae.git
    git remote add upstream https://gitlab.com/ae-group/ae_ae.git
    
  3. checkout out a new local feature branch and update it to the latest version of the develop branch:

    git checkout -b <new_feature_or_fix_branch_name> develop
    git pull --rebase upstream develop
    

    please keep your code clean by staying current with the develop branch, where code will be merged. if you find another bug, please fix it in a separated branch instead.

  4. push the branch to your fork. treat it as a backup:

    git push origin <new_feature_or_fix_branch_name>
    
  5. code

    implement the new feature or the bug fix; include tests, and ensure they pass.

  6. check

    run the basic code style and typing checks locally (pylint, mypy and flake8) before you commit.

  7. commit

    for every commit please write a short summary in the first line followed with a blank line and then more detailed descriptions of the change. for bug fixes please include any issue number (in the format #nnn) in your summary:

    git commit -m "issue #123: put change summary here (can be a issue title)"
    

    Note

    never leave the commit message blank! provide a detailed, clear, and complete description of your changes!

  8. publish your changes (prepare a Merge Request)

    before submitting a merge request, update your branch to the latest code:

    git pull --rebase upstream develop
    

    if you have made many commits, we ask you to squash them into atomic units of work. most issues should have one commit only, especially bug fixes, which makes them easier to back port:

    git checkout develop
    git pull --rebase upstream develop
    git checkout <new_feature_or_fix_branch_name>
    git rebase -i develop
    

    push changes to your fork:

    git push -f
    
  9. issue/make a GitLab Merge Request:

    • navigate to your fork where you just pushed to

    • click Merge Request

    • in the branch field write your feature branch name (this is filled with your default branch name)

    • click Update Commit Range

    • ensure the changes you implemented are included in the Commits tab

    • ensure that the Files Changed tab incorporate all of your changes

    • fill in some details about your potential patch including a meaningful title

    • click New merge request.

release to PyPI

the release of a new/changed project will automatically be initiated by the GitLab CI, using the two protected vars PYPI_USERNAME and PYPI_PASSWORD (marked as masked) from the users group of this namespace, in order to provide the user name and password of the maintainers PyPI account (on Gitlab.com at Settings/CI_CD/Variables).

create new namespace

a PEP 420 namespace splits the codebase of a library or framework into multiple project repositories, called portions (of the namespace).

the id of a new namespace consists of letters only and has to be available on PYPI. the group-name name gets by default generated from the namespace name plus the suffix '-group', so best choose an id that results in a group name that is available on your repository server.

register a new namespace portion

follow the steps underneath to add and register a new module as portion onto the ae namespace:

  1. open a console window and change the current directory to the parent directory of your projects root folders.

  2. choose a not-existing/unique name for the new portion (referred as <portion-name> in the next steps).

  3. run grm --namespace=ae --project=<portion_name> new-module to register the portion name within the namespace, to create a new project folder ae_<portion-name> (providing initial project files created from templates) and to get a pre-configured git repository (with the remote already set and the initial files unstaged, to be extended, staged and finally committed).

  4. run cd ae_<portion-name> to change the current to the working tree root of the new portion project.

  5. run pyenv local venv_name (or any other similar tool) to create/prepare a local virtual environment.

  6. fans of TDD are then coding unit tests in the prepared test module test_ae_<portion-name>.py, situated within the tests sub-folder of your new code project folder.

  7. extend the file <portion_name>.py situated in the ae sub-folder to implement the new portion.

  8. run grm check-integrity to run the linting and unit tests (if they fail go one or two steps back).

  9. run grm prepare, then amend the commit message within the file .commit_msg.txt, then run grm commit and grm push to commit and upload your new portion to your personal remote/server repository fork, and finally run grm request to request the merge/pull into the forked/upstream repository in the users group ae-group (at https://gitlab.com/ae-group).

the registration of a new portion to the ae namespace has to be done by a namespace maintainer.

registered portions will automatically be included into the ae namespace documentation, available at ReadTheDocs.

registered namespace package portions

the following list contains all registered portions of the ae namespace, plus additional modules of each portion.

Hint

a not on the ordering: portions with no dependencies are at the begin of the following list. the portions that are depending on other portions of the ae namespace are listed more to the end.

ae.base

basic constants, helper functions and context manager

ae.deep

easy handling of deeply nested data structures

ae.django_utils

helpers for django projects

ae.droid

android constants and helper functions

ae.notify

send notifications via email, telegram or whatsapp

ae.valid

data validation helper functions

ae.files

generic file object helpers

ae.paths

generic file path helpers

ae.core

application core constants, helper functions and base classes

ae.lockname

named threading locks

ae.dynamicod

dynamic execution of code blocks and expressions

ae.i18n

internationalization / localization helpers

ae.parse_date

parse date strings more flexible and less strict

ae.literal

literal type detection and evaluation

ae.progress

display progress of long running processes

ae.updater

application environment updater

ae.console

console application environment

ae.sys_core

dynamic system configuration, initialization and connection

ae.sys_data

external system data structures

ae.sys_core_sh

SiHOT PMS system core xml interface

ae.sys_data_sh

Sihot system data interface

ae.db_core

database connection and data manipulation base classes

ae.db_ora

database system core layer to access Oracle databases

ae.db_pg

postgres database layer

ae.transfer_service

transfer client and server services

ae.sideloading_server

sideloading server

ae.gui_app

base class for python applications with a graphical user interface

ae.gui_help

main app base class with context help for flow and app state changes

ae.kivy_glsl

add glsl shaders to your kivy widget

ae.kivy_dyn_chi

dynamic children mix-in for kivy container widgets

ae.kivy_relief_canvas

inner/outer elliptic/square reliefs for any kivy widget

ae.kivy

core application classes and widgets for GUIApp-conform Kivy apps

ae.kivy.i18n

ae.kivy.i18n module

ae.kivy.widgets

ae.kivy.widgets module

ae.kivy.apps

ae.kivy.apps module

ae.kivy.behaviors

ae.kivy.behaviors module

ae.kivy.tours

ae.kivy.tours module

ae.kivy_auto_width

automatic width mix-in classes for kivy widgets

ae.kivy_file_chooser

extended kivy file chooser widget

ae.kivy_iterable_displayer

iterable displayer widget

ae.kivy_qr_displayer

qr code displayer widget

ae.kivy_sideloading

kivy mixin and widgets to integrate a sideloading server in your app

ae.kivy_user_prefs

user preferences widgets for your kivy app

ae.lisz_app_data

lisz demo app data handling

ae.enaml_app

enaml application widgets, helper functions and classes

ae.enaml_app.functions

enaml application helper functions.

indices and tables