ae.transfer_service
transfer client and server services
this ae portion is providing client and server services to transfer files and text messages between two devices in the same local network.
the number of parallel running file and message transfers is only limited by the available resources of the involved devices.
if a file transfers gets interrupted it can be recovered later and without the need to resend the already transferred file content.
standard file paths - like e.g. the documents or downloads folders - are getting automatically adopted to the specific path structures of each involved device and operating system.
transfer service life cycle
the transfer service can be invoked in different ways: standalone as a separate process or attached and embedded into a controlling application.
run transfer service in standalone mode
execute this module to run the transfer services as a separate standalone process via:
python transfer_service.py [--bind=...] [--port=...] [--buf_len=...]
the following command line options are overwriting the default server address and socket buffer length (see also
service_factory()
):
‘bind’ to restrict the incoming connections to an ip address/range (overwriting the default
SERVER_BIND
).‘port’ to specify the socket port (overwriting the default port
SERVER_PORT
).‘buf_len’ to specify the socket buffer length (overwriting the default buffer length
SOCKET_BUF_LEN
).
after that the transfer service will be able to receive files send from another process or device.
Note
on Android a standalone transfer service has to be started as android service.
run transfer service attached to any app
alternatively you can run the transfer service server in a separate thread within respectively attached to your application:
from ae.transfer_service import service_factory
transfer_service_app = service_factory()
transfer_service_app.set_option('port', 12345, save_to_config=False)
transfer_service_app.set_option('buf_len', 34567, save_to_config=False)
transfer_service_app.start_server(threaded=True)
pause or stop transfer service
to manually pause the transfer service, store the app instance of the transfer service app
(transfer_service_app in the example above) and call its stop_server()
method:
transfer_service_app.stop_server()
to fully stop the transfer service and terminate the transfer service app call instead its
shutdown()
method:
transfer_service_app.shutdown()
Hint
the shutdown()
method of the base app instance (AppBase
) automatically
ensures a clean shutdown of the transfer service server on app quit/exit.
send file to another transfer service server
to send files from one transfer server to another running transfer server, a separate client process has to be started on the device storing the file to be send.
to initiate the file transfer the client process has to make a tcp connection to the transfer server running on the same device, specifying the path of the file to send and the remote ip of the receiving transfer server and finally call the remote procedure send_file like shown in the following example:
import socket
from ae.transfer_service import connect_and_request
request_kwargs = dict(method_name='send_file', file_path='path_to_file/file_name.ext', remote_ip='192.168.3.123')
with socket.socket() as sock:
response_kwargs = connect_and_request(sock, request_kwargs)
if 'error' in response_kwargs:
# handle error (e.g. display to user or add to a log file)
if the file transfer failed then the transfer kwargs dict returned by connect_and_request()
will contain an
error key containing the error message text.
implemented remote procedures
the following remote procedures are provided by the transfer service server:
cancel_request: cancel running file transfer.
pending_requests: get log info the progress/status of all currently running file transfers.
recv_file: receive file from other transfer service server.
recv_message: receive text message from other transfer service server.
send_file: send file to other transfer service server.
send_message: send text message to other transfer service server.
Hint
the demo app ComPartY is using all provided remote procedures.
Module Attributes
default timeout (in seconds) to connect and request a server process |
|
error message string prefix if error happened directly in |
|
default encoding and encoding error handling strategy |
|
setting BIND to '' or None to allow connections for all available interfaces |
|
server listening port |
|
timeout (in seconds) to shutdown/stop the console app |
|
buf length for socket receives and sends |
|
kwarg name suffix for values automatically converted |
|
end of line/command character as string |
|
command/action format of requests and responses |
|
locking requests TransferKwargs in |
|
transfer service server app |
Functions
|
remove high-commas and backslashes from the passed string to add it to the logs (preventing duplicates). |
|
connect to remote, send first command/action and return response as transfer kwargs dict. |
|
receive all bytes from the passed client socket instance until connection lost or line end reached. |
|
create server app instance including the command line options bind and port. |
|
add/append error to transfer kwargs dict without overwriting any previous error. |
|
convert dict literal (created with transfer_kwargs_literal()) to transfer kwargs dict. |
|
convert dict to str literal to be sent via sockets (re-instantiable via transfer_kwargs_from_literal()). |
|
update multiple transfer dicts with the same keys/values (locking with requests_lock). |
Classes
|
server request handler. |
|
server service app class |
- CONNECTION_TIMEOUT = 2.7
default timeout (in seconds) to connect and request a server process
- CONNECT_ERR_PREFIX = 'transfer_service.connect_and_request() exception '
error message string prefix if error happened directly in
connect_and_request()
helper (no protocol error).
- ENCODING_KWARGS = {'encoding': 'UTF-8', 'errors': 'ignore'}
default encoding and encoding error handling strategy
- SERVER_BIND = ''
setting BIND to ‘’ or None to allow connections for all available interfaces
- SERVER_PORT = 36969
server listening port
- SHUTDOWN_TIMEOUT = 3.9
timeout (in seconds) to shutdown/stop the console app
- SOCKET_BUF_LEN = 16384
buf length for socket receives and sends
- TRANSFER_KWARGS_DATE_TIME_NAME_PARTS = ('_date', '_time')
kwarg name suffix for values automatically converted
- TRANSFER_KWARGS_LINE_END_CHAR = '\n'
end of line/command character as string
- TRANSFER_KWARGS_LINE_END_BYTE = b'\n'
- requests_lock = <unlocked _thread.lock object>
locking requests TransferKwargs in
TransferServiceApp.reqs_and_logs
- clean_log_str(log_str)[source]
remove high-commas and backslashes from the passed string to add it to the logs (preventing duplicates).
- connect_and_request(sock, request_kwargs, buf_len=16384, timeout=2.7)[source]
connect to remote, send first command/action and return response as transfer kwargs dict.
- Parameters:
request_kwargs¶ (
Dict
[str
,Any
]) – first/initial request kwargs dict. if the key ‘server_address’ is not provided (with the server address as (host, ip) tuple), then (‘localhost’, SERVER_PORT) is used. if the key ‘local_ip’ is not specified then the local ip address will be used.buf_len¶ (
int
) – socket buffer length. if not passed thenSOCKET_BUF_LEN
will be used. pass zero/0 to use the buf length defined via the ‘buf_len’ command line option.timeout¶ (
Optional
[float
]) – timeout in seconds or None to use socket/system default timeout. if not passed then the default timeout specified byCONNECTION_TIMEOUT
will be used.
- Return type:
- Returns:
response transfer kwargs dict.
- recv_bytes(sock, buf_len=16384)[source]
receive all bytes from the passed client socket instance until connection lost or line end reached.
- service_factory(task_id_func=None)[source]
create server app instance including the command line options bind and port.
- transfer_kwargs_error(transfer_kwargs, err_msg)[source]
add/append error to transfer kwargs dict without overwriting any previous error. :type _sphinx_paramlinks_ae.transfer_service.transfer_kwargs_error.transfer_kwargs:
Dict
[str
,Any
] :param _sphinx_paramlinks_ae.transfer_service.transfer_kwargs_error.transfer_kwargs: request/response transfer kwargs dict. :type _sphinx_paramlinks_ae.transfer_service.transfer_kwargs_error.err_msg:str
:param _sphinx_paramlinks_ae.transfer_service.transfer_kwargs_error.err_msg: error message to add.
- transfer_kwargs_from_literal(transfer_kwargs_lit)[source]
convert dict literal (created with transfer_kwargs_literal()) to transfer kwargs dict.
- transfer_kwargs_literal(transfer_kwargs)[source]
convert dict to str literal to be sent via sockets (re-instantiable via transfer_kwargs_from_literal()).
Note
to ensure security (and prevent injections) only the following basic types can be used: int, float, boolean, str, bytes, list, tuple and dict. date/time values are only allowed as dict value and of the type datetime.datetime; additionally the key of this dict value has to contain one of the fragments defined in
TRANSFER_KWARGS_DATE_TIME_NAME_PARTS
.
- transfer_kwargs_update(*variables, **kwargs)[source]
update multiple transfer dicts with the same keys/values (locking with requests_lock).
- class ThreadedTCPRequestHandler(request, client_address, server)[source]
Bases:
StreamRequestHandler
server request handler.
self.rfile is a file-like object created by the handler; to use e.g. readline() instead of raw recv() likewise, self.wfile is a file-like object used to write back to the client.
- class TransferServiceApp(app_title='', app_name='', app_version='', sys_env_id='', debug_level=0, multi_threading=False, suppress_stdout=False, cfg_opt_eval_vars=None, additional_cfg_files=(), cfg_opt_val_stripper=None, formatter_class=None, epilog='', **logging_params)[source]
Bases:
ConsoleApp
server service app class
-
server_instance:
Optional
[ThreadingTCPServer
] server class instance
- cancel_request(request_kwargs, handler)[source]
cancel running request.
- Parameters:
- Return type:
- Returns:
response kwargs, having error key if request could not be found/cancelled.
- static id_of_task(action, object_type, object_key)[source]
compile the id of a transfer request task.
- Parameters:
- Return type:
- Returns:
unique key identifying the task/log-entry.
- log(log_level, message)[source]
print log message and add it to reqs_and_logs (to be read by controller app).
Note
please note that you have to use
print()
or one of the console print methods of theAppBase
(like e.g.verbose_out()
, respective self.vpo) instead of this method for the logging of low level transport methods/functions (like e.g.pending_requests()
,response_to_request()
,handle()
orrecv_bytes()
). this will prevent the duplication of a log message, because each call of this method creates a new entry inreqs_and_logs
which will be sent to the controlling app via the low level transport methods (which would recursively grow the sent messages until the system freezes), especially if the transfer kwargs are included into the log message.
- pending_requests(request_kwargs, handler)[source]
determine currently running/pending server requests and debug log messages (in debug mode only).
- recv_file(request_kwargs, handler)[source]
receive binary file content from client.
- Parameters:
request_kwargs¶ (
Dict
[str
,Any
]) –request transfer kwargs dict with the following keys:
’file_path’: file path (can contain path placeholders).
’total_bytes’: total file length in bytes.
’series_file’: optionally, pass any value to ensures new file name.
handler¶ (
StreamRequestHandler
) – request handler class instance.
- Return type:
- Returns:
response transfer kwargs dict keys (request kwargs extended with additional keys):
’error’: error message string if an error occurred.
’transferred_bytes’: start offset on recovered transfer (previously received bytes).
- response_to_request(request_lit, handler)[source]
process request to this server and return response string.
Note
this method is running in a separate thread (created by the server to process this request).
- Parameters:
request_lit¶ (
str
) – request string, which is a dict literal with ‘method_name’ key.handler¶ (
StreamRequestHandler
) – stream request handler instance.
- Return type:
- Returns:
response string with transfer kwargs as literal. if an error occurred then the returned response kwargs contains an ‘error’ key which stores the related error message.
- shutdown(exit_code=0, timeout=None)[source]
overwritten to stop a running transfer service server/threads on shutdown of this app instance.
- Parameters:
exit_code¶ (
Optional
[int
]) – set application OS exit code - seeshutdown()
.timeout¶ (
Optional
[float
]) – timeout float value in seconds - seeshutdown()
.
- start_server(threaded=False)[source]
start server and run until main app
stop_server()
.
-
server_instance:
- server_app: TransferServiceApp | None = None
transfer service server app