gromozeka package

Module contents

class gromozeka.Gromozeka

Bases: object

logger

logging.Logger – Class logger

registry

dict of gromozeka.primitives.Task – Task registry

config

gromozeka.config.Config – config object

_broker_adapter

gromozeka.brokers.BrokerAdapter – Broker

_backend_adapter

gromozeka.backends.BackendAdapter – Broker

_scheduler

gromozeka.concurrency.Scheduler – Scheduler

backend_adapter
broker_adapter
config
config_from_dict(conf)

Configure Gromozeka with dict

Examples

conf={'app_prefix':'my_application','broker_reconnect_max_retries':3}
app=Gromozeka().config_from_dict(conf)
Parameters:conf (dict) – config dict
Returns:Configured application
Return type:gromozeka.Gromozeka
config_from_env()

Configure Gromozeka with environment variables

Examples

app=Gromozeka().config_from_env()
Returns:Configured application
Return type:gromozeka.Gromozeka
get_task(id_)

Get task by id

Parameters:id (str) – Unique task identification
Returns:registry task
Return type:gromozeka.primitives.RegistryTask
is_closing
logger
pid
register(task, broker_point, worker_class=None, max_workers=1, max_retries=0, retry_countdown=0, ignore_result=False, broker_options=None, deserializator=None)
Parameters:
registry
scheduler
start()

Start application

stop()
stop_signal(signum)
gromozeka.get_app()
Returns:
Return type:gromozeka.app.Gromozeka
exception gromozeka.Retry(*args, max_retries=None, retry_countdown=None)

Bases: gromozeka.exceptions.GromozekaException

Exception for retry.

max_retries

int – Maximum number of retries

retry_countdown

float – Pause between retries (seconds)

Parameters:
  • *args – Exception
  • max_retries (int) – Maximum number of retries
  • retry_countdown (float) – Pause between retries (seconds)

Use this exceptions in task to retry it

Examples

@task()
def some_task():
    try:
        a = 1 / 0
    except ZeroDivisionError as e:
        raise Retry(e)

or with custom number of retries and retry_countdown:

@task()
def some_task():
    try:
        a = 1 / 0
    except ZeroDivisionError as e:
        raise Retry(e,max_retries=10, retry_countdown=3)
exception gromozeka.MaxRetriesExceedException

Bases: gromozeka.exceptions.GromozekaException

Exception will raise if maximum number of retries is reached

exception gromozeka.GromozekaException

Bases: Exception

Base exception

class gromozeka.ProcessWorker(pool)

Bases: multiprocessing.context.Process, gromozeka.concurrency.worker.Worker

Worker class for gromozeka.concurrency.Pool. Uses as mixin.

logger

logging.Logger – class logger

pool

gromozeka.concurrency.Pool – parent pool instance

_stop_event

multiprocessing.Event or threading.Event

Parameters:pool (gromozeka.concurrency.Pool) – Parent pool instance
run()

Method to be run in sub-process; can be overridden in sub-class

stop(timeout=None)

Stop worker. multiprocessing or threading part

class gromozeka.ThreadWorker(pool)

Bases: threading.Thread, gromozeka.concurrency.worker.Worker

Worker class for gromozeka.concurrency.Pool. Uses as mixin.

logger

logging.Logger – class logger

pool

gromozeka.concurrency.Pool – parent pool instance

_stop_event

multiprocessing.Event or threading.Event

Parameters:pool (gromozeka.concurrency.Pool) – Parent pool instance
run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

stop(timeout=None)

Stop worker. multiprocessing or threading part

gromozeka.task(bind=False, max_retries=0, retry_countdown=1, ignore_result=None, reply_to=None)

Task decorator. Use to make usual function as task

After this, you must register task

Examples

Register task:

@task()
def hello_world():
    print('hello world')

app=Gromozeka()

app.register(
    hello_world(),
    broker_point=BrokerPointType(
        exchange="first_exchange",
        exchange_type='direct',
        queue='first_queue',
        routing_key='first'))

app.start()

or Task object can register self:

@task()
def hello_world():
    print('hello world')

app=Gromozeka()

hello_world().register(
    broker_point=BrokerPointType(
        exchange="first_exchange",
        exchange_type='direct',
        queue='first_queue',
        routing_key='first'))

app.start()
Parameters:
  • bind (bool, optional) – If True Task will be in function arguments as first argument
  • max_retries (int, optional) – Maximum number of retries
  • retry_countdown (int or float, optional) – Pause between retries
  • ignore_result (bool) – If False result will be saved in result backend
  • reply_to (BrokerPoint) –
Returns:

task object
Return type:

gromozeka.primitives.Task
class gromozeka.BrokerPoint(exchange=None, exchange_type=None, queue=None, routing_key=None)

Bases: object

gromozeka.ProtoTask

alias of gromozeka.primitives.task_pb2.Task

gromozeka.ProtoReplyToBrokerPoint

alias of gromozeka.primitives.task_pb2.ReplyToBrokerPoint

class gromozeka.TaskDeserializator

Bases: object

deserialize(raw_task)
Parameters:raw_task – raw task to deserialize
Returns:task properties, tuple task_id args kwargs uuid graph_uuid