gromozeka.primitives package¶

Submodules¶

gromozeka.primitives.base module¶

class gromozeka.primitives.base.BrokerPoint(exchange=None, exchange_type=None, queue=None, routing_key=None)¶: Bases: object

class gromozeka.primitives.base.RegistryTask(broker_point, task_id, func, bind, worker_class, max_workers, max_retries, retry_countdown, ignore_result)¶

Bases: object

Task registry class

task_id¶: str – Task identification

func¶: Task function

bind¶: bool – If True Task will be in function arguments as first argument

broker_point¶: gromozeka.primitives.BrokerPointType – Broker entry

worker_class¶: gromozeka.concurrency.Worker – Task workers will be this type

max_workers¶: int – Number of task workers

max_retries¶: int – Maximum number of retries

retry_countdown¶: int or float – Pause between retries

pool¶: gromozeka.concurrency.Pool – Task workers pool

Parameters:

broker_point (gromozeka.primitives.BrokerPointType) – Broker entry
task_id (str) – Task identification
func – Task function
bind (bool) – If True Task will be in function arguments as first argument
worker_class (gromozeka.concurrency.Worker) – Task workers will be this type
max_workers (int) – Number of task workers
retry_countdown (int or float) – Pause between retries

bind

broker_point

deserializator¶

func

ignore_result¶

max_retries

max_workers

pool

retry_countdown

task_id

worker_class

class gromozeka.primitives.base.Task(func, args=None, kwargs=None, bind=None, app=None, state=None, max_retries=None, retry_countdown=None, retries=None, delay=None, uuid_=None, broker_point=None, delivery_tag=None, ignore_result=None, graph_uuid=None, reply_to_exchange=None, reply_to_routing_key=None)¶

Bases: gromozeka.primitives.dag.Vertex

Task class.

Parameters:

func – Task function
args (list, optional) – Task function arguments
kwargs (dict, optional) – Task function keyword arguments
bind (bool) – If True Task will be in function arguments as first argument
state (str, optional) – Current task state
max_retries (int, optional) – Maximum number of retries
retry_countdown (int or float, optional) – Pause between retries
retries (int, optional) – retries counter
delay (str, optional) – Retry countdown (%Y-%m-%d %H:%M:%S.%f)
uuid (uuid.uuid4, optional) – Unique task runtime identification
broker_point (gromozeka.primitives.BrokerPointType) – Broker entry
delivery_tag (int, optional) – Original message delivery tag

a(*args)¶

app¶

apply_async(*args, **kwargs)¶

Run Task asynchronously

Examples

@task()
def some_task(a,b):
    print(a+b)

some_task().apply_async(1,2)

Parameters:	args – Task function arguments *kwargs – Task function keyword arguments

args¶

bind¶

broker_point¶

cancel()¶

Cancel task. Use this method in Task function. Bind option must be True

Examples

@task(bind=True)
def some_task():
    try:
        a = 1 / 0
    except ZeroDivisionError as e:
        self.cancel()

delay¶

delivery_tag¶

classmethod from_proto(raw_task, delivery_tag=None, deserializator=False)¶

Make Task from Request

Parameters:	raw_task (gromozeka.primitives.Request) – task Request delivery_tag (int) – use with deserializator, this arg from broker deserializator (TaskDeserializator) –
Returns:
Return type:	gromozeka.primitives.Task

func¶

graph_uuid¶

ignore_result¶

k(**kwargs)¶

kwargs¶

logger¶

manual_ack¶

max_retries¶

on_cancel()¶: Event when task canceled

on_fail()¶: Event when task failed with Exception

on_receive()¶: Event when task received in worker

on_reject()¶: Event when task rejected

on_retry(e)¶

Event when task on retry

Parameters:	e (gromozeka.exceptions.Retry) –
Returns:	True if task will retry more times
Return type:	bool

on_start()¶: Event when started tu run task

on_success(res)¶: Event when task finishes successfully

register(broker_point, worker_class=None, max_workers=1, max_retries=0, retry_countdown=0, ignore_result=False, broker_options=None, deserializator=None)¶

Register task in task registry

Parameters:	broker_point (gromozeka.primitives.ProtoBrokerPoint) – Broker entry worker_class (`gromozeka.concurrency.Worker`) – Task workers will be this type max_workers (int) – Number of task workers max_retries (int) – Maximum number of retries retry_countdown (`int` or `float`) – Pause between retries ignore_result (bool) – If False result will be saved in result backend broker_options – Specific broker options. See NatsOptions broker for example deserializator (TaskDeserializator) – Custom task deserializator
Returns:
Return type:	gromozeka.primitives.Task

reply_to_exchange¶

reply_to_routing_key¶

request¶

Task Request property

Returns:
Return type:	gromozeka.primitives.Request

retries¶

retry_countdown¶

state¶

task_id¶

update_graph_states(graph, vertex, graph_state=None, result=None, error=None)¶

Parameters:	graph (gromozeka.primitives.dag.Graph) – graph instance vertex (gromozeka.primitives.dag.Vertex) – vertex instance graph_state – result – Graph vertex result error (str) – error string
Returns:
Return type:	list of gromozeka.primitives.dag.Vertex

uuid¶

class gromozeka.primitives.base.TaskDeserializator¶

Bases: object

deserialize(raw_task)¶

Parameters:	raw_task – raw task to deserialize
Returns:	task properties, tuple task_id args kwargs uuid graph_uuid

gromozeka.primitives.base.task(bind=False, max_retries=0, retry_countdown=1, ignore_result=None, reply_to=None)¶

Task decorator. Use to make usual function as task

After this, you must register task

Examples

Register task:

@task()
def hello_world():
    print('hello world')

app=Gromozeka()

app.register(
    hello_world(),
    broker_point=BrokerPointType(
        exchange="first_exchange",
        exchange_type='direct',
        queue='first_queue',
        routing_key='first'))

app.start()

or Task object can register self:

@task()
def hello_world():
    print('hello world')

app=Gromozeka()

hello_world().register(
    broker_point=BrokerPointType(
        exchange="first_exchange",
        exchange_type='direct',
        queue='first_queue',
        routing_key='first'))

app.start()

Parameters:	bind (`bool`, optional) – If True Task will be in function arguments as first argument max_retries (`int`, optional) – Maximum number of retries retry_countdown (`int` or `float`, optional) – Pause between retries ignore_result (bool) – If False result will be saved in result backend reply_to (BrokerPoint) –
Returns:	task object
Return type:	gromozeka.primitives.Task

gromozeka.primitives.base.to_dict(o)¶

gromozeka.primitives.dag module¶

class gromozeka.primitives.dag.Graph(*args)¶

Bases: object

apply_async(graph_uuid=None)¶

as_dict()¶

draw()¶: Draw workflow graph

first¶: Returns – Vertex – First Vertex of Graph

first_uuid = None¶

classmethod from_dict(data)¶

is_chain_part(vertex)¶

is_chain_tail(vertex)¶

is_group_part(vertex)¶

last¶: Returns – Vertex – Last Vertex of Graph

last_uuid = None¶

next_vertex(from_uuid=None)¶

vertex_by_uuid(uuid)¶

Get Vertex by uuid

Parameters:	uuid (str) –
Returns:
Return type:	Vertex

class gromozeka.primitives.dag.Vertex(uuid=None, task_id=None, op_name=None, args=None, kwargs=None, bind=None, reply_to_exchange=None, reply_to_routing_key=None)¶

Bases: object

add_edge(to)¶

Parameters:	to (Vertex) –

apply(graph_uuid)¶

as_dict()¶

gromozeka.primitives.task_pb2 module¶

class gromozeka.primitives.task_pb2.ReplyToBrokerPoint(**kwargs)¶

Bases: google.protobuf.message.Message

ByteSize()¶

Clear()¶

ClearField(field_name)¶

DESCRIPTOR = <google.protobuf.descriptor.Descriptor object>¶

DiscardUnknownFields()¶

EXCHANGE_FIELD_NUMBER = 1¶

FindInitializationErrors()¶

Finds required fields which are not initialized.

Returns:	A list of strings. Each string is a path to an uninitialized field from the top-level message, e.g. “foo.bar[5].baz”.

static FromString(s)¶

HasField(field_name)¶

IsInitialized(errors=None)¶

Checks if all required fields of a message are set.

Parameters:	errors – A list which, if provided, will be populated with the field paths of all missing required fields.
Returns:	True iff the specified message has all required fields set.

ListFields()¶

MergeFrom(msg)¶

MergeFromString(serialized)¶

ROUTING_KEY_FIELD_NUMBER = 2¶

static RegisterExtension(extension_handle)¶

SerializePartialToString(**kwargs)¶

SerializeToString(**kwargs)¶

SetInParent()¶: Sets the _cached_byte_size_dirty bit to true, and propagates this to our listener iff this was a state change.

WhichOneof(oneof_name)¶: Returns the name of the currently set field inside a oneof, or None.

exchange¶: Magic attribute generated for “exchange” proto field.

routing_key¶: Magic attribute generated for “routing_key” proto field.

class gromozeka.primitives.task_pb2.Task(**kwargs)¶

Bases: google.protobuf.message.Message

ARGS_FIELD_NUMBER = 3¶

ByteSize()¶

Clear()¶

ClearField(field_name)¶

DELAY_FIELD_NUMBER = 6¶

DELIVERY_TAG_FIELD_NUMBER = 8¶

DESCRIPTOR = <google.protobuf.descriptor.Descriptor object>¶

DiscardUnknownFields()¶

FindInitializationErrors()¶

Finds required fields which are not initialized.

Returns:	A list of strings. Each string is a path to an uninitialized field from the top-level message, e.g. “foo.bar[5].baz”.

static FromString(s)¶

GRAPH_UUID_FIELD_NUMBER = 9¶

HasField(field_name)¶

IsInitialized(errors=None)¶

Checks if all required fields of a message are set.

Parameters:	errors – A list which, if provided, will be populated with the field paths of all missing required fields.
Returns:	True iff the specified message has all required fields set.

KWARGS_FIELD_NUMBER = 4¶

ListFields()¶

MergeFrom(msg)¶

MergeFromString(serialized)¶

REPLY_TO_FIELD_NUMBER = 10¶

RETRIES_FIELD_NUMBER = 5¶

static RegisterExtension(extension_handle)¶

STATE_FIELD_NUMBER = 7¶

SerializePartialToString(**kwargs)¶

SerializeToString(**kwargs)¶

SetInParent()¶: Sets the _cached_byte_size_dirty bit to true, and propagates this to our listener iff this was a state change.

TASK_ID_FIELD_NUMBER = 2¶

UUID_FIELD_NUMBER = 1¶

WhichOneof(oneof_name)¶: Returns the name of the currently set field inside a oneof, or None.

args¶: Magic attribute generated for “args” proto field.

delay¶: Magic attribute generated for “delay” proto field.

delivery_tag¶: Magic attribute generated for “delivery_tag” proto field.

graph_uuid¶: Magic attribute generated for “graph_uuid” proto field.

kwargs¶: Magic attribute generated for “kwargs” proto field.

reply_to¶: Magic attribute generated for “reply_to” proto field.

retries¶: Magic attribute generated for “retries” proto field.

state¶: Magic attribute generated for “state” proto field.

task_id¶: Magic attribute generated for “task_id” proto field.

uuid¶: Magic attribute generated for “uuid” proto field.

Module contents¶

class gromozeka.primitives.Task(func, args=None, kwargs=None, bind=None, app=None, state=None, max_retries=None, retry_countdown=None, retries=None, delay=None, uuid_=None, broker_point=None, delivery_tag=None, ignore_result=None, graph_uuid=None, reply_to_exchange=None, reply_to_routing_key=None)¶

Bases: gromozeka.primitives.dag.Vertex

Task class.

Parameters:

func – Task function
args (list, optional) – Task function arguments
kwargs (dict, optional) – Task function keyword arguments
bind (bool) – If True Task will be in function arguments as first argument
state (str, optional) – Current task state
max_retries (int, optional) – Maximum number of retries
retry_countdown (int or float, optional) – Pause between retries
retries (int, optional) – retries counter
delay (str, optional) – Retry countdown (%Y-%m-%d %H:%M:%S.%f)
uuid (uuid.uuid4, optional) – Unique task runtime identification
broker_point (gromozeka.primitives.BrokerPointType) – Broker entry
delivery_tag (int, optional) – Original message delivery tag

a(*args)¶

app¶

apply_async(*args, **kwargs)¶

Run Task asynchronously

Examples

@task()
def some_task(a,b):
    print(a+b)

some_task().apply_async(1,2)

Parameters:	args – Task function arguments *kwargs – Task function keyword arguments

args¶

bind¶

broker_point¶

cancel()¶

Cancel task. Use this method in Task function. Bind option must be True

Examples

@task(bind=True)
def some_task():
    try:
        a = 1 / 0
    except ZeroDivisionError as e:
        self.cancel()

delay¶

delivery_tag¶

classmethod from_proto(raw_task, delivery_tag=None, deserializator=False)¶

Make Task from Request

Parameters:	raw_task (gromozeka.primitives.Request) – task Request delivery_tag (int) – use with deserializator, this arg from broker deserializator (TaskDeserializator) –
Returns:
Return type:	gromozeka.primitives.Task

func¶

graph_uuid¶

ignore_result¶

k(**kwargs)¶

kwargs¶

logger¶

manual_ack¶

max_retries¶

on_cancel()¶: Event when task canceled

on_fail()¶: Event when task failed with Exception

on_receive()¶: Event when task received in worker

on_reject()¶: Event when task rejected

on_retry(e)¶

Event when task on retry

Parameters:	e (gromozeka.exceptions.Retry) –
Returns:	True if task will retry more times
Return type:	bool

on_start()¶: Event when started tu run task

on_success(res)¶: Event when task finishes successfully

register(broker_point, worker_class=None, max_workers=1, max_retries=0, retry_countdown=0, ignore_result=False, broker_options=None, deserializator=None)¶

Register task in task registry

Parameters:	broker_point (gromozeka.primitives.ProtoBrokerPoint) – Broker entry worker_class (`gromozeka.concurrency.Worker`) – Task workers will be this type max_workers (int) – Number of task workers max_retries (int) – Maximum number of retries retry_countdown (`int` or `float`) – Pause between retries ignore_result (bool) – If False result will be saved in result backend broker_options – Specific broker options. See NatsOptions broker for example deserializator (TaskDeserializator) – Custom task deserializator
Returns:
Return type:	gromozeka.primitives.Task

reply_to_exchange¶

reply_to_routing_key¶

request¶

Task Request property

Returns:
Return type:	gromozeka.primitives.Request

retries¶

retry_countdown¶

state¶

task_id¶

update_graph_states(graph, vertex, graph_state=None, result=None, error=None)¶

Parameters:	graph (gromozeka.primitives.dag.Graph) – graph instance vertex (gromozeka.primitives.dag.Vertex) – vertex instance graph_state – result – Graph vertex result error (str) – error string
Returns:
Return type:	list of gromozeka.primitives.dag.Vertex

uuid¶

class gromozeka.primitives.RegistryTask(broker_point, task_id, func, bind, worker_class, max_workers, max_retries, retry_countdown, ignore_result)¶

Bases: object

Task registry class

task_id¶: str – Task identification

func¶: Task function

bind¶: bool – If True Task will be in function arguments as first argument

broker_point¶: gromozeka.primitives.BrokerPointType – Broker entry

worker_class¶: gromozeka.concurrency.Worker – Task workers will be this type

max_workers¶: int – Number of task workers

max_retries¶: int – Maximum number of retries

retry_countdown¶: int or float – Pause between retries

pool¶: gromozeka.concurrency.Pool – Task workers pool

Parameters:

broker_point (gromozeka.primitives.BrokerPointType) – Broker entry
task_id (str) – Task identification
func – Task function
bind (bool) – If True Task will be in function arguments as first argument
worker_class (gromozeka.concurrency.Worker) – Task workers will be this type
max_workers (int) – Number of task workers
retry_countdown (int or float) – Pause between retries

bind

broker_point

deserializator¶

func

ignore_result¶

max_retries

max_workers

pool

retry_countdown

task_id

worker_class

gromozeka.primitives.task(bind=False, max_retries=0, retry_countdown=1, ignore_result=None, reply_to=None)¶

Task decorator. Use to make usual function as task

After this, you must register task

Examples

Register task:

@task()
def hello_world():
    print('hello world')

app=Gromozeka()

app.register(
    hello_world(),
    broker_point=BrokerPointType(
        exchange="first_exchange",
        exchange_type='direct',
        queue='first_queue',
        routing_key='first'))

app.start()

or Task object can register self:

@task()
def hello_world():
    print('hello world')

app=Gromozeka()

hello_world().register(
    broker_point=BrokerPointType(
        exchange="first_exchange",
        exchange_type='direct',
        queue='first_queue',
        routing_key='first'))

app.start()

Parameters:	bind (`bool`, optional) – If True Task will be in function arguments as first argument max_retries (`int`, optional) – Maximum number of retries retry_countdown (`int` or `float`, optional) – Pause between retries ignore_result (bool) – If False result will be saved in result backend reply_to (BrokerPoint) –
Returns:	task object
Return type:	gromozeka.primitives.Task

gromozeka.primitives.ProtoTask¶: alias of gromozeka.primitives.task_pb2.Task

gromozeka.primitives.ProtoReplyToBrokerPoint¶: alias of gromozeka.primitives.task_pb2.ReplyToBrokerPoint

gromozeka.primitives package¶

Submodules¶

gromozeka.primitives.base module¶

gromozeka.primitives.dag module¶

gromozeka.primitives.task_pb2 module¶

Module contents¶

Table Of Contents

Table Of Contents

Related Topics