Branchprocess

The BranchProcess class is a special version of the Process class, capable of storing multiple versions of the same process. This comes in handy when the same set of tasks is to be run several times with different input. The branchprocess takes a list of tasks and the number of branches as input. Furthermore you can give it a meaningful name.

The tasks in the branchprocess are defined in a slightly different manner than for the process. To separate the arguments and keyword arguments for the different branches, they should be input as a list of lists and a list of dicts, respectively.

Under the hood of the BranchProcess

The BranchProcess automatically creates two extra objects that are added to your process: a Brancher and a Merger. The Brancher is put first, serves as a common starting point for all the tasks, and holds information about the number of branches. The Merger is put last and serves as a common ending point for the tasks.

Below is a slightly rewritten version of the previous example. Notice how the branchprocess wraps the tasks and that the branchprocess is further put in the process as a regular task.

main_branches.py

import caliber

import pasta_functions

# Define tasks
make_dough = caliber.Task(
    function=pasta_functions.make_dough,
    name='Make dough',
    args=[
        [3],
        [6],
    ],
)

rest = caliber.Task(
    function=pasta_functions.rest,
    name='Rest',
)

roll_and_shape = caliber.Task(
    function=pasta_functions.roll_and_shape,
    name='Roll and shape',
    kwargs=[
        {'pasta_type': 'tagliatelle'},
        {'pasta_type': 'ravioli'},
    ]
)

# Collect tasks in a branchprocess and put in process
pastas = caliber.BranchProcess(
    name='Variants of pasta',
    branches=2,
    tasks=[
        make_dough,
        rest,
        roll_and_shape,
    ]
)

pasta = caliber.Process(
    tasks=[
        pastas,
    ],
    name='Pasta',
)

# Create workflow
make_pasta = caliber.Workflow(
    process=pasta,
    name='Make pasta',
)

make_pasta.run()

When to use the BranchProcess?

The BranchProcess is a powerful option when the tasks take input files as arguments, e.g. when the tasks contain functions that call external programs that run the input files. In cases where tasks are repeated for different input, but where one task depends on the results from one of the previous, a better option could be to use the global data object and to handle the different input directly in the functions.