Group Pipeline
A group pipeline is a container of 0 or more individual job pipelines (also called child pipelines) that can be managed as a unit. The group can be scheduled to execute its job pipelines and provides a summary status of its jobs; the group execution can also be cancelled, stopping all current or scheduled executions of its child pipelines.
When a group is started manually or scheduled, its active job pipelines will execute in no specific order. The group will have a "running" status as long as one of its pipelines is running. However, individual jobs that belong to a group can also be started manually independently of the group they belong to.
When a job pipeline belongs to a group, its cron schedule setting is ignored; instead, the group cron schedule setting will be used to determine when its job pipelines should start. In addition, while individual job pipelines can have various timeout settings, the group timeout setting will take precedence. A group timeout will cancel all the individual job pipelines that have not yet completed upon its expiration.
A group cannot contain other groups.
Create a group
To create a group, use the New -> Group context menu option, or right-click in the jobs region and choose New -> New group.... You can also select a few jobs, right-click, and choose Group -> Create group from pipelines....
Settings
The source and target connections are used to specify default connections when creating a new job pipeline within the group; they do not override the individual job pipeline settings. In other words, each job pipeline in a group may use different connections.
ScheduleA group schedule overrides any job specific schedule. When a group has a schedule defined, all the active jobs within the group will start, in no specific order. However, the group must be active for the schedule setting to take effect.
Execution TimeoutA group execution timeout, if any, will have an overriding effect on individual job pipelines. Any job pipelines that do not complete within the group timeout period will be automatically cancelled. Setting the timeout value to 0 means the group can run indefinitely.
Keep Sync FilesThis setting is used as the default option selected when creating new job pipelines within the group; however, individual job pipelines can override this setting.
ActiveAn inactive group cannot be started on schedule nor manually. However, individual job pipelines can still be started manually if needed as long as they are active.
Sync File LocationThis setting is used as the default option used when creating new job pipelines within the group; however, individual job pipelines can override this setting.
Jobs
Selected Jobs
The top section lists the jobs currently assigned to this group (also referred to as child jobs). From this screen you can choose to create new jobs, using the default settings provided in the settings tab and perform a number of operations on these jobs, including enabling or disabling jobs individually.
Available Jobs
The list of available jobs is listed in the bottom part. By default, only jobs that use the same source connection will be displayed; however, you can check the show all jobs option to list all jobs.
Job Writers cannot be assigned to groups at this time; as a result, only Job Readers and Direct Jobs will be displayed. Also, a job can only be assigned to one group. You can see jobs that already belong to other groups in this list; moving them over to the upper part of this screen will reassign them to this group.
Triggers
Add/Remove Triggers
This screen allows you to define triggers on a job group. Triggers execute on completion of the entire group. If any child job fails, the group will execute the On Error triggers. If all child jobs succeed, the group will execute the On Success and On Success (with data) triggers. If at least one child job reports available records, the group will also execute the On Success (with data) triggers, as long as all the jobs succeeded.
Effects of Group Timeouts
If the group execution times out, the operation will be considered to have failed, and the On Error triggers will execute.
Cloud Agent Considerations
Depending on the type of cloud agent you have provisioned, certain limitations may be applied on the number of triggers that can be executed.
Monitor a group
To monitor a group, simply double-click on it. A group monitoring window will be displayed listing all the child pipelines that are currently assigned to it.
You can open multiple group monitoring windows.
Group Menu Options
The list of child pipelines provided in this screen can be managed like any other job pipeline. The context menu provides all available options, including enabling/disabling individual jobs, and even start/stop jobs individually and completely independently from the group execution.
Three additional menu options are available in this screen:
- Remove a job: removing a job does not delete it; is simply unassigns it from the group. Doing so will remove it from the monitoring screen.
- Edit group settings: clicking on this button opens the group settings screen, which allows you to modify scheduling options, assign/unassign jobs, and modify its triggers.
- Start group execution: clicking on this button starts the group execution; all its active child jobs will start almost at the same time, in no specific order.
Assign existing jobs
You can directly assign jobs to an existing group from the main screen listing all available pipelines; select one or more jobs (readers or direct jobs), right-click, and choose Group -> Add to existing group.... A screen will be displayed allowing you to choose an existing group.
The information provided on this screen is read-only. Choose a group to assign the selected jobs and click OK.