Scheduling Jobs in IRI Workbench
In addition to running IRI software from the command line, in batch scripts, DBMS schedulers like Oracle’s, or through third-party workflow automation suites like Stonebranch Universal Controller, the IRI Workbench development environment now has a built-in job scheduler.
IRI Workbench, and the Eclipse framework on which it is built, have always had a fairly powerful set of interfaces for configuring, saving, importing, and exporting both internal and external tasks, for later execution. However, the actual launching of these saved tasks has been pretty limited. Basically, the user could press a “Run” button, and launch a task. Period.
Due to popular demand from users, IRI has added a task scheduler feature to IRI Workbench, with enhanced support for easily creating saved tasks to launch batch files and shell scripts. The scheduler is fully supported for IRI Voracity® users, and is available as an unsupported feature for everyone, in all versions of IRI Workbench. The scheduler will let you configure the automatic, timed launch of any tasks defined and saved as a Workbench Run or External Tools Configuration.
The scheduled task can be run once or configured as a repeating task. Repeating tasks can be limited to a maximum number of launches. A list of scheduled tasks can be exported to a file or imported from a file. You can use this new feature to automate the start of any individual IRI software product or batch job you want to run, including:
- IRI data manipulation / masking products: CoSort®, FieldShield®, DarkShield®, NextForm®, and RowGen®
- IRI FACT™ for “fast extract” from databases
- IRI Voracity data integration (ETL job) workflows exported as batch files or shell scripts
- IRI products and workflows exported as MapReduce and other execution environments on Hadoop
- Any external program, such as SQL commands, DB load utilities, file transfer operations, Java apps, and more
Since IRI Workbench supports the execution and workflow management of IRI products on remote servers through SSH connections in the Eclipse Remote Systems Explorer, you can also set up and schedule remote jobs, too.
Opening the Scheduler View
If the scheduler view is not already open, you can select it from the Window→Show View menu item. If the Scheduler entry is not at the root level of the menu item, it can be found under Other…, in the IRI category.
Scheduling a New Task
Open the New Scheduled Launch Task wizard from the icon in the view toolbar. For a job to be available for scheduling, it must be saved in the Workbench, as either a Run/Debug, or External Tools Configuration.
These pre-configured tasks, collectively called launch configurations, are selected from a drop-down list for scheduling a new task. A launch configuration may have the same name as the file it runs, or the user may choose a more descriptive name. Along with the named launch configuration, choose a start date and time.
A pop-out calendar is available for picking the date. Optionally, tasks can be scheduled to repeat after a set amount of minutes, hours, days, months, or even years. Tasks that are scheduled to repeat can also have a limit on the maximum number of times they will launch.
Scheduler Menu Actions
You can create, edit, pause, or remove launch tasks. Completed tasks (that were not scheduled to launch again) can be edited to re-launch, and have their launch event limits changed. You can remove completed tasks from view, and from the scheduler, before they run.
Scheduled tasks may also be paused; they will not run even if the scheduled time is reached. When the scheduled task is resumed, the task reenters its waiting state until the scheduled launch time is reached.
If that time has already passed, and it is a repeating task, the launch will commence at its next scheduled interval. Tasks without a repeat interval will not be rescheduled, and will be marked as completed. You can also edit tasks to reschedule them to begin at a later start time.
The Timer Loop
A background process loop (timer loop) evaluates any tasks in the scheduler view every 5 seconds by default. However, the timer resolution can be configured for one or more seconds, in one second intervals, in the Workbench preferences.
Once a task has been added to the scheduler view, it will launch on, or after, the scheduled day and time, with the next execution of the timer loop. If the task has a repeat interval, it will update the next execution time after it has been run. During the timer loop, any tasks scheduled for or before the current time will launch.
Importing, Exporting, and Logging
The entire list of scheduled launch tasks can be exported to an XML file. You will be prompted to select a directory and filename for the exported list. These files can then be imported back into the scheduler view, optionally overwriting any existing launch tasks.
If enabled (the default) in the preferences, all launch events are recorded in the system Error Log. In addition to containing error and warning messages, it can also contain informational messages.
You can filter annotated log messages by any substring in the messages or originating plug-in identifier. The easiest way to see only those messages coming from the IRI Workbench is to filter on the term “pelican”, which is the codename for the IRI Workbench.