IRI Blog Articles

Diving Deeper into Data Management

 

 

Post image for Scheduling Jobs in IRI Workbench

Scheduling Jobs in IRI Workbench

by Don Purnhagen

In addition to running IRI software from the command line, in batch scripts, DBMS schedulers like Oracle’s, or through third-party workflow automation suites like Stonebranch Universal Controller, the IRI Workbench development environment now has a built-in job scheduler.

IRI Workbench, and the Eclipse framework on which it is built, have always had a fairly powerful set of interfaces for configuring, saving, importing, and exporting both internal and external tasks, for later execution. However, the actual launching of these saved tasks has been pretty limited. Basically, the user could press a “Run” button, and launch a task. Period.

Due to popular demand from users, IRI has added a task scheduler feature to IRI Workbench, with enhanced support for easily creating saved tasks to launch batch files and shell scripts. The scheduler is fully supported for IRI Voracity™ users, and is available as an unsupported feature for everyone, in all versions of IRI Workbench. The scheduler will let you configure the automatic, timed launch of any tasks defined and saved as a Workbench Run or External Tools configuration.

The scheduled task can be run once or configured as a repeating task. Repeating tasks can be limited to a maximum number of launches. A list of scheduled tasks can be exported to a file or imported from a file. You can use this new feature to automate the start of any individual IRI software product or batch job you want to run, including:

  • IRI data manipulation products: CoSort®, FieldShield®, NextForm®, and RowGen™
  • IRI FACT™ for “fast extract” from databases
  • IRI Voracity workflows exported as batch files or shell scripts
  • IRI products and workflows exported as MapReduce and other execution environments on Hadoop
  • Any external program, such as SQL commands, database load utilities, file transfer operations, and more

Since IRI Workbench supports the execution of IRI products on remote servers through ssh connections in the Eclipse Remote Systems Explorer, you can also schedule these types of remote jobs.

Opening the Scheduler View

If the scheduler view is not already open, you can select it from the Window→Show View menu item. If the Scheduler entry is not at the root level of the menu item, it can be found under Other…, in the IRI category.

Scheduler View in IRI Workbench

The view opens (by default) next to the console at the bottom of IRI Workbench.

Scheduling a New Task

Open the New Scheduled Launch Task wizard from the icon in the view toolbar. For a job to be available for scheduling, it must be saved in the Workbench, as either a Run/Debug, or External Tools, Configuration.

These pre-configured tasks, collectively called launch configurations, are selected from a drop-down list for scheduling a new task. A launch configuration may have the same name as the file it runs, or the user may choose a more descriptive name. Along with the named launch configuration, choose a start date and time.

A pop-out calendar is available for picking the date. Optionally, tasks can be scheduled to repeat after a set amount of minutes, hours, days, months, or even years. Tasks that are scheduled to repeat can also have a limit on the maximum number of times they will launch.

New Scheduled Launch Task

The IRI scheduler uses the same wizard for creating and editing launch tasks.

Scheduler Menu Actions

You can create, edit, pause, or remove launch tasks. Completed tasks (that were not scheduled to launch again) can be edited to re-launch, and have their launch event limits changed. You can remove completed tasks from view, and from the scheduler, before they run.

Available actions in context menu

Actions are available as view toolbar buttons and view context menu items.

Scheduled tasks may also be paused; they will not run even if the scheduled time is reached. When the scheduled task is resumed, the task reenters its waiting state until the scheduled launch time is reached.

If that time has already passed, and it is a repeating task, the launch will commence at its next scheduled interval. Tasks without a repeat interval will not be rescheduled, and will be marked as completed. You can also edit tasks to reschedule them to begin at a later start time.

The Timer Loop

A background process loop (timer loop) evaluates any tasks in the scheduler view every 5 seconds by default. However, the timer resolution can be configured for one or more seconds, in one second intervals, in the Workbench preferences.

Timer resolution in scheduler

Once a task has been added to the scheduler view, it will launch on, or after, the scheduled day and time, with the next execution of the timer loop. If the task has a repeat interval, it will update the next execution time after it has been run. During the timer loop, any tasks scheduled for or before the current time will launch.

Importing, Exporting, and Logging

The entire list of scheduled launch tasks can be exported to an XML file. You will be prompted to select a directory and filename for the exported list. These files can then be imported back into the scheduler view, optionally overwriting any existing launch tasks.

If enabled (the default) in the preferences, all launch events are recorded in the system Error Log. In addition to containing error and warning messages, it can also contain informational messages.

Unfiltered error log

This is a sample of an unfiltered log with some of all three types of messages visible.

You can filter annotated log messages by any substring in the messages or originating plug-in identifier. The easiest way to see only those messages coming from the IRI Workbench is to filter on the term “pelican”, which is the codename for the IRI Workbench.

Print Friendly

{ 0 comments… add one now }

Leave a Comment

Previous post:

Next post: