Scheduling and Schedule tracking

Owned by maimoona.kausar
Last updated: Jul 30, 2015
14 min read

OpenSRP uses motech library for all types of scheduling. Scheduling is broadly categorized into two main types

  • Simple task or event scheduling: It includes repeatable schedules, cron based jobs, and simple events which have no dependencies.
  • Trackable schedule: Also known as Health schedules among science folks. These include predefined schedules having different strategy or schedule for different periods of time from a given reference date, and also include any events which are dependent on these schedules.

OpenSRP wraps and hides the complexity of motech scheduling and architecture, and provides centrally managed scheduler services for both broader categories i.e. org.opensrp.scheduler.TaskSchedulerService and org.opensrp.scheduler.HealthSchedulerService respectively in opensrp-core.

To work with OpenSRP scheduling you don't need to get into details of motech library but for developers interested in extending the functionality and learn motech capabilities, the detail about Event handling and Scheduling architecture of motech can be found here and the Schedule Tracking module is described in detail here.

Below is a diagram to explain opensrp core-scheduling architecture in detail

There are four types of schedulable tasks available in OpenSRP

  1. Periodically Repeating Schedule  (Simple task or event scheduling)
  2. Simple Event - Event based tasks  (Simple task or event scheduling)
  3. Trackable schedule (Trackable schedule)
  4. Hooked or Dependent Event - Events dependent on Trackable schedule (Trackable schedule)

Scheduling explained with real world scenarios:

The most important part is to understand when to choose what type of schedule for any particular scenario. All four types of schedules are explained around the example below:

Example: Lets consider a Mortality Registry maintained by CHWs (Community Health Workers) under supervision of CDG (City District Government) to reach out to community living in under privileged area of city. The register also helps to facilitate citizens, gain access to CRVS services  at their home and in-turn provides completeness of data for NDRA (National Database and Registration Authority) department. Community Worker registers death in community via OpenSRP Mortality Registry and the scheduling services applies to this register as follows.

1- Periodically repeating schedule:

Implemented by org.opensrp.scheduler.RepeatingSchedule and org.opensrp.scheduler.RepeatingCronSchedule. The repeating schedule is basically a fixed, periodically repeating task, that is mostly used for cleaning up or systematic work, consistently running without any need to have variable schedule or to keep any tracking for previous schedules. The task runs repeatedly with defined repeating interval for specified or infinite period of time. The repeating schedules are mostly attached to startup listeners or entry point of the application to make sure that these are instantiated only once.

Assume that for our Mortality Registry, Data Supervisors keep a strict check on few important aspects of data completeness, and activities comprise of

  • Followup on a report updated daily where Death notifications by CHWs which have not been followed up by a detailed Verbal Autopsy (VA) form even after a month via a web interface which displays such records in defaulter list
  • Analyze data dump reports weekly generated by system to make sure that previous month`s data is clean and consistent before being pushed to Dataware House at the end of the month

Hence we have three types of repeated activities i.e. three different schedules

  • Generate VA-Missing report daily
  • Generate data dump report weekly
  • Push past month data to external system at the end of the month

The schedules are simpler and donot need to extend any functionality and could be handled like below on spring ApplicationStartupListener

 

public static class SchedulerConstants {   
    public static final VA_MISSING_SCHEDULE_SUBJECT = "VA Missing SCHEDULE unique subject";
    public static final DUMP_SCHEDULE_SUBJECT = "Data dumper SCHEDULE unique subject";
    public static final DATA_PUSHER_SCHEDULE_SUBJECT = "Data pusher SCHEDULE unique subject";
}
Repeating Schedule Example
@Component
public class ApplicationStartupListener implements ApplicationListener<ContextRefreshedEvent>
{
    public static final String APPLICATION_ID = "org.springframework.web.context.WebApplicationContext:/opensrp";
	private TaskSchedulerService scheduler;
	private RepeatingSchedule vaMissing;
    private RepeatingCronSchedule dataDumper;
	private RepeatingCronSchedule dataPusher;

	@Autowired
    public ApplicationStartupListener(TaskSchedulerService scheduler) {
        this.scheduler = scheduler;
    }

    public void onApplicationEvent(ContextRefreshedEvent contextRefreshedEvent) {
        if (APPLICATION_ID.equals(contextRefreshedEvent.getApplicationContext().getId())) {
			// schedule a job that runs after every 24 hours and generates report for missing VA forms
        	vaMissing = new RepeatingSchedule(SchedulerConstants.VA_MISSING_SCHEDULE_SUBJECT, 10, TimeUnit.MINUTES, 24, TimeUnit.HOURS); 
        	scheduler.startJob(vaMissing);

			// schedule a job that runs every Sunday at 11pm and generates data dump reports for past month
        	dataDumper = new RepeatingCronSchedule(SchedulerConstants.DUMP_SCHEDULE_SUBJECT, 10, TimeUnit.MINUTES, "? ? 11 * SUN"); 
        	scheduler.startJob(dataDumper);

			// schedule a job that runs every last day of month at 11pm and pushes data to dataware house for past month.
        	dataPusher = new RepeatingCronSchedule(SchedulerConstants.DATA_PUSHER_SCHEDULE_SUBJECT, 10, TimeUnit.MINUTES, "? ? 11 * SUN"); 
        	scheduler.startJob(dataPusher);
		}
	}
}

Create and start the schedule once and only once with schedule`s unique SUBJECT and add a listener for the schedule which specifies the action to be carried out when schedule is triggered. This is done by adding an annotation to any method @MotechListener with the same SUBJECT as on passed during the creation of scheduler  

Motech Listener for Repeating Schedule
@MotechListener(subjects = SchedulerConstants.VA_MISSING_SCHEDULE_SUBJECT)
public void handleMissingVA(MotechEvent event) {
	// Get missing VA forms for Death notifications filled past two months
	// Create CSV with pre-defined template
	// Email to admin or save to disk as required by system
}

@MotechListener(subjects = SchedulerConstants.DUMP_SCHEDULE_SUBJECT)
public void handleDataDumper(MotechEvent event) {
	// Get Data required to be included in dumps
	// Create CSV with pre-defined template
	// Email to admin or save to disk as required by system
}

@MotechListener(subjects = SchedulerConstants.DATA_PUSHER_SCHEDULE_SUBJECT)
public void handleDataPusher(MotechEvent event) {
	// Get Data of past month to be pushed to DWH
	// Get access to services and push data
	// Notify any errors to admin as configured
}

2- Simple Event or event based schedule:

Sometimes we want to perform different tasks when some event occurs in the system. Unlike fixed repeating schedule events are unpredictable and can happen at any point of time. We need to have a logic that is independent and self encapsulated instead of calling the required methods to fulfill the task. This becomes requirement when system comprises of different module working independent of each other. OpenSRP allows event based tasking via org.opensrp.scheduler.SystemEvent. Based on certain events supposed to drive the application behavior, corresponding event listeners are triggered to fulfill the requirements of business flow. SystemEvent also allows to pass data from calling class to the event so that listener class can take decisions based on dynamic logic. In simple words the flow is whenever something happens in system that is a business logic event notify via TaskSchedulerService and implement listeners in module where event derived logic is required.

Lets say that for our Mortality Registry, the death notification and data is used by two other registers, Infant Registry and Crime Investigation Registry. Both registers are  independent of this Mortality Registry but still want to be notified asap when a death happens. The only way that it should be done is both registers listening to death notification notified by Mortality Registry. Infant Registry, incase of infant death, would close all vaccinations due, infant stipend account closing would happen immediately. Crime Investigation Registry, incase of accidental death would automatically open a case, alert the corresponding personnel in that area for further investigation.

If the event has multiple listeners, a separate event is raised for each listener and sent to the outbound event gateway. The original event object is not handled by a listener in this case.

 

public static class SchedulerConstants {   
    public static final DEATH_NOTIFICATION_EVENT_SUBJECT = "Death notification EVENT unique subject";
}
Event based schedules
@Component
public class FormSbmissionHandler
{
	private TaskSchedulerService scheduler;

    @Autowired
    public FormSbmissionHandler(TaskSchedulerService scheduler) {
        this.scheduler = scheduler;
    }

    public void submitDeathForms(List<Form> forms) {	
		// process forms and save to DB
		// perform any other action required for form submission

		// Notify that Death has occurred so that any listeners waiting for event can perform their logic
 		scheduler.notifyEvent(new SystemEvent<>(SchedulerConstants.DEATH_NOTIFICATION_EVENT_SUBJECT, forms));
    }
}

Implement an event listener that needs to be invoked when event happens.

Motech listener for OpensrpEvent
// in Infant Registry
@MotechListener(subjects = SchedulerConstants.DEATH_NOTIFICATION_EVENT_SUBJECT)
public void handleDeathNotification(MotechEvent event) {
    // check if death is infant
	// find registered case by identifier
	// if no case found register a new case
	// close all vaccinations, reminders, and infant stipend account if applicable
 }

// in Crime Investigation Registry
@MotechListener(subjects = SchedulerConstants.DEATH_NOTIFICATION_EVENT_SUBJECT)
public void handleDeathNotification(MotechEvent event) {
    // check if case comes under the department
	// find and match record against any existing entry
	// fetch other required data from CRVS system
	// open a case for investigation and flag the priority and the urgency of action
	// alert the personnel responsible in that area 
}

3. Trackable schedule or Health schedule:

org.opensrp.scheduler.HealthSchedulerService exposes services to access ScheduleTrackingApi and Action or Alert management of OpenSRP. This schedule allows a client to be enrolled in a clearly defined schedule. Each schedule consist of specific "milestones."  i.e. criteria that should be fulfilled before moving on to the next milestone. Clients enrolled in these schedules are sent alerts when they are due, late, or past due on the period of schedule fulfillment. The details about schedule tracking is here.

Lets assume for our Mortality Registry we need to generate reminders and alerts for CHW to fill a Verbal Autopsy form with in 60 days after death. The CHW has a 30 days time period to call to relative ofn deceased and notify the appointment and fill the VA form before 60th day of death  passes and close the schedule. The schedule aims to remind CHW biweekly for first month, and weekly from start of 2nd month until the last week of 2nd month, and daily in the last week of 2nd month after death

Create a schedule in folder opensrp-web/resources/schedules similar to json below (read more on JSON schedule structure here)

{
            "name":"Verbal Autopsy Fill Schedule",
            "absolute": true,
            "milestones":[
                {
                    "name":"relativeAppointmentDue",
                    "scheduleWindows":{
                        "earliest":["0 Week"],
                        "due":["4 Weeks"],
                        "late":["5 Weeks"],
                        "max":[""]
                    },
                    "data":{},
                    "alerts":[
                        {
                            "window":"due",
                            "offset":["0 Weeks"],
                            "interval":["2 Weeks"],
                            "count":"3"
                        }
                    ]
                },
                {
                    "name":"VAFormDue",
                    "scheduleWindows":{
                        "earliest":["5 Weeks"],
                        "due":["7 Weeks"],
                        "late":["8 Weeks"],
                        "max":["9 Weeks"]
                    },
                    "data":{ },
                    "alerts":[
                        {
                            "window":"earliest",
                            "offset":["0 Weeks"],
                            "interval":["1 Week"],
                            "count":"3"
                        },
                        {
                            "window":"late",
                            "offset":["7 Weeks", "1 Day"],
                            "interval":["1 Day"],
                            "count":"7"
                        },
                        {
                            "window":"max",
                            "offset":["0 Day"],
                            "interval":["1 Day"],
                            "count":"1"
                        }
					 ]
                }
            ]
        }

Enroll person into schedule on registration, with org.opensrp.HealthSchedulerService like below

public static class SchedulerConstants { 
    public static final VA_FORM_SCHEDULE_SUBJECT = "Verbal Autopsy Fill SCHEDULE unique subject";
}

// Register person
// Enroll into schedule
scheduler.enrollIntoSchedule(caseId, SchedulerConstants.VA_FORM_SCHEDULE_SUBJECT, enrollmentDate);

When CHW notifies appointment then we may need to cancel further alerts for getting an appointment. This is done by marking milestone as fulfilled

// Got appointment and save form with data
// Mark "relativeAppointmentDue" milestone as fulfilled to cancel further alerts for the schedule
if (scheduler.isNotEnrolled(entityId, SchedulerConstants.VA_FORM_SCHEDULE_SUBJECT)) {
	logger.warn(format("Tried to fulfill milestone {0} of {1} for entity id: {2}", "relativeAppointmentDue", SchedulerConstants.VA_FORM_SCHEDULE_SUBJECT, entityId));
    return;
}
scheduler.fullfillMilestoneAndCloseAlert(entityId, chwId, SchedulerConstants.VA_FORM_SCHEDULE_SUBJECT, "relativeAppointmentDue", fulfillmentDate);

In the same way when VA form is filled we need to mark 2nd milestone "VAFormDue" as fulfilled so that all corresponding alerts are cancelled or made inactive.

Sometimes we need to cancel or unenroll person from schedule for any reason, lets say VA form was not applicable or CHW did not find anyone as VA respondent then to unenroll or cancel the schedule call unenroll method like below

scheduler.unEnrollFromSchedule(entityId, anmId, SchedulerConstants.VA_FORM_SCHEDULE_SUBJECT);

For other methods available in HealthSchedule see the class diagram above

4. Hooked Event:

Many times we need to trigger an action based on fulfillment of any particular milestone of a schedule in the system. OpenSRP allows to hook your business logic to achieve this via org.opensrp.scheduler.HookedEvent.

Lets assume that for our scenario any CHW has not filled the VA form despite having 2 months time period and/or assume that Crime Investigation Registry was enrolled into similar kind of schedule to investigate and close the Death Case within 2 years but despite passing 2 years the case has not been closed. CDG wants to get notified about such issue asap to take the appropriate action against the responsible person or department. Both logics are not predictable and independent of both registers. We may not want to keep adding the logic based on if and else at a central point into Mortality Registry. Rather we would like to "hook" our logic to the schedule alert on reaching the max time window. i.e. when max window is reached perform the action implemented by custom class.

Hooked Event
@Component
@Qualifier("VAMissedHookedEvent")
public class VAMissedHookedEvent implements HookedEvent {
    DataService dataService;

    @Autowired
    public VAMissedHookedEvent(DataService dataService) {
        this.dataService = dataService;
    }

    @Override
    public void invoke(MilestoneEvent event, Map<String, String> extraData) {
		// Get/Fetch details of deceased from DB or any other system
		// Send alert to CHW about the missed VA
		// Send alert to supervisor about the missed VA
		// Send alert to CDG to take appropriate action
    }
}

Similar would be implemented for Crime Investigation Registry where case has not been closed within 2 years.

add this Hooked Event to org.opensrp.scheduler.HealthScheduleService

@Component
public class HookedEventsHandler {
	private HookedEvent vaHookedEvent;
    private HookedEvent caseCloseHookedEvent;
	private HealthSchedulerService scheduler;

 	@Autowired
    public AlertHandler(HealthSchedulerService scheduler, @Qualifier("VAMissedHookedEvent") HookedEvent vaHookedEvent, 
			@Qualifier("CaseCloseHookedEvent") HookedEvent caseCloseHookedEvent) {
		this.vaHookedEvent = vaHookedEvent;
		this.caseCloseHookedEvent = caseCloseHookedEvent;
 		this.scheduler = scheduler;
    }

	public void addHookedEventsToSystem(){
		// Add hooked event to the max window of schedule VA_FORM_SCHEDULE_SUBJECT when last milestone "VAFormDue" max time has passed
		scheduler.addHookedEvent(eq(SchedulerConstants.VA_FORM_SCHEDULE_SUBJECT), any(), eq(max.toString()), vaHookedEvent);

		// Add hooked event to the late window of schedule CASE_CLOSE_SCHEDULE_SUBJECT for milestone "lastWarningForCaseClose"
        scheduler.addHookedEvent(eq(SchedulerConstants.CASE_CLOSE_SCHEDULE_SUBJECT), eq("lastWarningForCaseClose"), eq(late.toString()), caseCloseHookedEvent);
	}
}

NOTE:

  • Subject for all types of schedules should be a final, static, unique string throughout the system
  • We can implement multiple listeners for same schedule by adding multiple @MotechListener for subject of that schedule
  • If the event has multiple listeners, a separate event is raised for each listener and sent to the outbound event gateway. The original event object is not handled by a listener in this case.