This scoping document defines the narrative and technical requirements for dynamic task generation and entity persistence for the Reveal platform. The contents of this document relates to dynamic form configuration, but is not wholly dependent on that feature set. In essence dynamic task generation and entity persistence can take place independently of which form is opened when a task is assigned.
Critical Point: Plans will be evaluated in multiple locations across the platform to generate tasks. This will be used in combination with other platform features to support configurable workflows across the platform.
Background
Definitions
Operational Area: Operational area and Jurisdiction are synonymous. An operational area is the standardized geographic unit in which an intervention is executed. Operational areas are defined per implementation and are most often the lowest existing administrative organization (i.e. village), or lower.
Intervention Unit: An intervention unit is the unit of which an intervention is executed. An intervention unit can be an operational area, a location, a residential structure, a family or a family member. Intervention units are defined per plan action as part of the configuration of the system.
Intervention Type: An intervention type is defined in a plan. The system currently supports IRS, MDA and FI intervention types. This metadata is used to dynamically change how the Android app responds.
Entities
This specification depends on a fundamental understanding of the Reveal data model. An entity is a generic term for a data element that persists and changes state over time. Reveal has a number of core entities that all relate to each other.
Location
Jurisdiction - Locations with the jurisdiction tag represent geographic areas. They are technically multi-polygon GeoJSON with properties that are similar to FHIR locations.
Non-Jurisdiction (Structure) - Locations without the jurisdiction tag represent structures on the ground. These can be multi-polygon or point GeoJSON.
Form - Forms are the primary unit of data entry in Reveal in which the user interacts.
Event - Events are individual instances of forms that have been saved. When a form is saved it creates an event, which is the information at that point of time. Events also include embedded task information that so they can be linked to the appropriate location and plan.
Client - A client is an entity in the system that is acted upon by events. We technically represent many of the different entities as clients so that it’s able to conveniently track state over time.
Family - These are families that are registered in the system. These have a relationship to the non-jurisdiction (structure) where the family is registered.
Family member - These are family members registered in the system. These have a relationship to the family and structure where the family is registered.
Task - Tasks are units of work that a user is assigned or has completed. Tasks have a relationship to the plan definition (planIdentifier), plan.action (focus), jurisdiction (groupIdentifier), entity being acted upon (for), and the type of form to open (code)
Plan - Plans define a set of actions to completed in a list of jurisdictions over a period if time. Each plan lists actions that must be executed and provides the baseline for generating tasks.
Relationships
The following relationships are available in the system.
Plans
Intervention Type (IRS, MDA, or FI)
Focus Investigation Specific
Reason
Status
BIOPHICS case number and OpenSRP Event Id representing that case
Jurisdictions
Actions
Task
Plan Identifier
Jurisdiction location
Entity to be acted upon
structure
Family
Family Member
Operational Area
Which form to open when the task is activated (abstracted slightly as a code)
Event
Task Id associated with this event
By association, the plan and plan.action associated with this event
Task Business Status
Task Code
Non-jurisdiction location where this event was linked
Family
Event that created this family
Non-jurisdiction location where this family is linked
Family member
Event that created this family member
Family
Non-jurisdiction location where this family member is linked.
Generic Flow at this Time
User creates a plan with jurisdiction(s) and action(s)
User sets plan to active
System generates tasks in the server for each entity in the named jurisdiction
The action defines which entities should receive a task and performs logic based on entity relationships. For example:
If a family is registered at a structure, a “register family” task will not be generate.
If a structure is not residential it will not receive an action that is associated with a residential structure.
The user logs into their Android client and automatically downloads all plans, tasks, and events that they have access to.
The Android client applies all events and creates the clients locally.
Users navigate to the appropriate operational area/plan combination.
User can drop a point and automatically generate a task based on the type of the structure
User can click on any point and collect the appropriate form based on the task code.
Tasks are automatically created when a specific event has been created. For example, when a register family form is complete, the Android client creates a bednet distribution task and a blood screening task for the head of household that was registered as part of that form.
User can add family members once they are registered and a task is automatically generated once saved.
User syncs their device, which syncs updated clients, events, and tasks to the server.
Locations are not synced up
Plans are not changed and are, therefore, not synced
Conceptual Framework
This section outlines a conceptual framework of the changes we need to make to Reveal based on patterns adopted from HL7 FHIR workflows and implementation guides. Specifically, this section relies on the following links pasted here for reference:
Loosely from the Clinical Reasoning Module’s profiles
Critical Point: Plans will be evaluated in multiple locations across the platform to generate tasks. This will be used in combination with other platform features to support configurable workflows across the platform.
We have three major problems to solve in order to deliver these features in Reveal:
How do we generate tasks the first time a plan is activated?
How do we validate a task against the existing state when it is downloaded to the Android client?
How do we generate new tasks when an event is processed?
1. Task generation on plan activation
We presently generate tasks when a plan is activated with Apache Nifi. Nifi evaluates each action in a plan, queries each jurisdiction in the plan and performs logic based on the information in the action.
Sample IRS Logical Flow
Get each jurisdiction from the plan
For each jurisdiction
Get all active structures
Evaluate if they are residential and discard all others
For each residential structure
Determine if it has an active task already assigned for this plan.
If not, create an active task
Sample Focus Investigation Logical Flow
Get the jurisdiction from the plan (only 1 allowed)
Evaluate all actions in the plan and evaluate the subject
Operational Area
Generate one task for the operational area
Breeding Site
Get all structures with Breeding Site type
Determine if it has an active task already assigned for this plan.
If not, create an active task
Mosquito Collection Point
Get all structures with Mosquito Collection Point type
Determine if it has an active task already assigned for this plan.
If not, create an active task
Residential Structure
Get all structures with Residential Structure type
Determine if a family is registered
If registered
Determine if it has an active task already assigned for this plan.
If not, create an active task
Query to identify all active family members associated wit this family
For each family member, determine if it has an active task already assigned for this plan.
If not, create an active task
If not registered
Determine if it has an active task already assigned for this plan.
If not, create an active task
Determine if it has an active task already assigned for this plan.
If not, create an active task
We need to streamline this process so that any plan action combination can generate tasks in the OpenSRP server based on the server’s state at the time the plan is made active. This will involve evaluating the criteria on each action to see if it should.
Below is a description of this new process:
A plan is made active which triggers the task generator
The task generator loops through each action
The generator queries the database for all entities in the jurisdiction(s) of the type defined in the subjectCodableConcept
The generator evaluates the entity against action.condition.expression which evaluates to true or false.
Notes:
This expression may require additional information. For example, a register family action will only evaluate to true if the location.type=”residential_structure” AND there isn’t an existing family registered.
This criteria should evaluate to see if a task already exists for this action to accommodate any rerunning of the plan.
Generate a task if the condition evaluates to true
This will generate tasks based on the state of the server at the time they are generated.
2. Task validation to accommodate local state when downloaded to the Android client
We need to validate each task when it gets downloaded to the Android client so that we can confirm it is correctly placed against any unsynced entities. This allows us to ensure that local, unsynced changes on the device are reflected in the plan.
(This is still a work in progress)
3. Task generation during event processing
The event processor will need to be updated to include an evaluation of the plan to generate tasks in the Android client. OpenSRP includes an event processor that creates an event from the form. The following takes place in the app.
Create or update the client in the client entity table (EC Data model)
Update the task based on the business_status from the event if the event was triggered from a task
Evaluate the plan’s actions against the entities that was acted upon by this event.
Loop through each action in the plan for each entity that was acted upon by this event (i.e. a register family event creates a family, family member, and establishes a relationship between the location and the family)
Evaluate the criteria of the action
Generate a task if it evaluates to true
Note that the third step is the exact same process that’s done on the server, it’s just done on the entities that are transacted against during event processing.
Fundamental Shifts in How Reveal Works
This section defines some core characteristics of Reveal that need to shift in order to deliver Dynamic task generation and entity persistence. These changes will have implications across the platform, which we will break down into a set of features further in this document.
We currently use tasks as a way to measure work that is assigned. We are shifting this to include work that was performed in order to streamline reporting and standardize processing and changes to task state.
An example of this is when we have a register family task, which ultimately results in 3 family members being registered. Our current task paradigm would generate a register family task, but no tasks for the registration of the other two family members. As a result, we have events submitted that can’t be linked to plans and have to use events to track counts of work performed. Now, we will generate a task with a status “complete” so that we can retain this link and better measure work performed.
Plans will be updated to include action criteria that supports defining templated logic that should be evaluated when generating tasks in the server and client. This will be used at multiple steps across the system.
We will move bulk task generation from Nifi into the OpenSRP server, adding a business layer and queue that supports the evaluation of plans and generation of tasks. This process will walk the relationship tree for all affected entities that are affected in the jurisdiction, evaluating the logic defined in the plan action at each step.
(CRAIG CHECK THIS AGAIN AFTER YOU FINISH THIS DOC) We will not implement a state machine in the OpenSRP Server.
Configurable forms will be able to define the task’s business status as a hidden value in the bottom of the form labeled business_status.
This allows for configuration of the business logic, color coding, and task processing
We will implement a task state machine in the Android client that is able to execute logic on task state changes.
This allows us to respond to changes in task state independent of form submissions. A post save action on a form may or may not change the state of an assigned task. This allows us to chain actions in the event we need to execute multiple, independent, actions based on task state changes.
Plans will be evaluated in the Android client after a task state has changed to identify if follow-on actions should be taken. Each action will include evaluation criteria that determines the action.
The Android client will be able to display multiple tasks assigned to a structure in the map view. This will allow implementers to assign multiple action items and independent forms at the structure level.
(I’m not sure if we want to do this or solve it another way) We will provide access to view and edit locations, families and family members in the Android client even if a task is not assigned to that entity.
The location is the primary unit of navigation from the map view. A user can not access the information at a location if a task is not assigned. We will lift that constraint and allow users to view the underlying entity.
Configurable Items
This section defines items that can be configurable in the server with a description of what information will be configurable.
Forms (Upload Web UI)
System administrators will be able to add, edit, and update fields for forms that are available in the system. This includes changing the business status that impacts task generation
Plan Templates (No Web UI)
This is a new item that will allow system administrators to define plan templates for certain workflows. We currently hard code this into the OpenSRP Web UI for IRS and Focus Investigation. These templates allow a user to select from a drop down menu and control the actions that show up. Ultimately, the template creator has the capacity to create any action they wish.
Creating, updating, and archiving Plan Templates will not be accessible through the web UI.
Plans (Web UI)
Users are currently able to create plans in the Web UI. We will expand this to support adding criteria to action.
Task State Change Logic (No Web UI)
This will allow us to define logic that executes when the task state changes
(Placeholder: I’m not sure if we need to do this) Post form save logic (No Web UI)
We want to standardize the post form save logic so that it can be templated and eventually defined server side. This would allow us to define what happens after a form is saved.
(I’m not sure if we want to do this, or if MapBox provides this for us) Map View Styles (No Web UI)
System administrators will be able to define color coding for business status combinations available in the Android client and Web UI’s map views. This will allow system administrators to define color coding combinations on state
New End-to-End Workflow Samples
This section defines the new end-to-end workflow on how Reveal will implement. It assumes the system has been configured to support the appropriate flow.
IRS
Configured Items:
IRS Zambia 2020 plan template
Forms
Task State Change Logic
Map View Styles
Steps:
User creates a plan in the Web UI
They select from a drop down of plan templates, choosing (IRS Zambia 2020 plan template)
They complete the online form, confirm the default dates are correct and click Save
(optional) The user walks through the map based planning UI
The plan’s status is changed to “active” which triggers an action that generates all tasks for the plan in the OpenSRP Server
The user assigns a team to the plan if it wasn’t done in the map based planning UI
The spray operator logs into the Android client and the data syncs to their device
The user drops a point to add a location
The form is saved
The post form save action generates a task with status=”complete” and business_status defined by the form.
The task state change triggers an evaluation of the actions in the plan
The spray action evaluates to true because the location is marked as a residential structure.
The task state machine generates a “spray” task based on the spray task template that was configured with status “Ready”
The color coding is reflected as it was defined for this type of task in the Map View Styles configuration
The user taps the structure
If multiple tasks were assigned, they would be presented with a list of those tasks so they could choose which one to complete. The user should also be able to archive a task if it’s not relevant.
The form opens, they complete the form, and save it.
The post save action executes, updating the business status of the task
The task state change in status and business_status triggers an evaluation of the actions of the plan
All criteria evaluates to false for each action, so nothing is changed in the system
The user is returned to the map view. The color coding is reflected as it was defined for this type of task in the Map View Styles configuration.
Focus Investigation - Routine
Configured Items:
Thailand Routine FI 2020 plan template
Forms
Task State Change Logic
Map View Styles
Steps:
User creates a plan in the Web UI
They select from a drop down of plan templates, choosing (Thailand Routine FI 2020 plan template)
They complete the online form, confirm the default dates are correct.
They make sure to check the plan as active
The plan’s status is changed to “active” which triggers an action that generates all tasks for the plan in the OpenSRP Server
The user assigns a team to the plan
The frontline worker logs into the Android client and the data syncs to their device
The user drops a point to add a location
The form is saved
The post form save action generates a task with status=”complete” and business_status defined by the form.
The task state change triggers an evaluation of the actions in the plan
The spray action evaluates to true because the location is marked as a residential structure.
The task state machine generates a “register family” task based on the register family task template that was configured with status “Ready”
The color coding is reflected as it was defined for this type of task in the Map View Styles configuration
The user taps the structure
If multiple tasks were assigned, they would be presented with a list of those tasks so they could choose which one to complete. The user should also be able to archive a task if it’s not relevant.
The form opens, they complete the form, and save it.
The post save action executes, updating the business status of the task.
The task state change in status and business_status triggers an evaluation of the actions of the plan
The criteria for the “Bednet Distribution” action evaluates to true so a bednet distribution task is created
The criteria for the “Blood Screening” action evaluates to true so a blood screening task is created for the head of household.
The criteria for all other actions evaluate to false.
The user is forwarded to the household view
The user registers the spouse of the head of household and saves the form.
The post save action executes, creating a task with status=”complete” and business_status based on the register family member form. (This allows us to count that work, even though it wasn’t initially assigned)
The task state change in status and business_status triggers an evaluation of the actions of the plan
The criteria for the “Blood Screening” action evaluates to true so a blood screening task is created for the recently added family member.
The criteria for all other actions evaluate to false.
The user returns to the household view.
The user completes the blood screening form for the head of household.
The post save action executes, updating the business status of the task.
The task state change in status and business_status triggers an evaluation of the actions of the plan
The criteria for all actions evaluate to false.
The user returns to the household view.
The user edits the blood screening form for the head of household.
The post save action executes, updating the business status of the task.
The task state change in business_status triggers an evaluation of the actions of the plan
The criteria for all actions evaluate to false.
The user returns to the household view.
The user navigates back to the map view and the color of that structure changes to the color defined in the configurable Map View Style
Thought Exercise: Child Immunization
This thought exercise is here to test the logic of these features against another prominent health area. These generic features may be combined or slightly improved upon to unlock other frontline health workflows. This thought exercise is here to test that idea.
Configured Items:
Liberia Child Health Immunization plan template
These plans are structured slightly differently than the malaria based plan. The first action is to generate or update a care plan. Each immunization is represented as an action with the criteria defined by the EPI program’s the recommended immunization schedule.
Forms
Registration Form - This form registers the child
Care Plan - This form keeps track of the care plan for the child and represents the state of their immunization over time
Immunization Event - This form captures an immunization event
Task State Change Logic
Steps:
User creates a plan in the Web UI
They select from a drop down of plan templates, choosing (Liberia Child Health Immunization plan template)
They complete the online form, confirm the default dates are correct.
They make sure to check the plan as active
The plan’s status is changed to “active” which triggers an action that generates all tasks for the plan in the OpenSRP Server
The plan evaluates each action for each child already registered in the system and generates a care plan
The plan evaluates each immunization to see if it has already been completed based on the criteria defined in the action. For example, it will create a task for immunizations that are presently due based on the immunizations they have already received and the child’s age. It will not create a task for immunizations that are not yet due.
The user assigns a team to the plan
The spray operator logs into the Android client and the data syncs to their device
The user navigates to the child’s record and taps an immunization in the care plan which records an immunization and saves the form.
The post save action executes, updating the status and business status of the task.
The task state change in status and business_status triggers an evaluation of the actions of the plan
The care plan is updated
A task is generated for the next sequence
The user returns to the child’s profile page
(We need to accommodate for date based logic)
That night, on a regular schedule:
All care plans are evaluated on the Android client in a batch job to see if schedule based logic needs to be updated (i.e. the child has now reached 6 weeks old and they should receive their immunization)
The Care plan is evaluated and updated
A task is generated for each immunization that is due.
Existing tasks that are overdue get a change in business status
Changes to Plans
We need to change the plan definition so that it is responsive and supports conditions following the FHIR specification. We do this so that we can generate tasks based on actions when a particular condition is met for an action.
Major Changes
We need to remove the idea of an Intervention Type across the system. Instead of an intervention type, we need to depend on the task actions that are set. Ultimately, we want any action that is added to the plan to be supported in the Android client.
There’s a difference between the form that’s filled to create the plan and the plan itself. For example, there are specific use contexts related to FI plan types. This should remain optional to be filled in as a template for FI specific plans, but we do not need to depend on intervention type for internal business logic.
We need to add support for conditions on actions so they can be evaluated during generation.
specifically
condition.kind:”applicability”
condition.expression: (See https://www.hl7.org/fhir/metadatatypes.html#Expression)
We need to add a definitionUri that points to the form to be filled out by form name. This will be evaluated when a task is opened to determine which form to open. (i.e. the IRS Spray Structures action will have a definitionUri that points to the spray_form.json)
Don’t evaluate anything below this
Dynamic Task Generation
We need to update the flow in the Android client so that tasks are generated based on the actions in the plan. In order to do this, we need to develop the following feature:
Evaluate all plan actions during post form save action
Develop a way to identify the template for tasks based on the action. In other words, if you have a “register family” action, provide a way to identify how to fill out that task before it’s generated
We will likely need to accomplish this by generating templates in the server that are synchronized down to the Android client.
Loosen the constraints on the plan action so that we can configure the action criteria and codes that are available in the server. This will allow us to provide actions on different intervention units.
Identify how to handle multiple tasks assigned to a single structure for different task actions. In other words, what should we do if action 1 creates a task on the structure and action 2 creates a task on that structure. Reveal doesn’t currently handle this in the Android client.
Support entity persistence in the local task generation process.
Link the task information to the configurable form name and version that’s developed for the configurable form feature
Entity Persistence
We inherently create a conflict in the Android client across plans if a task has been generated server side and the Android client hasn’t synced. Additionally, users change entities in one plan and expect that to persist across new plans when offline. For example, if a user has two plans in the same operational area and they register a family at structure A, they expect that family to show up in structure A in both plans. The current process requires a duplicate registration of that family in the second plan.
Below is a list of entities that must persist across plans:
Jurisdiction Location (We may need to edit jurisdiction locations at some time in the future in the Android client)
Non-Jurisdiction Location
Family
Family Member
The following features need to be supported:
Identify a subset of post form save actions that create, update, or archive an entity in the system that should be persisted.
When the form is saved, go through all plans that have an active start and end date and represent the update state based on the actions in that plan.
For example, if a family is registered in plan A, query the database for any tasks that are assigned to that structure with code “register family” and mark them as complete. Then, identify the plan associated with those tasks, evaluate the actions and
This may pose conflicts where a plan already has a family registered. We need to handle these conflicts when they arise.
Make sure all persisted entities sync from the Android client to the server including locations.
Add the ability to edit and archive non-jurisdiction locations in the Android client so they are archived and do not show up in the future.
(Nice to have) Remove locations that do not have an active task in the Android client. They currently show up as grey.
Alternatively, we could find a way to display information about the location and possibly allow the user to assign a task to that location
Notes from Call with Matt 20 May 2020
Make the change on the task trigger an action (not sure why this is different from the event submission)
Task status change triggers one type of action based on the business status at that time
Loop in the goal to show how calculations will change
Model the scenarios (they need to be modeled anyway) around server-side generated tasks vs client-side generated tasks.
Consider any plan action to any logic, to any task template to any form. This is the most dynamic
Entity persistence falls within the logic queries that determine next steps.
Walk the chain if an entity is found, try to find the next entity relationship
This will require maxing sure the indexing is on point
There isn’t a FHIR operation for task state changes in the state machine. This is fine, but maybe something we could contribute in the long run
Think through an association with CarePlan resources.
Brainstorming Section
Thailand Specific
Recently Pedro highlighted that a routine FI plan had been created on the WebUI that only included the Register Family task.
When users looked at the plan on the Android device, a number of structures were marked as grey and could not be accessed
Ona investigated and validated that no tasks for those structures were created as they already had families registered
Should these structures still have tasks created, with a status of ‘Completed’ so that they show up as green rather than grey, and can be accessed to view and edit family information?
Does it make sense to have these tasks with no corresponding events?
Should the original family registration event be linked to the task somehow?
General Placeholder
Craig
Can we use the same logic in the server in the Android client to evaluate entities when an event is evaluated?
The core assumption is yes, we just need to make sure we architect it correctly.
Thinking through sample plan actions and criteria
Register Family Action
Description: This action focuses on registering families if they don’t already exist at each residential structure
subjectCodableConcept: location
Criteria
location.type=”residential_structure” AND
(location.status=”active” or “pending”) AND
a family isn’t registered against this structure AND
a task hasn’t already been generated for this plan and action combination
Notes:
It would be easier from a logic perspective for end users if we keep the subjectCodableConcept basic on jurisdiction, location, family, family member and then add that to a criteria
Complex MDA example
Mass Drug Administration Round 1 Administration Action
Description: This action supports the initial administration of the medication to the person
subjectCodableConcept: person (family member?)
Criteria
person.dob>=today - 5 years
Mass Drug Administration Round 1 Adherence Action
Description: This action supports the adherence follow-up of the medication to the person
subjectCodableConcept: person (family member?)
Criteria
person.dob>=today - 5 years AND
person has an event for MDA Round 1 Administration
Mass Drug Administration Round 2 Administration Action
Description: This action supports the second administration of the medication to the person who already received round 1
subjectCodableConcept: person (family member?)
Criteria
person.dob>=today - 5 years AND
person has a complete task with business status=Adherent for task.focus=action.id for the Mass Drug Administration Round 1 Administrative Action
Notes:
I don’t know if we want to set the criteria to see if an event exists or if a task exists with the complete status