This scoping document defines the narrative and technical requirements for dynamic task generation and entity persistence for the Reveal platform. The contents of this document relates to dynamic form configuration, but is not wholly dependent on that feature set. In essence dynamic task generation and entity persistence can take place independently of which form is opened when a task is assigned.
...
Plans will be updated to include action criteria that supports defining templated logic that should be evaluated when generating tasks in the server and client. This will be used at multiple steps across the system.
We will move bulk task generation from Nifi into the OpenSRP server, adding a business layer and queue that supports the evaluation of plans and generation of tasks. This process will walk the relationship tree for all affected entities that are affected in the jurisdiction, evaluating the logic defined in the plan action at each step.
(CRAIG CHECK THIS AGAIN AFTER YOU FINISH THIS DOC) We will not implement a state machine in the OpenSRP Server.
Configurable forms will be able to define the task’s business status as a hidden value in the bottom of the form labeled business_status.
This allows for configuration of the business logic, color coding, and task processing
We will implement a task state machine in the Android client that is able to execute logic on task state changes.
This allows us to respond to changes in task state independent of form submissions. A post save action on a form may or may not change the state of an assigned task. This allows us to chain actions in the event we need to execute multiple, independent, actions based on task state changes.
Plans will be evaluated in the Android client after a task state has changed to identify if follow-on actions should be taken. Each action will include evaluation criteria that determines the action.
The Android client will be able to display multiple tasks assigned to a structure in the map view. This will allow implementers to assign multiple action items and independent forms at the structure level.
(I’m not sure if we want to do this or solve it another way) We will provide access to view and edit locations, families and family members in the Android client even if a task is not assigned to that entity.
The location is the primary unit of navigation from the map view. A user can not access the information at a location if a task is not assigned. We will lift that constraint and allow users to view the underlying entity.
...
Forms (Upload Web UI)
System administrators will be able to add, edit, and update fields for forms that are available in the system. This includes changing the business status that impacts task generation
Plan Templates (No Web UI)
This is a new item that will allow system administrators to define plan templates for certain workflows. We currently hard code this into the OpenSRP Web UI for IRS and Focus Investigation. These templates allow a user to select from a drop down menu and control the actions that show up. Ultimately, the template creator has the capacity to create any action they wish.
Creating, updating, and archiving Plan Templates will not be accessible through the web UI.
Plans (Web UI)
Users are currently able to create plans in the Web UI. We will expand this to support adding criteria to action.
Task State Change Logic (No Web UI)
This will allow us to define logic that executes when the task state changes
(Placeholder: I’m not sure if we need to do this) Post form save logic (No Web UI)
We want to standardize the post form save logic so that it can be templated and eventually defined server side. This would allow us to define what happens after a form is saved.
(I’m not sure if we want to do this, or if MapBox provides this for us) Map View Styles (No Web UI)
System administrators will be able to define color coding for business status combinations available in the Android client and Web UI’s map views. This will allow system administrators to define color coding combinations on state
...
User creates a plan in the Web UI
They select from a drop down of plan templates, choosing (IRS Zambia 2020 plan template)
They complete the online form, confirm the default dates are correct and click Save
(optional) The user walks through the map based planning UI
The plan’s status is changed to “active” which triggers an action that generates all tasks for the plan in the OpenSRP Server
The user assigns a team to the plan if it wasn’t done in the map based planning UI
The spray operator logs into the Android client and the data syncs to their device
The user drops a point to add a location
The form is saved
The post form save action generates a task with status=”complete” and business_status defined by the form.
The task state change triggers an evaluation of the actions in the plan
The spray action evaluates to true because the location is marked as a residential structure.
The task state machine generates a “spray” task based on the spray task template that was configured with status “Ready”
The color coding is reflected as it was defined for this type of task in the Map View Styles configuration
The user taps the structure
If multiple tasks were assigned, they would be presented with a list of those tasks so they could choose which one to complete. The user should also be able to archive a task if it’s not relevant.
The form opens, they complete the form, and save it.
The post save action executes, updating the business status of the task
The task state change in status and business_status triggers an evaluation of the actions of the plan
All criteria evaluates to false for each action, so nothing is changed in the system
The user is returned to the map view. The color coding is reflected as it was defined for this type of task in the Map View Styles configuration.
...
User creates a plan in the Web UI
They select from a drop down of plan templates, choosing (Thailand Routine FI 2020 plan template)
They complete the online form, confirm the default dates are correct.
They make sure to check the plan as active
The plan’s status is changed to “active” which triggers an action that generates all tasks for the plan in the OpenSRP Server
The user assigns a team to the plan
The frontline worker logs into the Android client and the data syncs to their device
The user drops a point to add a location
The form is saved
The post form save action generates a task with status=”complete” and business_status defined by the form.
The task state change triggers an evaluation of the actions in the plan
The spray action evaluates to true because the location is marked as a residential structure.
The task state machine generates a “register family” task based on the register family task template that was configured with status “Ready”
The color coding is reflected as it was defined for this type of task in the Map View Styles configuration
The user taps the structure
If multiple tasks were assigned, they would be presented with a list of those tasks so they could choose which one to complete. The user should also be able to archive a task if it’s not relevant.
The form opens, they complete the form, and save it.
The post save action executes, updating the business status of the task.
The task state change in status and business_status triggers an evaluation of the actions of the plan
The criteria for the “Bednet Distribution” action evaluates to true so a bednet distribution task is created
The criteria for the “Blood Screening” action evaluates to true so a blood screening task is created for the head of household.
The criteria for all other actions evaluate to false.
The user is forwarded to the household view
The user registers the spouse of the head of household and saves the form.
The post save action executes, creating a task with status=”complete” and business_status based on the register family member form. (This allows us to count that work, even though it wasn’t initially assigned)
The task state change in status and business_status triggers an evaluation of the actions of the plan
The criteria for the “Blood Screening” action evaluates to true so a blood screening task is created for the recently added family member.
The criteria for all other actions evaluate to false.
The user returns to the household view.
The user completes the blood screening form for the head of household.
The post save action executes, updating the business status of the task.
The task state change in status and business_status triggers an evaluation of the actions of the plan
The criteria for all actions evaluate to false.
The user returns to the household view.
The user edits the blood screening form for the head of household.
The post save action executes, updating the business status of the task.
The task state change in business_status triggers an evaluation of the actions of the plan
The criteria for all actions evaluate to false.
The user returns to the household view.
The user navigates back to the map view and the color of that structure changes to the color defined in the configurable Map View Style
...
We need to remove the idea of an Intervention Type across the system. Instead of an intervention type, we need to depend on the task actions that are set. Ultimately, we want any action that is added to the plan to be supported in the Android client.
There’s a difference between the form that’s filled to create the plan and the plan itself. For example, there are specific use contexts related to FI plan types. This should remain optional to be filled in as a template for FI specific plans, but we do not need to depend on intervention type for internal business logic.
We need to add support for 0…* triggers within plan actions so that we can quickly evaluate whether we want to trigger the evaluation of this action based on a value in the system. If a trigger doesn’t exist, we will evaluate to false.
specifically (following https://www.hl7.org/fhir/metadatatypes.html#TriggerDefinition)
plan.action.trigger should contain the following:
type: “named-event” (required)
name:”” (required)
options:
“planActivation” - This event represents the one time a plan is activated
“locationAdded” - This event represents when a location is added
“familyRegistered” - This event represents when a family is registered
“familyMemberRegistered” - This event represents when a family member is registered
(Any encountertype that’s in the json form definition. We do not evaluate the condition of this trigger if it’s planActivation
“eventSubmission” - This signals that we are going to evaluate the conditions.
condition:””
Add an expression that identifies the encounterType and we can handle it from there. So, you could fill out a “MDA Adherence” form that creates an “mdaAdherence” event
). To put it another way, you can
We need to add support for conditions on actions so they can be evaluated during generation .using a FHIRpath evaluator
specifically (following http://hl7.org/fhirpath/)
condition.kind:”applicability”
condition.resourceType:””
This isexpression: We will implement FHIRPath with an extension to
FHIR to make the evaluation of the expression easier based on the resource typeSupported resourceTypes:
location - This references a location that is not a jurisdiction (is_jurisdiction=false)
jurisdiction - This references a location that is a jurisdiction (is_jurisdiction=true)
family - This references a family
familyMember - This references a family member
condition.expression: We will implement a light version of FHIRPath in the initial implementation to support boolean expressions that evaluate to true or false
$this.tree1.tree2.treeN We will support a dot notation that can go down different levels of a JSON object
Required expressions for this MVP
$this.properties.type = ““ and ($this.properties.status = “active” or $this.properties.status = “pending”) (resourceType = “location”)
We use this expression to validate the type of location and act when the location status is active or pending so we can trigger actions. The user can enter the type of location such as residential_structure.
$this.exists() (resourceType = “jurisdiction”)
This expression will evaluate to true if the jurisdiction exists. We will use this to trigger jurisdiction level tasks
$this.relationship(family,'status!=”active”') (resourceType = “location”)
This expression identifies if a location has an existing relationship with a family resourceType and that family’s status is active, not archived. We will use this to determine if we want to generate a register family task.
familyMember.age >= “5”
This expression identifies if the family member’s age is greater than or equal to 5
Supported operators
= (equals) This operator evaluates to true or false
!= (not equals) This operator determines if the values are not equivalent
> (greater than) This operator is used to evaluate if a value is greater than a number
< (less than) This operator is used to evaluate if a value is less than a number
>= (greater than or equal) This operator is used to evaluate if a value is greater than or equal to a number
<= (less than or equal) This operator is used to evaluate if a value is less than or equal to a number
and (and) This operator evaluate to true if all expressions evaluate to true
or (or) This operator evaluates to true if one or more expression evaluates to true
() (parenthesis) We use parenthesis to group logic
'' (single quotes) We use single quotes to segment expressions within a function
Supported functions to get additional information
exists() - This function identifies if the entity exists and evaluates to false if it does not
relationship(resourceType,'expression') - This function supports a nested expression so you can search expressions for existing relationships and evaluate the entity
We need the ability to have functions that determine if a relationship is available between entities
We do not support looping through different entities in these expressions. A user can create a plan action for each entity in the jurisdiction if they wish to
We need to add a definitionUri that points to the form to be filled out by form name. This will be evaluated when a task is opened to determine which form to open. (i.e. the IRS Spray Structures action will have a definitionUri that points to the spray_form.json)
New PlanDefinition
We need to create a new entity in OpenSRP server called PlanDefinition which acts as a template for plans so that we can make and reference plan templates from the web UI. So, the IRS plan would have a particular type of planDefinition template and the MDA would have another one. When a user creates a plan in the web UI, they would read from a dropdown of available plandefinitions and that would populate the form in the web UI. When they hit “save” it creates the plan as per usual.
We don’t need a UI for creating planDefinitions, but we need a REST API to POST, PUT and GET these templates.
Software to be Developed
This section outlines the software packages that need to be developed in order to deliver these features. The target is to break out these items into tangible chunks of work that can be scheduled and delivered within two weeks.
(This section is still a work in progress)
1. Develop Plan Evaluator Java Library
We need to develop a Java library that can be run both in the Android client and OpenSRP server. This library will be responsible for evaluating plans and generating tasks. The inputs vary depending on the location where the library is deployed. This library is responsible for consistently executing the logic that generates tasks.
Inputs:
planIdentifier - This is the identifier of the plan that needs to be executed
trigger -
(optional) Entity - This is an entity to be evaluated against the plan. This allows us to evaluate a specific
Triggers:
Server - This library gets triggered the single time that a plan transitions from status:draft to status:active
Android Client - This library gets triggered through an asynchronous call
Logic: (See section above)
2. Update Event Processor to Evaluate Plans
We need to update the event processor to evaluate plans for a given trigger.
3. Update App logic to make sure it’s truly dynamic
We need more info on this from Sam G.
4. Update
5. Add a the ability to define plan templates so the web UI can load these in
Don’t evaluate anything below this
Dynamic Task Generation
We need to update the flow in the Android client so that tasks are generated based on the actions in the plan. In order to do this, we need to develop the following feature:
Evaluate all plan actions during post form save action
Develop a way to identify the template for tasks based on the action. In other words, if you have a “register family” action, provide a way to identify how to fill out that task before it’s generated
We will likely need to accomplish this by generating templates in the server that are synchronized down to the Android client.
Loosen the constraints on the plan action so that we can configure the action criteria and codes that are available in the server. This will allow us to provide actions on different intervention units.
Identify how to handle multiple tasks assigned to a single structure for different task actions. In other words, what should we do if action 1 creates a task on the structure and action 2 creates a task on that structure. Reveal doesn’t currently handle this in the Android client.
Support entity persistence in the local task generation process.
Link the task information to the configurable form name and version that’s developed for the configurable form feature
Entity Persistence
We inherently create a conflict in the Android client across plans if a task has been generated server side and the Android client hasn’t synced. Additionally, users change entities in one plan and expect that to persist across new plans when offline. For example, if a user has two plans in the same operational area and they register a family at structure A, they expect that family to show up in structure A in both plans. The current process requires a duplicate registration of that family in the second plan.
Below is a list of entities that must persist across plans:
Jurisdiction Location (We may need to edit jurisdiction locations at some time in the future in the Android client)
Non-Jurisdiction Location
Family
Family Member
The following features need to be supported:
Identify a subset of post form save actions that create, update, or archive an entity in the system that should be persisted.
When the form is saved, go through all plans that have an active start and end date and represent the update state based on the actions in that plan.
For example, if a family is registered in plan A, query the database for any tasks that are assigned to that structure with code “register family” and mark them as complete. Then, identify the plan associated with those tasks, evaluate the actions and
This may pose conflicts where a plan already has a family registered. We need to handle these conflicts when they arise.
Make sure all persisted entities sync from the Android client to the server including locations.
Add the ability to edit and archive non-jurisdiction locations in the Android client so they are archived and do not show up in the future.
(Nice to have) Remove locations that do not have an active task in the Android client. They currently show up as grey.
Alternatively, we could find a way to display information about the location and possibly allow the user to assign a task to that location
Brainstorming Section
Thailand Specific
Recently Pedro highlighted that a routine FI plan had been created on the WebUI that only included the Register Family task.
When users looked at the plan on the Android device, a number of structures were marked as grey and could not be accessed
Ona investigated and validated that no tasks for those structures were created as they already had families registered
Should these structures still have tasks created, with a status of ‘Completed’ so that they show up as green rather than grey, and can be accessed to view and edit family information?
Does it make sense to have these tasks with no corresponding events?
Should the original family registration event be linked to the task somehow?
General Placeholder
Craig
Can we use the same logic in the server in the Android client to evaluate entities when an event is evaluated?
The core assumption is yes, we just need to make sure we architect it correctly.
Thinking through sample plan actions and criteria
Register Family Action
Description: This action focuses on registering families if they don’t already exist at each residential structure
subjectCodableConcept: location
Criteria
location.type=”residential_structure” AND
(location.status=”active” or “pending”) AND
a family isn’t registered against this structure AND
a task hasn’t already been generated for this plan and action combination
Notes:
It would be easier from a logic perspective for end users if we keep the subjectCodableConcept basic on jurisdiction, location, family, family member and then add that to a criteria
Complex MDA example
Mass Drug Administration Round 1 Administration Action
Description: This action supports the initial administration of the medication to the person
subjectCodableConcept: person (family member?)
Criteria
person.dob>=today - 5 years
Mass Drug Administration Round 1 Adherence Action
Description: This action supports the adherence follow-up of the medication to the person
subjectCodableConcept: person (family member?)
Criteria
person.dob>=today - 5 years AND
person has an event for MDA Round 1 Administration
Mass Drug Administration Round 2 Administration Action
Description: This action supports the second administration of the medication to the person who already received round 1
subjectCodableConcept: person (family member?)
Criteria
person.dob>=today - 5 years AND
person has a complete task with business status=Adherent for task.focus=action.id for the Mass Drug Administration Round 1 Administrative Action
Notes:
I don’t know if we want to set the criteria to see if an event exists or if a task exists with the complete status
Links to Dynamic Forms
We are adding the ability to define the form name in a plan action so that we can link that particular action to the data collection instrument. This will be in the action.definitionUri.
We need to create a table in the SQLite database that can be quickly referenced to identify which forms should be opened by which task
task.focus links to the plan.action.id.
We need to create a table that updates for each plan action that’s downloaded to the device that links the task.focus to the plan.action.definitionUri so that we can open the right form in a performant way.
We will not define the form version in the plan. That will be managed by the configurable forms manifest file.
within the condition to help identify the additional entity in the condition.subjectCodableConcept
condition.subjectCodableConcept.text: ““
(optional) This is an extension of the expression that we will use to identify which additional entities are available for evaluating the expression. This allows you to query to see if a relationship exists between the entity that was passed in and the subjectCodableConcept.
Options:
jurisdiction
location
family
family member
We need to add a definitionUri that points to the form to be filled out by form name. This will be evaluated when a task is opened to determine which form to open. (i.e. the IRS Spray Structures action will have a definitionUri that points to the spray_form.json)
New PlanDefinition
We need to create a new entity in OpenSRP server called PlanDefinition which acts as a template for plans so that we can make and reference plan templates from the web UI. So, the IRS plan would have a particular type of planDefinition template and the MDA would have another one. When a user creates a plan in the web UI, they would read from a dropdown of available plandefinitions and that would populate the form in the web UI. When they hit “save” it creates the plan as per usual.
We don’t need a UI for creating planDefinitions, but we need a REST API to POST, PUT and GET these templates.
Entity Persistence
We inherently create a conflict in the Android client across plans if a task has been generated server side and the Android client hasn’t synced. This is the case when we have multiple plans working in the same operational area and a new plan is created in that same operational area. Additionally, The current process requires a duplicate registration of that family in the second plan.
Below is a list of entities that must persist across plans:
Jurisdiction Location (We may need to edit jurisdiction locations at some time in the future in the Android client)
Non-Jurisdiction Location
Family
Family Member
Illustrative Use Case
Plan A (Activation Date 2020-01-01 T00:00.000+00:00) for Jurisdiction X, which has 10 residential structures.
The user synced on January 1st at 10:00AM and got to work.
The user worked offline registering 10 families and 20 family members (2 at each residential structure)
The user found 5 additional residential structures, dropped points in the field and registered families against them, each with 2 family members.
(Current totals: 15 residential structures with 15 families and 30 family members)
The user is still offline and Plan B kicks off with Activation Date 2020-01-05 T00:00.000+00:00 for Jurisdiction X
The server runs through the plan and created 10 register family tasks because the state still shows 10 residential structures.
The user gets online and syncs on 2020-01-10 T10:00.000+00:00
The following events sync up: 5 Register_Structure events, 15 Family Registration and 15 Family Member Registration
5 new locations, 15 families and 30 family members sync up to the server (families and family members as clients)
Plan B syncs down with 10 register family tasks.
The user, looking at Plan B does not have access to the underlying families that were created in Plan A. Also, no tasks are assigned to the 5 new locations that they dropped in Plan A
Problems we need to solve
There is a disparity between the local state and the server state of locations, families and family members when a plan is generated. Plans are generated at a point in time and we don't evaluate them against entities that may already exist in the Android client at any time in the future. We need to provide a way to update tasks based on
Proposed Solutions
We need to develop a pattern that generically identifies existing tasks that should be canceled based on differences in state from the time a plan was turned to active and the time an entity reached the server. Ultimately, we would like the OpenSRP server to evaluate the events or clients that were recently received against Plan B, cancel the 10 register family tasks and evaluate the plan against all 15 locations, 15 families and 30 family members.
In order to do this, we will add the ability to create plan actions that support updating tasks. We will accomplish this by creating a plan.action.type: “Update” and implementing a plan.action.type: “create” for the existing plans that are generated for the dynamic tasking feature. These new plan types are going to be responsible for updating tasks based on the events that are passed in. Ultimately, these new action types will allow us to update tasks in the current plan.
Below is a sample of the Register Family Action from Plan A
Notice that we added a high level “type” to the action with value “create” This signifies that we will create a task when we evaluate the plan.
| Code Block | ||
|---|---|---|
| ||
{
"identifier": "3802af3a-f40c-4705-849e-1727b603bd4e",
"prefix": 1,
"title": "Family Registration",
"description": "Register all families and family members in all residential structures enumerated (100%) within the operational area",
"code": "RACD Register Family",
"type": "create",
"trigger": [
{
"type": "named-event",
"name": "plan-activation"
},
{
"type": "named-event",
"name": "event-submission",
"expression": {
"expression": "questionnaire = 'Register_Structure'"
}
}
],
"condition": [
{
"kind": "applicability",
"expression": {
"description": "Structure is residential",
"reference": "plan-activation",
"expression": "Location.type.where(id='locationType').text = 'Residential Structure'"
}
},
{
"kind": "applicability",
"expression": {
"description": "Family does not exist for structure",
"reference": "plan-activation",
"expression": "$this.contained.exists()",
"subjectCodableConcept": {
"text": "Family"
}
}
},
{
"kind": "applicability",
"expression": {
"reference": "event-submission",
"expression": "questionnaire = 'Register_Structure' AND item.where(linkId='structureType').answer.value ='Residential Structure'"
}
}
],
"timingPeriod": {
"start": "2020-06-04",
"end": "2020-10-01"
},
"reason": "Routine",
"goalId": "RACD_register_all_families",
"subjectCodableConcept": {
"text": "Location"
},
"definitionUri": "family_register.json"
}, |
Below is a sample action that updates register family tasks
Note that the expressions in this sample are just to illustrate and we have not clearly defined a patter at this moment.
| Code Block | ||
|---|---|---|
| ||
{
"identifier": "NEW_IDENTIFIER",
"prefix": 10,
"title": "Cancel Duplicate Register Family Tasks",
"description": "Cancel tasks that are generated against locations that already have families against them",
"code": "RACD Register Family",
"type": "update",
"trigger": [
{
"type": "named-event",
"name": "event-submission",
"expression": {
"expression": "questionnaire = 'Register_Structure'"
}
}
],
"condition": [
{
"kind": "applicability",
"expression": {
"reference": "event-submission",
"expression": "task.code = 'RACD Register Family' and task.for = QuestionnaireResponse.location_id",
"subjectCodableConcept": {
"text": "global.task"
}
}
}
],
"timingPeriod": {
"start": "2020-06-04",
"end": "2020-10-01"
},
"reason": "Routine",
"goalId": "RACD_register_all_families",
"subjectCodableConcept": {
"text": "Location"
},
"definitionUri": "family_register.json",
"dynamicValue": [
{
"expression": {
"expression": "INSERT_SOME_EXPRESSION_HERE"
}
}
]
}, |
Business Logic
We expect to execute business logic generically as follows.
The Family registration Event will evaluate false to all triggers except for the “Cancel Duplicate Register Family Tasks” action. At this moment, we only have the event itself. We don’t have any client information.
The system will evaluate the condition and identify that we need to query the database for all tasks that have the following fields:
task.groupIdentifier = QuestionnaireResponse. jurisdiction (Whatever field this is mapped to in the Questionnaire)
task.code = plan.action.code (For this action)
task.for = QuestionnaireResponse.item.where(linkId=”locationId”).answer.text (This is the id of the location)
task.status = “Ready” (We only want to query ready tasks because they can be adjusted. Completed and Cancelled tasks can not be changed in this way)
The system returns the results and each task is evaluated against the criteria.
If evaluated to true, the system changes the status of the task to Cancelled regardless of the plan that it was assigned to.
Next, the event submission is evaluated against the task.planIdentifier if the status is “active” and the effectivePeriod is still valid. This allows us to ensure that any downstream “create” plan.action.type actions are honored with this new entity.
Illustrative Example:
Unsynced events from Plan A hit the OpenSRP server.
Each event is evaluated against Plan A’s triggers.
(We need to decide if we need to augment the trigger in some way to ensure we don’t duplicate tasks by evaluating all actions and re-firing the “create” action.type in this step)
The Family Registration Event for Plan A, Location 123 in Jurisdiction X is evaluated against the entire Plan A
The “Cancel Duplicate Register Family Tasks” action trigger evaluates to true.
The system queries all tasks with the following criteria:
task.groupIdentifier = “Jurisdiction X”
task.code = “RACD Register Family”
task.for = “Location 123”
task.status = “Ready”
The system returns one task with this criteria where task.planIdentifier: “Plan B”
The condition is evaluated against this task and it evaluates to true.
The system reads the dynamicValue.expression.expression and identifies that we need to change task.status to “Cancelled” and task.businessStatus to “Cancelled-Duplicate” (or something like that)
The system saves the task change and queries the database for plans where plan.id = task.planIdentifier (“Plan B”), status = “Active” and the event dateTime falls within the plan.effectivePeriod
The system returns one plan
The event submission is evaluated against all “create” actions
The task creates a Blood screening task for the head of household and a bednet distribution task
Software to be Developed
This section outlines the software packages that need to be developed in order to deliver these features. The target is to break out these items into tangible chunks of work that can be scheduled and delivered within two weeks.
(This section is still a work in progress)
1. Develop Plan Evaluator Java Library
We need to develop a Java library that can be run both in the Android client and OpenSRP server. This library will be responsible for evaluating plans and generating tasks. The inputs vary depending on the location where the library is deployed. This library is responsible for consistently executing the logic that generates tasks.
Inputs:
planIdentifier - This is the identifier of the plan that needs to be executed
trigger -
(optional) Entity - This is an entity to be evaluated against the plan. This allows us to evaluate a specific
Triggers:
Server - This library gets triggered the single time that a plan transitions from status:draft to status:active
Android Client - This library gets triggered through an asynchronous call
Logic: (See section above)
2. Update Event Processor to Evaluate Plans
We need to update the event processor to evaluate plans for a given trigger.
3. Update App logic to make sure it’s truly dynamic
Sam G There are post save actions that executed on the device
Update of the task business status and status. This will be derived from the business_status hidden status
Evaluate tasks that should be generated after events are submitted
Create tasks that should be generated after events are submitted
Refresh the map or task list
4. Update
5. Add a the ability to define plan templates so the web UI can load these in
...
Don’t evaluate anything below this
...
Dynamic Task Generation Discussion Notes
We need to update the flow in the Android client so that tasks are generated based on the actions in the plan. In order to do this, we need to develop the following feature:
Evaluate all plan actions during post form save action
Develop a way to identify the template for tasks based on the action. In other words, if you have a “register family” action, provide a way to identify how to fill out that task before it’s generated
We will likely need to accomplish this by generating templates in the server that are synchronized down to the Android client.
Loosen the constraints on the plan action so that we can configure the action criteria and codes that are available in the server. This will allow us to provide actions on different intervention units.
Identify how to handle multiple tasks assigned to a single structure for different task actions. In other words, what should we do if action 1 creates a task on the structure and action 2 creates a task on that structure. Reveal doesn’t currently handle this in the Android client.
Support entity persistence in the local task generation process.
Link the task information to the configurable form name and version that’s developed for the configurable form feature
Entity Persistence Discussion Notes
This section includes notes on the Entity Persistence discussion
A “register family” task has been created against a location in the server, but the Android client already has a family registered against that location when the new plan is activated
User is operating on FI Plan1 offline
User has a task for Plan1 to register family for Structure A
User registers this family
Plan2 is created on the server
StructureA has a new task generated against Plan2
User goes online
Conflict:
The structure has two tasks associated with the same entity (family) against different plans
the task for the new plan should not have been generated (if the device had been online and the family registration had synced before the new plan was created)
Solution Options:
We could add logic server side when new clients are synced, we could analyze any tasks that exist against the family registration when the client hits the OpenSRP server and changes “status”: “Cancelled” for the task that was created.
We have an issue with follow-on task generation. So if you registered a family, you need to run through the plan again and generate the appropriate tasks.
Business Logic: Evaluate against a single entity server side.
We sync an event
For this specific event type, we evaluate it against all known plans that were active when the event was created
Identify if there are any tasks linked to it with a status “Ready” for any plan that is active (this includes doing a structure look-up)
We cancel the associated task based on the task.code and then immediately trigger plan evaluation for plans that are affected
Note:
It’s possible that users have done double family registrations.
It would be valuable to configure this in the plan, including adding a look-up for other tasks that exist
Events are synced by jurisdiction, so that means other devices have a copy of all events against that jurisdiction over time
We need to do this for the following eventTypes
Family Registration
Family Member Registration
Register_Structure
Feature:
Evaluate the family member event in the OpenSRP server
Evaluate the action against
The task that was first marked as Done first wins, the other task is archived?
Archive needs to cascade down to all devices
A point is added locally and it hasn’t been synced
Is this an issue?
Point in added locally
Task is created locally
Family is registered
Device syncs location, task and event to server
location is synced down to other devices
Problem: A new plan is generated and this point isn’t evaluated
We don’t provide access to perform life cycle events on locations (update, archive)
We don’t provide access to underlying families if we don’t have a register family task
Note
We need to consider the color coding for families that are registered but don’t have a task
We don’t have a clear process for handling duplicate registrations and tasks that are generated on multiple offline devices
The current strategy is latest wins. That should be fine, but we need a clear audit path to show what happened. If possible, see if we can reverse it.
Users change an entity in one plan and expects that to persist across other plans in the operational area when offline. For example, if a user has two plans in the same operational area and they register a family at structure A, they expect that family to show up in structure A in both plans.
if this is happening locally when offline, this should be catered for by removing the restriction on accessing entities without tasks. The location is associated to the jurisdiction, the entity (family) is associated to the location.
A user removes a family member in one plan and expects it to persist across plans
Ditto above
There’s a case
Business Logic
A “register family” task has been created against a location in the server, but the Android client already has a family registered against that location when the new plan is activated
Is it possible to generate the same task client and server side?
We don’t provide generic access to the underlying family or family member registered at a location regardless of task assignment for that particular plan.
We need to be able to
We need to execute business logic as follows in the Android client:
Issue: We don’t evaluate the difference in state between the OpenSRP server and the Android client upon sync. There’s an underlying Issue where we evaluate plans only once when they are made active based on the state in the server at that time. (PD: does ‘evaluate’ equate to ‘generate tasks for’?)
PD: What currently happens if there is a task of the same type for the same location, one generated server side and one generated on the client? Is this even possible? - should tasks be marked as 'generated server side or generated android? seems useful))
Option 1:
Evaluate the entity state of certain task types before they are opened. For example, if you have a register family task type, you could evaluate it and see that there’s already a relationship available for that structure. If so, run through the plan locally and generate the appropriate tasks.
Option 2:
Evaluate all tasks of a certain type when they are received by the Android client and update the state as part of the sync process
We don’t address state of entities across plans. In other words, the completion of certain types of events could trigger changes in other plans.
We aren’t sure if it should. I need help with this
e.g there a dynamic ‘ante-natal visit’ task created for pregnant women when a family member who is pregnant is registered….
If we stick to the set of entities that are persistent (locations, families, family members) then this is is not an issue - events are plan specific.
Ideally, an entity (specifically clients in this case UPDATE: I had originally thought that ‘Clients’ referred to families and family members only, but this seems to be incorrect as for IRS implementations the ‘sprayed_structure’ is a client, I don’t think spray information should be persisted) can be recreated by replaying all of the events that have that entity as the baseEntity (this is not currently the case as some client information does not get captured as part of the event)
Structures and Jurisdictions are also not generated through a task, ever. Register structure tasks are ‘after the fact’ events that are generated when a user drops a pin (Craig Appl is this true?)
It feel like there should be some events which are able to modify baseEntitiy attributes, and some that aren’t … does this fit into the DHIS2 paradigm of tracked entities with attribute values, who have events with data elements ? in that case, events CANNOT modify tracked entity attributes - it is a different process to set the attributes
Issue: We don’t provide generic access to the underlying family or family member registered at a location regardless of task assignment for that particular plan
Logic:
Any user should be able to tap on any location to open a view of the family regardless of task assignment.
We need to determine the impact of this across plans and in the reporting system
We don’t have a clear process for handling duplicate registrations and tasks that are generated on multiple offline devices
Logic:
We don’t allow users to archive locations or change the state of a location
Logic:
Any user should be able to tap a location regardless of task status, view and edit information about the location.
We need to determine the impact of this across plans and in the reporting system
Features
Identify a subset of post form save actions that create, update, or archive an entity in the system that should be persisted.
When the form is saved, go through all plans that have an active start and end date and represent the update state based on the actions in that plan.
For example, if a family is registered in plan A, query the database for any tasks that are assigned to that structure with code “register family” and mark them as complete. Then, identify the plan associated with those tasks, evaluate the actions and
This may pose conflicts where a plan already has a family registered. We need to handle these conflicts when they arise.
Make sure all persisted entities sync from the Android client to the server including locations.
Add the ability to edit and archive non-jurisdiction locations in the Android client so they are archived and do not show up in the future.
(Nice to have) Remove locations that do not have an active task in the Android client. They currently show up as grey.
Alternatively, we could find a way to display information about the location and possibly allow the user to assign a task to that location
Brainstorming Section
Thailand Specific
Recently Pedro highlighted that a routine FI plan had been created on the WebUI that only included the Register Family task.
When users looked at the plan on the Android device, a number of structures were marked as grey and could not be accessed
Ona investigated and validated that no tasks for those structures were created as they already had families registered
Should these structures still have tasks created, with a status of ‘Completed’ so that they show up as green rather than grey, and can be accessed to view and edit family information?
Does it make sense to have these tasks with no corresponding events?
Should the original family registration event be linked to the task somehow?
General Placeholder
Craig
Can we use the same logic in the server in the Android client to evaluate entities when an event is evaluated?
The core assumption is yes, we just need to make sure we architect it correctly.
Thinking through sample plan actions and criteria
Register Family Action
Description: This action focuses on registering families if they don’t already exist at each residential structure
subjectCodableConcept: location
Criteria
location.type=”residential_structure” AND
(location.status=”active” or “pending”) AND
a family isn’t registered against this structure AND
a task hasn’t already been generated for this plan and action combination
Notes:
It would be easier from a logic perspective for end users if we keep the subjectCodableConcept to a resourceType such as on jurisdiction, location, family, family member and then add that to a criteria
Complex MDA example
Mass Drug Administration Round 1 Administration Action
Description: This action supports the initial administration of the medication to the person
subjectCodableConcept: person (family member?)
Criteria
person.dob>=today - 5 years
Mass Drug Administration Round 1 Adherence Action
Description: This action supports the adherence follow-up of the medication to the person
subjectCodableConcept: person (family member?)
Criteria
person.dob>=today - 5 years AND
person has an event for MDA Round 1 Administration
Mass Drug Administration Round 2 Administration Action
Description: This action supports the second administration of the medication to the person who already received round 1
subjectCodableConcept: person (family member?)
Criteria
person.dob>=today - 5 years AND
person has a complete task with business status=Adherent for task.focus=action.id for the Mass Drug Administration Round 1 Administrative Action
Notes:
I don’t know if we want to set the criteria to see if an event exists or if a task exists with the complete status
Links to Dynamic Forms
We are adding the ability to define the form name in a plan action so that we can link that particular action to the data collection instrument. This will be in the action.definitionUri.
We need to create a table in the SQLite database that can be quickly referenced to identify which forms should be opened by which task
task.focus links to the plan.action.id.
We need to create a table that updates for each plan action that’s downloaded to the device that links the task.focus to the plan.action.definitionUri so that we can open the right form in a performant way.
We will not define the form version in the plan. That will be managed by the configurable forms manifest file.
We should be able to identify the difference of jurisdictions before it’s saved and generate tasks for that plan
Register Family Logic Brainstorming
This section is the result of a conversation between Sam G an Craig on 23rd June 2020
We have three different scenarios where we need to evaluate and generate a register family task.
planActivation trigger, when the location is passed in to the evaluator
eventSubmission trigger when a “Register_Structure“ eventType is submitted
eventSubmission trigger when a “Archive_Family” eventType is submitted
The event is passed into the eventSubmission as a QuestionnaireResponse
We need to build the logic for three cases within the condition to accommodate for these scenarios.
($this.is(FHIR.Location) and $this.properties.type = 'Residential Structure' and $this.hasFamily().exists() = false) or ($this.is(FHIR.QuestionnaireResponse) AND (($this.questionnaire='https://fhir.smartregister.org/questionnaire/Register_Structure' and $this.item.where(linkId='structureType').value = 'Residential structure') or ($this.questionnaire='https://fhir.smartregister.org/questionnaire/Archive_Family')))
The first section evaluates whether the passed in item is a location with the residential structure. Then, it evaluates whether it’s a questionnaireResponse. Then, it evaluates if the questionnaire response is a residential structure if it’s a Register_Structure eventType.
Reveal Location to FHIR Location mapping
This section maps the Reveal FHIR location elements to the FHIR specification
Reveal Location Entity | FHIR Location |
|---|---|
type | N/A |
“resourceType”:”Location” | |
id | id |
geometry | (not represented) |
properties.status | status |
properties.parentId | partOf |
properties.name | name |
properties.geographicLevel | |
properties.version | meta.versionId |
properties.OpenMRS_Id | identifier.[{“system”:”OpenMRS_Id”,”value”:””}] |
properties.externalId | identifier.[{“system”:”externalId”,”value”:””}] |
properties.name_en | location.alias |
serverVersion | meta.lastUpdated as YYYY-MM-DDTHH:mm.sss+00:00 |
isJurisdiction? | true = physicalType:jdn false=physicalType:bu |
“mode”: “instance” |
Existing Application Logic that needs to be reworked
...