This document defines the software requirements for the distribution point features for Reveal. A distribution point is a centralized location where individuals meet to receive care. The primary workflows are funded under a grant for Neglected Tropical Diseases (NTDs) which focuses on distributing medication among children who are attending schools in Eswatini. The functional requirements are outlined on NTD School Workflow and the data dictionary. This document applies those functional requirements into the Reveal technical architecture.
...
(Outside of the system) The user updates a template based on the structure that’s defined for the project
User logs into the system
User chooses a plan (draft or active)User chooses a distribution that they want to upload the CSVselects “Admin” in the navigation and finds “student lists”
User clicks a button to find the file on their computer
User clicks the upload button and the CSV is uploaded to the server
Details are passed in with the CSV to show what types of entities to generate and the mapping of the template.
CSV is parsed server side and validated (See CSV Validator section)
An error is returned to the user if the CSV is invalid or it doesn’t pass a security check
An event is created for each row in the OpenSRP server
Each event is tagged to the plan and location
A task is created for each even that’s uploaded if the plan is active. Otherwise, tasks are created when the plan becomes active.
(Existing functionality) A client is created in the Android client for each event when it’s received
The
...
We need to build a REST API in the OpenSRP server user can upload more than one csv - list of all csv uploads shown
User can download a csv of students - they can do this at each level of the location hierarchy down to school
Mock created by Roger for csv upload functionality:
...
The OpenSRP server needs to be enhanced in the following ways in order to deliver this workflow:
We need to build a REST API in the OpenSRP server that can receive the CSV file and perform the event generation
We need to build the foundation for a server side task generator in OpenSRP that can generate tasks as a service when the CSV is uploaded. This task generator will be improved upon in the coming months so that we can move the Nifi flows that generate tasks for Focus Investigation, IRS and MDA to Java server code so that the task generation system functions better.
Note that this will involve engaging the queuing software in OpenSRP so that we don’t overwhelm the server with a spike of activity.
We need to build an import template pattern that is configurable for OpenSRP. This would allow us to create a server side configuration that validates the template that was received and create the events. Ultimately, this will be able to be used generically to import events and, optionally, tag them to the appropriate jurisdiction and plan.
We need to be able to define settings that map the event/client metadata tags to the labels in the CSV template, sync them down to the Android client and make them available for display and filter in the Android client. We have three points that need to be dynamically assigned in the app. We need to “tag” each client in the register view to up to 3 of those points so they can be searched against.
See the subsection labeled CSV Upload to Event Generator
We need to be able to generate clients by using a CSV upload. Each row will present an event
...
How do we roll back these items in the event someone messes up?
We need to define the logic behind generating
We need to define the error handling.
(Business question) Users don’t have a view into all of the entities that are registered to a distribution point. Therefore, they have no idea if their list was uploaded correctly. How should we mitigate this?
In other words, how will we handle mistakes when a user uploads information that they don’t want to be there. What’s the life cycle we should create to remove the likelihood of duplicates?
We need to come up with unique id generation so that we can
We are building a bulk event generator in this CSV upload process. It would be valuable to support JSON format as well as CSV.
We need to determine if we should have different variations of the We are building a bulk event generator in this CSV upload process. It would be valuable to support JSON format as well as CSV.
We need to determine if we should have different variations of the template and optional fields so that we don’t need different templates here.
Consider removing the upload after the plan is generated.How will we provide a list of children to the user.
OpenSRP Web UI
We need to
...
Do we generate a task for each child and mark not eligible kids as not eligible, or not generate a task for them?
OpenSRP Web UI
We need to develop the following develop the following features in the OpenSRP Web UI in order to deliver the features above.
...
Dashboards will follow the current IRS model
Drill down to school level with the same set of stats (Region, Inkhundla, School)
At the school level, cut statistics by age category
6-10
11-15
16-18
The reporting needs are defined in the data dictionary from Akros. This will require changes to the Canopy flow to make sure the data is available to the data warehouse and Superset.
CSV Upload → Event Generator
...
Web UI Reporting Needs
Dashboards will follow the current IRS model
Drill down to school level with the same set of stats (Region, Inkhundla, School)
At the school level, cut statistics by age category
6-10
11-15
16-18
The reporting needs are defined in the data dictionary from Akros. This will require changes to the Canopy flow to make sure the data is available to the data warehouse and Superset.
Canopy Components
A NiFi and Superset instance will need to be deployed (along with all necessary backend data stores) to support the OpenSRP Web UI dashboards and task generation features. The NiFi instance will have two main flows:
Latest version of the Reveal OpenSRP connector, or a generic version if at all possible
Customized NTD task generation logic. Initially this will be single-pass generation of all tasks when a plan becomes active.
If it is impossible to guarantee all clients affected by a plan will be registered in OpenSRP when the plan becomes active, a second iteration which does some form of dynamic task generation in response to client changes will need to be researched and built.
The Superset instance will have the following slices exposed:
Aggregate target statistics per juridiction+level
Non-aggregated target statistics
Due to the the complexity of these statistics, they will probably need to be recomputed on a (smart or dumb) periodic basis vs aggregated for every request. As a first iteration a simple time interval will be used, but if this is not sufficient we can iterate with marked-as-dirty tables and triggers or other solutions.
CSV Upload → Event Generator
We need to develop a server side event generator that generates events server side for each row that’s available in the import. We aim to make the CSV event generator generic so that we can import different types of entities into OpenSRP in the future. We need to reference the event template with the CSV so that we can identify which fields should be mapped to which events when they are created. This template should be defined within the OpenSRP server so that it’s CSV to event objects or make it so that system administrators can configure these entities and evolve their generation over time.
...
Perform a security validation to reduce risks on import (see the Security section below)
Validate the structure of each field in the template
(Stretch) identify if there is an existing client in the system with the same information. Provide feedback to the user.
...
information. Provide feedback to the user.
We do not allow the user to delete children from the system using the csv upload because that could affect active plans and reporting. We do, however, allow them multiple csv uploads in order to add children that have been missed or update information on children already in the system.
In order to prevent duplicates, the user will be asked to download the csv roster from the system, make changes or add students in that sheet, and re-upload it. This downloaded csv will contain a unique ID for all students already in the system. When that csv is uploaded, the system will run a check for these IDs and update those children’s data, but not create new entities.
Plan Definition and Task changes
...
This section defines the business logic for each task type that needs to be executed when we generate tasks either in the Android client or OpenSRP server/Nifi.
MDA Dispense
...
These tasks are generated in the OpenSRP server when the plan is made active.
...
One task is generated for each eligible child registered at each distribution point
Child eligibility is defined as
Isabel Shaw please complete this
...
One task is generated in the Android client when a child is registered following the child eligibility above
Business Status and Color coding
...
server/Nifi.
MDA Dispense
These tasks are generated in the OpenSRP server when the plan is made active.
One task is generated for each eligible child registered at each distribution point
Child eligibility is defined as
Generate task if 6.0 years<=age<=18.0 years
One task is generated in the Android client when a child is registered following the child eligibility above
Business Status and Color coding
Status one [Green] = Child is eligible, has a Mass Medicine Administration form, and mmaDrugAdmin = 'Yes' (aka drugs were in fact dispensed)
Status two [Red] = Child is eligible, has a Mass Medicine Administration form, and mmaDrugAdmin = 'No' (aka drugs were not dispensed)
Status three [Yellow] = Eligible, and does not have a Mass Medicine Administration form - this is the default status
MDA Adverse Event
These tasks are generated in the Android client only when an adverse event form is complete
One task is generated after the form is saved and the status is marked as complete immediately
Business Status (No Color Coding in the system) (Data element from dictionary)
Severe Reaction
Moderate Reaction
Mild Reaction
Changes to the Nifi Flow
...
Users should not have the capability to download large lists that contain personally identifiable information about the clients.
Access to download rosters will, at maximum, provide identifiers that can be linked to people registered in the system.
All uploads need to be validated to include the following:
Valid CSV architecture with UTF-8 encoding
Individual fields are parsed and validated to remove exposure to SQL injection threats
Individual fields are parsed and validated to remove exposure to cross-site scripting threats
(We probably need to add more here)
Things to be discussed from scoping so far
We need to validate the assumption that we will support importing rosters against active plans.
...
Do we only import rosters against active plans?
...
Should we support importing rosters against plans in draft state?
...
Out of Scope
Linking people who were registered in the distribution point workflows to families or structures
...