End-to-End Data Flow

The Implementer's Guide front page describes the high level architecture and systems used in OpenSRP. This document defines the movement of data across the different systems from a user's perspective. Technical details should be available in the Developer's Guide.

Core Architecture (Android App, OpenSRP Server, OpenMRS)

The core OpenSRP implementation includes the Android App, the OpenSRP Server, and OpenMRS. The Android App and OpenMRS are the two primary locations for data entry in this core implementation. The OpenSRP server acts as the mobile deployment and synchronization manager between these systems. It's also appropriate to note that the OpenSRP server aims to have 100% replication of information between the OpenSRP Server and OpenMRS server. In other words, all transactions that happen in OpenSRP server are pushed to OpenMRS and all transactions in OpenMRS are pulled into the OpenSRP server. This synchronization ensures that all systems are up to date at all times.

Connections

There are multiple technologies in use to make synchronization possible. The Android App to OpenSRP server connection uses a REST Application Programming Interface (API) to exchange information with the OpenSRP Server's CouchDB. These technologies provide an automatic synchronization system that provides two-way data synchronization between the Android app and the OpenSRP server. 

The OpenMRS to OpenSRP connection uses two technologies, atom-feed and REST API. OpenMRS has a robust REST API that allows developers to interact with the information in OpenMRS in a structured way. The REST API allows other computers to GET data from OpenMRS and POST data to OpenMRS using well documented commands. The OpenSRP server does just this. Every time there is a change in the OpenSRP server, it POSTs data to the OpenMRS server. If it needs to get a single piece of information it can also GET data from OpenMRS.

OpenMRS has an atom-feed technology that constantly monitors changes to the database. Every time there is a change in OpenMRS, the atom-feed module creates a record and posts it to the atom-feed webpage that's read by the OpenSRP server every few seconds. When the OpenSRP server reads that something has been changed, it queries the REST API to get the change and updates the OpenSRP database. When there's an update to the OpenSRP database, there will also automatically be an update to the Android App the next time the Android device synchronizes.

Sample Data Movement - Android App to OpenMRS

When a document is stored on the mobile device and there's a sync, the Android App pushes the change to the OpenSRP server and queries for updates, making sure the device is 100% synchronized with the data it should have access to that's stored in the OpenSRP server. The diagram below shows how data moves from the Android App to OpenMRS:

Web Sequence Diagram Code
title Android App to OpenMRS

FLW->Android\nApp: Save patient form
Android\nApp->+Android\nApp: Recognize change and initialize sync
Android\nApp->+OpenSRP\nServer: Send new data from app
OpenSRP\nServer->OpenSRP\nServer: Receive data and store in database
OpenSRP\nServer->OpenSRP\nServer: Identify data not in app
OpenSRP\nServer->-Android\nApp: Send new data not in app
Android\nApp->-Android\nApp: Receive new data and store in database

OpenSRP\nServer->OpenSRP\nServer: Identify and convert data to OpenMRS format
OpenSRP\nServer->+OpenMRS: POST data to REST API
OpenMRS->OpenMRS: Store data in the database
OpenMRS->-OpenSRP\nServer: Respond with success

Sample Data Movement - OpenMRS to Android App

When a record is changed in OpenMRS, the OpenSRP server reads a change in the atom feed and initiates a REST query to get the latest information. The following diagram starts on the right side with a change in OpenMRS by an Admin User, moving right to left as you proceed down the diagram.

Web Sequence Diagram Code
title OpenMRS to Android App

participant Android\nApp
participant OpenSRP\nServer
participant OpenMRS
participant Admin\nUser

Admin\nUser->+OpenMRS: Creates Patient in OpenMRS
OpenMRS->OpenMRS: Save patient in the database
OpenMRS->-OpenMRS: Update Atom Feed
loop Every 10 seconds
    OpenSRP\nServer->OpenMRS: Read atom feed for changes
end
alt if change in atom feed
OpenSRP\nServer->+OpenMRS: Query REST API for change
OpenMRS->-OpenSRP\nServer: Return data
OpenSRP\nServer->OpenSRP\nServer: Convert and store data in database
end alt
note over Android\nApp: Initiate Sync

Android\nApp->OpenSRP\nServer: Send new data in app
OpenSRP\nServer->Android\nApp: Send new data not in app
Android\nApp->Android\nApp: Store data in database

Connecting Other Systems

The core system architecture (Android App, OpenSRP Server and OpenMRS) can be extended to include a number of other systems that allow for new workflows and cross-system reporting. This section includes information on known workflows and connections and can be extended in the future. Currently, we are able to connect OpenMRS to DHIS2, Electronic Data Warehouses (EDWs) and RapidPro. Each of these connections adds value to an implementation.

OpenMRS and DHIS2

OpenMRS and DHIS2 have a number of connection points based on the needs of both communities who implement these platforms. OpenSRP's use of OpenMRS allows implementers to push aggregate reports from OpenMRS to DHIS2 using the OpenMRS reporting module and DHIS Report module. The OpenMRS reporting module allows users to create aggregate reports as Data Set Definitions based on the patient information that's stored in OpenMRS. These data set definitions are made available to the DHIS Report module and mapped to the appropriate indicators in DHIS2. Users are then able to push their reports directly from OpenMRS to DHIS2 after reviewing them in the OpenMRS user interface.

ETL

It's appropriate to note some challenges faced by reporting on the OpenMRS transactional data model. The OpenMRS data model was built to allow for quick retrieval of data in the patient-provider interaction. It stores all patient information in a fixed set of tables that are excellent for extracting one record at a time, but not optimized for reporting at national scale. Due to this, a number of OpenMRS implementations have created reporting solutions that extract, transform and load (ETL) the data into a different database structure or a third-party reporting tool. A number of OpenSRP implementations are working on ETL-based solutions at this time and we expect to have a set of standard solutions for OpenSRP implementers in the future.

OpenSRP Server and DHIS2

OpenSRP server is able to push data sets in to DHIS2 that were directly created from the user's Android App. This reporting feature allows frontine users to submit monthly reports that were previously only able to be submitted on paper. Additionally, OpenSRP server is able to push individual client information directly into DHIS2 tracker.

Electronic Data Warehouse, OpenSRP Server and OpenMRS

Electronic Data Warehouse (EDW) technology is able to connect to the OpenSRP Server and OpenMRS databases. It uses an ETL process to extract individual patient data from these systems into their server for processing and analysis. EDW provides a graphical interface for understanding all data that's in both systems, allowing for cross-platform reporting. When connected, EDW can provide dashboards on the implementation, based on information from the OpenSRP server, track individual patient records over time based on OpenMRS and create ad-hoc reports through a user interface. We should note that EDW isn't the only reporting system that's able to do this task. Other systems like Jasper Reports and Pentaho are also able to function in this way.

RapidPro with OpenSRP Server

RapidPro is able to connect directly with the OpenSRP server through form submissions. RapidPro is able to query the OpenSRP server for new information, collect information from the end user and push the data to the OpenSRP's form submission endpoint. This allows frontline health workers to interact with the data through SMS, Facebook Messenger, and the Telegram app. This integration also opens up a number of new direct-to-patient workflows that could allow patients to have access to their child's record and be reminded of their appointments. This integration focuses on sharing individual patient records that are in the OpenSRP Server and does not interact with OpenMRS.