Logical Walk Through of Client and Event Information

Owned by Craig Appl
Last updated: Mar 07, 2018Version comment

As outlined in the End-to-End Data Flow document, data moves bidirectionally between the Android client, OpenSRP server and OpenMRS. Data is represented in different structures, databases and data models in each of these systems. This document provides a logical walk through, with examples, of a standard client record and their encounters from the point of origin to the OpenMRS and reporting systems.

Note: This walk through is based on the event client data model that is deployed in the Zambia Electronic Immunization Registry (ZEIR). Other implementations may be following a different model.

All Event Types:

  • Out of Area Service - Vaccination
  • New Woman Registration
  • Birth Registration
  • Out of Area Service - Growth Monitoring
  • Vaccination
  • Growth Monitoring
  • Recurring Service
  • Death
  • Move To Catchment
  • Update Birth Registration
  • AEFI (Adverse Event Following Immunization)


Steps:

  • Android Client:
    • Child Registration Form:
      • When a child registration form is created, the form is saved as an event with eventType "Birth Registration"
      • The form is saved as JSON and a number of processes kick off:
        • A child is created in the ec_child client table based on that birth registration form. The child is assigned a zeir_id.
        • A mother is created in the ec_mother client table based on that birth registration form. The mother is assigned a m_zeir_id in the database.
        • A relationship is created in ec_child client table pointing to ec_mother client table.
        • The view is updated and the child is shown in the register.
    • Update Child Registration Form:
      • Any updates to the child registration form are captured as events as eventType "Update Birth Registration"
      • The changes are stored in the event as JSON and any changes to the child or mother update the associated clients in the clients table(ec_child or ec_mother)
    • Capture Weight or Length:
      • When the user captures a weight or length, an event is saved in the system as eventType "Growth Monitoring"
      • This weight is stored in the event table as JSON and also in the weights table in a normalized form by the Client Processor
    • Recurring Service:
      • When the user captures a recurring service like Vitamin A supplement, Deworming and ITN, they are stored in the events table as eventType "Recurring Services"
      • Each recurring service is stored as JSON in the event table and also in the recurring_service_records table in a normalized form by the Client Processor
      • Business logic is performed to update the schedule and view
    • Vaccinations:
      • When a user captures a vaccination event, the event is stored in the events table as eventType "Vaccination"
      • Each recurring service is stored as JSON in the event table and also in vaccines table in a normalized form by the Client Processor
      • Business logic is performed to update the schedule and view
  • Sync:
    • On a regular interval the Android client syncs with the OpenSRP Server through the REST API
    • The device pushes all of its events with each event being accompanied by its respective client to the server
      • When the event hits the OpenSRP server, it is assigned a serverVersion that is effectively a millisecond timestamp so we can query for all events after a particular time
      • The event also gets an event id that when its stored in couch, this is used as the primary key on the client side
    • The device then queries the server to pull down all events that it should know about.
      • This query is generated from the Android client based on the user ID, location IDs associated with the user's team and the last sync time (serverVersion)
    • All events that have happened based on the query credentials are compressed and returned back to the Android client
    • The Android client receives these events and applies them to the local SQLite database and runs the Client Processor to extract the various properties from the EC models and save in the nomalized tables as explained above for each event Type
  • OpenSRP Server:
    • A number of actions happen for each client and event that is received by the OpenSRP server:
      • For each birth registration or update birth registration OpenSRP pushes the form(client) to OpenMRS to create a new person and patient as well as a patient relationship between the mother and child.
      • For each event, OpenSRP server pushes an encounter with the associated encounterType/eventType and observations to OpenMRS
    • Each time a person is created or an encounter is created in OpenMRS, it raises an entry in the OpenMRS atom feed
    • On a regular schedule, OpenSRP server queries the patient and encounter atom feeds and performs processes on each:
      • For each new entry in the atom feed, OpenSRP queries the OpenMRS REST API
      • The person or encounter is returned and OpenSRP queries the couch database to see if that person already exists in the client table or if the encounter exists in the event table.
      • If they don't exist, OpenSRP creates the client or event.
      • If the person already exists, OpenSRP updates the identifier information in the Client table.
  • ETL:
    • On a regular schedule, an ETL process is run to flatten the Event Client model for data analytics.
    • This ETL queries the OpenSRP Server's couch database and stores information in PostgreSQL database
    • On a regular schedule (nightly), CSV files are dumped from the PostgreSQL and forwarded to the data analytics platform