Procedure for migrating to faster forms

The OpenSRP client uses a light weight version of enketo core to render forms (https://github.com/OpenSRP/enketo-opensrp), the bundle constitutes enketo core and ziggy (https://github.com/OpenSRP/ziggy) enketo core is in charge of xform rendering while ziggy does data binding and database level operations.

Since initial release OpenSRP uses enketo core as the engine to render the xls authored forms this has worked well over years and is now an integral part of OpenSRP, however there has been some performance related concerns hence the need to develop a faster form technology. The new form technology still uses enketo core to render the xforms since its still crucial maintain xls form authoring, however unlike the previous approach the new approach uses static template of enketo core library. The library is loaded to android assets folder and accessed whenever required at runtime. The save logic is also moved outside the form such that the xml submission obtained from the form is first converted into a form submission then handed over to ziggy which in turn inserts the items to the respective tables based on the declared bind type.

The procedure for rendering the form reads the static html together with its javascript and css resources from the assets folder then injects the model and form string. e.g if we are rendering form1 then form1 model and form tag is injected to static template.

If you have already have an OpenSRP client implementation and you want to migrate to the new form technology the procedure for doing so is as follows:

The new form loading involves preloading of Fragments inside a viewpager, so all Activities should extend android.support.v7.app.ActionBarActivity instead of Activity

SecuredActivity.java

public abstract class SecuredActivity extends ActionBarActivity {
	// class specific code here
}

The associated activity syle is also changed to be descendant of AppCompact theme e.g on the android manifest make the following changes:

AndroidManifest.xml

		<activity
            android:name=".view.activity.NativeECSmartRegisterActivity"
            android:screenOrientation="landscape"
            android:theme="@style/AppThemeNoTitle" />

Where style/AppThemeNoTitle is defined under styles.xml file as:

styles.xml

	<style name="AppThemeNoTitle" parent="Theme.AppCompat.Light.DarkActionBar">
        <!-- Customize your theme here. -->
        <item name="android:windowNoTitle">true</item>
        <item name="android:windowActionBar">false</item>
        <item name="android:windowFullscreen">false</item> <!-- this is useful to prevent the keyboard from covering up the input fields -->
        <item name="android:windowContentOverlay">@null</item>
	</style>

The logic that is contained in the initial activity that loads the form is then moved to a fragment, the base fragment, the fragment should be as self sufficient as possible i.e only the functions that are handled better by activity rather than fragment are left, the rest are moved to the fragment.

see org.ei.opensrp.mcare.fragment.HouseHoldSmartRegisterFragment; most of the the code moved over from the initial activity i.e org.ei.opensrp.mcare.household.HouseHoldSmartRegisterActivity.

The other modification required by the activity goes to manifest file i.e since we are switching the activity orientation during runtime depending on the fragment on display, we need to prevent the activity from starting a fresh each time (would otherwise cause an exception)

AndroidManifest.xml

		<activity
            android:name=".household.HouseHoldSmartRegisterActivity"
            android:configChanges="keyboardHidden|orientation|screenSize"
            android:theme="@style/AppThemeNoActionBarAndTitle" />

The default form launch functionality is also overridden such that we show the FormFragment instead of launching a new activity whenever a form is requested, the new form launch functionality accepts three parameters:

formName - the name of the form we want to launch
entityId - the id of the record we want to load to the form e.g if we are editing
meta data (fields overrides) - data structure that holds the fields we would like to override

xRegisterActivity.java

@Override
public void startFormActivity(String formName, String entityId, String metaData) {
    Log.v("fieldoverride", metaData);
    try {
        int formIndex = FormUtils.getIndexForFormName(formName, formNames) + 1; // add the offset
        if (entityId != null || metaData != null){
            String data = null;
            //check if there is previously saved data for the form
            data = getPreviouslySavedDataForForm(formName, metaData, entityId);
            if (data == null){
                data = FormUtils.getInstance(getApplicationContext()).generateXMLInputForFormWithEntityId(entityId, formName, metaData);
            }

            DisplayFormFragment displayFormFragment = getDisplayFormFragmentAtIndex(formIndex);
            if (displayFormFragment != null) {
                displayFormFragment.setFormData(data);
                displayFormFragment.loadFormData();
                displayFormFragment.setRecordId(entityId);
                displayFormFragment.setFieldOverides(metaData);
            }
        }

        mPager.setCurrentItem(formIndex, false); //Don't animate the view on orientation change the view disapears

    }catch (Exception e){
        e.printStackTrace();
    }

}

If the entityId parameter isn’t null the provided id is used to preload the form with the entities’ data.

org.ei.opensrp.util.FormUtils class contains the logic needed while saving/retrieving the form, while saving the form the class converts the xml input obtained from the form into a form submission which can be handled by ziggy. When retrieving the form, e.g if the form needs to be preloaded with existing data, the class has utility method that uses the supplied id parameter to build an xml input string to be supplied to the form.

Navigation to and from the FormDisplay is achieved by simply switching the viewpager index to show the desired fragment, e.g initially the register activity shows the base fragment (the fragment with the records list), when new form flow is triggered the view pager index is moved to point to the FomFragment, 1 if its single form flow, or n if we can load multiple forms from the same base fragment.

Example in code is:

SmartRegisterActivity

mPager.setCurrentItem(1, false); //Don't animate the view on orientation change the view disappears

Form display fragment

The form display fragment contains the logic required to load, edit and save a form. the important fields are:

DisplayFormFragment

	private String formName;
   
    public String getFormName() {
        return formName;
    }

    public void setFormName(String formName) {
        this.formName = formName;
    }

Whenever a new DisplayFormFragment is initialized the formName is also set (this parameter is Required)

DisplayFormFragment

	String formData;

    public void setFormData(String data){
        this.formData = data;
    }

    public void loadFormData(){
        formData = formData != null ?  formData.replaceAll("\"","\\\"") : "";
        webView.post(new Runnable() {
            @Override
            public void run() {
                webView.loadUrl("javascript:loadDraft('"+ formData + "')");
            }
        });
    }

The above method is used whenever we need to preload data to the form, the first method setFormData sets the xml string required to be used. loadFormData method is then triggered to reload the form with the data.

The fragment also contains javascript interface for communicating to and fro between the fragment/activity and the enketo form, currently there are only two functions:

showFormErrorToast - get called if user attempts to save a form that has errors.
processFormSubmission - gets called whenever save operation is triggered.

DisplayFormFragment

	
public class MyJavaScriptInterface {
    Context mContext;

    MyJavaScriptInterface(Context c) {
        mContext = c;
    }

    @JavascriptInterface
    public void showFormErrorToast(){
        // code to show an error message if the user attempts to save a form that contains errors 
    }

    @JavascriptInterface
    public void processFormSubmission(String formSubmission){
        showTranslucentProgressDialog();
        ((SecuredNativeSmartRegisterActivity)getActivity()).saveFormSubmission(formSubmission, recordId, formName, getFormFieldsOverrides());
    }

    @JavascriptInterface
    public void javascriptLoaded(){
        //Toast.makeText(mContext, "Javascript loaded", Toast.LENGTH_LONG).show();
        javascriptLoaded = true;
    }

    @JavascriptInterface
    public void savePartialFormData(String partialData){
        //Toast.makeText(mContext, "saving un-submitted form data", Toast.LENGTH_LONG).show();
        ((SecuredNativeSmartRegisterActivity)getActivity()).savePartialFormData(partialData, recordId, formName, getFormFieldsOverrides());
    }

}

FormUtils.java class

FormUtils is a utility class concerned mainly with two main roles:

When saving the form it takes the xml input and converts it to a form submission which ziggy can save to the database
When editing a form it reads the record being edited from the database and builds an xml input string that the form can ingest

The two functions are:

FormUtils.java

	/**
 	*  Generates a form submision from a given xml input string
	*
 	* @param entity_id - id specified then the record with the given is used to preload non specified values
 	* @param formData - the xml input received from the form
 	* @param formName - the name of the form, determines which form definition we should load 
 	* @param overrides - the field overrides 
 	* @return
 	* @throws Exception
 	*/
	public FormSubmission generateFormSubmisionFromXMLString(String entity_id, String formData, String formName, JSONObject overrides) throws Exception{
        JSONObject formSubmission = XML.toJSONObject(formData);
        System.out.println(formSubmission);
        // use the form_definition.json to iterate through fields
        String formDefinitionJson = readFileFromAssetsFolder("www/form/" + formName + "/form_definition.json");
        JSONObject formDefinition = new JSONObject(formDefinitionJson);
        // ...
    }

	// ...

	/**
 	* Builds an xml input string that can be passed to an enketo form
 	* 
 	* @param entityId - the id of the entity being generated, if not null the record with matching id is used to generate the xml string
 	* @param formName - the name of the form being edited/displayed
 	* @param overrides - overrides for default field values 
 	* @return
 	*/
	public String generateXMLInputForFormWithEntityId(String entityId, String formName, String overrides){
    	try{
     		//get the field overrides map
   			JSONObject fieldOverrides = new JSONObject();
       		if (overrides != null){
        		fieldOverrides = new JSONObject(overrides);
        		String overridesStr = fieldOverrides.getString("fieldOverrides");
        		fieldOverrides = new JSONObject(overridesStr);
       		}
       		// use the form_definition.json to get the form mappings
            //...
    }