Question Types Explained

The best way to understand the different question types is to open up the Sample Form which has a question of each type

Text

A text field. Allows you to enter free text as the data input.

The mobile and web client will allow you to enter as much text as you want, and it will get saved to the database. However, you will not be able to process responses greater than 255 characters (i.e. you will see the red warning for unprocessed data and you will not be able to see your responses in the Dashboard). See Known Issues for further information.

Number

Accepts integers or whole numbers – so 1235 is valid but 12.23 is not. Negative numbers are accepted. Only ne

There is a limit to the size of integer accepted that is documented here as with Text a number longer than 10 digits will be accepted and saved, it just can’t be exported by default.

Decimal

Accepts numbers including numbers with decimals like 123.345

Date

Accepts a date.

On the mobile client, a prompt with a Select Day option will be available that takes the user to a calendar, the exact format depends on your phones settings
On the web form, a pop-up calendar will appear

The display and submit Date Format can be changed through the Admin Console -> Settings

Time

Accepts a time.

On the mobile client the format is HH:MM or HH:MM AA (where AA = am or pm) – depending on your phone settings
On the web form the format is HH:MM:SS AA

The display and submit Time Format can be changed through the Admin Console -> Settings

Date and Time

Both Date and Time must be recorded for one question answer

Boolean

Yes / No – user must select one or the other. Using this is the same as creating a Single Select question with two options "Yes" and "No." It is a useful shortcut for english forms, but as the Yes and No are not editable it is not useful for non-English forms.

Single Select

Add a series of options to this question type and the user can only pick one.

On the mobile phone it is presented as a list with a "radio" button
On the web form it is presented as a dropdown box

Multiple Select

Add a series of options to this question type and the user can pick as many as they want

Repeat

A repeat question contains a group of questions that can be repeated multiple times. Clicking Add Child will add a new question. In a web form, repeats are shown by default as rows of a table where new rows can be added
On the mobile client, the option New is available which takes the user into a new list of questions, this can be repeated as many times as necessary

Tip: it is possible to validate the "length" of a repeat question to see how many rows or sets of answers you have – see the sample form for an example.

Picture

Image question

On the mobile phone this accesses the phones camera funcitonality
On the web capture, this allows the upload of a picture

Video

Video question – video format must be 3gp

On the mobile phone this accesses the phones camera in movie mode
On the web capture, this allows upload of a file – it does not check on upload whether the file is the correct format or not.

Audio

Audio question – audio format must be 3gp

On the mobile phone this accesses the phones microphone
On the web capture, this allows upload of a file – it does not check on upload whether the file is the correct format or not.

Single Select Dynamic

Single Select Dynamic questions present a shortlist of a longer list of options, based on answers to previous questions. Single select dynamic questions require a Single Select Question to be present before they can be used.

A simple example is to narrow down a list of countries by first selecting a continent. The continent question would be a standard single select question. The country question would be a Single Select Dynamic question. You add all the countries of the world, but you map them to a continent. Once the user has selected a continent, only countries in that continent are displayed.

Single Select Dynamic questions appear like Single Select Questions

There is no limit to the nesting of Single Select Dynamic questions
e.g. to narrow down all the pupils in school in the entire world:
Continent -> Country -> District -> Village -> School -> Teacher -> Pupil

GPS

Get or enter the GPS co-ordinates (longitude, latitude, altitude) for a place

On the mobile phone, if GPS functionality is available it will launch the GPS to get the current location. If GPS functionality is not available it will present three windows to enter the data

On the web form, a single text box is presented which doesn’t validate that the input is a valid GPS co-ordinate.

Barcode

Not implemented in 1.16, but available in ~1.18 onwards

On the web form it just presents a free text box.

On the mobile client, if you have the -barcode midlet which includes a barcode reader, the barcode reader is launched which then uses mobile phone’s camera to take an image and process the barcode.

Roles and Permissions

openXdata controls access through roles and permissions

openXdata uses Roles and Permissions to control access to different components of the openXdata interface.

Permissions

Permissions are the finest grain of control. They control individual functionality. For example, within forms there are pemissions for the following functionality:

  • Perm_Add_Forms – Ability to add forms in the system
  • Perm_Delete_Forms – Ability to delete forms from the system
  • Perm_Edit_Forms – Ability to edit forms in the system
  • Perm_View_Forms     – Ability to view forms in the system

Roles

Roles are groups of permissions. openXdata comes with 7 pre-defined groups of permissions that cover many common functional needs:

  • Administrator – has permission to do everything
  • Study Manager – Creates studies and designs forms
  • Data Manager – Review and edits collected data
  • Data Collected – Collects data through the web dashboard
  • Mobile User – Collects data through the mobile client
  • Report Manager & Report User – these are not currently functionally useful but are there for future implementations of reporting functionality

Assigning users roles and permissions

Users are given privileges to different functionality by assigning them roles (covered in Users – New and Users – Edit).

If a role does not have the set of permissions you’d like to assign your user, you can create a new role and customize the privileges in the Admin interface.

Users can be assigned more than one role at once.

Note, as a user you cannot add or edit permissions as this functionality is set inside the code base. Of course openXdata is open-source so you are welcome to modify the code as you see fit. Or, you can file a ticket for a feature request if you feel something is missing.

Further information

On the developer wiki, all the permissions are described in detail along with the standardized set of roles – https://trac.openxdata.org/wiki/permissions

How openXdata stores your data

In a MySQL database

All the data you collect in openXdata is stored in the MySQL database that you setup in the first step. It is worth noting here that openXdata (the software) does not do anything to secure your database for you, you must take your own steps to ensure the integrity and security of your database (and your backups). In addition openXdata (the organization) cannot see or access or control your data – it is yours to own, use and protect.

Inside the database, it goes into two or more places:

  1. standard table: form_data
  2. standard table: form_data_version (only used when data is edited)
  3. custom table: table of data per form
  4. custom table: table of data for any repeat questions (only when form has repeat questions)

form_data

wpid1245-media_1343975285162.jpg

The table form_data stores the raw xml of your data in the column data.

The xml for your data will contain all of the fields in your form and any data you’ve stored, for example:

<?xml version="1.0" encoding="UTF-8"?>
<example_study_sample_form_v1 formKey="example_study_sample_form_v1" id="1" name="Sample Form_v1" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<patient_id/>
<title/>
<first_name/>
<last_name>Testing Form 1</last_name>
<start_time/>
<endtime>09:49:27 PM</endtime>
</example_study_sample_form_v1>

Each data entry is given a unique ID, and it retains that ID when you edit it.

The version of your data entry in the table form_data will be the latest version of that particular data entry

form_data_version

wpid1247-media_1343975779830.jpg

The table form_data_version stores old versions of the data when a data entry is edited. It also includes information useful for audit trails like who changed the data and when.

custom table

wpid1246-media_1343976414665.jpg

For each form, a unique table, or tables is created.

These tables have the same name as the form version binding or the repeat question binding. (See the Glossary for more information on bindings)

These tables have the data broken out into columns with the column headings being the question bindings.

In 1.16 there are some protections to prevent a clash of table names but they can still happen. In particular:

  • Importing a form that already exists in the database
  • Creating a repeat question with the same binding

Requirements

Hardware and software requirements for openXdata web application.

Software

openXdata runs on freely available open-source tools. To run openXdata server you will need:

  • Apache Tomcat 6 with Sun Java 6 (open-jdk not supported)
  • MySQL

Hardware

The above software can run on a very low spec machine. OpenXdata has been successfully run on virtual machines with a 4GB hard drive and 384MB RAM running Ubuntu Linux (using Sun VirtualBox OSE). Any laptop or PC less than 5 years old with more than 1GB of hard disk free and 512MB RAM should be sufficient to run openXdata. However, a full hard drive or other programs running with intensive requirements may affect performance.

If you wish your phones to connect via GPRS you will require a public IP address. Contact your internet service provider to find about this service. They may call it “dedicated IP” or “static IP.”

  • If you are part of an organization you should discuss this with your IT team as if you don’t know what you are doing, deploying openXdata could open up your computer network to the outside world. (This is not because of openXdata but because you are deploying a web-based service.)
  • There are also Dynamic DNS services which can give you a static domain even if you have a dynamic IP. To use services like this at home, you may need to configure your internet modem / wireless router.

If you wish to use SMS to receive data you will need an SMS gateway/modem. This can be as simple as a phone connected to your server with a USB cable but there are also custom pieces of hardware that act as SMS gateways. The use of sms is currently not widespread throughout the openXdata community so we welcome contributions on experiences, set-up and use of sms.

For phones see Compatible Phones

openXdata Glossary

Description of terms used in openXdata

A – E

Binding = Each form version and each question in openXdata has a binding. The binding can be viewed in the Form Designer in the properties tab under the row binding. It is used in a number of ways. The form version binding is used to create a table name in the openXdata database. The question binding is used as the column name in that table, it is also the xml tag identifier in the raw xml. The binding for an option in a single or multiselect question becomes the value stored for that question in the xml.

Designer = A graphical interface that opens in a new window which allows you to design your form including adding and remove questions, defining skip logic, viewing & editing raw xml, and desiging a web form

F – J

Form = A set of Questions – the basic unit of data collection in openXdata

form_binding = A long string used to identify the form. Specific to each form version. Used as the name for the table export in database. Used in xml, including to form skip logic. The form_binding is defined the first time a form version is opened in the Designer the default is StudyName_FormName_FormVersion. It is better not to manually edit the form_binding

Form Version = Forms can have multiple versions, they are usually similar, involving just an upgrade to a subset of questions or the addition or removal of questions. They are unique and have their own database table in openXdata.

K – O

P – T

Published / Unpublished =
A published form is:

  • Shown by default in the Dashboard view
  • Downloaded to the mobile client if the user has permission for the Form or Study

An unpublished form is:

  • Hidden by default in the Dashboard view – view unpublished forms it by selecting "Show all Versions" in the List of Forms box in the Dashboard
  • NOT downloaded to the mobile client

Study = A group of Forms

U – Z

Unpublished = see Published

Version = see Form Version

Import an existing form

How to import a form from a .xml file

Create a new form and open form designer

wpid1041-media_1321342052847.jpg

Follow the steps in the lesson "Adding a new Form" to add a new form.

To Edit the form:

  1. select the form
  2. press Edit
  3. then Design Form

From empty form designer go to Xforms Source

wpid1044-media_1321342336981.jpg

You will see a blank box

Open your .xml file in a text editor

wpid1043-media_1321342391118.jpg

Then select all the text (Ctl+A) and copy and paste it into the Xforms Source Box

Press the Open button to import the .xml into a form

wpid1042-media_1321342489025.jpg

After you have pasted the text into Xforms Source box, press the Open button

openXdata will render the form

wpid1040-media_1321342638781.jpg

Assuming that there are no errors in your xml and it is compatible with openxdata, then you should see your questions appear in the left column of the form designer. Depending on your .xml you may also have an existing web form under the Design Surface

Adding a new form

Adding a new Form to an existing Study.

Select Study you wish to add form too (Optional)

wpid1033-media_1321340261154.jpg

A study is selected by clicking on a Form within that study. You cannot directly highlight the line which lists the study. It is not necessary to select a study in order to add a new Form and if you select the wrong study, you can change it in Step 1 of the Wizard.

Open the New wizard

wpid1034-media_1321340403033.jpg

Step 1 – Choose to add a new Study or use an existing Study

wpid1037-media_1321340540932.jpg

If you select use an existing Study, as shown in the above picture, the Wizard will automatically use the one you previously selected. But you can change it, by using the dropdown box.

Step 2 – Add a New Form

wpid1035-media_1321340784989.jpg
  1. Select Add a new form
  2. Enter the name for your new form. If you enter a form name that has already been taken, you will see an error, as shown at the bottom of this tutorial.
  3. Optional – Enter a description of your form
  4. Click Next

If you do not click Next, the form will be created, but if will not have a version and will not be published. In order to use a form, it must have a version. If you decide to create a form without a version, you can always return later and repeat the above steps, but then "select an existing form" at this step and go to Step 3.

To view a form with no version in the Dashbaoard you must select "Show all versions" & "All Forms"

Step 3 – Create a version and choose whether to publish

wpid1036-media_1321341394238.jpg

The version number will automatically be set for you. You can add a version description, perhaps to note any changes you make between versions.

All forms, Published or not, will be visible on the web interface. However, only "Published" forms are downloaded to the mobile phone. This allows you to select which forms are usable by your mobile data collectors and to allow you to keep them up-to-date with the latest form changes. In order to see an un-Published form on the Dashboard, you must select "Show all Versions"

Click Design to go straight to the Form Designer, or Save & Exit to return to the Dashboard to see your new form.

Error – Form already exists

wpid1038-media_1321340944201.jpg

Within each study, form names must be unique. If you enter a form name that already exists for the chosen study then you will see the above error. Form names can be the same across different studies, but Study names must also be unique.

Nerd note: The form binding is used as the name of the MySQL database table containing the uploaded (& processed) data. The form binding is study_binding_form_binding_version#

Installing pre-requisites (Linux)

 

Installing MySQL & Tomcat on Ubuntu Linux

Installing MySQL

Full instructions regarding the install and running of MySQL can be found at the MySQL home page. MySQL is a very widely used database and you should find many, many resources online.

In ubuntu, to install MySQL

$ sudo apt-get install mysql-server

During the installation process you will be asked to set a root password. For your own security:

  1. do not leave this blank
  2. do not set this to be the same as your computer’s user or root password
  3. make the password secure and keep if safe

Once installed, to access mysql type

$ mysql -u root -p

you will be prompted for your root password that you set during installation and you will then enter the mysql command line environment where you will be able to use commands to administer your mysql databases.

Installing Tomcat

Again, full instructions can be found on the apache tomcat website http://tomcat.apache.org. On Ubuntu, Tomcat and the Tomcat Web Application Manager (useful if you’re not a fan of the command line) can be installed as follows:

$ sudo apt-get install tomcat6 tomcat6-admin

To use the Web Application Manager you will need to set-up a user to do this.

Browse to tomcat6 conf directory
$ cd /var/lib/tomcat6/conf
Edit the file tomcat-users.xml (you need to be root to do this on a default Ubuntu installation)
$ sudo nano -w tomcat-users.xml
Modify the file so it looks like this (replacing TOMCATUSER and PASSWORD for your own secure choices):

<tomcat-users>
<role rolename=”manager”/>
<user username=”TOMCATUSER” password=”PASSWORD” roles=”manager”/>
</tomcat-users>

On Ubuntu (this may vary by distribution), once tomcat is installed via the package manager apt (method described above), then you can access all the required folders through the folder /var/lib/tomcat

  1. Tomcat logs (If you’re having problems you may be asked for output from files in this folder) – /var/lib/tomcat6/logs -> /var/log/tomcat6
  2. Tomcat configuration (Find files to configure custom aspects of your tomcat setup such as custom port configuration) – /var/lib/tomcat6/conf -> /etc/tomcat6
  3. Tomcat webapps directory (The directory where all your web applications live, including openXdata) – /var/lib/tomcat6/webapps

On Ubuntu to start and stop tomcat you will need sudoer privileges and the commands are:

To start tomcat
$ sudo /etc/init.d/tomcat6 start
To stop tomcat
$ sudo /etc/init.d/tomcat6 stop
To restart tomcat
$ sudo /etc/init.d/tomcat6 restart

If you do a manual installation by downloading the tomcat zip file from the tomcat website, when you unzip it all the the folders listed above will be found in whichever location you extract your zip file. To start/stop/restart tomcat from a manual installation use /path/to/tomcatfolder/bin and then startup.sh and shutdown.sh e.g.

$ cd /home/myhome/tomcat/bin
$ ./startup.sh

 

Upgrading openXdata

 

Upgrading your existing openXdata instance

Check the release notes

If there are any for a release, release notes are on the Download page. The release notes should outline whether the compatibility of upgrading between versions. Most likely problems are:

  • mobile client protocols – you must make sure your mobile phones can talk to the newer version of your server.
  • database inconsistencies – changes may have occured that require your database structure to be upgraded

Back-up your database

The mysqldump command comes with your MySQL server setup. Full instructions can be from the MySQL Documentation page e.g. http://dev.mysql.com/doc/refman/5.5/en/mysqldump-sql-format.html (make sure you have the documentation that corresponds to your MySQL version).

The basic command that can be run from the command line is:
mysqldump -u root -p myopenxdatadb > myoxdbackup.sql

This produces a .sql file in the location you run the command from which can be used to restore your database in the future.

Please note, if running a production system, you should be taking regular backups of your openxdata database anyway.

Back-up your current openXdata war

In your tomcat webapps home folder, you will find a folder named openxdata.war and a directory openxdata

Before proceding further, take a copy of the folder and its contents as a backup and put it somewhere other than your tomcat/webapps folder.

Download new openxdata.war

Do you want your new version of openXdata to be at the same or a new URL? This lesson assumes that you do want your upgrade at the same URL http://myhost:8080/openxdata

If you want it at a new URL, you can see the lesson on multiple openXdata instances for further information

If your existing openXdata instance is at /openxdata, then save your newly downloaded war as openxdata.war

Undeploy your existing openxdata web application

wpid1011-media_1321251166228.jpg

This step will delete your currently deployed web application, but will not touch your database

In the Tomcat Web Application Manager, find the row with your openxdata instance and click Undeploy

openXdata un-deployed

wpid1012-media_1321251234567.jpg

Repeat steps from installation

wpid1013-media_1321251547656.jpg

Now repeat the steps from the installation, to deploy your new web application and then set the database settings in the OPENXDATA_SETTINGS.properties file to point your new openXdata instance to your existing database

If something goes wrong

If something goes wrong:

  1. Undeploy your new openXdata
  2. Restore your MySQL database from your backup
  3. Redeploy your old openXdata. Don’t forget to update the OPENXDATA_SETTINGS.properties

 

Creating MySQL database and user (All Platforms)

A run through of the command line steps required to add a new database and user for openXdata on your MySQL. A variety of graphical interfaces for MySQL exist for various platforms including MySQL Administrator and phpMyAdmin, there are a wide number of resources online to cover these programs.

Enter MySQL command line

Add a new user

mysql> CREATE USER ‘myoxduser’@'localhost’ IDENTIFIED BY ‘securepassword’;

Don’t forget the ; at the end of the line. If successful, you will see:

  • Query OK, 0 rows affected (x sec)

Add a new database

mysql> CREATE DATABASE myopenxdatadb;

Don’t forget the ; at the end of the line. If successful, you will see:

  • Query OK, 1 row affected (x sec)

Grant new user access to new database

mysql> GRANT ALL ON myopenxdatadb.* to ‘myoxduser’@'localhost’;
mysql> FLUSH PRIVILEGES;

Don’t forget the ; at the end of the line. If successful, after each command you will see:

  • Query OK, 0 rows affected (x sec)

Exit MySQL

mysql> quit

To confirm your setup worked re-enter MySQL as your new user

From the command line / command prompt:
~$ mysql -u myoxduser -p
(then enter the password created in step 2)

mysql> USE myopenxdatadb;

If you get the response "Database changed" then your setup has worked