A while ago, on the formhub mailing list, a user asked how formhub data could be view on the phone. He was doing Registration of households, and he was wondering how to load up the data for these households so that surveyors could see which houses to visit when they were going back.

Avid formhub user Erwin Olario thought that viewing .gpx files through an application like OsmAnd would be a great solution, and has been generous enough to write up his process for the formhub blog:

Formhub lets export your data as a KML file. If you are looking for an option to display this data in your phone, one way to go about it using Open Source tools is to use your Android device for offline navigation. I particularly like OsmAnd for this purpose but it doesn’t read KML files, so the data needs to be converted prior to transferring the data to your device.

Exporting to KML

If your form collects geographic coordinates, you will find that there’s a KML download option available for you on Formhub. This link is visible on the front page of your user account, under “Download” in the “Published Surveys” table.

Click the “kml” option to go to the Export KML page. It may take several seconds to generate the file, longer if you have lots of records. When the export is complete, you will find a link you can click for downloading the generated KML file which you can now save to your computer.

Converting to GPX

We need to convert this KML file to GPX format. GPSBabel converts files to or from many default GPS formats, including those devices made by Garmin or Magellan.

If you’re using some flavor of Ubuntu like I do, (make sure the Universe repository is enabled) from the command line type and run:

$ sudo aptitude install -y gpsbabel

If you are not faimilar with the command line, you can install the same program using Ubuntu’s Software Center. For other platforms, please visit GPSBabel’s official download page for binaries you can install. To convert a KML file, run from the command line:

$ gpsbabel -w -i kml -f "in.kml" -x nuketypes,tracks,routes -o gpx -F "out.gpx"

The command issue above tells GPSBabel to ignore tracks and routes and just use the waypoints found in “in.kml” (or whatever file name you used when you downloaded it to your computer) and save them in GPX format as “out.gpx” (or choose your own.)

Now, if you don’t want to install anything in your computer, there’s a GPSBabel web interface hosted by a third-party: GPS Visualizer: GPSBabel. Just make sure you set the option to convert to Waypoints, and that your input file format is Google Earth (Keyhole) Markup Language and the output format is GPX XML, upload your file and hit the Convert the file button.

View it in your device

Your file is almost ready for use in your Android device. Make sure to have a GPX reader and import the data. If you don’t have any, I recommend OsmAnd - and as Prabhas pointed out, not only can you view your POIs, you also view the map offline, using the excellent and community provided data from OpenStreetMap.

If you use OsmAnd, make sure that you copy your GPX file to the following location because it doesn’t have a built-in import capability, so it expects GPX tracks in the same folder like so:

$ osmand/tracks/out.gpx

Launch OsmAnd, and configure the view to display Favorites and GPX tracks. Your waypoints are displayed as stars, the same icons used for favorites. For details on how to setup your OsmAnd app, please visit the very good collection of How-Tos in the OsmAnd wiki.

For a few screenshots of how I use this in OsmAnd, I have an album here.

Encrypted forms

We are excited to add support for encrypted forms to Formhub. Encrypted forms work by encrypting the data on the phone the moment it is saved. Data sent to formhub is encrypted and completely inaccessible to anyone not possessing the private key. In this case, Formhub serves simply as a storage locker for your encrypted files - a place to upload and then download for later for local decryption (using ODK Briefcase). Since the form submissions are encrypted, it means, however, anything that requires access to the data like the map view or data export won’t work within Formhub. The extra level of security makes using Formhub in a way to collect sensitive data while meeting IRB protocols possible. This has been a major request from our University colleagues so we are particularly excited to announce support for this important feature.

How it Works?

ODK Collect supports the ability to encrypt the content of a form the moment it is marked as completed and ready for submission on the phone. To take advantage of this requires the use of a public encryption key which you include in the XLSForm and a private key (which you never share) that is used by ODK Briefcase to decrypt the data locally after you’ve downloaded it from Formhub. The public key is used to encrypt data while the private key decrypts it. Only a person who has the private key, can decrypt the data encrypted with the public key. To understand more about public and private key infrastructure see http://en.wikipedia.org/wiki/Public-key_cryptography/ .

How to encrypt XLS forms

1. In your XLSform, add a worksheet called ‘settings’
2. In this worksheet create three columns namely: ‘ id_string’, ‘submission_url’ (is your submission url), and ‘public_key’ (is the base64RsaPublicKey). See below for more information on generating the required public key. Do not set a password when generating the key. The public key is the key that is on the ‘public_key’ column on the ‘settings’ worksheet of your xlsform. The syntax for the form is as shown by the image below:
3. Upload the xlsform to formhub as usual. You should see a label with the text “encrypted” next to your form name on formhub as shown below:

We recently just hit our 1 millionth form submission! To celebrate we decided to create this map to share where and to what extent people have been using Formhub to map the world around them. We are thrilled to see the usage spread.

It’s particularly exciting to see Formhub being used in several national level mapping efforts. It’s a testament to the fact we build formhub from the ground up to gracefully scale and handle as much data as you can throw at it.

Here’s to the next 10 million!

For a full size map, please click here.

ODK Briefcase is used to gather and export data from an ODK Aggregate server e.g formhub.org or from ODK Collect when you are offline

ODK Briefcase can:

1. Export forms with submissions from formhub (ODK Aggregate) - PULL
2. Make bulk submission to formhub (ODK Aggregate) - PUSH
3. CSV exports of submission data
4. Pull forms and submissions collected by ODK Collect from a mobile phone
5. Supports encrypted forms: it can make encrypted submissions to formhub as well as decrypt encrypted submissions from formhub when exporting the data to csv.

If you have encrypted forms and making submissions to formhub, you’ll realise that you do not have the ability to browse the submitted data on the site. Formhub stores the submissions in encrypted form, using your private key and ODK Briefcase you can be able to pull your data from formhub and export it to a csv file.

For the past two years, we’ve been workign to support an amazing group of kids in Kolkata, India known locally as the “Dare Devils” to help teach them how to use mobile phones to map social issues in their community. Their inspiring story which features the incredible work of Prayasam has recently been featured in a documentary called the Revolutionary Optimists which just aired publically on PBS on Monday.

To help promote the message that children can be power change agents in their community, the film makers Nicole and Maren, have launched a project called Map Your World to allow youth groups from around the world to use mobile phones to map issues that concern them in their own communities. The website for the project is close to launching but for now the following trailer provides good background on the project:

Map Your World from Grainger-Monsen Newnham on Vimeo.

Behind the scenes, Map Your World will be powered by formhuband ODK Collect. To do this, we are working to provide the MYW team with formhub API’s that will allow them to build the MYW on top of the formhub as a service. This will allow them to focus on creating powerful vizualizations while formhub will handle all the form submissions, user permissions and data storage. We think this a model that will make sense for a lot of organizations who want to leverage the power of mobile data collection but create an experience more adapted for their project needs. If you interested in doing the same and want early access to our API’s please let us know.

Leveraging publically available satelitte imagery, MapBox recently released a stunning new satellite map that is cloudless. Yes, you heard us right. Cloudless!

To do this, they basically went through archives of satelitte imagery and on a pixel-by-pixel basis assembled a map that is essentially cloud free.

Starting today we are excited to offer MapBox Cloudless Atlas as one of the core basemaps avaiable on formhub. This joins the existing MapBox Streets base layers - Open Street Map powered and available in English, French and Spanish.

Since formhub is used extensively by many groups for mapping. We hope this new, clearer picture of the world will help extra insight to your data.

Want to tell a friend what formhub is, but unable to give them a full demo? Maybe the following video will help!

We’re happy to release a new formhub User’s Guide that provides an introductory overview of some of the functionalities formhub provides.

Thanks to Leigh Jaschke for her help in putting this together!

Many data collection projects have strict research protocols on how data can be accessed or modified. To help with with this, we’re excited to announce that we’ve added activity/audit logs to formhub. This allows you to view what actions have been performed on your formhub account, by which user(s) and at what times. To access this log, click on the little arrow next to your username and select “Account Activity” from the list.

The Account Activity page

The activity list has these 4 columns:

1. Performed On - The date and time that the action was performed
2. Action - The name of the action performed
3. Performed By - The username of the user who performed the action
4. Description - A short description of what was exactly done and what it affected

Initially, the list will display the most recent activity first, this sort order can be changed by clicking on the “Performed On” column header (or any of the other columns headers to sort by them).

By Jonathan Cabral, Stephane Keil and Edwin Adkins

As a part of an ongoing project to support the Liberian government in energy related data-gathering and planning, three members of the Modi Lab conducted a two-week training in Monrovia to prepare key energy and data specialists from Liberian government, the electric utility, and related energy organizations in the use of mobile rapid data-gathering tools to estimate the urban and rural energy demand in the country.