100 Years

The Code Cadets are a group of students from Canberra Grammar School who are interested in software development and computer programming.

As part of the GovHack 2012 competition, Lochie Ferrier (Year 10) and Matthew Purcell (Software Development Teacher) combined forces and formed a team to write an iOS application from scratch during the 48 hour competition period.


Who are we?

Lochie Ferrier – Canberra Grammar School Year 10 Student. I enjoy programming and other nerdy hobbies. I have written for a blog at http://lochieferrier.com about making everything from synthesizers to robots. I am doing iPhone and iPad development as a school course this year and I am a member of the CGS Code Cadets. I am entering govhack because I think it will be a challenge to work under a time limit and great opportunity to learn something new.

Matthew Purcell – Canberra Grammar School Teacher. This year I am writing and teaching two new courses at Canberra Grammar School – Year 10 iPhone and iPad Application Development, together with Year 9 Web Design and Application Development. I am also the coordinator of the CGS Code Cadets group, who are embarking on a trip to the Macworld 2013 Conference in San Francisco during January 2013.


What are we doing?

We are writing an iPad application which provides a visual representation (data visualisation) of the past 100 years of rainfall and temperature data provided by the Australian Bureau of Meteorology. While many applications provide jumpy and difficult to follow visualisations, we intend to provide a much more appealing and easily understood representation of this historical information.

News Flash: Our app is now officially called “100 Years”. Want to find out more? Check back later today (Sunday) for more screen shots and information. In the meantime, here is the promised sneak-peek of our app:

This is not the finished version, but a work in progress. There are plenty of nice little other things that we have included to really make the finished version look polished and stunningly designed.

We are using the BOM daily rainfall and temperature data sets. Many thanks to data set mentors Martin, Clinton, and Laurent for their help in understanding and effectively using these data sets.


App screenshots

We’re done, with 28 minutes to spare! Here are some screen shots of our creation.


Source code now available!

The full source code for 100 Years is now available at http://govhack.codecadets.com/100_Years_Final.zip


What have we learned?

GovHack has been a massive learning experience for us – having never competed in a Hackathon before, requiring an app to be developed from scratch in a set amount of time, this was a fantastic experience which also presented unique challenges that we enjoyed overcoming. Here we will be discussing some of these since they all (in some way) influenced the finished product that we produced.

1. How not to store 4.1 million records in a database…

As our app is storing 100 years of temperature and rainfall data this equates to approximately 4.1 million individual data points (records) that need to be stored. The storage mechanism we are using is Core Data, essentially a framework allowing iOS applications to easily access an SQLite database. At the moment, our database schema looks like the following:

While the database itself can easily handle that number of records, querying the records is another matter. Our app requires the data for a particular date to be accessed for display on the map and graph. This means that when we need to fetch all data for a particular date all 4.1 million records need to be queried. For example, if we want to get the rainfall data for the 2 June 2011 then the date attribute of all objects in the RainFall data entity must be checked to find which objects match. Obviously this is a substantial performance bottleneck which takes approximately 17 seconds to complete on an iPad 2…every-single-time the user wishes to view data from a different date. Considering that we want a fluid interaction, allowing the user to seamlessly scroll between dates, this is unacceptable performance.

Unfortunately this is something that we only found after doing initial testing, given that we didn’t have much experience in large data sets, but we have found a solution which we haven’t implemented yet (given that it takes a very long time to import 4.1 million data points, which we can’t/couldn’t do before judging today) but we will be making these modifications later.

So, after analysing the problem we found that a much better way to approach the data model which we have redesigned below:

Using this new schema, there is an object representing each date which is related to the rainfall or temperature objects which record data for that particular date. This means that instead of having to parse all 4.1 million records to find the data for a date, the app simply needs to locate the EventDate object for the date (out of 36500 records – 365 days per year for 100 years) which would be fairly quick, and then traverse the relevant relationship (rainfallData or temperatureData) to find the data records related to that date. This is substantially quicker and results in no noticeable lag when retrieving the data for a particular date.

2. Do not hax0r all night

Since we hadn’t done a major Hackathon before we decided to hack through Friday night and Saturday morning. So, we came straight to GovHack from school on Friday afternoon and only left on Saturday morning at 8:30am. That was pretty rough, and found by about 2am that the coding machine was definitely out of steam.

Point and case:

On reflection, it would have been much better to head home late Friday night (early Saturday morning) to be recharged and then hack through Saturday during the day. That said, even with minimal sleep we found that hacking from home on Saturday was extremely productive allowing us to come along on Sunday ready to put the final touches onto the app.


Follow Us

You are welcome to follow our progress at GovHack 2012 on our Twitter feed @CodeCadets

Comments

Add a comment

You must be logged in to comment.

  1. Algunas notas del #GovCamp 2012- Canberra | Digital BA06-15-12
Who’s behind GovHack? This is a non-profit event proudly run by volunteers who form the GovHack Coordination Team. Our ongoing thanks to everyone who gets involved and makes GovHack awesome! That is, the hackers, data providers, sponsors, mentors and a special thanks to the volunteers who run Local GovHack events.
Sponsoring GovHack 2016 If you are interested in sponsoring this event, please contact GovHack Australia at sponsors@govhack.org or GovHack New Zealand at sponsors@hacknz.org.
The GovHack Global Operations Team Click through to find us on Twitter and Google+ Richard TubbJan Bryson,  Alex Sadleir, Alysha Thomas, Keith Moss,  Gavin Tapp

Privacy Statement/Disclaimer
GovHack is run under the auspices of Linux Australia.