The Code Cadets are a group of students from Canberra Grammar School who are interested in software development and computer programming.
As part of the GovHack 2012 competition, Lochie Ferrier (Year 10) and Matthew Purcell (Software Development Teacher) combined forces and formed a team to write an iOS application from scratch during the 48 hour competition period.
Who are we?
Lochie Ferrier – Canberra Grammar School Year 10 Student. I enjoy programming and other nerdy hobbies. I have written for a blog at http://lochieferrier.com about making everything from synthesizers to robots. I am doing iPhone and iPad development as a school course this year and I am a member of the CGS Code Cadets. I am entering govhack because I think it will be a challenge to work under a time limit and great opportunity to learn something new.
Matthew Purcell – Canberra Grammar School Teacher. This year I am writing and teaching two new courses at Canberra Grammar School – Year 10 iPhone and iPad Application Development, together with Year 9 Web Design and Application Development. I am also the coordinator of the CGS Code Cadets group, who are embarking on a trip to the Macworld 2013 Conference in San Francisco during January 2013.
What are we doing?
We are writing an iPad application which provides a visual representation (data visualisation) of the past 100 years of rainfall and temperature data provided by the Australian Bureau of Meteorology. While many applications provide jumpy and difficult to follow visualisations, we intend to provide a much more appealing and easily understood representation of this historical information.
News Flash: Our app is now officially called “100 Years”. Want to find out more? Check back later today (Sunday) for more screen shots and information. In the meantime, here is the promised sneak-peek of our app:
This is not the finished version, but a work in progress. There are plenty of nice little other things that we have included to really make the finished version look polished and stunningly designed.
We are using the BOM daily rainfall and temperature data sets. Many thanks to data set mentors Martin, Clinton, and Laurent for their help in understanding and effectively using these data sets.
We’re done, with 28 minutes to spare! Here are some screen shots of our creation.
Source code now available!
The full source code for 100 Years is now available at http://govhack.codecadets.com/100_Years_Final.zip
What have we learned?
GovHack has been a massive learning experience for us – having never competed in a Hackathon before, requiring an app to be developed from scratch in a set amount of time, this was a fantastic experience which also presented unique challenges that we enjoyed overcoming. Here we will be discussing some of these since they all (in some way) influenced the finished product that we produced.
1. How not to store 4.1 million records in a database…
As our app is storing 100 years of temperature and rainfall data this equates to approximately 4.1 million individual data points (records) that need to be stored. The storage mechanism we are using is Core Data, essentially a framework allowing iOS applications to easily access an SQLite database. At the moment, our database schema looks like the following:
While the database itself can easily handle that number of records, querying the records is another matter. Our app requires the data for a particular date to be accessed for display on the map and graph. This means that when we need to fetch all data for a particular date all 4.1 million records need to be queried. For example, if we want to get the rainfall data for the 2 June 2011 then the date attribute of all objects in the RainFall data entity must be checked to find which objects match. Obviously this is a substantial performance bottleneck which takes approximately 17 seconds to complete on an iPad 2…every-single-time the user wishes to view data from a different date. Considering that we want a fluid interaction, allowing the user to seamlessly scroll between dates, this is unacceptable performance.
Unfortunately this is something that we only found after doing initial testing, given that we didn’t have much experience in large data sets, but we have found a solution which we haven’t implemented yet (given that it takes a very long time to import 4.1 million data points, which we can’t/couldn’t do before judging today) but we will be making these modifications later.
So, after analysing the problem we found that a much better way to approach the data model which we have redesigned below:
Using this new schema, there is an object representing each date which is related to the rainfall or temperature objects which record data for that particular date. This means that instead of having to parse all 4.1 million records to find the data for a date, the app simply needs to locate the EventDate object for the date (out of 36500 records – 365 days per year for 100 years) which would be fairly quick, and then traverse the relevant relationship (rainfallData or temperatureData) to find the data records related to that date. This is substantially quicker and results in no noticeable lag when retrieving the data for a particular date.
2. Do not hax0r all night
Since we hadn’t done a major Hackathon before we decided to hack through Friday night and Saturday morning. So, we came straight to GovHack from school on Friday afternoon and only left on Saturday morning at 8:30am. That was pretty rough, and found by about 2am that the coding machine was definitely out of steam.
Point and case:
On reflection, it would have been much better to head home late Friday night (early Saturday morning) to be recharged and then hack through Saturday during the day. That said, even with minimal sleep we found that hacking from home on Saturday was extremely productive allowing us to come along on Sunday ready to put the final touches onto the app.
You are welcome to follow our progress at GovHack 2012 on our Twitter feed @CodeCadets