Using big data to understand how the world works

My intentions for this blog are three fold:

1. to explain "how the world works" by extracting information from the wealth of publicly available data using data mining techniques;
2. to present my applied mathematics research in means digestible by the wider community; and
3. to give my mum something to direct people towards when they ask her what her son does!

As tempting as it is for me to include mathematical formulations, I have instead attempted to explain the key concepts without such details, and provide links to my more technical documentation.

Qualitatively I see the world evolving as a result of a series of complex interactions between the Earth, biological, and socio-economic systems. However, if you ask the question "how does the world work" to a 100 different people, you're likely to get a 100 different responses. Depending on your training and background you are likely to focus on different interactions present in the world in which we live. For example:

• astrophysicists focus on the physical interactions between planets;
• particle physicists focus on the physical interactions between sub-atomic particles;
• climate scientists focus on the physical interactions between the atmosphere and ocean;
• anthropologists focus on the social interactions between people;
• economists focus on the socio-economic interactions between governments, commercial enterprises, non-government organisations, and consumers; and
• politicians focus on the political interactions required to implement the policy instruments necessary to modify the behaviour of the institutions and constituents under their jurisdiction.

There is one key difference between physical systems and biological / socio-economic systems. The difference is that physical systems are governed by fundamental equations of motion, which we can simulated on high performance computers for analysing past events, and for making predictions of the future; for example numerical weather prediction. Socio-economic and biological systems are not governed by fundamental evolution equations, so we rely on developing models from existing data to understand them.

The following posts will focus on the application of data mining and scientific visualisation techniques to various physical, biological, socio-economic data sets to explain "how the world works". 

Finally in case you're wondering the background image was taken at the Pinnacles, in the Nambung National Park in Western Australia.