Recently I spoke at TradeTech Europe, an important event for global financial and trading markets. One of the hot topics from the event and something that is (or should be) on everyone’s mind at the moment is ‘big data.’

I’m confident that trends around speed and proximity won’t change and equally confident that technology will evolve to meet the demands. What I think is the real challenge is the nature and structure of the data – or rather the lack of structure – and how this will require us to alter the way in which decisions, including investment and trading decisions are made. Simply stated, the challenge for the future is how, in an environment of ‘big data’, people can work smarter.

This is not simply about creating pictures from structured data but about the ability to integrate non-structured data, whether in the form of social network information or video. Artificial intelligence now allows such unstructured data to be interrogated in the search for patterns. Bringing structured and unstructured data together, for example on a trading turret, creates new levels of insight as well as a clarity that we haven’t had before.

In financial markets, this offers traders who are willing to make the investment superior ways of generating investment returns – possibly at the expense of those who don’t make the same commitment. Of course, some firms struggle to make sense of the need to invest and compete in the low latency, high frequency, big data environment that exist today around structured data and relatively simple algorithms.