Data tapestry

Map of languages connections on the web. Credit: project GitHub: https://github.com/norvigaward/naward25/wiki/Babel-2012---Web-Language-Connections

Yesterday I was involved in a panel at the ICT Days 2014 in Trento discussing Big Data and the territory. I'd like to share some of the thoughts I socialised there.

We live in a connected world and technology has increased this connectivity. More than that. At a perceptual level we tend to see everything in terms of networks. The casualty principle (at the core of Science for hundreds of years) is yet another example of a network, a chain of events resulting in something we detect.

Infrastructures are all around us, be it roadways, power lines, and of course, Internet and the web. 
Social behaviours are also described in terms of networks and we are looking at brain connectivity trying to understand how it works.

However, several scientists are pointing out that this reductionist approach in very complex systems (not complicated but complex) will lead nowhere. For  them we won't be able to explain a thought by looking at the axons and dendrites, we won't be able to explain the mood of a city by looking at the interaction among citizens, the innovation capability of a company by looking at its processes and so on. In quantum physics, and we hate it for this, we have learnt that we cannot, simply cannot, explain the behaviour of a single photons by looking at its (very weak) interaction.

Why this very long preamble if the topic of the panel was "big data for the territory"?

Because I feel that the Big Data may represent a change of paradigm, from the perception of infrastructures as a web of links as infrastructures as a tapestry where of course there are links (warp and weft) but the result is an emerging property, in the same way as thoughts are an emerging property of the neuronal connections, their status, the neuro-modulators and neurotransmitters. The total is greater then the sum of its parts.

I see Big Data, and the computation made on these that creates new information as well as visual representations able to trigger specific perception, as a paradigm change. A shirt might feel tight or loose because of its threads, but also because of the model and the fabric...It is the whole that takes precedence over the set of its components.

The territory generates data, huge volume of them, sometimes with significant variety and velocity. However, once they become Big Data, through processing and correlation, they create a perception of the territory itself that is nowhere in the raw data produced. A high traffic volume can result in a mood of frustration or a mood of joy, depending on several aspects. And what Big Data is telling us is the broad picture, the holistic view. 

Reductionism does not work on Big Bata, it shouldn't work, otherwise we would just be dealing with massive data sets. Being able to create an holistic view, to stimulate perception as emerging properties is, I feel, the magic of Big Data, and what differentiate them from a statistical analyses.

We have just started in this new space, at the EIT ICT Labs we are about to launch a High Impact Initiatives in this area. Don't fool yourself that it is business as usual. It is not.

Author - Roberto Saracco

© 2010-2018 EIT Digital IVZW. All rights reserved. Legal notice. Privacy Policy.