Big data is one of the big topics of the past few years and whilst copious amounts of money has been invested in the collection of and storage, the analytics and real life applications have been slightly slower in coming.
However, big data is changing the world and CBR has created a list of some of the main ways that it is being used to shape the world we live in.
1. Security
Big data is being heavily used by law enforcement, particularly by national organisations such as GCHQ and NSA (ignoring the legality issues). These organisations have access to vast amounts of data which they use to catch criminals, foil terrorist plots and make us all feel safe.
Although cases of thwarted terrorist attacks are shrouded in mystery, the big data breaches in the past 10 years have also significantly created increases in data protection. The Snowden leaks and others have created great changes in the way data is accessed and the security measures behind storage.
2. Health
Big data analytics is accelerating the speed at which researches can work, for example DNA strings can now be decoded in minutes which can lead to the faster creation of cures and the ability to predict disease patterns.
Big data is being used to monitor premature and sick babies in some specialist units, with the techniques allowing the doctors to analyse every heart beat and breathing patterns. This leads to the development of algorithms which now allow for the prediction of infections 24 hours before any physical symptoms occur.
3. Science
Big Data analytics has made scientific research such as CERN possible, with the Large Hadron Collider producing astronomical amounts of data designed to unlock the secrets of the universe. CERN has its own data centre which contains 65,000 processors, however, it also relies upon thousands of computers across 170 data centres to analyse the data. The processing power is necessary to be able to analyse the 30 petabytes of data that the Hadron Collider produces annually.
Content from our partners
Big data is also aiding with space exploration – the Square Kilometre Array generates 700 terabytes of data a second and at full capacity is expected to produce 100 times more data than the internet. This obviously creates a huge big data problem, with the data useless if it cannot be processed and alaysed.
Partnerships with startups and large companies such as Amazon working with NASA with the NASA Exchange Platform will help to manage the data. The technology which can detect airport radar on a planet 50 light years away could eventually help to discover life on another planet.
4. Retail
Advertising and marketing has been one of the leaders in the use of big data analysis to improve the sector. Amazon and Netflix have taken advantage of big data collection by producing "recommendation engines." These work by making future purchase suggestions based upon the prior interests that were displayed by the individual customer. This tailored marketing allows firms to be much more specific and direct with product advertising.
Other insights allows retailers to significantly improve inventory management, for example new data insights show that after unusually cool weather, beer sales will decrease in some cities but increase in others. Using real-time analysis of current and historical weather, buying histories and personal history, retailers can more effectively plan ahead.
5. Finance
The financial industry is another area which has sought to make significant improvements through the use of big data. High Frequency Trading now uses big data algorithms to make trading decisions, with the majority of equity trading taking place by using data algorithms. These take into account information from social media networks as well as news websites so that buy and sell decisions can be made in split seconds.
Big data is also helping to keep financial organisations in check, regulations have been introduced to make sure that all pre-trade communications data is reported. Big data solutions can analyse the unstructured pre-trade communications and apply a number of rules to match the data to the right financial transactions. These systems can also flag anomalies and can help to prevent rogue trading and trading errors.
