Outlining the challenges of managing big data

Introduction

As the question and paper from the above-mentioned writers, suggests above, big data and the ability to analyse it is, and will continue to have a huge impact on how businesses work and evolve in the future. Big data is changing not juts computer savvy companies like Google and Facebook but companies from different industries. This answer will first define what big data is and it will then discuss how this is impacting businesses of all kinds.  

 

Big Data Intro

There are many different definitions of what big data is. As the name suggests it is when a lot of data needs to be processed but there is no exact cut of point when data as we have known in the past enters the world of “big data” that we have had in recent years. The general consensuses is that we ca consider a dataset to be “big data” when the dataset to too large to be managed by people or basic databases like Microsoft access. The data is impossible too big represent with a few simple graphs. 

 

This change has led to an increased need for data scientists in a world where it said that we as humans have produced more data in the last two years than we have in all of human history. This need will make all organisations more data driven but those who get the best analysts will be the ones that thrive. Services such as Amazon Web Services have democratised data management but getting strategic value from this data is the key challenge. In other words, 10 people could read a book, but only one person might understand what the writer was trying to say and puts the knowledge to effective use. This is big data analytics and its competitive advantage. Having the data does not guarantee the competitive advantage. 

 

Issues with Big Data

Not having an understanding of the data, having incorrect data or even not understanding what data should be collected can lead to terrible decision making. This is why it is vital to get the right data scientists even if they cost a premium. They will help an organisation avoid data silos and poor data quality. Big data can also be a big expense in terms of storage costs and integration costs. 

 

Finally, data is not the answer to everyone’s problems. We are not going to perfectly rational as economists wish, because we have all the data, if such a reality is even possible. Data cannot account for black swans written about by Nassim Taleb. The recent Covid-19 pandemic is a stark reminded that while data helps with decision making, it can’t see every possible possibility and at times a gut instinct will be needed. 

 

Big Data and the move from Analytics 1.0 - Analytics 3.0

As already mentioned traditional analytics also known as 1.0 was descriptive analytics. These datasets are small and don’t do a lot more beyond what has already happened. This was valued to a point by firms and still is but it does not add any strategic benefit to the firm. Analytics 2.0, like its web counterpart saw the emergence of new techniques of data gathering and data types. This is where unstructured data forms such as video, pictures and even social media started to become part of the data that firms analysed to gain a strategic advantage. Fiinally, we are now in the Analytics 3.0 where data is not seen as an extra resource but rather central to everything a business down from resource management to decision making. Data is not a by-product of business but rather the most important part of decision making. In fact, many entire business models are solely based on data. 

 

 

Case Study: Aviation Industry

In the past, data and Analytics 1.0 simply described what had happened in the past. But since then we have seen the start of diagnostic data analysis which looks at why x has happened. Predictive analysis looks at what will happen and finally prescriptive analysis asks what we should do now.  A real-world example written about by the writers in question was of PASSUR Aerospace who offer support for decision making in the aviation industry with the use of data analytics and big data. Pulling in data from datasets such as past flights, weather information and that of flight schedules they could predict better than past techniques as to when it would be safe for flights to take off and land. 

 

The use of big data let them predict exactly when flights would arrive which meant flights spent less time in the air which was a fantastic money saver for airlines and airports alike. It does this my reducing the amount of time planes spend in the air as the longer a flight is in the air the more money it loses the airline and if airports can use this analysis to get more flights to land safety they will make more money also. This is an example of an organisation optimization the time in the day and space on a run way with the use of big data and analysis to maximise earnings. As the writers mention in their piece “big data has the potential to transform traditional businesses” and the aviation industry is an example of this. 

 

Case Study: Betting and the Football industry

It goes without saying that big data has had an impact on data companies like Facebook and Amazon and we have seen already that it has impacted traditional industries such as aviation. But modern data analytics has also had a major impact on football. While baseball and Billy Beane was the first to catch on, big data and analysis now plays a huge role in the world’s most popular game.

 

The betting industry has been impacted greatly as fans are much more educated about the sport than they once were. Besides getting told what to think by pundits and newspaper columnists, big data football statistic websites are given better much more data than they had before. Companies such as Opta, Squawka, WhoScored, Statszone and so on have given everyone detailed datasets on player and team performance. With the use of tools like Tableau and Excel, betters can make they create their own datamaps and datasets. This has drastically reduced the asymmetric information advantage betting companies had over their customers. To stay ahead they will need to employ the best and brightest data scientists. With this in mind, betting companies will hope that very few catch onto the data revolution. 

 

Data has also impacted football in a more direct sense. As we learned in class, clubs like AC Milan used data to predict when players were at risk of injury. They have had the descriptive data, but now they are immersed in analytics 3.0 and predictive modelling. This helps the manager manage their squad of players to reduce the amount of injuries. From a financial point of view, as an injured player is an expensive and unproductive asset. Keeping your best assets in top-working condition maximises the financial investment in terms of wages and transfer fees. It also reduces financial liabilities when players are injured. These predictive analytics based on big data collected from body sensors from companies such as Titan Sensor, helps predict further threats and saves clubs lots of money. Football scouting and tactics have also been impacted but this answer will not have the time to speak about these.