Definition of Big Data

 Definition of Big Data


Today, we introduced the concept of Big Data and its place in today's world.

Big Data can be defined as 'data so large and complex that no traditional data management tools can process or store it effectively.' 

It comes in three types:

  • Structured- Large amounts of data organised in a structured format, such as tables with clear rows and columns.

  • Semi-structured- A Vast amount of data that is not stored in traditional, organised ways such as tables but has some organisation such as tags or metadata, making it easier to analyse than without structure. 

  • Unstructured- This is a large amount of data that doesn't have any specific format or structure and is, therefore, difficult to analyse effectively.

An example of Big Data could be the data collected from the New York Stock Exchange, which generates about one terabyte of data daily, or data collected by Facebook, which stores over 500 terabytes of data every day.



1. https://www.geeksforgeeks.org/types-of-big-data/

Comments

Post a Comment

Popular posts from this blog

Limitations of Traditional Statistics

Image
 Limitations of Traditional Statistics In my last post, I spoke about the use of traditional statistics. Now, I will cover some of the drawbacks of using traditional statistics to make predictions in the real world. Traditional data analysis has been practised with structured data, often in a relational database; however, Big Data isn’t as manageable with this structured format. Traditional statistics is about analysing and summarising data and is suited to processing smaller amounts of linear, repeatable data to draw a conclusion. The easiest environments are when the data has a stable relationship – e.g. regular exercise increases, obesity decreases. Therefore, the practicality of traditional data analysis is limited when applied to the real world. More recently, there is now the availability of new large data sets and cheaper and faster computing power. In addition, there has been a surge in machine learning research and development, which aims to automatically discover re...
Read more

Application of Big Data Techniques to a Problem

 Application of Big Data Techniques to a Problem From the previous posts, it is clear that Big Data is collected, stored and utilised in the real world at enormous scales by huge companies. Now I want to show directly how these companies use this data to address their problems in the modern world. Netflix Netflix is known for its great, vast choice of shows that users love. But how does Netflix know what each individual customer might want to watch? Netflix recommends specific titles for each user on the home page. It uses multiple algorithms for personalised rankings, searches, similarity ratings, ranking and more to narrow down the shows to display on the recommended screen for each individual customer. It looks at what titles the user has watched, searched for, and rated good or bad so the system can show them similar content. Additionally, if user A shows the same interests as user B, Netflix will show user A the user B's recommended shows. Each show is also put into some of th...
Read more

Traditional Statistics

 Traditional Statistics Today we learned about types of traditional statistics and how they're used to draw conclusions.  Statistics is the discipline that concerns the collection, organisation, analysis, interpretation, and presentation of data. It can be broken up into descriptive and inferential statistics. Descriptive Statistics Only the subjects that have been analysed have findings implied upon them. If we had the political views of 20 people, we may be interested in the voting intentions of these people or the distribution or spread of the votes, but the findings are limited to only these 20 people. Inferential Statistics If you don’t have access to the whole population you’re interested in, but only a small portion. If you want to study the political voting intentions of everybody in Scotland. From what I've learnt about the two types, I would say that descriptive is more accurate as conclusions are drawn from what the data had been directly collected from. Howev...
Read more