Big Data

 Types of Visualisation Today, we covered how data can be presented visually. I have spoken about how data is used to draw conclusions and make predictions, however, I haven't yet discussed how data can be visualised to draw these conclusions. It would be impossible to understand vast data through numerical spreadsheets of data or even just large stores of unorganised data. This is why it's important to use data visualisation. Data visualisation is the graphical representation of data. We can use charts, graphs, maps, and other visualisation tools to see and understand trends, patterns and outliers. Some examples are: Tables These are a collection of related data organised in columns and rows. Graphs These are diagrams that show the relationship between variables across two axes. Maps These are diagrams to display location data in their geographic locations. Infographics These are representations of information or data in a graphic format to make the data easily understandable....

 Data Mining Techniques I have previously spoken about data analysis and the real world impact of it. Today, I discovered a website which highlighted the common data mining techniques which are used to discover hidden patterns and correlations is vast amounts of data, which is then used for decision making and predictive modelling. Some of the most common data mining techniques I discovered are: Classification - A technique used to categorise data into predefined classes based on the data's features. Association Rule - Discovering interesting relationships or patterns among a set of items in transactional or market basket data. It helps identify frequently co-occurring items and generates rules to show associations between items, such as if  X, then Y. Anomaly detection - Aims to identify unusual data instances that deviate significantly from the expected patterns. Some use cases may be detecting fraudulent actions or network intrusions. Clustering - To group similar data inst...

 Types of Problems Suited to Big Data Analysis Today, we covered the problems that arise with analysis from Big Data. We discussed why Big Data might be less predictive and powerful than first thought. For example, Big Data that has been collected over the past 5 years sounds like it could be trustworthy enough to form strong predictions from; however, the longer the Big Data collection has gone on, the better. If data has been collected for 20 years, it will hold much more accuracy than data collected over 5 years. This is because it can pick up trends that have appeared, then disappeared, then perhaps reappeared.  The issue that follows on from that is if you want to create accurate predictions, you have to find data that has been collected at a wide scale for decades, which is still rare in todays world but also expensive, especially if you want to start collecting it today. You would require large expensive data collection methods, long term, backed up storage facilities a...

Strategies for Limiting the Negative Effects of Big Data As previously discussed, there are negative effects of Big Data in modern society. There are strategies that are employed to limit these effects. Today, we discussed the Data Protection Act 2018 as one of the strategies. It incorporates the European Union’s General Data Protection Regulations (GDPR), which prohibit processing personal data unless it is expressly allowed by law or consent is given by the subject. It helps to ensure companies protect the data they collect from external forces. We also read an extract from the National Union of Journalists which described another strategy being the emphasis on free press and investigative journalism. This is a right that allows individuals to express, publish and share information and ideas without fear of censorship or government interference. I believe that this is just as important as explicit legislation, as it too holds companies accountable.

Implications of Big Data in Society Previously, I covered Big Data in Science, today, we looked at how Big Data is utilised in Society. Big Data has been incorporated in modern societies for years, and it is only going to play a larger role as its uses are better understood and its technology develops further. Today, Big Data is used in society across most departments, including water, energy, communication, housing and mobility to improve people’s quality of life. In policing, for example, it has been used to predict and identify where crimes are most likely to occur. This means the limited resources can be utilised more efficiently. It can also be used to identify the risks associated with particular individuals, such as those at risk of reoffending or being the victims of crimes. The police may also grasp a better potential of data collected through surveillance, such as CCTV or automatic number plate recognition. Finally, Big Data collected from social media allows police to ...

Implications of Big Data for Individuals   Firstly, it can be argued that Big Data leaves societies vulnerable and breaches civil liberties, especially in authoritarian governments that may utilise Big Data sets for immoral reasons, such as against political opponents. Big Data that has been collected by companies can be analysed by security services. These could include very personal data such as NHS health records. We watched a really good video which simplified the Investigatory Powers Act 2016 and it explained what the act would allow authorities to carry out. Regulation of Investigatory Powers Legislation related to the implications of Big Data for Individuals includes the Investigatory Powers Act 2016, which allows: ·         Security services to legally bug computers and phones with a warrant. ·         Security services to analyse bulk collections of communications data. ·...