DDMA Level III
Data Management and Analytics cheat sheet
Every chapter's key points, formulas and definitions on one page, for the last week before the exam. Each heading links to the chapter that explains it.
Big data management concepts
- Big data is data too large, fast-moving or varied for traditional tools to capture, store and process effectively; big data management is the practice and technology for storing, organising and governing it throughout its lifecycle.
- The five Vs describe big data: volume (scale), velocity (speed), variety (range of formats), veracity (trustworthiness and accuracy) and value (the worth that can be extracted).
- Structured data fits a fixed schema of rows and columns, semi-structured data such as JSON or XML has some structure without a rigid schema, and unstructured data such as free text, images and video has no predefined format.
- Data moves through a life cycle of capture, storage, processing, analysis and visualisation, from raw generation to actionable insight.
- Predictive analytics, collective intelligence and machine learning are the techniques of big data analytics; they bring better decisions and earlier detection of trends, against infrastructure cost, data quality and privacy concerns and a shortage of skilled analysts.
Key terms
- Big data
- Datasets too large, fast-moving or varied for traditional tools to capture, store and process effectively.
- Big data analytics
- Examining big data to uncover patterns, correlations and insights beyond what conventional analysis can handle.
- Big data management
- The practices and technology for reliably storing, organising and governing big data throughout its lifecycle.
- Volume
- The sheer scale of data generated and stored.
- Velocity
- The speed at which data is generated and needs to be processed.
- Variety
- The range of formats and types data arrives in, structured, unstructured and everything between.
- Veracity
- The trustworthiness and accuracy of the data.
- Value
- The actual business or analytical worth that can be extracted from the data.
- Structured data
- Fits a fixed schema of rows and columns, e.g. a relational database table (Databases).
- Unstructured data
- Has no predefined format, free text, images, video.
- Semi-structured data
- Has some organisational structure without a rigid schema, e.g. JSON or XML.
- Metadata
- Data describing other data, its source, format and meaning.
- Big data life cycle
- Data moves through capture, storage, processing, analysis and visualisation, from raw generation to actionable insight.
- Predictive analytics
- Uses historical patterns to forecast future outcomes.
- Collective intelligence
- Draws insight from the aggregated behaviour of many users or agents.
- Machine learning
- Algorithms that improve their performance on a task automatically from data, without being explicitly reprogrammed.
The rest of the sheet covers
- Visualising big data problems
- Statistical tools for big data
- Managing big data with R
- Data analytics with R
- Exam technique and revision
- Glossary