Naicents
Sign inGet started
DDMA Level IIIData Management and Analytics
Chapters
Chapters0 of 7 readSign in to track progress
Chapter 1

Part A: Big data concepts

Big data management concepts

Last updated November 2026about 4 min read

Key takeaways

  • Big data is data too large, fast-moving or varied for traditional tools to capture, store and process effectively; big data management is the practice and technology for storing, organising and governing it throughout its lifecycle.
  • The five Vs describe big data: volume (scale), velocity (speed), variety (range of formats), veracity (trustworthiness and accuracy) and value (the worth that can be extracted).
  • Structured data fits a fixed schema of rows and columns, semi-structured data such as JSON or XML has some structure without a rigid schema, and unstructured data such as free text, images and video has no predefined format.
  • Data moves through a life cycle of capture, storage, processing, analysis and visualisation, from raw generation to actionable insight.
  • Predictive analytics, collective intelligence and machine learning are the techniques of big data analytics; they bring better decisions and earlier detection of trends, against infrastructure cost, data quality and privacy concerns and a shortage of skilled analysts.
Big data
Datasets too large, fast-moving or varied for traditional tools to capture, store and process effectively.
Big data analytics
Examining big data to uncover patterns, correlations and insights beyond what conventional analysis can handle.
Big data management
The practices and technology for reliably storing, organising and governing big data throughout its lifecycle.

Characteristics of big data, commonly summarised as the five Vs:

Volume
The sheer scale of data generated and stored.
Velocity
The speed at which data is generated and needs to be processed.
Variety
The range of formats and types data arrives in, structured, unstructured and everything between.
Veracity
The trustworthiness and accuracy of the data.
Value
The actual business or analytical worth that can be extracted from the data.

Different types of data:

Structured data
Fits a fixed schema of rows and columns, e.g. a relational database table (Databases).
Unstructured data
Has no predefined format, free text, images, video.
Semi-structured data
Has some organisational structure without a rigid schema, e.g. JSON or XML.
Metadata
Data describing other data, its source, format and meaning.
Big data life cycle
Data moves through capture, storage, processing, analysis and visualisation, from raw generation to actionable insight.

Examples of big data: social media activity streams, sensor and IoT device readings, transaction logs from large e-commerce platforms, and telecommunications call records.

Techniques of big data analytics:

Predictive analytics
Uses historical patterns to forecast future outcomes.
Collective intelligence
Draws insight from the aggregated behaviour of many users or agents.
Machine learning
Algorithms that improve their performance on a task automatically from data, without being explicitly reprogrammed.

Big data analytics business uses span fraud detection, personalised recommendations, supply chain optimisation, and customer sentiment analysis, among many others.

Big data analytics brings both benefits and challenges:

  • Benefits:
    1. Better-informed decisions.
    2. New revenue opportunities.
    3. Earlier detection of problems and trends.
  • Challenges:
    1. The cost and complexity of the infrastructure required.
    2. Data quality and privacy concerns.
    3. The shortage of skilled analysts to interpret it well.

Exam tip

The five Vs (volume, velocity, variety, veracity, value) are the standard framework for describing why a dataset counts as 'big data' rather than just 'large data', a question describing a dataset usually expects you to identify which V or Vs it illustrates.

Test Your Knowledge

3 questions on this chapter
Question 1 of 3A fraud team at a Kenyan bank finds that many customer records hold mistyped ID numbers and duplicated accounts, so it cannot trust what its analysis shows. Which of the five Vs is the problem?
Pick an option first
Question 2 of 3Customer feedback is stored as JSON files that carry named fields but follow no rigid schema. What type of data is this?
Pick an option first
Question 3 of 3In the big data life cycle, which stage comes immediately after storage?
Pick an option first