CISSE Advanced Level
Big Data Management glossary
Every term this guide defines, A to Z, each with the chapter that teaches it. Open it when a word in a question is unfamiliar, and test yourself on it in the last week: cover the definition and say it in one sentence.
14 terms from the free first chapter
B
- Batch processing
Processing a large, bounded set of data in one run, such as a nightly job that recalculates every customer's credit score. High throughput, but results are hours old.
Taught in Introduction to big data
D
- Data locality
Moving the computation to the machine that holds the data rather than moving large data across the network to the computation.
Taught in Introduction to big data
- Data mining and machine learning
Techniques that search large datasets for patterns, groups, associations and predictive relationships, such as fraud scoring or market-basket analysis.
Taught in Introduction to big data
- Distributed storage
Splitting data into pieces and storing them across many machines, with copies kept so that a failed machine loses nothing. HDFS is the classic example, covered in the chapter on the Hadoop ecosystem.
Taught in Introduction to big data
I
- In-memory processing
Keeping working data in RAM rather than on disk so that repeated calculations and interactive queries run much faster, the approach Spark takes.
Taught in Introduction to big data
N
- NoSQL databases
Non-relational stores (key-value, document, wide-column and graph) that relax the fixed schema and scale out across a cluster.
Taught in Introduction to big data
P
- Parallel processing
Dividing one job into many tasks that run at the same time on different machines, then combining the partial results. MapReduce and Spark both work this way.
Taught in Introduction to big data
S
- Schema-on-read
Storing raw data first and applying a structure only when it is read for a particular analysis, the opposite of a relational database's schema-on-write.
Taught in Introduction to big data
- Stream processing
Processing each record, or a small window of records, as it arrives, such as flagging a suspicious transfer within seconds. Low latency, covered with velocity management in the chapter on big data mining and warehousing.
Taught in Introduction to big data
V
- Value
The business benefit extracted from the data through analysis, such as insight discovered, patterns recognised and decisions improved. Data that is stored but never turned into value is only a cost.
Taught in Introduction to big data
- Variety
The range of forms and sources the data comes from: database tables, log files, emails, JSON from mobile apps, images, audio, video and GPS positions from smartphones, often all feeding the same analysis.
Taught in Introduction to big data
- Velocity
The speed at which data is generated and must be processed. Card payments, mobile money transfers and sensor readings arrive continuously and often need a response in seconds, not at the end of the month.
Taught in Introduction to big data
- Veracity
The trustworthiness of the data: how accurate, complete and consistent it is. Big data is often noisy, with duplicates, missing fields and conflicting records, so its quality must be measured before it is used.
Taught in Introduction to big data
- Volume
The quantity of data generated and stored, measured in terabytes, petabytes and beyond. Volume is the reason data must be spread across many machines rather than kept on one disk.
Taught in Introduction to big data
The full glossary adds every term from the other 12 chapters, A to Z, each with the chapter that teaches it.