Part A: Framing the research
Introduction to research
Key takeaways
- Scientific research is systematic, controlled, empirical and critical, and it is public and self-correcting, which is what separates it from opinion, however expert.
- An operational definition turns a construct into a measurable variable by stating exactly how it will be measured.
- The unit of analysis decides how many cases a study really has: 300 employees sampled to conclude about 12 firms is 12 cases, not 300.
- Reliability is consistency and validity is truth: an instrument can give the same wrong answer every time.
- Steps 1 to 6 of the research process are the proposal and steps 7 to 10 are the project, so a proposal never contains findings.
Research is what a quality professional does when the answer is not already known. Auditing tells you whether a requirement is met. Inspection tells you whether a unit conforms. Research tells you why the process behaves as it does, and it is the only one of the three that produces new knowledge.
What scientific research is
Scientific research is the systematic, controlled, empirical and critical investigation of a question, guided by theory and evidence rather than by authority, intuition or precedent. Each of those four words does real work and each one is a place where workplace investigations usually fail.
Two further properties follow from these. Research is public: methods and data are described fully enough that a competent stranger could repeat the work. And research is self-correcting: findings are provisional, held only until better evidence arrives. Neither property belongs to opinion, however expert.
Basic terms
Reliability without validity is the commonest measurement failure
A thermometer reading 3 degrees low is perfectly reliable and completely invalid: it gives the same wrong answer every time. In quality work this is the calibration problem, and it is why a consistent measurement system is necessary but not sufficient.
Types of research
The same study can be classified several ways at once, and examiners expect the classification axis to be named rather than assumed.
Classifying a study by six independent axes
| Axis | Types | What distinguishes them |
|---|---|---|
| Purpose | Basic (pure, fundamental) and applied | Basic research seeks knowledge for its own sake, testing or building theory. Applied research solves a stated practical problem. Almost all CQP research is applied. |
| Objective | Exploratory, descriptive, explanatory (causal), predictive | Exploratory maps an unfamiliar problem; descriptive documents what is; explanatory establishes why; predictive forecasts what will be. |
| Data type | Quantitative, qualitative, mixed methods | Quantitative measures and counts; qualitative interprets meaning in words, images and observation; mixed methods deliberately combines both to offset each one's weakness. |
| Setting | Laboratory, field, simulation | Laboratory maximises control at the cost of realism; field research reverses that trade-off. |
| Time | Cross-sectional and longitudinal | Cross-sectional observes at one point in time; longitudinal follows the same cases over time and is the only design that can show change within a case. |
| Involvement | Non-interventionist and action research | Action research deliberately intervenes and studies the effect, with the researcher inside the system. Most quality improvement research is of this kind. |
The same quality problem, researched six ways
PROBLEM: a milk processor in Nakuru is rejecting 6% of incoming raw
milk on the alcohol stability test, against a 2% industry benchmark.
Exploratory Ten unstructured interviews with transporters, farmers
and graders to find out what anyone thinks is going on.
Output: a list of candidate causes, not an answer.
Descriptive Twelve months of rejection records analysed by route,
season, collection centre and time of day. Output: the
rejections cluster on two routes and in the dry season.
Explanatory Measure ambient temperature, time from milking to
chilling, and rejection outcome on 400 consignments;
test whether time-to-chilling predicts rejection with
temperature controlled. Output: a causal claim.
Predictive Build a model that flags consignments likely to fail
before they reach the plant. Output: a decision rule.
Action Install chilling at the two problem centres, then
measure the rejection rate before and after against
unchanged centres. Output: whether the fix worked.
Qualitative Spend three weeks riding the collection routes and
observing practice. Output: the finding that the
measured 'time to chilling' is systematically
understated because drivers record departure time
rather than milking time, which invalidates the
explanatory study above.
NOTE what the qualitative work did: it did not add colour to the
quantitative finding, it destroyed it. Mixed methods exists for
this reason.
Advantages of research
- It replaces opinion with evidence. In an organisation where decisions follow seniority, a defensible study is the only instrument a junior quality officer has for changing a senior manager's mind.
- It reduces the cost of being wrong. A pilot study costing KES 200,000 that shows an intervention does not work has saved whatever the full rollout would have cost.
- It generalises. A properly sampled study lets you say something about 4,000 batches from 60, which no amount of inspecting individual batches achieves.
- It uncovers causes rather than symptoms. Corrective action addressed to a symptom recurs; research is how the actual cause is found.
- It creates organisational memory. A written study survives the departure of the person who did it, which tacit knowledge does not.
- It supports certification and regulatory arguments. Evidence of the effectiveness of controls is precisely what an ISO 9001 or a KEBS surveillance audit asks for.
- It contributes to the wider body of knowledge, which is how the discipline of quality improves at all.
Disadvantages of research
- It is expensive in money and in the scarcest resource, competent attention. A good study consumes months of a quality professional's time.
- It is slow. Management frequently needs an answer this week, and the honest research answer is that a defensible one will take three months.
- Findings can be wrong, and wrong findings carry the authority of the method. A badly sampled study is more dangerous than no study, because it is believed.
- Results may not transfer. A finding established in a Thika factory may not hold in a Mombasa one, and the study itself usually cannot say whether it does.
- Research can be captured. Whoever pays chooses the question, and the question chosen determines much of the answer.
- It can disturb what it studies: the act of measuring operator performance changes operator performance, which is the Hawthorne effect.
- Ethical costs are real. Data collected on people creates obligations that outlast the study.
The disadvantages are reasons to do research well, not reasons to avoid it
Every alternative to research (custom, authority, intuition, anecdote) shares all of these weaknesses and lacks the mechanism that corrects them. The honest comparison is not research against certainty; it is research against guessing.
The research process
The process is presented as a sequence because it must be written down in some order, but in practice it iterates: the literature review changes the problem, the pilot changes the instrument, and the analysis sometimes sends you back for more data. What must not iterate is the hypothesis after the results are seen.
The ten steps and where each one usually goes wrong
| Step | What happens | The commonest failure |
|---|---|---|
| 1. Identify the problem | A gap between what is known and what needs to be known is stated precisely. | Stating a topic instead of a problem. 'Quality management' is a topic; 'why certified firms in Athi River report no fewer customer complaints than uncertified ones' is a problem. |
| 2. Review the literature | What is already established, and where the gap actually lies, is documented. | Reviewing after the data is collected, so the study repeats work already done. |
| 3. Formulate objectives, questions and hypotheses | The problem is broken into answerable pieces with testable statements. | Objectives that no possible data could achieve. |
| 4. Design the study | Design, population, sample, instruments and analysis plan are chosen and justified. | Choosing the method first and fitting the question to it. |
| 5. Obtain approvals | Institutional clearance, NACOSTI permit, county authorisation and organisational consent are secured. | Leaving this to the end and losing a month. |
| 6. Pilot | Instruments are tested on a small group and revised. | Skipping it, then discovering an ambiguous question after 300 responses. |
| 7. Collect data | The plan is executed and departures from it are recorded. | Silent substitution of accessible respondents for sampled ones. |
| 8. Process and analyse | Data is cleaned, coded, analysed and presented. | Analysing until something is significant, then reporting only that. |
| 9. Interpret and conclude | Findings are related back to the objectives and the literature. | Concluding beyond what the design supports, especially causally. |
| 10. Report and disseminate | The work is written up and made available for scrutiny and use. | A study that is never read changes nothing. |
Steps 1 to 6 are the proposal; steps 7 to 10 are the project
This is exactly the split KASNEB uses between the proposal a candidate submits and the completed research project. If your proposal contains findings, you have misunderstood the split.
Research and religion
The syllabus asks for this comparison because both are systems for arriving at held beliefs, and a candidate who cannot distinguish their grounds will confuse conviction with evidence in their own work. The comparison is about method, not about worth: a claim about the natural world and a claim about ultimate meaning are answerable in different ways, and neither method is competent in the other's domain.
Two ways of holding a belief, compared on method
| Dimension | Scientific research | Religion |
|---|---|---|
| Source of knowledge | Observation, measurement and reasoning from evidence | Revelation, scripture and religious authority |
| Basis of acceptance | Evidence that others can examine and reproduce | Faith, which does not depend on empirical demonstration |
| Status of claims | Provisional; every finding is open to revision by better evidence | Held as enduring truth, not contingent on new data |
| Method of resolving disputes | Further evidence, replication and peer scrutiny | Interpretation of authoritative texts and teaching |
| Domain of questions | Questions answerable by observation: what is, how much, why, what follows | Questions of ultimate meaning, purpose, moral obligation and the transcendent |
| Attitude to doubt | Doubt is the engine; a study is designed so it could fail | Doubt is typically something to be resolved rather than sought |
| Test of a claim | Falsifiability: it must be possible to say what would show it wrong | Not framed as falsifiable, and not intended to be |
This is not a ranking
A great many working scientists, including Kenyan ones, hold religious beliefs without contradiction, because the two answer different classes of question. What the researcher must not do is import the epistemology of one into the other: accepting a research finding on authority, or demanding empirical proof of a claim that was never empirical.
Research and tradition
Tradition is knowledge transmitted because it has been transmitted. It is not worthless: indigenous knowledge encodes a great deal of genuine observation, and in Kenya the traditional preservation of milk in smoked gourds and the use of specific plants as antimicrobials both turned out to have real microbiological basis. The problem is that tradition has no mechanism for detecting its own errors.
Research and tradition compared on method
| Dimension | Scientific research | Tradition |
|---|---|---|
| Justification | Because the evidence supports it | Because it has always been done this way |
| Transmission | Written, explicit, with the method recorded | Oral and practical, with the reasoning usually lost |
| Error correction | Built in through replication and revision | Absent; an error transmitted faithfully is transmitted forever |
| Scope of validity | Stated explicitly with its limits | Assumed universal within the community |
| Response to contradiction | The claim is revised | The contradiction is explained away or the source discredited |
| Speed of change | As fast as evidence accumulates | Generational, and often resisted |
| Value in quality work | Establishes what actually causes a defect | Encodes accumulated practical skill that no manual captures |
Tradition as data, not as authority
A honey processor in Baringo finds that operators consistently
reject certain combs on sight, before any test. Asked why, they
say only that the honey 'is not right' and that this is what
they were taught.
THE WRONG RESPONSES
Accept it The operators are experienced, so the practice
stands. But nobody can train a new operator, the
rejection rate cannot be audited, and if the rule
is wrong good product is being destroyed.
Dismiss it It is unscientific, so it is banned. This is how
organisations destroy tacit knowledge that took a
generation to accumulate.
THE RESEARCH RESPONSE
Treat the practice as a hypothesis. Have operators mark 120
combs as accept or reject, then test every comb for moisture
content, HMF and diastase activity, with the laboratory blind
to the operator's call.
RESULT: operator-rejected combs averaged 23.4% moisture
against 18.1% for accepted combs, and 31 of 34 rejections
exceeded the KS EAS 36 limit of 21%. The operators were
detecting moisture by the appearance of the capping.
WHAT THIS BUYS: the tacit skill is now an explicit, teachable
and auditable screening rule with a known error rate. The
tradition was right, and research is what made it usable.
Self-check
Distinguish research from routine problem solving, and explain why a quality professional needs both. (10 marks)
Explain six characteristics of scientific research and, for each, give one way in which a typical workplace investigation falls short of it. (10 marks)
Show worked answer
Research and routine problem solving (10 marks).
Routine problem solving applies a known method to a known class of problem to restore a standard: a nonconformity is raised, root cause analysis is applied, corrective action is taken and effectiveness is verified. The knowledge required already exists in the organisation. Research is undertaken when the knowledge does not exist: the cause is genuinely unknown, or the effectiveness of a proposed intervention has never been established, or the question is about the population rather than the case. Research is therefore systematic, controlled, empirical and critical in a way routine problem solving need not be, it is designed before the data is seen, its method is documented so that others may repeat it, and its conclusions are stated with their limits and their uncertainty. Both are needed because most quality failures are of the routine kind and applying research machinery to them is wasteful, while the minority that recur despite repeated correction are exactly the ones where the organisation's existing knowledge is wrong, and only research will find that out.
Six characteristics of scientific research, and how workplace investigation falls short (10 marks).
Systematic: the steps follow a plan set out before data is collected. Workplace investigations typically improvise, deciding what to look at after seeing the first results, so they cannot state in advance what evidence would have changed the conclusion.
Controlled: competing explanations are ruled out deliberately. Workplace investigations usually change several things at once, so when the defect rate falls nobody can say which change caused it, and all of them are retained at permanent cost.
Empirical: conclusions rest on observation others could repeat. Workplace investigations frequently rest on the recollection of an experienced operator, which may well be right but cannot be checked.
Critical: the investigator looks for weaknesses in their own work. Workplace investigations are usually presented to management as findings rather than as claims with stated limits, because admitting uncertainty is read as incompetence.
Public: the method is described fully enough for a stranger to repeat it. Workplace investigations are commonly reported as a conclusion and a corrective action with no method section at all, so nobody can audit the reasoning.
Self-correcting: findings are provisional and revised when better evidence arrives. Workplace investigations, once closed and signed off, are rarely revisited even when the problem recurs, and the recurrence is treated as a new incident rather than as evidence the first conclusion was wrong.