The Measure of a Food Safety Program Is How Much Risk It Removes

In food safety, we often spend a lot of time talking about food testing. These conversations often revolve around sample mass, number of grabs, lot size, timing, etc. But, instead of getting lost in the specifics of sample mass, grabs, etc., we should start with the meaningful question is: What contamination is this program designed to detect, with what probability, and what happens when it finds it?

The mechanics of testing matter-the sample mass, the number and distribution of grabs, the lot they represent, and the limitations of the test. But their value depends on the design and execution of the whole sampling and testing program: how intensively we sample, where and when samples are collected, how well they represent the product being evaluated, how confident we are that collection followed the sampling plan, and how reliably the analytical method detects the target pathogen.

A simple number in grams cannot describe that system. Yet too often, that is where we end up discussing and evaluating the "value" of a sample.

What Are We Actually Measuring?

Grabs

Pathogens are not necessarily distributed evenly across a field, irrigation water source, a production run, or a finished product lot. Contamination can be, and usually is, sporadic, clustered, and difficult to capture. Increasing the mass collected from one location does not necessarily provide the same information as collecting increments across locations and times. We usually call these increments grabs or 'n'.

Think of searching a beach for lost keys. One large, deep scoop (a large grab) in one part of the beach examines a lot of sand in one specific area but covers little of the entire beach. Many small scoops that are spread across the beach cover far more ground, giving us higher odds of finding the keys. The figure below puts this into perspective. In our simulated lot, a single large scoop finds a hot spot covering 1% of the lot 3.5% of the time, while 60 small grabs of the same total size find it 53.0% of the time (assuming every grab that touches the hot spot detects it).

Mass

Mass is the amount of material each grab, and the whole sample, actually examines. A bigger grab is more likely to contain the organism if it lands in contamination, and it touches a slightly larger patch. But mass collected in one place is still one chance. For a fixed total sample, the design question becomes how to split it: fewer, larger grabs, or more, smaller ones.

There is a limit. Splitting a sample into more pieces covers more of the lot, but each piece holds less material, so it is less likely to contain any organism at all.

Intensity

Intensity is how much sample we collect for the ground it represents: total mass per total unit we are trying to represent. Grabs and grab mass describe the design of a sample, but intensity describes how thinly that sample is stretched over the lot.

To see the effect, hold a small hot spot fixed at 1% of a one-acre lot (0.01 acre) and change only the intensity, as in the figure below. Detection rises with intensity at every contamination level, but the contamination level sets how much intensity is needed. At 100 CFU/lb (a very high level of contamination) the plan on one acre finds the hot spot 56% of the time; stretched over a theoretical 10 acres it falls to 8% of the time. At 10 CFU/lb (10x lower contamination level) the same two cases are 23% and 2.5%, and at 1 CFU/lb (level calculated to have been representative of an outbreak) they are 3% and 0.3%.

Sampling is a probability exercise; the figure below and explanation above shows that clustered contamination is hard to find. This doesn't mean a plan is flawed. It shows that mass, grabs, and the area they represent work together, and that no single number describes how well a plan performs. Intensity matters, so mass and grabs cannot be spoken as independent numbers.

Putting it all together

The reality is, to evaluate the value of a sample, we need to understand the grabs, the amount of material examined (mass), the area/pounds/size of the lot it represents (intensity), and the opportunities the sampling design creates to encounter contamination if it is present. Just as importantly, and perhaps more so, is that we also need to understand the uncertainty in using those results to draw conclusions about the larger population (i.e., lot, field, water source, input). Codex recognizes this limitation: a sampling plan can describe the probability of detecting contamination, but no sampling plan can establish that a particular organism is absent from an entire lot.

A negative result means the target was not detected in the material examined, under the conditions of that test. The confidence we place in that result must come from the design and performance of the whole program. That makes "which sample tests more?" an inadequate and incomplete measure of food safety performance.

We should be asking what each testing and sampling program can detect, what it is likely to miss, how confident are we in the performance of the sampling and testing, and whether that performance is appropriate for the hazard and the exposure involved.

Testing and prevention serve different purposes.

Testing provides information. It can identify contaminated product, reveal emerging problems, verify controls, and support decisions that prevent exposure. Those are important functions, but the test itself does not remove a pathogen or correct the conditions that allowed contamination to occur. It tells us something (mostly when it finds something), but it doesn't tell us anything more than what the program was designed to tell us.

Let's explore what a positive test result can do. When a positive is found, the product is held and diverted so that the pathogen doesn't go into the supply chain, and ultimately to a consumer. The protective action is the containment and disposition of that product. Its effectiveness as a mitigation still depends entirely on whether the sampling program finds the contamination in the first place.

This is why testing alone should rarely carry the primary burden of risk mitigation. Trying to intercept contamination after it has occurred can require substantial sampling, time, and expense while still leaving opportunities for contaminated product to pass undetected. Codex places the foundation of food safety in source control, hygienic practices, and process controls, recognizing that preventive approaches provide greater control than microbiological testing alone. The ultimate value of a food safety program is the system, and the power (or intensity) of that program to address specific thresholds of risk.

The question for investment by the food industry should therefore be: Where will the next dollar produce the greatest reduction in consumer risk?

Sometimes better sampling is necessary. Sometimes there would be better impact if we improved the underlying infrastructure more - addressing the uncertainty around the sampling, the testing, the variables that we are measuring with the testing (inputs, practices, etc.) that may interact with the test outcomes. Generally, the greater benefit comes from improving the entire program to understand and manage more effectively agricultural water use, sanitation practices, equipment design, or other controls that addresses minimizing contamination (or allowing it to reduce - die off, etc.) at its source. We should be designing the testing programs to help us better characterize our systems - highlighting what we know, what remains to be known, and ultimately to identify and evaluate the effect of improvements once made.

Zero tolerance does not mean zero uncertainty.

Our goal should be to prevent consumers from being exposed to foodborne pathogens.

But a zero-tolerance criterion for a target pathogen is not a demonstration that a testing program can detect every instance of its presence. Requiring action whenever a pathogen is detected and being capable of detecting all contamination are different things. Every practical sampling program has both theoretical and real limits. It most definitely has some probability of detecting a specified contamination scenario, but it also has some probability of missing it. We should make those limits explicit and intentional.

We should aim to be able to explain the prevalence, concentration, and distribution of contamination it is designed to detect, with a stated probability and under stated assumptions. In short, what type of scenario are we aiming to find (1 CFU/lb, gross contamination in x% of lot/servings/pounds, etc.), and how well do we actually know our program executed to find it.

Those characteristics describe detection performance and the overall efficacy of the testing and sampling program. Without those connections, we risk treating testing activity as evidence of protection, and that is rarely, if ever, the best way to address the challenge.

Outcomes must challenge our assumptions.

A food safety program cannot be evaluated solely by whether its procedures were followed. It must also be evaluated against what happens to the people it is intended to protect. Illnesses and outbreaks are evidence that unacceptable levels of exposure have occurred. Positive findings and recalls provide additional information, although their meaning depends on context: a product test result or recall initiated through effective detection before consumption tells a different story from one prompted by an outbreak.

These signals deserve investigation individually and collectively. Their significance is not captured by a count alone, and we should not wait for a succession of outbreaks before improving a program. When the evidence shows that consumer harm occurred, this remains unacceptable (too many outbreaks/illnesses)and we need to reconsider both our controls and the residual risk our system permits. That may mean redesigning surveillance to detect more contamination than was previously designed for (same sample in smaller lots, less contamination per pound like 1 CFU/0.5lb, 1 CFU/lb in a smaller lot, etc.), responding earlier to warning signs by designing a program that teaches more earlier so that we can intervene (index genes, etc.), strengthening preventive controls, or some combination of these actions. Increasing testing alone cannot substitute for identifying why contamination exists, continues, and/or how to reduce it in the first place.

A program can meet its existing specifications and still need to improve. The specifications themselves may no longer provide sufficient protection.

Continuous improvement needs direction and pace.

Continuous improvement requires more than a commitment to do better. It requires decisions about how much risk to remove, which actions are most likely to remove it, and how quickly those actions must occur.

Testing helps inform those decisions. Its value grows when results lead to investigation, corrective action, and measurable improvements in control. That is the conversation we should be having.

What level of contamination can our program reliably detect? How much uncertainty remains? What exposure can occur before we recognize a problem? And when the evidence shows that the remaining risk is unacceptable, how quickly will we change to address that?

Let's start evaluating the risk our programs are designed to detect, the exposure we are willing to tolerate, and the actions we will take to reduce both.

Have questions? The WG Science team is here to help on how to do this evaluation.