~V
1237_C08.fm Page 263 Wednesday, November 12, 2003 1 :06 PM
Q Predictive Microbiology
in Quantitative Risk
Assessment
Anna M. Lammerding and Robin C. McKellar
CONTENTS
8.1 Introduction
8.2 Assessing Microbial Risks
8.3 Role of Predictive Microbiology in QRA
8.4 Scope of Risk Assessments
8.5 Process Risk Modeling
8.6 Examples of Risk Modeling
8.6.1 PRM for E. coli 0157:H7 in Ground Beef
8.6.2 Poultry Farm Model
8.7 Modifying Risk: Concentration vs. Prevalence
8.8 What Is the Right Model to Use?
8.9 Future Directions
8.10 Conclusions
References
8.1 INTRODUCTION
Food-borne disease arises from the consumption of microbial pathogens, microbial
toxins, or both, by a susceptible individual. The risk of food-borne disease is a
combination of the likelihood of exposure to the pathogen, the likelihood of infection
or intoxication resulting in illness, and the severity of the illness. In a system as
complex as the production and consumption of food, many factors affect both the
likelihood and the severity of the occurrence of food-borne disease. Many of these
factors are variable and often there are aspects for which little information is cur-
rently available. To manage food safety effectively, a systematic means of examining
these factors is necessary.
Historically, the production of safe food has been based on numerous codes of
practice and regulations enforced by various governing bodies worldwide. With the
increased concern regarding the existence of microbial hazards in foods, a more
objective approach is warranted, which has led to the introduction of the Hazard
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 264 Wednesday, November 12, 2003 1:06 PM
Analysis Critical Control Point (HACCP) system. HACCP as a tool for safety
management consists of two processes: building safety into the product and exerting
strict process control. 1 The principles of HACCP have been set out by the Codex
Alimentarius Commission 2 and consist of seven steps: hazard analysis; determination
of critical control points (CCP); specification of criteria; implementation of moni-
toring system; corrective action; verification; and documentation. 3 HACCP processes
as defined for various food products are often based on qualitative information and
expert opinion. Moreover, the microbiological criteria underlying HACCP are often
poorly understood or defined. 4
The concept of risk assessment as defined by the Food and Agriculture Organi-
zation (FAO) and the World Health Organization (WHO) 5 provides a more quanti-
tative approach to food-borne hazards. Quantitative risk assessment (QRA) is the
scientific evaluation of known or potential adverse health effects resulting from
human exposure to food-borne hazards. 1 ' 6 Risk assessment is a systematic framework
and process that provides an estimate of the probability and impact of food-borne
disease. In doing so, exposures to food-borne pathogens are translated into actual
human health outcomes.
Quantifying the human health risks associated with the ingestion of specific
pathogens in specific foods has been considered feasible only within the last decade.
Historically, until the mid-90s, risks associated with foods were estimated, at best,
qualitatively, largely with reliance on epidemiological evidence and expert opinion
to determine "high risk" vs. "low risk." Evaluations of the risks associated with
food-borne hazards, in general or attributable to specific foods, have been predom-
inantly qualitative descriptions of the hazard, routes of exposure, handling practices,
consequences of exposure, or all of these. Quantifying any of these elements is
challenging, since many factors influence the risk of food-borne disease, complicate
interpretations of data about the prevalence, numbers, and behavior of microorgan-
isms, and confound the interpretations of human health statistics. Consequently,
policies, regulations, and other types of decisions concerning food safety hazards
have been largely based on subjective and speculative information.
Today, advances in our knowledge, analytical techniques, and public health
reporting, combined with increased consumer awareness, global trade consider-
ations, and realization of the real economic and social impacts of microbial food-
borne illness, have moved us toward the threshold of using QRA to support better
prioritizing and decision-making.
Developments in the field of microbial risk assessment have some resemblance
to the growth characteristics of a microbial population (see Chapter 2). During what
might be termed the lag phase, few researchers attempted to define and model the
food chain quantitatively. Today, the field can be described as entering the log phase,
as efforts increase internationally to develop sophisticated models in response to
risk managers' needs in decision-making.
The recent ratification of the World Trade Organization (WTO) agreement is
having a major impact on the development of new approaches for the regulation of
food. Countries are encouraged to base their procedures on Codex standards and
guidelines to maintain and enhance safety standards. 7 This will lead to the develop-
ment of harmonized risk assessment and risk management frameworks, providing
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 265 Wednesday, November 12, 2003 1:06 PM
input into HACCP, which is the primary vehicle for achieving enhanced food safety
goals. 7 As the use of HACCP increases, there will be a need for a clear understanding
of the relationship among HACCP, microbiological criteria, and risk assessment. 4
Regulators will be called upon to participate in all aspects of HACCP development,
in particular to establish public health-based targets, elucidate microbiological cri-
teria, develop improved techniques in microbiological risk assessment, and develop
the means for evaluating the relative performance of HACCP systems. 4 Harmoniza-
tion of international rules will clearly require standardized approaches. 8
For microbial pathogens in foods, formal risk assessment has evolved from the
traditional fields of application such as toxicology and environmental health risks,
but with distinct differences. In particular, survival and/or inactivation of pathogens,
and the growth of bacteria, must be accounted for, and assessors require predictive
models to estimate these parameters. Second, human responses to microbial patho-
gens can vary significantly, depending on characteristics of the host's immunity and
other defense factors; the pathogen's characteristics and survival and virulence
mechanisms; and the characteristics of the food matrix in supporting growth, or in
protecting the microorganism from inactivation by processing, or in the human after
ingestion. However, the focus of this paper will consider the parameters that are
driven in part by our ability to predict exposures.
8.2 ASSESSING MICROBIAL RISKS
There are different approaches that can be taken in microbial risk assessment;
however, the basic sections of formal risk assessment are hazard identification,
hazard characterization, exposure assessment, and risk characterization. 5,6 These
describe, respectively, the nature of the food, the contaminant, and associations with
human illness; the characteristics of the disease, the pathogen-host interaction, and
if data are available, a mathematical model that quantifies the dose-response rela-
tionship; an evaluation of the likely intake of the agent in the food; and an integration
of the foregoing information to provide a risk estimate, i.e., the likelihood and
severity of the adverse effects in a given population. Risk characterization should
also delineate the uncertainties and variability in the data used, and in our under-
standing of the food system, pathogen behavior and human health response. QRA
is also considered to be part of the larger concept of risk analysis, which includes,
in addition, risk management and risk communication steps. 9
QRA and HACCP have some common parameters. Process risk models (see
later) may help to identify CCPs and specify where significant risks exist. QRA can
have input into specification of criteria for CCPs (step 3 of HACCP), as shown in
Figure 8.1. 3 Risk assessment is intended to provide a scientific basis for risk man-
agement decisions, while HACCP is a systematic management approach to the
control of potential hazards in food operations. 10 Thus, risk assessment concerns the
overall product safety, while HACCP enhances overall product safety by assuring
day-to-day process control. 10 The view of risk assessment being associated with one
step of HACCP may be a limited one; in a contrasting view, both HACCP and risk
assessment are encompassed in risk analysis, with HACCP representing one man-
agement strategy (Figure 8.1). 10 Nauta 11 has recently clarified this relationship by
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 266 Wednesday, November 12, 2003 1:06 PM
HACCP
r
Step3:
Specification of criteria I
Hazard identification
~\
Hazard characterization
<
~\
>
Exposure assessment
Risk characterization
J
Risk
Assessment
>
Risk management
(including HACCP)
Risk communication
Risk
Analysis
J
FIGURE 8.1 Relationship among HACCP, quantitative risk assessment, and risk analysis.
(Modified from Notermans, S., Gallhoff, G., Zwietering, M.H., and Mead, G.C., Food Micro-
biol, 12, 81, 1995. With permission.)
stating that while HACCP is typically linked to industrial processes, QRA is used
for public health purposes and to help set hazard targets for industry as a whole.
Traditionally, food-borne pathogens and the risk of human illness are often
described by descriptive hazard assessments, which typically do not actually provide
a measure of the risk in terms of likelihood of occurrence and extent of illness (or
other endpoint) expected in a population. However, if appropriate, this type of
approach can be useful because the information can usually be compiled and sum-
marized quickly if necessary. Expert knowledge has also often been relied upon to
help decision-makers; however, even experts can misinterpret data, and may be
biased towards certain conclusions.
Formal risk assessment based on the four-step framework relies on the basic
elements of data, models, and assumptions. The variability and uncertainty in all
three elements must be described, either quantitatively or descriptively. The risk
assessment must be well documented and transparent; that is, all the data, assump-
tions, calculations, and technical descriptions should be presented to allow others
to understand completely how the conclusions were reached, using what data, and
what types of analyses.
We refer to qualitative assessments, in which the information used for the
assessment is described in general terms, as categories or ratings. For example,
ratings of "high," "medium," "low," or "negligible" may be assigned for the various
parameters (e.g., pathogen concentration; prevalence, extent of growth/inactivation,
or both; amount of food eaten; severity of illness) and for the final risk estimate,
based on defined ranges of values for each rating and for each parameter. There are
few examples of comprehensive hazard ranking systems. The ICMSF book Micro-
organisms in Foods 7: Microbiological Testing in Food Safety Management 12 gives
a good table (in its Chapter 8 appendix) on the ranking of food-borne hazards or
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 267 Wednesday, November 12, 2003 1:06 PM
FIGURE 8.2 Example of a log normal distribution.
toxins into hazard groups. In a similar manner, various seafood products have been
ranked qualitatively into risk categories. 13
By contrast, quantitative risk assessments require mathematical equations to
describe the relationships among all the factors that influence the risk. Quantitative
assessments can be point estimates or stochastic (or probabilistic). Point-estimate
and stochastic models can be differentiated along the lines of their treatment of
randomness and probability. Point-estimate models do not include any form of
randomness or probability in their characterization of a system, whereas these are
fundamental characteristics of probabilistic assessments.
Point-estimate models use a single number for each data set that is used as an input
into the model analyzed. For example, the mean concentration (i.e., colony forming
units [CFUs] per gram) of Salmonella in raw ground beef is a point estimate; the 95th
percentile value from a collection of data points would be a "worst-case" point estimate.
Probabilistic analyses consider the entire possible range of the numbers of Salmonella
that may be in the raw product, with the likely frequency at which the various concen-
trations might occur. Thus, the distribution curve may range from 1.0 to 4.0 logs per
gram CFU per gram in product that is positive for the pathogen. Some values will be
more likely than others, and this is represented by the height of the distribution curve
at those values. This information is derived from one or more sets of laboratory data,
or, if few data are available, estimations using sound scientific rationale will be required.
The outcome of a point-estimate risk assessment is a single value for the risk estimate,
such as 1 in 100,000 probability of illness. A probabilistic risk estimate is a range of
values, and how probable each value is likely to be, again depicted by a distribution
curve. An example of a lognormal distribution is given in Figure 8.2.
8.3 ROLE OF PREDICTIVE MICROBIOLOGY IN QRA
Most of the risk model development takes place within exposure assessment and
dose-response assessment (part of hazard characterization). For some agents,
particularly those involving voluntary exposure, such as prescription drugs, exposure
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 268 Wednesday, November 12, 2003 1:06 PM
assessment is relatively straightforward. But for other agents, such as environmental
or food contaminants, an exposure assessment is usually based on considerable
uncertainties. It is often not possible to measure exposures directly; rather they must
frequently be predicted, for example, by monitoring data, using mathematical mod-
eling, and reconstructing historical exposure patterns. There are two broad types of
mathematical models used in exposure assessment: those that predict probable expo-
sure to the agent and those that predict the probable concentration of the agent.
Exposure models can be used to estimate population exposures based on small
numbers of representative measurements. Models that predict concentrations can be
combined with information on human time-activity patterns to estimate exposures.
Key components of assessing exposures may include:
The microbial ecology in relation to food
Intrinsic and extrinsic microbial growth requirements
Prevalence of infection in food animals
The initial contamination of the raw materials
The impact of production, processing, cooking, handling, storing, distri-
bution steps, and preparation by the consumer on the microbial agent
The variability in processes involved and the level of process control
Slaughter or harvesting practices and the level of sanitation
The potential for contamination or recontamination
The conditions for packaging, distribution, and storage of the food, and
the food attributes that could influence growth, toxin production, or both
Implicit in the concept of exposure assessment is the influence of processing and
environmental factors on the survival and growth of food-borne pathogens. Mathe-
matical models can predict the extent of impact of unit operations on the numbers of
microorganisms, which in turn determines the exposure. 14 Specific mathematical func-
tions to quantitate microbial growth and death can be incorporated into risk assess-
ments. 14-17 For example, the Gompertz function is used to evaluate growth parameters:
log x{t) = A + Cexp{- exp[-B(t - M)] }
(8.1)
where x(t) is the number of cells at time t, A is the asymptotic count as t decreases
to 0, C is the difference in value of the upper and lower asymptotes, B is the relative
growth rate at M, and M is the time when the absolute growth rate is maximum. 1819
Thermal death models can be used to establish the D-value for a microorganism:
log S,=-
D
(8.2)
where S t is the survival ratio at time t. Much information on microbial growth and
survival has been documented, and resulting predictive software such as Food Micro-
Model has been used to predict the influence of food composition and environmental
conditions on growth and survival of potentially hazardous microorganisms. 20 Mod-
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 269 Wednesday, November 12, 2003 1:06 PM
els can therefore be used to develop CCPs, and show where data for risk assessments
are missing. 21 In addition, models can support regulations and optimize product
formulations and support process control. 21 Mathematical modeling can also support
quantitation in dose-response assessment. For example, the Beta-Poisson is a com-
monly used distribution model for dose-response: 14
P. =\-
V K J
P
(8.3)
where P { is the probability of infection, N is the exposure, and a and p are coefficients
specific to the pathogen.
In QRA, mathematical models are used to estimate the ultimate risk to the
consumer as a function of input values taken from various points along the "farm-
to-fork" continuum. Because of heterogeneity of microorganisms, variability around
single point estimates of risk can be significant. Thus, point estimates give limited
information, describing single instances such as worst-case scenarios without any
insight into how likely, or unlikely, this is to occur. 1422 Improvements in prediction
can be made by incorporating uncertainty. Uncertainty is an important factor in risk
analysis, since failure to account for it limits our ability to make reliable predictions
of risk. Uncertainty may arise from inherent variability in the biological system, or
from lack of information or understanding of the mechanisms involved. 15 Uncertainty
due to lack of information or understanding vs. uncertainty due to variability can
sometimes be minimized by obtaining more, high quality data; however, as this is
not always feasible, alternatives must be sought. One approach is to use probability
distributions to represent parameter values. These distributions can be built from
empirical data, knowledge of underlying biological phenomena, or expert opinion. 22
Using distributions as inputs leads to an output where risk is expressed as a proba-
bility distribution. Risk analysis software such as @RISK™, which uses Monte
Carlo analysis to simulate output distributions of risk on the basis of variability of
input data, can facilitate the risk assessment process. 14 ' 22 In Monte Carlo analysis,
the point-estimate relations are replaced with probability distributions. Samples are
randomly taken from each distribution in a series of iterations, and the results of
each iteration are tallied, usually in the form of a probability density function, or
cumulative distribution function. This approach yields an output, the risk estimate,
that reflects the uncertainty and variability in the data used for the assessment.
As probabilistic models include components of randomness within their defi-
nition, these result in outputs that are in fact estimates of the true system. Proba-
bilistic assessments attempt to capture the variability that is naturally present in a
biological system. Such models tend to be a better representation of natural systems,
given the randomness inherent in nature itself. Clearly, a point-estimate model to
describe a natural system is a significant simplification of a biological system;
however, with these caveats, a point-estimate model could be entirely appropriate
for the problem at hand, and with given resources. What is important is that
assessors and managers alike acknowledge the limitations of the information
derived from any such risk model.
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 270 Wednesday, November 12, 2003 1:06 PM
Nauta 23 has emphasized the need to separate true biological variability (due to
heterogeneity of populations) from uncertainty, the lack of perfect knowledge of
the parameter values. This is commonly neglected in risk assessment studies.
Working with data on growth of Bacillus cereus in pasteurized milk, Nauta 23 showed
that prediction of outbreak size may depend on the way that uncertainty and
variability are separated. Using a deterministic estimate, the exposure assessment
model predicted that there was no risk. A stochastic model without separation of
uncertainty and variability predicted individual risk, but no major outbreak. In
contrast, when uncertainty and variability were differentiated, a potential major
outbreak was predicted.
8.4 SCOPE OF RISK ASSESSMENTS
In addition to different modeling approaches, risk assessments can also differ in
their scope. In risk ranking, several foods may be compared within the assessment
to determine which pose higher or lower risk. This type of assessment is useful for
setting priorities for risk management. Farm-to-fork (production-to-consumption)
models describe each stage of the food: growing, harvesting, processing, distribution,
retail, and preparation pathway. Alternatively, assessments may focus only on the
stages after retail distribution. Typically, risk assessments are constructed in a mod-
ular sequence of relevant stages in food harvesting/processing/handling/consump-
tion. Submodels within the individual modules, including ones that integrate pre-
dictive equations for growth, inactivation, or both, are defined as appropriate. These
may be simple or complex equations, reflecting the precision necessary to estimate
significant parameters and changes in pathogen number.
8.5 PROCESS RISK MODELING
Evaluating the microbial safety of a food typically requires consideration of multiple
factors that influence the prevalence and numbers of a microbial pathogen in the
product. As a tool for strategic decision-making, the scope of a risk assessment
should include activities that provide relevant information for the risk manager. This
approach has been taken for many microbial risk assessments to describe the pro-
duction-to-consumption pathway. The main goal of such work is to develop a tool
that can be used to analyze the relationship between the factors that affect the
presence, behavior, and ultimately concentration of microorganisms and the proba-
bility of human illness.
The phrase "Process Risk Model" (PRM) has been introduced to describe such
risk assessments. 2425 The basis of a PRM is the mathematical model that predicts
the probability of an adverse impact as a function of multiple process parameters.
By manipulating the parameters of the sequential stages of food production, the
effect of hypothetical risk-reducing strategies that are based upon changing some
component of the system is estimated by the changes in the risk prediction under
different scenarios. For example, the probabilistic model allows the prediction of a
change in a health effect endpoint, such as the expected number of illnesses within
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 271 Wednesday, November 12, 2003 1:06 PM
a defined population and time frame, under different HACCP or other types of
intervention/control strategies. As a result the model acts as a predictive tool for
evaluating future scenarios, rather than presenting a static picture of the present risk
to health. Simulation provides this important link between HACCP and QRA.
In addition, intermediate results or outputs from the risk model may be of
interest. For example, an intermediate output might be a prediction of the distribution
of cell numbers in a package of ground beef, simulated from input parameters of
initial carcass contamination, and modeling the changes that occur during slaughter
and fabrication. Further analysis of the probabilistic model provides information
about key inputs or uncertainties that most significantly influence the risk outcome,
thereby identifying potentially effective interventions or research opportunities.
Manipulation of the model, by altering input values in "what if scenarios, can
readily provide insight into the effectiveness of proposed risk interventions. 24
Nauta's 11 modular process risk model (MPRM) constitutes a further improve-
ment on the PRM. In this approach, it is assumed that in any food pathway, all
processing steps can be described by six process modules: two microbiological
processes (growth or inactivation) and four product handling steps (mixing, parti-
tioning, removal, and cross-contamination). This approach highlighted the impor-
tance of including variability in microbial growth models, and provided a tool to
identify the most important gaps in knowledge along a food pathway.
8.6 EXAMPLES OF RISK MODELING
There have been very few examples of well-developed QRA for specific microbial
hazards in foods. Much of the work published to date is about quantitative models
that describe either exposure or dose-response relationships. Schlundt 26 reviewed
several microbial risk assessments published between 1996 and 1998 with a view
to assessing the state of the art. He noted that few of the studies comprised a full
Codex-based risk assessment. 6 Often the purpose of these studies did not relate
directly to risk analysis, and the factors that determined the risk were not identified.
The driving force for many of these studies was the use and application of mathe-
matical models; thus the focus was largely on exposure assessment. Examples of
some recently published food safety assessments are given in Table 8.1.
One of the important developments in QRA is the establishment of risk assess-
ment simulations that can be easily accessed by users. A few examples of these
follow. Oscar 27 has developed an interactive Microsoft® Excel-based spreadsheet
called Poultry Farm Assess Risk Model (Poultry FARM). This model uses the risk
analysis software @RISK™ to provide poultry companies and regulatory agencies
with the tools they need to make informed public health decisions, van Gerwen et
al. 28 have described a system for microbiological QRA of a cheese spread. Predictive
models were incorporated with a decision support expert system called SIEFE:
Stepwise and Interactive Evaluation of Food Safety by an Expert System. This
approach combined quantitative information on the production processes with qual-
itative expert knowledge expressed as a series of rules. Ross and Sumner 29 have
also developed a Microsoft Excel spreadsheet model for QRA. In this model,
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 272 Wednesday, November 12, 2003 1:06 PM
TABLE 8.1
Examples of Currently Published Microbial Food Safety Exposure,
Dose-Response, and Risk Assessments
Microorganism
Bacillus cereus
B. cereus, C.
perfringens
Listeria
monocytogenes
Salmonella
Enteritidis
Salmonella spp.
Mycobacterium
paratuberculosis
Escherichia coli
0157:H7
Commodity
Pasteurized milk
Chinese-style rice
Vegetable puree
Cooked chilled
vegetables
Pasteurized milk
Pate and soft cheese
Ready-to-eat meat
and smoked fish
Raw milk soft cheese
Smoked or gravad
fish
Ready-to-eat foods
Cracked eggs
Pasteurized liquid
Shell eggs
Poultry and products
Pasteurized milk
Ground beef
Raw fermented
sausage
Type of Assessment
Semiquantitative, from retail to consumer
Probabilistic risk estimation, raw product
to consumer
Exposure assessment, retail to consumer
Probabilistic exposure assessment,
product preparation and storage
Point-estimate hazard/exposure
assessment
Quantitative (simple), retail to consumer
Quantitative dose-response assessment
Reference
33
34
35
36
37
38
39
Probabilistic risk estimation, from farm to 38
consumer
Quantitative, from retail to consumer 40
Probabilistic risk ranking 41
Semiquantitative, from eggs to consumer 42
Probabilistic risk estimation, from eggs to 43
consumer
Probabilistic risk estimation, from farm to 31
consumer 44, 45
Probabilistic risk estimation from 27, 31,
processing to consumer 46
Probabilistic vs. point-estimate exposure 47
assessments
Probabilistic risk estimation, from cattle 25, 48
to consumer
Probabilistic risk estimation, retail to 49
consumer
Probabilistic exposure assessment, cattle 50
to retail
qualitative inputs are converted into numerical values, and then combined with
quantitative inputs in a series of mathematical and logical steps. It was designed as
a generic model, to give a quick and simple means of comparing food-borne risks
from diverse products.
There are different approaches to risk modeling; thus it is appropriate to discuss
here some illustrative examples in more detail. The PRM for Escherichia coli
0157:H7 in ground beef reported by Cassin et al. 25 is one of the best developed and
most detailed models available. The PRM is limited to a particular food production
system, and predicts the distribution of probability of illness attributable to E. coli
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 273 Wednesday, November 12, 2003 1:06 PM
0157:H7 in a particular ground beef scenario. 25 In contrast, the Poultry FARM model
described by Oscar 27 is a packaging-to-consumption model that assesses the risk
and severity of Salmonella spp. and Campylobacter jejuni infections from chicken.
Outputs include the concentrations of pathogens at each stage of the process, and
the health outcome assessment, which takes into account the number of patients
who might seek medical treatment, and suffer death or chronic sequelae.
8.6.1 PRM for F. com 0157:H7 in Ground Beef
The model of Cassin et al. 25 describes the probability of becoming ill with E. coli
0157:H7 as a result of consuming undercooked ground beef. It models the produc-
tion of beef trimmings by a hypothetical abattoir, which are subsequently ground
and sold by retailers. Figure 8.3 shows the flow diagram of the process. E. coli
0157:H7 is the primary microbial hazard identified with ground beef. Cattle are
known to be a reservoir for this pathogen, which is shed in feces, and can then
subsequently contaminate the carcass during slaughter. E. coli 0157:H7 is a human
pathogen that can cause severe infection, often resulting in death or permanent
damage. There have been a number of food-borne outbreaks attributed to under-
cooked ground beef.
As mentioned earlier, exposure assessment is one of the most important aspects
of QRA. In this step, the potential exposure to the pathogen was determined in a
single-serving meal. In this model, multiple stages of product handling were
described, with appropriate probability distributions assigned to each step, based on
available data. The various stages include production; processing and grinding;
postprocessing conditions such as microbial growth and thermal inactivation; and
consumption.
The production stage concerns the potential concentration of fecal material on
the beef carcass. This depends on the level of E. coli 0157:H7 in feces, which is
affected by many factors including season, age of animal, and feeding practices.
Prevalence relates to the relative number of animals that shed the pathogen, both
within and between herds. Processing includes skinning, evisceration, and trimming.
During the skinning process, fecal material from the hide can contaminate the
carcass. Previous studies have shown that the various decontamination steps such
as trimming of visible contamination and washing using a variety of methods have
limited effect on the level of contamination. During the subsequent chilling of the
carcass, some microbial growth can occur.
Trimmings collected during the deboning stage are then combined into 5-kg
lots, and sent to retailers for grinding. During storage of the ground beef, some
microbial growth can occur, and this can be modeled using common functions such
as the Gompertz equation (see Chapter 2). The effect of temperature can be modeled
using Food MicroModel (see Chapter 6). Finally, cooking is the most effective barrier
against E. coli 0157:H7 exposure, and modeling was based on the cooking prefer-
ence of the consumer.
A dose-response model based on the Beta-Poisson model was constructed. It
was assumed that the virulence of the pathogen is similar to that of Shigella dysen-
teriae, and model parameters were selected based on human feeding studies. The
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 274 Wednesday, November 12, 2003 1:06 PM
Prevalence in
faeces
Prevalence in
fresh ground beef
Processing and Grinding
Mass of package of fresh ground beef
Mass of a package of trimmings
Mass of a trimming
Cross contamination factor
Surface area of a trimming
Number of trimmings in package
Faecal dilution factor
Reduction due to spray washing
Reduction due to trimming
Growth during processing
Concentration
in faeces
Prevalence in
cooked hamburger
Storage
Time in storage
Maximum temperature experienced during storage
Maximum growth rate
Lag time
Maximum population density
Cooking
Cooking preference
Final internal temperature
Thermal inactivation regression parameters
Concentration in
fresh ground beef
Consumption
Mass consumed (age dependent)
Concentration in
cooked hamburger
Probability of
exposure
Dose-Response
Host susceptibility
Probability of infection from a single organism
Conditional probability of severe outcomes
Probability of
illness
Probability of
HUS
Probability of
mortality
FIGURE 8.3 Flow diagram of the mathematical model of exposure assessment and
dose-response for E. coli 0157:H7 in hamburgers. (From Cassin, M.H., Lammerding, A.M.,
Todd, E.C.D., Ross, W., and McColl, R.S., Int. J. Food Microbiol., 41, 21, 1998. With
permission.)
dose-response curve for an adult population is shown in Figure 8.4. The susceptible
population, i.e., young children, was assumed to have a similar vulnerability, but an
increased propensity for more severe outcomes. As a final step in development of
the model, the probability of illness was the product of the probability of a nonzero
exposure and the output of the dose-response model.
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 275 Wednesday, November 12, 2003 1:06 PM
C
o
>«
SI
(0
SI
o
0)
o
>
<
.5th percentile
median
-95th percentile
2 4 6 8
Ingested Dose(Log 10 CFU)
FIGURE 8.4 Beta-Binomial dose-response model — uncertainty in average probability of
illness vs. ingested dose of E. coli 0157:H7. (From Cassin, M.H., Lammerding, A.M., Todd,
E.C.D., Ross, W., and McColl, R.S., Int. J. Food Microbiol, 41, 21, 1998. With permission.)
A simulated distribution of probability of illness per meal is shown in Figure
8.5. It is not a simple matter to determine the risk of a specific health outcome, since
a wide range of scenarios can exist. The risk for most scenarios is less than 1 in
10,000 (Figure 8.5). The expected value of risk was also calculated, which is a point
estimate of the probability of a particular health effect occurring. This value is often
used to compare with regulatory objectives to meet standards of acceptable risk;
however, the range of risk experienced by the population is lost.
1 6% -
14%
12%
10%
I 8%
o
i — i
i —
i — i
i — i
i —
i —
— i
i —
— i
i — i
i —
—i 1 n .
i —
p — i^ — i — i —
6%
4%
2%
0%
-23 -21 -19 -17 -15 -13 -11 -9 -7 -5 -3
Log Probability of Illness
FIGURE 8.5 Probability distribution for probability of illness from a single hamburger meal
predicted by the E. coli 0157:H7 Process Risk Model (PRM). (From Cassin, M.H., Lammer-
ding, A.M., Todd, E.C.D., Ross, W., and McColl, R.S., Int. J. Food Microbiol., 41, 21, 1998.
With permission.)
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 276 Wednesday, November 12, 2003 1:06 PM
o
CO
>
o
'■v
Concentration in feces
Host susceptibility
Carcass contamination factor
Cooking preference
Retail storage temperature
Reduction due to decontamination
Growth during processing
Retail storage time
Prevalence in feces
Mass ingested
Number of trimmings in a package
Mass of a trimming
Surface area of trimming
Cross contamination factor
Mass of ground beef package
-0.4
-0.2
0.2 0.4
Rank Correlation
0.6
0.8
FIGURE 8.6 Spearman rank correlation between the estimated probability of illness and the
15 most important predictive factors of the Process Risk Model (PRM). (From Cassin, M.H.,
Lammerding, A.M., Todd, E.C.D., Ross, W., and McColl, R.S., Int. J. Food Microbiol., 41,
21, 1998. With permission.)
CCPs can be identified from a PRM using importance analysis. Importance anal-
ysis includes the sensitivity of the outcome to a factor, and the uncertainty and
variability of that factor. The Spearman rank correlation coefficient was used to mea-
sure importance, and a tornado graph showing the 15 predicting factors most highly
correlated with risk is shown in Figure 8.6. The concentration of E. coli 0157:H7 in
the feces was the most highly correlated factor, which points out the importance of
animal prescreening prior to slaughter, or some intervention that reduces numbers in
the feces of the live animal. Host susceptibility (probability of illness from a single
organism), carcass contamination factor (relationship between concentration in the
feces and on the carcass), and cooking preference were also important risk factors.
The ability to propose appropriate risk mitigation strategies is an important
outcome of a PRM. Hypothetical strategies such as improvements in storage tem-
perature, better preslaughter screening, and institution of a consumer information
program were simulated using the E. coli 0157:H7 PRM. These were defined to
have an assumed level of compliance with the intervention. It was found that reducing
the average temperature of storage at the retail level from 10 to 8°C, with the
maximum expected of 13 instead of 15°C, reduced the risk of illness by 80%. In
contrast, the effectiveness of consumer education on the importance of fully cooking
hamburgers was predicted to reduce risk by only 16%.
8.6.2 Poultry Farm Model
This model predicts the change in concentration of Salmonella spp. or C. jejuni in
a single serving of chicken from packaging at the processing plant, through to
consumption by the consumer, as well as adverse health outcomes. It is structured
as one simulation model (using @RISK in a Microsoft® Excel spreadsheet) and four
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 277 Wednesday, November 12, 2003 1:06 PM
TABLE 8.2
Simulation Conditions for Poultry FARM Model
Describing the Fate of Salmonella spp. on Chicken
from Production to Consumption
Incidence
Extent 3
Node
(%)
Minimum
Most Likely
Maximum
Packaging
20
0.0
2.5
4.5
Cold storage
100
-2.0
-0.3
0.0
Distribution
40
0.0
0.3
3.0
Cooking
15
-9.0
-6.0
0.0
Cooling
20
0.0%
0.1%
0.5%
Consumption
20
0.0
0.2
2.0
Infection
1.3
3.3
8.3
a Units are: Packaging, log number/serving; Cold Storage, Distribution,
Cooling, Consumption (log change/serving); Cooling (% transfer/serving);
Infection (log number).
predictive models. In this example, we will address the prediction of only Salmonella
spp. during the process.
The various nodes described in the model are given in Table 8.2. At each stage,
the change in numbers per serving is calculated by drawing a value from the given
extent, ranging from the minimum to the maximum expected value, with an inter-
mediate value representing the most likely outcome. The value for incidence is used
to specify the number of servings predicted to be contaminated with Salmonella
e.g., 20% of 100,000, or 20,000 (Table 8.2). During cold storage, the numbers are
expected to decrease by a minimum of -2.0, a maximum of 0.0, and a most likely
value of -0.3 log cfu per serving in 100% of the servings (Table 8.2).
The other nodes work in a similar fashion. The incidence value in the cooking
node represents the 15% of consumers who are expected to undercook their chicken,
and during cooling, it is expected that 20% of consumers will expose the cooked
chicken to temperature abuse. Changes in the Salmonella content of each serving
of the chicken were accumulated over the whole process, and infection was expected
to occur if the cumulative numbers exceeded the minimum infective dose. This
calculation assumed that one Salmonella was capable of causing an infection, and
that resulted in 100% probability of developing salmonellosis. This calculation is
similar to an exponential dose-response model.
Health outcomes were further calculated from the infection incidence. It was
assumed that 45.4% of those infected became sick, that 20.7% of infected victims
visited a doctor, 4.1% were admitted to hospital, 2.3% experienced chronic sequelae,
and 0.1% died. 30
The @RISK model was simulated 100,000 times to represent the number of
servings being considered, and the outputs are given in Figure 8.7 and Figure 8.8.
In Figure 8.7, the exposure assessment is presented as the change in log number of
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 278 Wednesday, November 12, 2003 1:06 PM
10
o
o
o
m-
o
o
1—
o»
E
o
8.5-1
8.0-
7.5-
7.0-
6.5-
6.0-
5.5-
5.0-
4.5-
4.0-
3.5-
T
T
i0
T
T
T
^Control
-°— Cooking/Cooling
^^ Sensitive Population
T
x<^ JS aJDP ^ .^ jvO'
^ jer N^y ^y jfir
tr .&' ^ G°* ^ ^
^ ^ ^ ,o* e?* JT
«~ r& <P
o
^
Node
FIGURE 8.7 Exposure assessment for Salmonella spp. on chicken for the Poultry FARM
model of Oscar. 27
Salmonella per 100,000 servings over the whole process. In Figure 8.8, the health
outcome assessment is given as number of cases per 100,000 consumers. When the
simulation was performed using the values given in Table 8.2, the exposure assess-
ment (Figure 8.7; control) showed that the cooking step had the greatest impact on
numbers of Salmonella. Under these conditions, the number of consumers infected
was 43 out of 100,000, with <0.1 deaths (Figure 8.8).
The simulation was repeated with changes made in the initial assumptions. To
simulate cases of abuse, it was assumed that the proportion of consumers under-
cooking their chicken was 75% rather than 15%, and the proportion of consumers
exposing the cooked chicken to temperature abuse during cooling was 75% rather
than 20%. In the exposure assessment (Figure 8.7), the combined abuse treatments
resulted in an increase of 0.5 log numbers by the end of the cooling step. This
translated into an increase in infections to 138 per 100,000 consumers, with a 0.14
death rate (Figure 8.8). In a further example, the incidence of cooking or cooling
abuse was kept the same as in the control, and the influence of exposure to a more
susceptible population was simulated. This was achieved by decreasing the assumed
infection level (minimum, most likely, and maximum) by 1 log. The results of this
simulation showed that, as expected, this change did not influence the exposure
assessment (Figure 8.7); however, the infection rate increased to 168 per 100,000,
and the death rate to 0.17 (Figure 8.8). With other simulations based on predicted
changes in processing or consumption patterns, it would be possible to determine
those factors that have the greatest impact on health outcomes.
8.7 MODIFYING RISK: CONCENTRATION
VS. PREVALENCE
In addition to identifying and quantitating risks, risk assessments can also provide
information on the relative impact of intervention strategies. The number of
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 279 Wednesday, November 12, 2003 1:06 PM
200-i
100-
1
#
a
I Control
3 Cooking/Cooling
] Sensitive Population
/?
7^
6-
5-
4-
3-
2-
1
■s
#
T
■._
>**
.©
o?
e
G«'
^
T
vO
^
Outcome
FIGURE 8.8 Health outcome assessment for Salmonella spp. on chicken for the Poultry
FARM model of Oscar. 27
microorganisms present in a sample of raw food has a direct impact on the number
finally consumed; however, initial numbers may be influenced by either concentra-
tion (number per unit weight) or prevalence (proportion of units contaminated). This
can be demonstrated by reference to data used for a FAO/WHO risk assessment on
Salmonella in broiler chickens. 31 In this study, the prevalence of Salmonella was
determined after immersion in the chill tank with and without chlorine. Data show
that carcass cross-contamination was significantly reduced by inclusion of chlorine.
Reducing prevalence of Salmonella-contaminated carcasses was estimated to have
a one-to-one effect on risk reduction.
The effects of reducing the numbers of Salmonella on poultry carcasses without
changing the prevalence of contaminated carcasses was also assessed using the risk
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 280 Wednesday, November 12, 2003 1:06 PM
assessment model. A change in concentration does not necessarily have a linear
relationship with risk outcome, as is found for prevalence. Assuming a constant
prevalence of 20%, and reducing the concentration, gave a reduction in illnesses per
million servings from 11.3 to 4.28. 31 However, these observations pertain to indi-
vidual units that will be prepared by a consumer vs. raw material units that would
be comingled during processing and before consumption.
8.8 WHAT IS THE RIGHT MODEL TO USE?
Clearly, there are many options available to microbial risk assessors, from simple
descriptive evaluations to highly complex and detailed analyses. In reality, a com-
bination of techniques and analyses will often be incorporated into a single assess-
ment, for example, qualitative information, expert knowledge, and quantitative anal-
yses of available data, when appropriate. The decision of what approach to use, what
analytical techniques are needed, and the scope and level of detail of the assessment
will be dependent on the nature of the risk management question, and practical
issues such as time, expertise, and other resources that are needed. In international
trade disputes, the demands are for quantitative microbial risk assessments with
some measure of variability and uncertainty. At the national level, the urgency and
nature of the risk issue will dictate what approach is needed.
Microbiological models to predict pathogen growth, survival, or inactivation can
differ in mathematical complexity, but a complex model may not necessarily be the
best choice to answer a particular risk management question. 28 The need for an
accurate prediction needs to be offset by consideration of whether the model is easy
to use, whether it is robust and precise, and whether it has been validated against
independent data. For example, if the objective of a risk assessment is to identify
the most significant risk factors in a process, a simple model may have advantages
over a complex model. However, if an accurate prediction of bacterial numbers is
necessary, a more complex and accurate model may be preferable. In the choice of
a suitable model, one must also consider the quality of data that are going to be
used to generate a prediction. If the temperature data on a process are poor, it may
not be appropriate to use a complex model for the predictions. Often, this can lead
to a misinterpretation of the accuracy of the final prediction. The most appropriate
model would be the simplest model possible for a given purpose and the given data
quality, provided that it is validated and precise. A good model should also be
subjected to an analysis that quantifies the accuracy and bias of its predictions. 32
Ideally, a model should be both accurate and unbiased. Models in risk assessment
must adequately reflect reality.
8.9 FUTURE DIRECTIONS
At the present time, microbial risk assessors acknowledge many limitations in
providing exact estimates of risk, and in the elements of any one risk assessment:
the data available, the models developed to describe both the physical and mathe-
matical aspects, and the assumptions necessary to construct these assessments.
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 281 Wednesday, November 12, 2003 1:06 PM
Validating the outcomes of a risk assessment also provides a challenge; in many
cases, any and all available data for a particular food/pathogen combination are used
for assessing exposures and dose-response relations. This leaves a risk estimate, or
intermediate outputs, that cannot be validated against independent data.
An additional challenge will be to facilitate the incorporation of existing and
new mathematical models into the QRA framework. Many potentially useful models
have been developed and published; however, these often exist independent of the
specific needs of the regulators and the food industry. There is no definitive process
by which these models can be combined with expert opinion and knowledge and
other data on (for example) prevalence to clearly and unequivocally define the risk
of consumption of a particular food. Some mathematical model databases exist, but
these seldom describe the stochastic aspects of the underlying data. It is clear that
in future researchers must work closely with regulators and the industry to improve
the technology transfer.
8.10 CONCLUSIONS
Currently, there are many aspects of microbial contamination of foods, and the human
health responses to pathogens, for which there are few data. However, the develop-
ment of even preliminary quantitative risk models in a systematic way will help to
identify what critical information is lacking, and help to guide future data gathering
efforts. Finally, it is worth recognizing that risk assessment models should be con-
sidered "dynamic." As modeling tools improve, better data become available, and as
we learn more about pathogen-food-host relationships and microbial responses, risk
models will be updated and refined to provide better risk estimates. One continuing
challenge will be to make sure that QRA models are kept current, so that they continue
to be relevant to the changing needs of the food industry. The process of QRA is
still in its infancy, however, and standards have yet to be developed. There is a clear
advantage to the food industry and consumers to further develop the concepts of
QRA and apply them to both common and novel food processes, and it is expected
that significant advances will be made in this field over the next decade.
REFERENCES
1 . Notermans, S. and Jouve, J.L., Quantitative risk analysis and HACCP: some remarks,
Food Microbiol., 12, 425, 1995.
2. C AC (Codex Alimentarius Commission), Guidelines for the Application of the Hazard
Analysis Critical Control Point System, Codex Alimentarius (CCFH) Alinorm
9 3/ 13 A- Appendix II, FAO, Rome, 1993.
3. Notermans, S., Gallhoff, G., Zwietering, M.H., and Mead, G.C., The HACCP concept:
specification of criteria using quantitative risk assessment, Food Microbiol, 12, 81,
1995.
4. Buchanan, R.L., The role of microbiological criteria and risk assessment in HACCP,
Food Microbiol., 12, 421, 1995.
5. FAO/WHO, Application of Risk Analysis to Food Standards Issues, in Report of the
Joint FAO/WHO Expert Consultation, WHO, Geneva, 1995, pp. 13-17.
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 282 Wednesday, November 12, 2003 1:06 PM
6. CAC (Codex Alimentarius Commission), Principles and Guidelines for the Conduct
of a Microbiological Risk Assessment, CAC/GL-30, FAO, Rome, 1999.
7. Hathaway, S.C. and Cook, R.L., A regulatory perspective on the potential uses of
microbial risk assessment in international trade, Int. J. Food Microbiol., 36, 127, 1997.
8. Lammerding, A.M., An overview of microbial food safety risk assessment, J. Food
Prot, 60, 1420, 1997.
9. Notermans, S. and Teunis, P., Quantitative risk analysis and the production of micro-
biologically safe food: an introduction, Int. J. Food Microbiol., 30, 3, 1996.
10. Foegeding, P.M., Driving predictive modelling on a risk assessment path for enhanced
food safety, Int. J. Food Microbiol., 36, 87, 1997.
1 1 . Nauta, M.J., Modelling bacterial growth in quantitative microbiological risk assess-
ment: is it possible? Int. J. Food Microbiol., 73, 297, 2002.
12. ICMSF, Microbiological Testing in Food Safety Management, Kluwer Academic/Ple-
num, New York, 2002.
13. Huss, H.H., Reilly, A., and Embarek, P.K.B., Prevention and control of hazards in
seafood, Food Control, 11, 149, 2000.
14. Buchanan, R.L. and Whiting, R.C., Risk assessment and predictive microbiology,
J. Food Prot., 31 (Suppl.), 1996.
15. McNab, W.B., A literature review linking microbial risk assessment, predictive
microbiology, and dose-response modeling, Dairy Food Environ. Sanitation, 17,
405, 1997.
16. Walls, I. and Scott, V.N., Use of predictive microbiology in microbial food safety
risk assessment, Int. J. Food Microbiol., 36, 97, 1997.
17. van Gerwen, S.J.C. and Zwietering, M.H., Growth and inactivation models to be used
in quantitative risk assessments, J. Food Prot, 61, 1541, 1998.
18. McMeekin, T.A., Olley, J.N., Ross, T., and Ratkowsky, D.A., Predictive Microbiol-
ogy: Theory and Application, John Wiley & Sons, New York, 1993.
19. Skinner, G.E., Larkin, J.W., and Rhodehamel, E.J., Mathematical modeling of micro-
bial growth — a review, J. Food Safety, 14, 175, 1994.
20. Panisello, P.J. and Quantick, PC, Application of Food MicroModel predictive soft-
ware in the development of hazard analysis critical control point (HACCP) systems,
Food Microbiol., 15, 425, 1998.
21. Baker, D.A., Application of modelling in HACCP plan development, Int. J. Food
Microbiol., 25, 251, 1995.
22. Lammerding, A.M. and Fazil, A., Hazard identification and exposure assessment for
microbial food safety risk assessment, Int. J. Food Microbiol., 58, 147, 2000.
23. Nauta, M.J., Separation of uncertainty and variability in quantitative microbial risk
assessment models, Int. J. Food Microbiol., 57, 9, 2000.
24. Cassin, M.H., Paoli, G.M., and Lammerding, A.M., Simulation modeling for micro-
bial risk assessment, J. Food Prot., 61, 1560, 1998.
25. Cassin, M.H., Lammerding, A.M., Todd, E.C.D., Ross, W., and McColl, R.S., Quan-
titative risk assessment for Escherichia coli 0157:H7 in ground beef hamburgers,
Int. J. Food Microbiol., 41, 21, 1998.
26. Schlundt, J., Comparison of microbiological risk assessment studies published, Int.
J. Food Microbiol., 58, 197, 2000.
27. Oscar, T.P., The development of a risk assessment model for use in the poultry
industry, J. Food Safety, 18, 371, 1998.
28. van Gerwen, S.J.C, te Giffel, M.C., Van't Riet, K., Beumer, R.R., and Zwietering,
M.H., Stepwise quantitative risk assessment as a tool for characterization of micro-
biological food safety, J. Appl. Microbiol., 88, 938, 2000.
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 283 Wednesday, November 12, 2003 1:06 PM
29. Ross, T. and Sumner, J., A simple, spreadsheet-based, food safety risk assessment
tool, Int. J. Food Microbiol., 11, 39, 2002.
30. Mead, P.S., Slutsker, L., Dietz, V., McCaig, L.F., Bresee, J.S., Shapiro, C, Griffin,
P.M., and Tauxe, R.V., Food-Related Illness and Death in the United States, Emerging
Infectious Diseases, 5, http://www.cdc.gov/ncidod/eid/vol5no5/mead.htm, 2002.
31 . FAO/WHO, Risk Assessments of Salmonella in Eggs and Broiler Chickens: Interpre-
tative Summary, World Health Organization, Geneva, 2002.
32. Ross, T., Indices for performance evaluation of predictive models in food microbiol-
ogy, /. Appl. Bacteriol., 81, 501, 1996.
33. Notermans, S., Dufrenne, J., Teunis, P., Beumer, R., Giffel, M.T., and Weem, P.P., A
risk assessment study of Bacillus cereus present in pasteurized milk, Food Microbiol.,
14, 143, 1997.
34. McElroy, D.M., Jaykus, L.A., and Foegeding, P.M., Validation and analysis of mod-
eled predictions of growth of Bacillus cereus spores in boiled rice, J. Food Prot., 63,
268, 2000.
35. Nauta, M.J., Litman, S., Barker, G.C., and Carlin, F., A retail and consumer phase
model for exposure assessment of Bacillus cereus, Int. J. Food Microbiol., 83, 205, 2003.
36. Carlin, R, Girardin, H., Peck, M.W., Stringer, S.C., Barker, G.C., Martinez, A.,
Fernandez, A., Fernandez, P., Waites, W.M., Movahedi, S., van Leusden, F., Nauta,
M., Moezelaar, R., DelTorre, M., and Litman, S., Research on factors allowing a risk
assessment of spore-forming pathogenic bacteria in cooked chilled foods containing
vegetables: a FAIR collaborative project, Int. J. Food Microbiol., 60, 117, 2000.
37. Peeler, J.T. and Bunning, V.K., Hazard assessment of Listeria monocytogenes in the
processing of bovine milk, J. Food Prot., 57, 689, 1994.
38. Bemrah, N., Sanaa, M., Cassin, M.H., Griffiths, M.W., and Cerf, O., Quantitative risk
assessment of human listeriosis from consumption of soft cheese made from raw
milk, Prev. Vet. Med., 37, 129, 1998.
39. Buchanan, R.L., Damert, W.G., Whiting, R.C., and van Schothorst, M., Use of
epidemiologic and food survey data to estimate a purposefully conservative
dose-response relationship for Listeria monocytogenes levels and incidence of liste-
riosis, J. Food Prot., 60, 918, 1997.
40. Lindqvist, R. and Westoo, A., Quantitative risk assessment for Listeria monocytogenes
in smoked or gravad salmon and rainbow trout in Sweden, Int. J. Food Microbiol.,
58, 181, 2000.
41. FDA/Center for Food Safety and Applied Nutrition, USDA/Food Safety and Inspec-
tion Service, and Centers for Disease Control and Prevention. 2001. Draft assessment
of the public health impact of foodborne Listeria monocytogenes among selected
categories of ready-to-eat foods. www.foodsafety.gov/~dms/lmrisk.html.
USDA/FSIS, Washington, DC, 20250.
42. Todd, E.C.D., Risk assessment of use of cracked eggs in Canada, Int. J. Food
Microbiol., 30, 125, 1996.
43. Whiting, R.C. and Buchanan, R.L., Development of a quantitative risk assessment
model for Salmonella enteritidis in pasteurized liquid eggs, Int. J. Food Microbiol.,
36, 111, 1997.
44. USDA/FSIS, Salmonella Enteritidis Risk Assessment. Shell Eggs and Egg Products.
Final report, USDA/FSIS, Washington, DC, 1998. www.fsis.usda.gov/ophs/risk/
index.htm.
45. Whiting, R.C, Hogue, A., Schlosser, W.D., Ebel, E.D., Morales, R.A., Baker, A., and
McDowell, R.M., A quantitative process model for Salmonella Enteritidis in shell
eggs, /. Food Scl, 65, 864, 2000.
2004 by Robin C. McKellar and Xuewen Lu
1237_C08.fm Page 284 Wednesday, November 12, 2003 1:06 PM
46. Brown, M.H., Davies, K.W., Billon, C.M.P., Adair, C, and McClure, P.J., Quantitative
microbiological risk assessment: principles applied to determining the comparative
risk of salmonellosis from chicken products, J. Food Prot., 61, 1446, 1998.
47. Nauta, M.J. and van der Giessen, J.W.B., Human exposure to Mycobacterium paratu-
berculosis via pasteurized milk: a modeling approach, Vet. Record, 143, 293, 1998.
48. USDA/FSIS, Draft Risk Assessment of the Public Health Impact of Escherichia coli
0157:H7 in Ground Beef, USDA/FSIS, Washington, DC, 2002. www.fsis.usda.gov/
oppde/rdad/frpubs/00-23nreport.pdf.
49. Marks, H.M., Coleman, M.E., Lin, C.-T.J., and Roberts, T., Topics in microbial risk
assessment: dynamic flow tree process, Risk Anal., 18, 309, 1998.
50. Hoornstra, E. and Notermans, S., Quantitative microbiological risk assessment, Int.
J. Food Microbiol., 66, 21, 2001.
2004 by Robin C. McKellar and Xuewen Lu
Live Music Archive
Librivox Free Audio
Metropolitan Museum
Cleveland Museum of Art
Internet Arcade
Console Living Room
Books to Borrow
Open Library
TV News
Understanding 9/11