Statistical Prediction of the Type of Gastric Aspiration Lung Injury Based on Early Cytokine/Chemokine Profiles

Calculations used data on inflammatory mediators in BAL recently published from our laboratory in 96 adult male Long-Evans rats,1278 pathogen-free adult C57/BL6 wild-type mice,13and 73 transgenic mice on a C57/BL6 background that lacked the gene for MCP-1 (MCP-1 [−/−] mice).13Detailed assessments of the severity of lung injury in these animals are given in the original publications,12,13and a brief summary of specific aspirates and mediators is as follows. All animal procedures were conducted in accordance with protocols approved by the Institutional Animal Care and Use Committee at the State University of New York at Buffalo. In brief, animals were anesthetized with halothane and given one of four aspirate types (1.2 ml/kg body weight) by intratracheal instillation: (1) normal saline (NS), pH = 5.3 (controls); (2) NS adjusted to pH = 1.25 with HCl (ACID); (3) SNAP (40 mg/ml) in NS (pH = 5.3); and (4) small gastric particles (40 mg/ml) in NS adjusted to pH = 1.25 with HCl (CASP).12,13SNAP particulates (mean diameter, 10–20 μm) were prepared from the stomach contents of healthy animals of the same species and strain and were washed in normal saline, coarse-filtered through gauze, and sterilized by autoclaving before use. After tracheal instillation of aspirates, animals were maintained in room air and housed in the vivarium for subsequent assessments of lung injury and inflammatory mediator concentrations in cell-free BAL fluid at 6 and 24 h (rats) or 5 and 24 h (mice). BAL fluid recovered from experimental animals was immediately centrifuged at 1,500g  for 3 min to pellet cells, and the supernatant was stored at −80°C for cytokine/chemokine measurements. Immunogenic concentrations of IL-1β, MIP-2, CINC-1, KC, IL-6, IL-10, MCP-1, and IFN-γ were determined using a multiplex Luminex (Luminex Corporation, Austin, TX) antibody-based microsphere assay12and/or by standard enzyme-linked immunosorbent assay. TNF-α levels were determined by bioassay using WEHI 164, subclone 13 cells, a TNF-α–sensitive cell line derived from a mouse fibrosarcoma.11 

The prediction of aspirate type (NS, ACID, SNAP, and CASP) was performed using an approach recently defined by Hutson15to extend standard ROC methodology.16In brief, lung injury type was modeled as a 2 × 2 outcome with injury groups recoded as NS = (ACID = 0, particles = 0), ACID = (ACID = 1, particles = 0), SNAP = (ACID = 0, particles = 1), or CASP = (ACID = 1, particles = 1). In the foregoing, the two factors in injury type are ACID and particles, and the two levels for each factor are no = 0 and yes = 1. The basic algorithm used in predictive modeling considered the following population probability parameters given by Π⁰⁰= Probability (NS) = Pr (ACID = no, particles = no); Π¹⁰= Pr (ACID) = Pr (ACID = yes, particles = no); Π⁰¹= Pr (SNAP) = Pr (ACID = no, particles = yes); and Π¹¹= Pr (CASP) = Pr (ACID = yes, particles = yes). These population probability parameters were modeled as a function of a linear combination of cytokine levels under the constraint that they sum to 1, i.e. , Π⁰⁰+Π¹⁰+Π⁰¹+Π¹¹= 1. Related estimated (cell) probabilities were defined during modeling as p⁰⁰, p¹⁰, p⁰¹, and p¹¹, and the final predicted group assignment corresponded to the maximum of these estimated probabilities. For example, if the maximum cell probability from among p⁰⁰, p¹⁰, p⁰¹, and p¹¹was found to be p⁰⁰, the predicted injury was NS. Modeling was initiated by considering the marginal probabilities Π^1.and Π^.1, which corresponded to Pr (ACID = yes) and Pr (particulate = yes) as determined from standard logistic regression with covariate cytokine levels (marginal probability estimates for Π^1.and Π^.1were denoted as p^1.and p^.1). The probability of the combined injury, Pr (ACID = yes, particles = yes), was then assessed as a function of cytokine levels under the constraint p⁰⁰+ p¹⁰+ p⁰¹+ p¹¹= 1 conditional on the marginal probabilities held fixed. Evaluations assuming that no synergistic injury was present used independent standard ROC curve methodology considering ACID alone and particles alone with the probability of the combined injury set equal to the product of the marginal probabilities (Π¹¹=Π^1.Π^.1).

By hypothesis, analyses tested the predictive accuracy of models incorporating a maximum of two inflammatory mediators at a fixed time of injury and for a fixed type of aspiration. For such analyses, initial assessments used a series of individual cytokines based on data in rats, wild-type mice, and MCP-1 (−/−) mice as described in the statistical modeling method. Log transformed cytokine/chemokine levels used in modeling were lcy1 =‘log [TNF-α] (pg/ml)'; lcy2 =‘log [IL-10] (pg/ml)'; lcy3 =‘log [IL-1β] (pg/ml)'; lcy4 =‘log [IFN-γ] (pg/ml)'; lcy5 =‘log [MIP-2] (pg/ml)'; lcy6 =‘log [CINC-1] (pg/ml)' in rats or ‘log [KC] (pg/ml)' in mice; lcy7 =‘log [MCP-1] (pg/ml)'; and lcy8 =‘log [IL-6] (pg/ml).' After assessments of the predictive accuracy of the individual mediators, additional modeling assessed the ability of each of the relevant mediator pairs in discriminating aspiration injury. Calculations were limited to a maximum of two mediators per margin and two mediators for the CASP cell to minimize overfitting the data. For purposes of analysis, mediator concentrations below the limit of assay detection were defined at the detection limit (an easy option to apply to low cytokine/chemokine levels in future clinical applications). Formal details on the statistical predictive model and computer algorithms used in calculations have recently been detailed by Hutson.15 

As shown in table 1, modeling based on the levels of one or two inflammatory cytokines/chemokines in BAL at 6 h after aspiration in this species provided accurate predictions of the type of aspirate used. The cytokines with the greatest predictive utility in rats at 6 h varied with the type of aspiration, with TNF-α and MCP-1 found to be best for ACID, with MIP-2 best for SNAP, and with MCP-1 best for CASP. Only 3 of 48 rats were misclassified for the type of aspiration based on modeling with cytokine levels at 6 h (1 ACID rat was misclassified as CASP, and 2 CASP rats were misclassified as ACID; table 1). Calculated summary probabilities and (lower, upper) exact 95% confidence intervals were as follows: Pr (predicted NS|true NS) = 12/12 = 1.00 (0.67, 1.0) sensitivity; Pr (not predicted NS|not true NS) = 36/36 = 1.00 (0.86, 1.00) generalized specificity; Pr (predicted ACID|true ACID) = 11/12 = 0.92 (0.58, 1.0) sensitivity; Pr (not predicted ACID|not true ACID) = 34/36 = 0.94 (0.81, 0.97) generalized specificity; Pr (predicted SNAP|true SNAP) = 12/12 = 1.00 (0.67, 1.0) sensitivity; Pr (not predicted SNAP|not true SNAP) = 36/36 = 1.00 (0.86, 1.00) generalized specificity; Pr (predicted CASP|true CASP) = 10/12 = 0.83 (0.50, 1.0) sensitivity; and Pr (not predicted CASP|not true CASP) = 35/36 = 0.97 (0.83, 1.0) generalized specificity. The wide confidence intervals are a function of the relatively small sample sizes used in the model. Individual predictions including the estimated probabilities of different types of aspiration for each rat at 6 h are given in figure 1.

Predictions of aspiration type based on cytokine/chemokine data at 24 h after injury are also shown in table 1. Predictions based on these data generated a slightly higher incidence of misclassification, with correct identifications made in 10 of 12 NS rats, 9 of 12 ACID rats, 10 of 12 SNAP rats, and 11 of 12 CASP rats. Mediators with the best predictive value were CINC-1/MCP-1 for ACID, TNF-α for SNAP, and MCP-1 for CASP (table 1). Calculated summary probabilities and (lower, upper) exact 95% confidence intervals were as follows: Pr (predicted NS|true NS) = 10/12 = 0.83 (0.50, 1.0) sensitivity; Pr (not predicted NS|not true NS) = 33/36 = 0.92 (0.75, 0.97) generalized specificity; Pr (predicted ACID|true ACID) = 9/12 = 0.75 (0.42, 0.92) sensitivity; Pr (not predicted ACID|not true ACID) = 34/36 = 0.94 (0.81, 0.97) generalized specificity; Pr (predicted SNAP|true SNAP) = 10/12 = 0.83 (0.50, 1.00) sensitivity; Pr (not predicted SNAP|not true SNAP) = 35/36 = 0.97 (0.83, 1.0) generalized specificity; Pr (predicted CASP|true CASP) = 11/12 = 0.92 (0.58, 1.0) sensitivity; and Pr (not predicted CASP|not true CASP) = 34/36 = 0.94 (0.81, 0.97) generalized specificity.

Predictions for wild-type mice based on cytokine/chemokine data at 5 and 24 h are shown in table 2. Correct assignments for predicted aspiration group based on inflammatory mediator data at 5 h were made in 7 of 9 NS mice, 9 of 9 ACID mice, 7 of 9 SNAP mice, and 10 of 10 CASP mice (table 2). Correct assignments of aspiration group based on mediator data at 24 h were made in 9 of 10 NS mice, 9 of 9 ACID mice, 8 of 9 SNAP mice, and 13 of 13 CASP mice (table 2). The false predictions in wild-type mice at both 5 and 24 h were due to misassignments between the SNAP and NS groups. Mediators with the greatest predictive utility in wild-type mice at 5 h were TNF-α/KC for ACID, MIP-2 for SNAP, and TNF-α/MIP-2 for CASP. Mediators with the greatest predictive utility at 24 h were TNF-α/MCP-1 for ACID, TNF-α/MIP-2 for SNAP, and TNF-α/MCP-1 for CASP.

Predictions for MCP-1 (−/−) mice based on cytokine/chemokine data at 5 and 24 h are shown in table 3. Correct assignments for predicted aspiration group based on inflammatory mediator data in MCP-1 (−/−) mice at 5 h were made in 9 of 9 NS mice, 7 of 9 ACID mice, 3 of 8 SNAP mice, and 10 of 10 CASP mice (table 3). Correct assignments of aspiration group based on mediator data at 24 h were made in 6 of 9 NS mice, 8 of 8 ACID mice, 5 of 9 SNAP mice, and 11 of 11 CASP mice (table 3). The most prevalent false assignments in MCP-1 (−/−) mice involved misclassifications between SNAP and NS (table 3). Mediator pairs with the greatest predictive utility in MCP-1 (−/−) mice at 5 h were TNF-α/IL-1β for ACID, IL-10/MIP-2 for SNAP, and TNF-α/MIP-2 for CASP. Mediator pairs with the greatest predictive utility in MCP-1 (−/−) mice at 24 h were TNF-α/KC for ACID, TNF-α/IL-1β for SNAP, and TNF-α/KC for CASP.

This study has used predictive models incorporating inflammatory mediator concentrations in BAL to estimate the multinomial probability of several clinically relevant forms of aspiration injury in rats, wild-type mice, and MCP-1 (−/−) mice. The predictive modeling approach has recently been detailed by Hutson15and is an extension of standard ROC analytical methodology.16The standard ROC model develops predictions for binary outcomes such as the presence or absence of disease as a function of a continuous marker expression or linear combinations of marker expressions. Our approach extends this notion by considering aspiration diagnosis as a function of inflammatory mediator levels in the context of the marginal contributions of acid (ACID, yes/no) and nonacidified gastric particles (SNAP, yes/no) plus the possibility of an interactive component of acid and gastric particles (combined acid + particles, CASP). The case of normal saline aspiration was modeled as a control (ACID = no; SNAP = no). The overall modeling approach provided a refined diagnosis of aspiration lung injury as 2 × 2 cross classifications of acid and/or particulate injury based on parameters of inflammation that are accessible to measurement in a clinical setting. Modeling assessments used cytokine data from three types of adult animals (rats, wild-type mice, and MCP-1 [−/−] mice) and two acute time points of injury (5–6 and 24 h after aspiration) to more completely gauge the feasibility and accuracy of the predictive approach. In addition, results with MCP-1 (−/−) mice assessed predictability in the presence of genetic variability in the inflammatory response.

Inflammatory mediators used in analyses were chosen based on two recent studies from our laboratory in rodents demonstrating the synergistic severity of lung injury after the aspiration of CASP in rats and mice.12,13Overall, the four types of aspiration studied (CASP, ACID, SNAP, NS) were correctly discriminated in 85 of 96 rats (89%), 72 of 78 wild-type mice (92%), and 59 of 73 MCP-1 (−/−) mice (81%) by models that used a maximum of only two mediators (tables 1–3). CASP aspiration was correctly predicted from cytokine/chemokine data in 65 of 68 rodents (96%; 21 of 24 rats, 23 of 23 wild-type mice, and 21 of 21 MCP-1 [−/−] mice). Predictive modeling was also accurate in identifying ACID, SNAP, and NS aspiration in rats (table 1) and wild-type mice (table 2) at 5–6 and 24 h. Predictive modeling was least accurate in terms of discriminating between the aspiration of SNAP and NS in MCP-1 (−/−) mice, although CASP and ACID aspiration were identified with high precision in these animals (table 3). Inaccuracies in discriminating NS and SNAP may in part reflect the fact that nonacidified food particles can generate a delayed inflammatory response that is less pronounced immediately after aspiration,12,17and thus may appear similar to saline aspiration in some animals during this early period. This effect may have been exacerbated in MCP-1 (−/−) mice, where an important component of the inflammatory response was ablated, leading to an increased percentage of SNAP/NS misclassifications in these animals (table 3).

Acute pulmonary injury in rats has previously been shown to lead to inflammatory mediator responses that differ in magnitude and pattern depending on the type of aspiration (ACID, SNAP, CASP).11,12,14,17In rats, levels of TNF-α in BAL are reduced in aspiration of CASP compared with SNAP alone, whereas levels of IL-1β, CINC-1, MCP-1, and IL-10 in BAL are increased during the 24-h period after the initiation of injury.12Combined data from rats given these three aspirates (ACID, SNAP, or CASP) indicate that levels of MCP-1 in BAL correlate strongly with the severity of lung injury based on lavaged albumin levels at 6 and 24 h after aspiration.12In addition, IL-10 levels in BAL from the combined groups correlate with albumin levels and inflammatory cell numbers at 24 h after aspiration, and cluster dendrograms reveal that MCP-1 and CINC-1 in BAL also correlate with inflammatory neutrophil numbers at 6 and 24 h after aspiration for the combined groups.12The current study analyzed inflammatory mediator data in a different way, i.e. , in terms of the ability of levels of each mediator and pair of mediators in BAL to discriminate among different forms of aspiration in diagnostic assessments. Viewed from this perspective, several different cytokines and chemokines were useful in predictive analyses (tables 1–3).

Depending on time and animal type, TNF-α, CINC-1 (rats) or KC (mice), MIP-2, and MCP-1 were found to be important in discriminating among the different aspirates studied at different times. CASP aspiration was best discriminated by log concentrations of CINC-1 at 6 h and MCP-1 at 24 h in rats, by TNF-α/MIP-2 at 5 h in wild-type mice and MCP-1 (−/−) mice, and by TNF-α/MCP-1 at 24 h in wild-type mice and TNF-α/KC at 24 h in MCP-1 (−/−) mice (tables 1–3). These same mediators (TNF-α, CINC-1 or KC, MIP-2, and MCP-1 alone or in pairs) were similarly best able to predict the aspiration of ACID or of SNAP in rats and wild-type mice at 5–6 and 24 h (tables 1 and 2). In MCP-1 (−/−) mice, the inflammatory cytokines IL-1β and IL-10 were in addition found to be useful in discriminating between different aspirates (table 3). Cellular and molecular mechanisms underlying the predictive ability of individual inflammatory mediators were not addressed in modeling here and should be the subject of further research. It is also emphasized that our current predictive assessments considered a relatively limited set of cytokines/chemokines, and additional mediators and inflammatory substances may prove to have even greater utility in future calculations. Nonetheless, the results found here are highly encouraging for the general approach of using predictive modeling based on levels of inflammatory mediators in BAL to accurately discriminate different forms of gastric aspiration. In future applications, it is possible that inflammatory mediators in blood as well as BAL may also be helpful, although blood measurements assess systemic responses as opposed to local (pulmonary) responses.

In terms of clinical significance, it is particularly important to be able to identify aspiration involving a combination of acid and gastric food particles (i.e. , CASP). Severe and progressive conditions such as ALI/ARDS or pneumonia are more likely in patients if gastric aspiration including both acid and food particulates is present.6–8As discussed previously, the degree of acute pulmonary injury at 4–6 h after aspiration of CASP in animals is greater than the sum of the two component injuries of ACID or SNAP alone based on histopathology and albumin leakage.11,12,14,17This increased pathology is consistent with a “two-hit” injury scenario, where acute caustic injury from acid sensitizes and exacerbates pulmonary inflammation in response to the detrimental effects of gastric food particles. Another related example of two-hit pathology involves the exacerbation of acid aspiration lung injury by the presence of milk products.18Other secondary injuries like hyperoxia have also been shown to interact with and exacerbate acid aspiration lung injury in rats and rabbits.10,19The ability to identify patients with severe forms of gastric aspiration associated with an increased risk of ALI/ARDS or secondary pulmonary infection is crucial in early prognostication, as well as in designing clinical studies for evaluating the efficacy of new therapeutic interventions for lung injury. In addition, by defining aspiration-specific patterns of inflammatory mediator responses, the kinds of analyses described here could also be applied in future to distinguish between gastric aspiration and other forms of lung injury. For example, an extension of the diagnostic strategy presented here could potentially be developed to discriminate between primary gastric aspiration and nosocomial bacterial pneumonia.

Although this study used cytokine/chemokine data from animals with known forms of aspiration to assess the feasibility of predictive modeling, clinical applications will ultimately require inflammatory mediator measurements and computational modeling over a rapid timescale in an “at-the-bedside” setting. Advances in medical technology and laboratory science are rapidly achieving an inpatient environment where this is feasible. For example, we have recently defined a multiplex microarray enzyme-linked immunosorbent assay to rapidly assess patient inflammatory cytokine profiles for use in predictive diagnostic models.20Also, current microchip and computer technology is compatible with the development of handheld diagnostic devices that incorporate the capability for multiplex microchemical analysis, as well as the necessary computational power for predictive algorithms such as those used here. It should be noted that our current results were estimated using only training data sets and, in general, will tend to slightly overestimate true diagnostic accuracy. In calculations, the number of mediators included in each model was limited to two per margin and two per synergistic injury to avoid overfitting the data. Further large-scale predictive and validation assessments will be necessary to refine the overall modeling approach to include a broader range of inflammatory cytokines and chemokines in prospective studies to better define the practical feasibility of this approach in a clinical setting.