Accuracy of Contrast-Enhanced MR Angiography in Predicting Angiographic Stenosis of the Internal Carotid Artery: Linear Regression Analysis

Abstract

BACKGROUND AND PURPOSE: We sought to assess whether contrast-enhanced MR angiography is able to predict the degree of angiographic stenosis of the internal carotid artery within a clinically acceptable margin of error, thereby decreasing the need for angiography. In addition, we sought to assess whether adding ultrasound peak systolic velocity (PSV) as an additional regressor improves the accuracy of prediction.

METHODS: A retrospective review of our institution’s records for a 4-year period was conducted to identify all patients who had undergone evaluation of their carotid arteries using digital subtraction angiography, contrast-enhanced MR angiography, and ultrasonography. All internal carotid artery stenoses ranging from 10% to 90% at carotid angiography were selected (n = 22). Measurements were then obtained based on the North American Symptomatic Carotid Endarterectomy Trial style by using the digital subtraction angiograms and contrast-enhanced MR angiograms in a blinded fashion. The correlation between digital subtraction angiography data and contrast-enhanced MR angiography data was assessed by conducting linear regression analysis. Multiple regression analysis was then conducted to determine whether the inclusion of ultrasound PSV as an additional regressor increased the accuracy of prediction.

RESULTS: The correlation between the degree of stenosis measured by digital subtraction angiography and that measured by contrast-enhanced MR angiography was r = 0.967. The 95% confidence interval for the line of means showed low errors bounds, ranging as low as ±2.8%. The 95% confidence interval for individual prediction of angiographic stenosis based on a given contrast-enhanced MR angiographic measurement, however, was significantly larger, being no less than ±13.6%. With the inclusion of PSV, the adjusted correlation was r = 0.965.

CONCLUSION: A clear linear relationship exists between digital subtraction angiographic and contrast-enhanced MR angiographic measurements of carotid stenosis. Increasing severity of stenosis as measured by contrast-enhanced MR angiography corresponds to increasing severity at angiography. Although the predictive value of contrast-enhanced MR angiography is excellent in the mean, it is less reliable for predicting the degree of angiographic stenosis in the individual patient, showing rather wide confidence intervals. Furthermore, the inclusion of PSV as an additional regressor does not improve the predictive accuracy beyond that of contrast-enhanced MR angiography alone.

Symptomatic patients with severe carotid artery stenosis (70–99%) derive a clear benefit from elective endarterectomy, resulting in overall decrease in stroke and mortality risk (1). Recent updated results from the North American Symptomatic Carotid Endarterectomy Trial (NASCET) reveal a small but statistically significant improvement in outcome for endarterectomy in symptomatic patients with stenoses in the 50% to 69% range as well, although the guidelines for patient selection for surgery are less clear (2). Results from the Asymptomatic Carotid Atherosclerosis Study (ACAS) also show a small benefit for endarterectomy performed in asymptomatic patients with stenoses >60% (3).

Considering the small benefit for mid-range symptomatic stenoses (50–69%), the lack of stratification of the ACAS data, and the small overall benefit for asymptomatic patients, we note that the vascular surgeons at our institutions have tended to develop an empiric standard of care based on the data and a subjective synthesis of the above considerations. For example, although no particular benefit stratification exists versus degree of stenosis in the ACAS data, asymptomatic patients with 60% stenosis are not routinely offered endarterectomy whereas those with 80% stenosis may be surgical candidates. Likewise, endarterectomy is more likely to be offered to a symptomatic patient with 65% stenosis rather than 55% stenosis. Therefore, accounting for both the data and the empirical practices, which combine the data with the “sense” the vascular surgeon has regarding whom should undergo surgery, it is clear that careful patient selection for endarterectomy is essential. The surgical procedure is associated with a finite risk of cerebrovascular events and other complications normally associated with general anesthesia and surgery (3).

Digital subtraction angiography is currently considered the “gold standard” for the evaluation of atherosclerotic lesions of the carotid arteries. This semi-invasive procedure carries a finite risk of complications, including but not limited to puncture site hematomas and pseudoaneurysms, thromboembolic events, contrast reactions, and nephrotoxicity (5). Noninvasive methods such as contrast-enhanced MR angiography, CT angiography, and ultrasonography are being evaluated as possible alternatives to digital subtraction angiography.

Studies in the past have consistently shown the ability of contrast-enhanced MR angiography to correctly classify the degree of angiographic stenosis, despite an occasional tendency toward overestimation (6–15). In general, previous studies compared digital subtraction angiography with contrast-enhanced MR angiography by stratifying stenoses into broad categories. The most commonly used classification scheme separated stenoses into the following groups: normal (0%), mild (1–29%), moderate (30–69%), severe (70–99%), and occlusion (100%). To date, it remains unclear whether contrast-enhanced MR angiography can estimate the degree of stenosis in the individual patient with a precision sufficient for surgical decision making in certain patient populations (eg, distinguishing a symptomatic 55% versus 65% stenosis or an asymptomatic 70% versus 80% stenosis).

Similar doubts have already arisen regarding the precision of ultrasonography in the grading of stenoses. Numerous studies have shown the ability of color duplex ultrasonography to grade carotid arterial stenoses with rates of accuracy reportedly ≥90% (16–22). A study recently performed at our institution, however, found that although ultrasonographic parameters such as ultrasound peak systolic velocity (PSV) are correlated with angiographic severity of stenosis and are excellent for identifying stenoses as above or below a single degree of severity (70%), Doppler examination does not function well in stenosis subclassification (23).

There is no question that on a broad level, contrast-enhanced MR angiography is strongly correlated with digital subtraction angiography. However, the precision may be inadequate at the level of the individual patient. In our study, we compared digital subtraction angiography and contrast-enhanced MR angiography by using a somewhat novel approach. Instead of assigning stenoses into broad categories, we used regression analysis and assigned confidence intervals to examine the precision of contrast-enhanced MR angiography in making predictions for the individual patient (ie, given a contrast-enhanced MR angiographic stenosis of 55%, what is the range of stenoses expected based on conventional angiography?). We also assessed whether adding ultrasound PSV as an additional regressor improves the accuracy of prediction to learn whether the combination of contrast-enhanced MR angiography and ultrasonography in a quantitative rather than qualitative sense can serve as adequate replacement for digital subtraction angiography within the confines of a simple linear regression model.

Methods

Participants

Hospital records from 1996 to 2000 were reviewed, and all patients who underwent evaluation of the carotid arteries with digital subtraction angiography, contrast-enhanced MR angiography, and ultrasonography were identified. All internal carotid artery stenoses measured to be between 10% and 90%, based on NASCET criteria, at the time of angiography were selected (n = 22 carotid arteries from 17 patients). For a carotid artery to be included in the study, the three types of examinations had to have been performed within 90 days of each other. Patients with intervening endarterectomies were excluded. Also, patients with angiographic stenoses >90% were not included because of concerns regarding distortions in the NASCET measurements due to distal luminal atrophy, possible paradoxical decreases in ultrasound PSV with very high grade stenoses, and the invariable presence of focal flow gaps on the contrast-enhanced MR angiograms at that degree of stenosis. Two potential candidates were excluded because of poor quality of their contrast-enhanced MR angiograms, one secondary to motion artifact and the other due to a missed bolus and venous contamination of the arterial phase images. Patients were initially referred for these examinations based on symptomatic (eg, stroke, transient ischemic attacks, or amaurosis fugax) or asymptomatic (eg, carotid bruit, preoperative for coronary artery bypass, or severe peripheral vascular disease) indications.

Imaging

Contrast-enhanced MR angiography.

All contrast-enhanced MR angiography was performed with a 1.5-T magnet using a 3D subtracted gradient-recalled echo sequence and turbo fast low angle shot sequence (4/1.6 [TR/TE]; flip angle, 25 degrees; matrix, 120 × 256). The total dose of gadolinium-based contrast material (ProHance; Bracco Diagnostics, Princeton, NJ) was 20 mL, injected by a power injector (Spectris; Medrad, Indianola, PA). The initial timing bolus consisted of 6 mL delivered at a rate of 3 mL/s and then 15 mL of saline flush. After the timing bolus was administered, power injection at a rate of 3 mL/s was performed. Measurements of angiographic stenosis were obtained in accordance with the NASCET methodology (1). The region of highest degree of stenosis was measured from the film using a jeweler’s magnifying glass with an embedded measuring scale marked in 0.1-mm increments. This was compared with the “normal” internal carotid artery distal to the stenosis. The formula for computation of the degree of stenosis was (1 − [minimum residual lumen / normal lumen]) × 100. Four carotid arteries with flow gaps and evidence of distal flow without luminal atrophy were assigned a stenosis of 80%, which is the average angiographic stenosis severity measured by using digital subtraction angiography in these particular examinations. This approach was adopted because it would artificially strengthen the predictive power of the regression model and hence make a conclusion of unreliability even more powerful. The images were reviewed independently and in blinded fashion by two neuroradiologists (G.M.H., S.M.E.), and the results were determined by averaging.

Ultrasonography.

Carotid ultrasonography was performed by experienced technologists in a single, accredited laboratory in a veterans hospital, and the ultrasonograms were interpreted by radiologists with specialization in ultrasound. Equipment included commercially available, state-of-the-art scanners (Advanced Technology Laboratories, Bothell, WA; Acuson, Mountain View, CA). Five- or 7.5-MHz linear array transducers were used as dictated by patient body habitus. All images were obtained according to a set laboratory protocol. Angle adjustment was based on flow direction as depicted by color Doppler. The highest angle-adjusted PSV was recorded from within each internal carotid artery. To perform linear regression, the relationship between PSV and angiographic stenosis was assumed to be linear in the range of stenoses in our study (10–90%).

Angiography.

Digital subtraction angiography was performed via a femoral artery approach with selective injections in the common carotid arteries. Two or more orthogonal views of each bifurcation were obtained. Delayed imaging and prolonged injections were performed for all patients. Our technique consisted of an exposure rate of one image per second for ≤20 s and a manual injection volume of ≤20 mL of contrast material (Isovue 300 [iopamidol], Bracco Diagnostics). The images were reviewed by two neuroradiologists (G.M.H., S.M.E.) in a blinded fashion, and the results were determined in consensus. The angiographic determination of the degree of stenosis was measured in accordance with NASCET methodology (1), in the same manner as that used for the contrast-enhanced MR angiographic measurements.

Statistical Analysis

Regression model.

We use the standard model of linear regression, assuming that there is a dependent variable, Y, which in this case is the measured digital subtraction angiographic stenosis, and an independent variable, X, which in this case is the measured contrast-enhanced MR angiographic stenosis from which Y is to be predicted. It is assumed that there is a true population regression line, such that: $\text{[math]}$ 1)

In other words, the expected value E(Y_i), or mean value μ_i, of the dependent variable is related to the independent variable by the least squares slope and intercept of the regression line. Of course, any individual Y_i measurement will deviate from its expected value by an error term, e_i, which is assumed to be normally distributed and to have a mean of zero and a variance σ²: $\text{[math]}$ 2)

None of these parameters (α, β, σ) is known for the population, and all must be estimated. The estimated regression line is thus: $\text{[math]}$ 3) where the variables $\text{[math]}$ and $\text{[math]}$ are calculated estimators of the true population parameters α and β, and Ŷ_i is the fitted value around the regression line. Likewise, σ is unknown and is estimated by the residual variance around the fitted line: $\text{[math]}$ 4)

It is then possible, as is shown in the accompanying appendix, to derive a 95% confidence interval for the mean value of Y₀, in other words, E(Ŷ₀) or u₀, the expected mean value of the dependent variable, given a certain X₀, or value of the independent variable (24): $\text{[math]}$ 5)

It is also possible to derive a 95% prediction interval for an individual observation Y₀, given a certain X₀, or value of the independent variable (24): $\text{[math]}$ 6)

In both cases, t is the Student’s t statistic, having (n − 2) degrees of freedom. Also, for mathematical convenience (as explained in the appendix), each independent variable X_i has been replaced by x_i, the deviation from the mean of the independent variables: $\text{[math]}$ 7)

Results

A total of 22 internal carotid arteries were included in the study, with digital subtraction angiographic measurements of stenosis ranging from 28% to 88%. Contrast-enhanced MR angiographic measurements of the arteries ranged from 32% to 85%. Ultrasound PSV ranged from 94 to 448 cm/s.

Digital Subtraction Angiography and Contrast-enhanced MR Angiography

Regression analysis showed a very strong correlation between stenoses measured by digital subtraction angiography and contrast-enhanced MR angiography, with r = 0.967 (slope = .9974, P < .0001; y intercept = 1.145). No clear tendency of contrast-enhanced MR angiography toward over- or underestimation of angiographic stenosis was observed (Fig 1). The regression line very closely reflected the true line of means, with 95% confidence intervals as low as ±2.83% (Fig 2). The 95% prediction interval for individual predictions of angiographic stenosis, given a contrast-enhanced MR angiographic measurement of stenosis, was relatively wide, being no less than ±13.6% (Figs 3 and 4).

Fig 1.

Scatter plot and regression line for digital subtraction angiographic measurement of stenosis versus contrast-enhanced MR angiographic measurement of stenosis. r = 0.967. The slope is very nearly 1, and the intercept is nearly 0, showing excellent linearity. DSA, digital subtraction angiography; Gd-MRA, contrast-enhanced MR angiography.

Fig 2.

Comparison of digital subtraction angiographic measurement of stenosis versus contrast-enhanced MR angiographic measurement of stenosis shows regression line with 95% mean confidence intervals, which reach a minimum of ±2.8%, showing excellent correlation between digital subtraction angiography and contrast-enhanced MR angiography in the mean. DSA, digital subtraction angiography; Gd-MRA, contrast-enhanced MR angiography.

Fig 3.

Comparison of digital subtraction angiographic measurement of stenosis versus contrast-enhanced MR angiographic measurement of stenosis shows regression line with 95% individual confidence intervals, which reach a minimum of ±13.6%. These intervals are significantly wider than the mean confidence intervals shown in Figure 2. DSA, digital subtraction angiography; Gd-MRA, contrast-enhanced MR angiography.

Fig 4.

Comparison of digital subtraction angiographic measurement of stenosis versus contrast-enhanced MR angiographic measurement of stenosis shows regression line with both 95% mean confidence intervals and 95% individual confidence intervals overlaid on the same graph to depict the difference in relative width. DSA, digital subtraction angiography; Gd-MRA, contrast-enhanced MR angiography.

PSV and Digital Subtraction Angiography

As seen in Figure 5, digital subtraction angiographic measurements of stenosis tend to increase with increasing ultrasound PSV. Linear regression showed moderate correlation between PSV and digital subtraction angiographic measurements of stenosis, with r = 0.8601 (slope = 0.1670, P < .0001; y intercept = 18.05). This correlation was weaker than that observed between contrast-enhanced MR angiography and digital subtraction angiography. The confidence intervals were much wider than that of contrast-enhanced MR angiography. The 95% confidence intervals for the true line of means showed minimum error bounds of ±5.70%. The precision of PSV for angiographic stenosis prediction at the individual level showed prediction intervals not less than ±27.3% (Fig 5).

Fig 5.

Comparison of digital subtraction angiographic measurement of stenosis versus ultrasound PSV shows regression line with 95% individual confidence intervals, which reach a minimum of ±27.3%. The fit is significantly less (r = 0.86) and the confidence intervals significantly wider than with contrast-enhanced MR angiography (Gd-MRA). DSA, digital subtraction angiography.

Digital Subtraction Angiography and Contrast-enhanced MR Angiography/PSV

Multiple regression analysis was conducted, describing digital subtraction angiography-measured stenosis as a function of both contrast-enhanced MR angiography and PSV, resulting in r = 0.968 and adjusted r = 0.965 (t ratio of contrast-enhanced MR angiography = 7.788 with P < .0001; t ratio of PSV = 0.8106, with P = .4276). This correlation was higher than that seen with PSV alone but was not significantly different from that of contrast-enhanced MR angiography alone. The coefficient for the intercept = 1.592, for contrast-enhanced MR angiography = 0.9150, and for PSV = 0.01794.

Discussion

It is well known that different surgical thresholds for endarterectomy apply for symptomatic and asymptomatic patients in accordance with the NASCET and ACAS studies. The complex threshold data are combined with subjective criteria on the part of vascular surgeons to arrive at decisions regarding which patients should be candidates for endarterectomy.

In symptomatic patients, for example, although the NASCET update on moderate (50–69%) stenosis showed some benefit for carotid endarterectomy, the benefit was mild, with overlapping of the confidence intervals of the stroke-free survival curves for the medical versus surgical arms and a higher perioperative complication rate compared with the severe stenosis group (25). The severe (≥70%) stenosis group, however, derives a clear benefit from endarterectomy, without overlap in the confidence intervals of stroke-free survival curves (25). These considerations are profoundly reflected in the numbers needed to treat to prevent one stroke: 19 for the moderate (50–69%) stenosis group versus six for the severe (≥70%) stenosis group. Meanwhile, the European Carotid Surgery Trial study showed a 9.8% perioperative complication rate for patients with moderate stenoses, an overall negative benefit for endarterectomy, and no calculable number needed to treat to prevent one stroke (25). Therefore, it becomes important to distinguish a 63% from a 73% stenosis, for example. Also, subjectively, our surgeons are thus more likely to take a symptomatic moderate stenosis to endarterectomy if the stenosis measures 65% rather than 55% because it is closer to the 70% cutoff at which there is unequivocal benefit. Furthermore, it has been shown that no benefit, and possibly harm, can occur from carotid endarterectomy in the <50% stenosis range (25). Therefore, it also becomes important to distinguish 46% from 54% stenosis, again by way of example.

In asymptomatic patients, the ACAS trial showed a reduction of stoke risk in patients with >60% stenosis. However, these results often are viewed with significant skepticism; the absolute average annual risk reduction in stroke was only 1%, yielding a 2-year number needed to treat to prevent one stroke of 83 (ie, it is necessary to operate on 83 patients to prevent one additional stroke in 2 years) (25). In light of this, our surgeons do not routinely consider asymptomatic endarterectomy until the stenosis approaches approximately 80%. Therefore, it becomes important to distinguish 70% from 80% stenosis, for example.

Hence, considering the necessity of accurate estimation of stenoses in certain patient populations, perhaps to within ±10% of a digital subtraction angiographic measurement, we pose the question of whether contrast-enhanced MR angiography is capable of this level of predictive accuracy. This, of course, begs the question of how the predictive accuracy is to be estimated or measured. Thus far, previous studies have shown the ability of contrast-enhanced MR angiography to successfully stratify patients into broad stenosis groups, with emphasis on the ability to detect lesions greater than or less than a 70% cutoff point (6–15). The predictive accuracy of contrast-enhanced MR angiography compared with digital subtraction angiography for more precise stenosis measurement in the individual patient is less clear and was the object of this study.

Previously, conventional 2D and 3D time-of-flight MR angiography had been evaluated for the delineation of carotid stenoses. Some of the difficulties associated with conventional MR angiography were related to lengthy acquisition time and movement artifact and the dependence on flow-related enhancement. Overestimation of stenoses and over-diagnosis of occlusions occurred as a result of saturation of slow flow spins and turbulent flow. One study revealed the general inaccuracy of conventional MR angiography, estimating that 23% of their study patients would have received non-indicated endarterectomies and that 33% would have been improperly denied clinically indicated endarterectomies if only MR angiographic predictions had been used (26).

Contrast-enhanced MR angiography has shown promise in addressing many of these issues, introducing the advantages of quick, breath-hold acquisitions and improved visualization of epiaortic and intracranial vasculature. The use of contrast material renders the examination more physiologically equivalent to conventional angiography, decreasing artifacts related to slow flow, and improving visualization of near occlusions and tandem lesions (6–15). However, the predictive accuracy in stenosis measurement in the individual remains unproved.

We attempted to answer this question using the theory and tools of simple linear regression analysis for a small series of patients, obtaining measurements of the contrast-enhanced MR angiograms based on the NASCET style rather than rendering qualitative readings, and attempting to assess predictive accuracy as compared with digital subtraction angiographic measurement of stenosis. The great advantage of linear regression analysis is that it not only allows the assessment of accuracy in the mean but also offers a method of calculating 95% prediction intervals of digital subtraction angiographically shown stenosis in the individual patient.

Our study shows several important results. First, the relationship between stenosis measurement by contrast-enhanced MR angiography and digital subtraction angiography is highly linear, with an excellent correlation coefficient (r = 0.967), a slope extremely close to 1.0 (0.9974, P < .0001), and a y intercept of nearly zero (1.145). Furthermore, there was no tendency to underestimate or overestimate, with the residuals distributed nearly equally above and below the regression line. This indicates that use of the linear regression theory should provide valid conclusions.

Second, the 95% confidence intervals for the mean around the regression line were extremely small, reaching a minimum of ±2.8%. This suggests that in the aggregate, contrast-enhanced MR angiographic stenosis measurement is an excellent predictor of digital subtraction angiographic stenosis measurement.

Finally, and most importantly, there may be some confusion regarding the significance of such numbers, which are sometimes the ones quoted to justify the accuracy of a test or study. These small error bounds imply accuracy on a large scale. For example, if a large group of patients is studied and has a mean contrast-enhanced MR angiographic stenosis of 55% (which was nearly the mean contrast-enhanced MR angiographic measurement in our patient group), our regression model predicts that if each of the patients undergoes conventional angiography and if the digital subtraction angiographic measurements of stenosis are tabulated, the mean digital subtraction angiographic measurement of stenosis for this group would be 56%. This is only a prediction, but with 95% confidence, the true mean of the digital subtraction angiographic measurement of stenosis would fall within ±2.8% of the predicted value; in other words, we can state with 95% confidence that the mean digital subtraction angiographic measurement of stenosis would fall between 53.2% and 58.8%. This agreement is excellent, and the small errors would not be expected to change surgical management. However, it must be emphasized that these results are valid only for the means of large groups. The clinically relevant question is if an individual patient has a contrast-enhanced MR angiographic measurement of stenosis of 55%, what are the 95% confidence intervals (or, more correctly, the prediction intervals) for the digital subtraction angiographic stenosis if angiography is performed? A comparison of equations 5 and 6 and Figures 2 and 3 shows that the individual error bars are much wider than the mean error bars. The minimum 95% prediction interval for the individual patient, occurring at a contrast-enhanced MR angiographic stenosis of 55%, is ±13.6%. For contrast-enhanced MR angiographic stenoses above and below the mean value of 55%, the error bars are even wider and the predictive accuracy is less.

The high degree of accuracy in the mean is comparable with, and theoretically equivalent to, the results that show that contrast-enhanced MR angiography is accurate at broad stratification of patients, in the statistical sense, into those with <70% and those with ≥70% stenosis. Our results show that it is significantly less accurate in the substratification of the degree of stenosis (eg, in 10% intervals) in the individual patient. This imprecision may preclude its use for the evaluation of lesions in “gray areas,” such as in the symptomatic patient with a moderate stenosis close to 50% or the asymptomatic patient with a questionable moderate versus severe stenosis, in which small errors in measurement may lead to large differences in clinical management. Unnecessary endarterectomies, along with their associated thromboembolic complications, could be the result; alternatively, beneficial endarterectomies in sufficiently stenotic arteries might be forgone as a result of an underestimation.

Our results also allow non-rigorous analysis of the possible contribution of ultrasound PSV to contrast-enhanced MR angiography in surgical decision making. It has been our clinical experience that for a large portion of patients, and depending on the institution, ultrasonography alone or a combination of ultrasonography and MR angiography is regularly used in lieu of arteriography before endarterectomy is performed. Clinical experience at our institution has shown that ultrasonography alone, although adequate for dividing patients into broad categories of above or below a 70% stenosis threshold, is inadequate for stratification of stenoses into subgroups, particularly for those stenoses that fall into the moderate level (50–69%) (23). The current study shows little additional benefit for the combination of both contrast-enhanced MR angiography and ultrasound PSV over contrast-enhanced MR angiography within a linear regression framework. Overall, contrast-enhanced MR angiography alone performed significantly better than did ultrasonography alone, showing much higher correlation with digital subtraction angiography, with smaller prediction intervals for individual measurements of true angiographic stenosis. This is consistent with previous results showing superior predictive accuracy for surgical decision making for MR angiography over ultrasonography. The work conducted by Johnston et al (27) revealed a misclassification rate of 18% for MR angiography used alone versus 28% for ultrasonography used alone in surgical decision making.

In our study, the correlation coefficient of contrast-enhanced MR angiography was not significantly improved when ultrasonography was included as an additional regressor. This is corroborated by the extremely small coefficient for ultrasound PSV in the multiple regression model (r = 0.01794), which is very nearly zero, indicating that it contributes essentially no added information compared with contrast-enhanced MR angiography alone. This, of course, is because of the phenomenon of colinearity and the high covariance between ultrasound PSV and contrast-enhanced MR angiographic measurement of stenosis; both are expected to increase together and are not independent variables. Although this result is not unexpected, it does highlight an important point regarding clinical decision making: concordance between the results of contrast-enhanced MR angiography and ultrasound PSV should not necessarily increase the confidence of angiographic stenosis prediction. Conversely, a discrepant PSV should not necessarily alter interpretation of the contrast-enhanced MR angiographic findings. Although this is only a tentative conclusion, it is significant in that it does contradict the findings of earlier work, which showed that a concordance between ultrasonography and MR angiography has a higher accuracy rate than does either test alone. In the work conducted by Johnston et al (27), the misclassification rate in surgical decision making was 28% for ultrasonography, 18% for MR angiography, and 7.9% for concordance of the two examination modalities. However, that earlier work relied essentially completely on conventional time-of-flight MR angiography rather than on contrast-enhanced MR angiography. Hence, further investigation of the contribution of ultrasonography to contrast-enhanced MR angiography in surgical decision making is warranted.

To avoid being overly pessimistic regarding the usefulness of contrast-enhanced MR angiography as an accurate noninvasive test, our results must be examined in the context of several factors. The first of these is the accuracy of digital subtraction angiography versus itself. For example, the largest studies examining the optimal guidelines for carotid endarterectomy, such as the NASCET, base their recommendations on measurements of carotid stenosis as determined by digital subtraction angiography. Although angiography currently remains the gold standard for evaluation of carotid artery stenosis, this method itself has been noted to have some difficulty in classifying lesions into categories as tight as 10% (28), even with an allowance for the minimal interobserver error usually ascribed to the NASCET method of measurement (29). Other studies have shown surgical misclassification rates of 3.4% to 7.3% and 3.8% to 12.4% for digital subtraction angiography when looking at the interobserver variations (30, 31). This inherent error margin in measurements of conventional angiograms should thus be considered when evaluating other noninvasive imaging methods, and, at a minimum, this level of error should be expected and considered baseline. Using this criterion, predictive intervals of 13.6% may be acceptable, especially at the ends of the spectrum, where surgical decisions are clear.

A second important consideration concerns the methodology for deciding “misclassifications” by noninvasive tests. In the work conducted by Johnston et al (27), for example, the surgical population was defined as symptomatic patients with 50% to 99% stenoses and asymptomatic patients with 60% to 99% stenoses. As stated earlier, the data regarding the efficacy of carotid endarterectomy in symptomatic patients with moderate stenoses and in asymptomatic patients in general raise serious concerns regarding the appropriateness of these cutoff points. The practice of our vascular surgeons also does not concur with these criteria. For example, symptomatic patients with 50% stenosis or asymptomatic patients with 60% stenosis rarely undergo endarterectomy. If the criteria are redefined in a less exact fashion, but one that is more reflective of clinical practice, contrast-enhanced MR angiography is probably sufficiently accurate in the majority of cases. For example, if the criteria for surgery are taken as ≥60% for symptomatic patients with an allowance for a 10% overcall (ie, an allowance that some of the stenosis measured to be 60% based on contrast-enhanced MR angiography would actually measure as low as 50% based on digital subtraction angiography) or as 80% in asymptomatic patients with an allowance for a 10% overcall, the error bounds of contrast-enhanced MR angiography would seem to be to be sufficiently accurate in most cases. This is so because although the 95% error bounds are large, only a small percent of the errors will be as large as the outer limits of the bounds, especially when the errors are normally distributed. Such criteria seem to be more consistent with the subjective element of current practice, although they do not coincide precisely with literature cutoffs for surgical populations.

The question arises regarding the role of digital subtraction angiography. Although our study shows large individual error bounds in individual prediction intervals, we do not advocate the abandonment of noninvasive tests; they are used successfully in the more subjective paradigm offered above. However, we think that there is a specific role for digital subtraction angiography: to avoid unnecessary endarterectomies when noninvasive studies are equivocal. The perioperative complication rate from the NASCET and Aspirin and Carotid Endarterectomy Trial was 6.2% and was higher in patients with moderate stenoses than in those with severe stenoses. Additionally, the European Carotid Surgery Trial study showed a 9.8% perioperative complication rate for patients with moderate stenoses, with an overall negative benefit for endarterectomy. Meanwhile, the data, as described above, are even less convincing regarding the asymptomatic patient, for whom any benefit at all is highly dependent on very low perioperative complication rates (25). The risks of direct angiography, however, are in the vicinity of a 1% overall incidence of neurologic complications and 0.5% incidence of persistent neurologic deficits (32). Therefore, we suggest that angiography be considered for symptomatic patients at the lower end of the moderate stenosis category and for asymptomatic patients who have less than severe stenosis who would otherwise undergo endarterectomy on the basis of noninvasive studies alone.

Finally, an important consideration regarding the validity of our conclusions is the effect of the small sample size in our study and the effect of sample size in similar studies in general. Sample sizes are likely to be somewhat limited because of considerations similar to our own: whether at the veterans hospital or the university hospital, most patients at our institutions do not undergo digital subtraction angiography before endarterectomy, and the MR angiographic examinations tend to be time-of-flight examinations. Therefore, the number of patients undergoing digital subtraction angiography, contrast-enhanced MR angiography, and ultrasonography is small. In the large retrospective study conducted by Johnston et al (27), only 11% of 569 patients had undergone digital subtraction angiography, MR angiography, and ultrasonography, rendering numbers in the same order of magnitude as for our population. Furthermore, only 1% of the MR angiograms in that study were contrast-enhanced MR angiograms. Also, for example, the contrast-enhanced MR angiography study conducted by Aoki et al (33) analyzed 20 patients with >0% stenoses using contrast-enhanced MR angiography and digital subtraction angiography. Therefore, it is important to examine what conclusions may be drawn from these populations.

A great advantage of the machinery of linear regression, once shown to be a valid paradigm, is the ability to extrapolate error estimates from small to very large samples. Recalling equation 6 for the individual prediction intervals, we see that the associated error bars for 95% prediction intervals are given by the following equation: $\text{[math]}$ 8)

The actual estimate depends on the degree of stenosis, x₀, and the sample size, which influences the 1/n term and the t statistic, for which the degrees of freedom = n − 2. In the optimum, if the contrast-enhanced MR angiographic measurement of stenosis happens to be the population mean, the rightmost term under the radical vanishes as x₀, previously defined as the deviation from the mean, becomes 0. Meanwhile, if the sample size becomes infinite, the 1/n term vanishes, the t statistic becomes the z statistic by the equivalence of z and Student’s t tests when the degrees of freedom in the t test are infinite, and s, the estimated standard error, becomes σ, the true population standard error. Therefore, under the most optimal of circumstances and with an infinite sample size, the error term for the 95% prediction intervals cannot fall below ±z_.025σ. Therefore, if we know σ, we can estimate the best-case scenario error bounds, regardless of the issue of sample size.

Once we establish the validity of a linear regression model, as discussed previously, this in turn allows the use of a modified χ² statistic to estimate error bounds around σ from our calculated s. The χ² statistic takes into account the sample size. Thus, using our data, for degrees of freedom = 20 (n − 2 degrees of freedom) and a standard χ² table, we can state with 95% confidence that: $\text{[math]}$ 9)

Inverting this equation, we can thus state with 95% confidence that: $\text{[math]}$ 10)

Our s value, or estimated error from our sample, was 6.36. Thus, with 95% confidence, we know that 4.9 < σ < 9.2. Therefore, using the value z_.025 = 1.96, the best-case scenario error bounds are 9.54 < z_.025σ < 18.03%. This is in excellent agreement with our result of 13.6% and clarifies that even with infinite sample sizes, the error bars for an individual 95% prediction interval would not fall below ±9.54% and would tend to be somewhere closer to the middle of the interval.

Conclusion

Although our data show excellent general correlation between contrast-enhanced MR angiography and digital subtraction angiography, we posit that the relevant clinical issue is the predictive accuracy for the individual patient. The best-case predictive interval for our data (±13.6%) for estimation of digital subtraction angiographic measurement of stenosis may be excessively wide in certain patient populations. It is known that such error intervals, however, exist with any technique and are, in general, due to two different sources: measurement error (ie, the physical limitations of the accuracy of digital subtraction angiographic or contrast-enhanced MR angiographic measurement of stenosis) and stochastic error (24). Thus, our somewhat negative conclusions must be tempered with the previously discussed limitations of digital subtraction angiography itself. If, as digital subtraction angiography is viewed as the gold standard, the uncertainty associated with digital subtraction angiographic measurements is viewed as the baseline minimum error, then perhaps the additional error introduced by the use of contrast-enhanced MR angiography is within clinically acceptable bounds in the majority of cases. This is especially true when the more subjective elements of surgical decision making, which have evolved around the published criteria, are taken into account in the accuracy analysis.

Appendix

Derivation of Mean and Individual Confidence Intervals for Least Squares Regression

A regression line is to be fitted such that the predicted observations, which will be called Ŷ_i, will best mirror the real observations, Y_i. Therefore, we wish to derive a regression line, Ŷ_i = $\text{[math]}$ + $\text{[math]}$ x_i, which will minimize least squares errors, ς(Y_i − Ŷ_i)², or ς(Ŷ_i − $\text{[math]}$ + $\text{[math]}$ x_i)². In this case, each independent variable, X_i, has been replaced by x_i, the deviation from the mean of the independent variables (x_i = X_i − X̄). This remapping of the independent variables as displacements from their own mean leads to the important simplification that ςx_i = 0.

To derive the least squares estimators, we set the partial derivatives of the squared-error term to zero and solve the resulting equations in standard fashion. For the slope, $\text{[math]}$ , the derivation proceeds as follows: $\text{[math]}$ 11)

Dividing by −2 and distributing the summation yields: $\text{[math]}$ 12)

Recalling that ςx_i = 0, this can be solved for the slope of the line $\text{[math]}$ to yield: $\text{[math]}$ 13)

A similar but simpler derivation yields the intercept of the regression line: $\text{[math]}$ 14)

Now, $\text{[math]}$ can be seen as a linear combination of the values of the dependent variable Y_i; $\text{[math]}$ 15) where $\text{[math]}$ 16)

This can further be rewritten as: $\text{[math]}$ 17) where $\text{[math]}$ 18)

Because the values of Y_i are independent of each other, the variance of $\text{[math]}$ can now be easily expressed as a weighted sum of the variance of Y_i: $\text{[math]}$ 19)

Of course, the variance of each term Y_i is just the variance of the dependent variable, previously defined as σ². Thus: $\text{[math]}$ 20)

Once again, a similar but simpler argument also establishes that: $\text{[math]}$ 21)

Now, we are ready to apply these results to the derivation of confidence intervals. We ask the question, for a given value of the dependent variable, say x₀, what is the mean value of the dependent variable Y₀ and the corresponding confidence interval surrounding it?

Using the parameters derived for the least squares regression line, it is easy to state that: $\text{[math]}$ 22) where Ŷ₀ is the regression estimate of the mean. In resetting (x_i = X_i − Ē), this makes $\text{[math]}$ and $\text{[math]}$ have a zero covariance. Therefore: $\text{[math]}$ 23) or, by substitution of the previously derived values, $\text{[math]}$ 24)

Because the true population variance σ² is unknown, it is estimated by the residual variance, $\text{[math]}$ 25)

Assuming normality for the error terms, 95% confidence intervals for Ȳ₀ can now be constructed: $\text{[math]}$ 26)

The confidence intervals for prediction of a single value of the dependent variable, say Y₀, based on a value of the independent variable x₀ follow an essentially identical derivation to that presented above. We start with the estimate of Y₀, given by the regression equation: $\text{[math]}$ 27)

To the variance of the estimate of the mean $\text{[math]}$ 28) we must now add the inherent variance of the individual Y observations around the mean, to get: $\text{[math]}$ 29)

Once again, using the residual variance to estimate the true population variance, we get: $\text{[math]}$ 30)

Footnotes

Original research was presented at the 2002 Annual Meeting of the Radiological Society of North America.

References

North American Symptomatic Carotid Endarterectomy Trial Collaborators. Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis. N Engl J Med 1991;325:445–453
Barnett HJ, Taylor DW, Eliasziw M, et al. Benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenosis. N Engl J Med 1998;339:1415–1425
Executive Committee for the Asymptomatic Carotid Atherosclerosis Study. Endarterectomy for asymptomatic carotid artery stenosis. JAMA 1995;273:1421–1428
Moore WS, Barnett HJ, Beebe HG, et al. Guidelines for carotid endarterectomy: a multidisciplinary consensus statement from the ad hoc committee, American Heart Association. Stroke 1995;26:188–201
Hessel SJ, Adams DF, Abrams HL. Complications of angiography. Radiology 1981;138:273–281
Enochs WS, Ackerman RH, Kaufman JA, Candia M. Gadolinium-enhanced MR angiography of the carotid arteries. J Neuroimaging 1998;8:185–190
Remonda L, Heid O, Schroth G. Carotid artery stenosis, occlusion, and pseudo-occlusion: first-pass, gadolinium-enhanced, three-dimensional MR angiography: preliminary study. Radiology 1998;209:95–102
Leclerc X, Gauvrit JY, Nicol L, Pruvo JP. Contrast-enhanced MR angiography of the craniocervical vessels: a review. Neuroradiology 1999;41:867–874
Kollias SS, Binkert CA, Ruesch S, Valavanis A. Contrast-enhanced MR angiography of the supra-aortic vessels in 24 seconds: a feasibility study. Neuroradiology 1999;41:391–400
Leclerc X, Martinat P, Godefroy O, et al. Contrast-enhanced three-dimensional fast imaging with steady-state precession (FISP) MR angiography of supraaortic vessels: preliminary results. AJNR Am J Neuroradiol 1998;19:1405–1413
Scarabino T, Carriero A, Giannatempo GM, et al. Contrast-enhanced MR angiography (CE MRA) in the study of the carotid stenosis: comparison with digital subtraction angiography (DSA). J Neuroradiol 1999;26:87–91
Huston J III, Fain SB, Wald JT, et al. Carotid artery: elliptic centric contrast-enhanced angiography compared with conventional angiography. Radiology 2001;218:138–143
Fellner FA, Fellner C, Wutke R, et al. Fluoroscopically triggered contrast-enhanced 3D MR DSA and 3D time-of-flight turbo MRA of the carotid arteries: first clinical experiences in correlation with ultrasound, x-ray angiography, and endarterectomy findings. Magn Reson Imaging 2000;18:575–585
Randoux B, Marro B, Koskas F, et al. Carotid artery stenosis: prospective comparison of CT, three-dimensional gadolinium-enhanced MR, and conventional angiography. Radiology 2001;220:179–185
El-Saden SM, Grant EG, Hathout GM, Zimmerman PT, Cohen SN, Baker JD. Imaging of the internal carotid artery: the dilemma of total versus near total occlusion. Radiology 2001;221:301–308
Fell G, Phillips DJ, Chikos PM, Harley JD, Thiele BL, Strandness DE Jr. Ultrasonic duplex scanning for disease of the carotid artery. Circulation 1981;64:1191–1195
Garth KE, Carroll BA, Sommer FG, Oppenheimer DA. Duplex ultrasound scanning of the carotid arteries with velocity spectrum analysis. Radiology 1983;147:823–827
Zwiebel WJ, Austin CW, Sackett JF, Strother CM. Correlation of high-resolution, B-mode and continuous-wave Doppler sonography with arteriography in the diagnosis of carotid stenosis. Radiology 1983;149:523–532
Jacobs NM, Grant EG, Schellinger D, Byrd MC, Richardson JD, Cohan SL. Duplex carotid sonography: criteria for stenosis, accuracy, and pitfalls. Radiology 1985;154:385–391
Bluth EI, Stavros AT, Marich KW, Wetzner SM, Aufrichtig D, Baker JD. Carotid duplex sonography: a multicenter recommendation for standardized imaging and Doppler criteria. Radiographics 1988;8:487–506
Robinson ML, Sacks D, Perlmutter GS, Marinelli DL. Diagnostic criteria for carotid duplex sonography. AJR Am J Roentgenol 1988;151:1045–1049
Grant EG, Duerinckx AJ, El-Saden S, et al. Doppler sonographic parameters for detection of carotid stenosis: is there an optimum method for their selection? AJR Am J Roentgenol 1999;172:1123–1129
Grant EG, Duerinckx AJ, El Saden SM, et al. Ability to use duplex US to quantify internal carotid arterial stenoses: fact or fiction? Radiology 2000;214:247–252
Wonnacott TH, Wonnacott RJ. Introductory Statistics for Business and Economics. 2nd ed. New York: John Wiley & Sons;1977 .
Barnett HJ, Meldrum HE, Eliasziw M. The appropriate use of carotid endarterectomy. CMAJ 2002;166:1169–1179
Wardlaw JM, Lewis SC, Collie DA, Sellar R. Interobserver variability of magnetic resonance angiography in the diagnosis of carotid stenosis: effect of observer experience. Neuroradiology 2002;44:126–132
Johnston DC. Goldstein LB. Clinical carotid endarterectomy decision making: noninvasive vascular imaging versus angiography. Neurology 2001;56:1009–1015
Dippel DW, van Kooten F, Bakker SL, Koudstaal PJ. Interobserver agreement for 10% categories of angiographic carotid stenosis. Stroke 1997;28:2483–2485
Gagne PJ, Matchett J, MacFarland D, et al. Can the NASCET technique for measuring carotid stenosis be reliably applied outside the trial? J Vasc Surg 1996;24:449–456
Young GR, Sandercock PA, Slattery J, Humphrey PR, Smith ET, Brock L. Observer variation in the interpretation of intra-arterial angiograms and the risk of inappropriate decisions about carotid endarterectomy. J Neurol Neurosurg Psychiatry 1996;60:152–157
Young GR, Humphrey PR, Nixon TE, Smith ET. Variability in measurement of extracranial internal carotid artery stenosis as displayed by both digital subtraction and magnetic resonance angiography: an assessment of three caliper techniques and visual impression of stenosis. Stroke 1996;27:467–473
Heiserman JE, Dean BL, Hodak JA, et al. Neurologic complications of cerebral angiography. AJNR Am J Neuroradiol 1994;15:1401–1407
Aoki S, Nakajima H, Kumagai H, Araki T. Dynamic contrast-enhanced MR angiography and MR imaging of the carotid artery: high-resolution sequences in different acquisition planes. AJNR Am J Neuroradiol 2000;21:381–385

Received March 19, 2003.
Accepted after revision April 12, 2003.

Main menu

User menu

Search

American Journal of Neuroradiology