The Revascularization Scales Dilemma: Is It Right to Apply the Treatment in Cerebral Ischemia Scale in Posterior Circulation Stroke?

Abstract

BACKGROUND AND PURPOSE: Although various revascularization scales are used in the angiographic evaluation of acute ischemic stroke, observer reliability tests of these scales have been rarely performed for posterior circulation stroke. We aimed to evaluate inter- and intraobserver variability of 2 scales, the modified Treatment in Cerebral Ischemia and the Arterial Occlusive Lesion, in posterior circulation stroke.

MATERIALS AND METHODS: Three independent readers interpreted pre- and postthrombolytic angiographies of 62 patients with posterior circulation stroke by using the modified Treatment in Cerebral Ischemia and Arterial Occlusive Lesion scales. The κ statistic was used to measure observer agreement for both scales, and κ > 0.6 was considered substantial agreement.

RESULTS: For the Arterial Occlusive Lesion scale, inter- and intraobserver agreement was >0.6. While intraobserver agreement of the modified Treatment in Cerebral Ischemia scale was >0.6 except for 1 reader, interobserver agreement was lower in dichotomized and original scales. In 49 cases with solely basilar artery occlusion, inter- and intraobserver agreement of both scales was similar to that in all 62 patients with posterior circulation stroke. In 2 consecutive readings, there was a significant decrease in the proportion of mTICI 2a reads (22.58% in the first versus 13.44% in the second session, P < .03) and a reciprocal increase in the sum of proportions for modified Treatment in Cerebral Ischemia 2b and modified Treatment in Cerebral Ischemia 3 reads (62.37% in the first versus 72.58% in the second session, P < .046).

CONCLUSIONS: In angiographic assessment of posterior circulation stroke, inter- and intraobserver agreement for the Arterial Occlusive Lesion scale was reliable, while the modified Treatment in Cerebral Ischemia failed to achieve substantial interobserver agreement. The clinical impact of this result needs to be validated in future studies.

ABBREVIATIONS:

AOL: Arterial Occlusive Lesion
BAO: basilar artery occlusion
mTICI: modified Treatment in Cerebral Ischemia
TICI: Thrombolysis in Cerebral Infarction
TIMI: Thrombolysis in Myocardial Infarction

In the treatment of acute ischemic stroke with large intracranial arterial occlusion, endovascular techniques are becoming the mainstream with higher revascularization rates.^1⇓–3 While various grading schemes, such as the Thrombolysis in Myocardial Infarction (TIMI) or Thrombolysis in Cerebral Infarction (TICI), are widespread, their reliability in angiographic assessment of anterior circulation stroke remains controversial.^{4⇓⇓⇓–8} Recently, the modified Treatment in Cerebral Ischemia (mTICI) and the Arterial Occlusive Lesion (AOL) scales were strongly recommended as standards of reperfusion and recanalization in the angiographic evaluation of anterior circulation stroke.⁹

Despite applying similar scales to the posterior circulation, it is also unclear which scales might be reliably implemented for the vertebrobasilar territory. Recently, Gerber et al¹⁰ questioned whether it is right to use the TIMI or TICI scale in posterior circulation stroke and demonstrated that interobserver variability tests of these scales had never been performed in case of posterior circulation stroke.

Therefore, we aimed to evaluate intra- and interobserver agreement of the mTICI and AOL scales in the angiographic evaluation of posterior circulation stroke.

Materials and Methods

Patients

After approval of the institutional review board for this retrospective study, 62 patients (men/women = 37:25, mean age = 68 ± 11.4 years) with acute posterior circulation stroke, who underwent intra-arterial thrombolysis from April 2004 to December 2013, were consecutively enrolled from a single institutional data base. All patients underwent digital subtraction angiography before and after the procedure.

In these patients, the mean NIHSS score at admission was 17 ± 8, and the levels of arterial occlusion in the basilar (n = 49) and vertebral arteries (n = 13) were 79% and 21%, respectively. At 3 months, 21 patients (33.9%) had a good outcome (modified Rankin Scale score 0–2) and 12 (19%) had died.

Image Acquisition

DSA images from angiograms of the bilateral vertebral arteries were acquired by using a biplane angiography system. Consecutive anteroposterior and lateral angiographic images before and after intra-arterial thrombolysis, from the arterial to the delayed venous phase, were obtained in JPEG format, and were converted to a movie file format (Adobe Flash authoring file). Each reader interpreted them via on-line storage by making a comparison between preoperative and postoperative DSA images.

Image Interpretation

Three experienced readers (S.H.S., C.J., and W.Y.), who worked in 3 different tertiary medical centers as interventional neuroradiologists (with >10 years of experience), independently reviewed all images of 62 cases twice, 3–4 months apart. There was no training or required consensus of readers to perform this task.

Regardless of the others' reads, each reader assessed his own read by using the mTICI and AOL scales. In Table 1, the mTICI scale is defined in 5 grades according to the Stroke Treatment Academic Industry Roundtable consensus⁹ and the AOL scale is classified into 4 grades.¹¹ For this study, we did not provide any special information to the readers.

View this table:

Table 1:

The modified Treatment in Cerebral Ischemia and the Arterial Occlusive Lesion scale scores

Statistical Analysis

As parameters of intraobserver and interobserver agreement, the κ statistic was used for the mTICI and AOL scales. The κ value was interpreted according to Landis and Koch¹² with a κ value of 0 = poor, 0.01–0.20 = slight, 0.21–0.40 = fair, 0.41–0.60 = moderate, 0.61–0.80 substantial, and 0.81–1.0 = almost-perfect agreement. The κ statistic was also calculated for 49 cases with basilar artery occlusion (BAO) and the dichotomized groups, including those with poor revascularization (AOL = 0–2, mTICI = 0–2a) versus good revascularization (AOL = 3, mTICI = 2b–3). Comparison of the κ values was performed by using 95% CIs for the difference between the κ statistics, with 1000 bootstrapped samples. The difference was statistically significant if 95% CIs did not include zero. Statistical analyses were performed by using SAS (Version 9.2; SAS Institute, Cary, North Carolina).

Results

Contrary to the AOL scale, all pair-wise κ values for the mTICI were lower than 0.6 in interobserver agreement (Table 2). Intraobserver agreement for the AOL and mTICI scales was substantial to almost perfect in most cases, except for 1 reader for the mTICI scale (reader A, κ = 0.444 in all and 0.462 in BAO, respectively; Table 3). Regardless of the scale used, intraobserver and interobserver agreement for 49 cases with BAO was not significantly different from that for all 62 patients. For the dichotomized scales, there was an increasing trend of κ values in the AOL, but not in the mTICI scale.

View this table:

Table 2:

Interobserver agreement of 62 cases and subset of 49 cases with basilar artery occlusion by using original versus dichotomized outcomes

View this table:

Table 3:

Intraobserver agreement between 62 cases and subset of 49 cases with basilar artery occlusion

Table 4 shows the distribution of all reads by using either the mTICI or the AOL scale. In 2 consecutive readings, there was a significant decrease in the proportion of mTICI 2a (42/186, 22.58%, in the first session versus 25/186, 13.44%, in the second session; P < .03) and a reciprocal increment in the sum of proportions for mTICI 2b and mTICI 3 (116/186, 62.37%, in the first session versus 135/186, 72.58%, in the second session; P < .046). In the proportion of unanimity, the mTICI was also significantly lower than the AOL (14/62, 23%, in mTICI, versus 44/62, 71%, in the AOL; P < .001).

View this table:

Table 4:

Proportions in all ratings of 3 readers by categories of the scales

Discussion

In the 17 recent studies regarding intra-arterial thrombolysis in acute posterior circulation stroke,^{13⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓⇓–29} 5 (29.4%) did not provide any scheme and 12 (70.6%) used the TIMI or TICI scale, of which 58% (7/12) used TIMI ≥ 2 and 42% (5/12), TICI ≥2b as a cutoff level of “successful reperfusion.” In 10 studies with only BAO cases,^{14,17,20⇓⇓–23,25,27⇓–29} the TIMI scale was used in 5 and the TICI scale in 4. However, observer reliability tests in those previous studies had never been performed for angiographic assessment of posterior circulation stroke. If such heterogeneous and inconsistent scoring systems are used in defining end points of revascularization success, it is difficult to compare or combine results of clinical studies.³⁰ Therefore, selection of a reliable biomarker for revascularization is relevant to the prediction of procedural efficacy and the outcome in posterior circulation stroke. To our knowledge, this is the first study to evaluate inter- and intraobserver agreement for 2 common scales in the angiographic evaluation following intra-arterial thrombolysis of posterior circulation stroke.

In this study, we found the observer variability of the mTICI inherent in the angiographic assessment of posterior circulation stroke. While inter- and intraobserver variability for the AOL was reliable, the mTICI failed to achieve substantial interobserver agreement with a low concordance rate of 23% (14/62 cases). We also demonstrated a significant difference in the proportion of mTICI 2a reads that had a direct influence on the sum of proportions of mTICI 2b and 3, which may affect the judgment of angiographic end points in intra-arterial thrombolysis. The mTICI was inferred inferior to the AOL for the following reason¹: the relative complexity of the mTICI with more responses and semiquantitative descriptors. Gaha et al⁴ explained that inter- and intraobserver disagreement in adjudicating treatment results may be caused by multiple problems: intrinsic ambiguities in the definitions of the classifications; discrepancies in the various ways the definitions are interpreted by various readers; and even if the definitions were understood in the same way, discrepancies in applying the definitions to individual cases. Kundel and Polansky³¹ also showed that the κ value in observer agreement was likely to increase as the number of categories decreased.² It is possible to have inconsistency in interpreting the mTICI 2 grade, such as mTICI 2a versus mTICI 2b, especially for posterior circulation. In fact, angiographic evaluation of the vertebrobasilar territory has some limitations, such as interference with abundant collateral flows, incomplete visualization of the perforating arteries to the brain stem, and the necessity to consider the antegrade flow from the anterior circulation.

Because endovascular revascularization therapy is becoming the main strategy for acute ischemic stroke, it is important to use the optimal scale with high reliability in decisions of revascularization end points. In fact, revascularization can be understood as angiographic recanalization of the primary arterial occlusive lesion or reperfusion in the arterial bed distal to the occlusion (TIMI, TICI, mTICI). The AOL scale has been the sole scoring system for measuring the degree of recanalization at the target arterial lesion since its introduction in the Interventional Management of Stroke trials,^32⇓–34 and the posterior circulation occlusions were categorized according to AOL recanalization in Interventional Management of Stroke III.³² Although Gaha et al⁴ reported that observer variability for the AOL was “moderate” in anterior circulation stroke, we found that this scheme had high reliability in posterior circulation stroke. Considering its ease of use and consistency, it is possible to evaluate the AOL as a recanalization scale in a further posterior circulation stroke study.

This study had some limitations. First, the study design was retrospective with a limited number of cases. Second, the predictive power of both scales in this study was not analyzed due to the sampling heterogeneity. In fact, whether to choose the recanalization or reperfusion scale as a determinant of clinical outcome in posterior circulation stroke is still controversial. Cho et al³⁵ reported that reperfusion was a reliable surrogate and the strongest predictor of clinical outcome in anterior circulation stroke, and Singer et al³⁶ reported that independent predictors of clinical outcome were not the TICI scale, but the collateral status in BAO. In contrast, Mourand et al³⁷ proposed a DWI brain lesion score for prediction of clinical outcome in patients with BAO by using brain MR imaging. Finally, unfortunately in this study, only bilateral vertebral angiographies were used for interpretation, which may cause underdiagnosis of the mTICI because of the imperfect evaluation of collateral flows from the circle of Willis circulation.

Conclusions

In angiographic assessment of posterior circulation stroke, this is the first study to evaluate inter- and intraobserver variability for 2 commonly used scales; while the AOL as a recanalization scale showed a higher reliability, the mTICI, as a reperfusion criterion, failed to achieve substantial interobserver agreement among readers. In future studies, it will be necessary to validate the clinical impact of this result in posterior circulation stroke.

Acknowledgments

We thank Kyung Hwa Han, MS, for her assistance with statistics for this study.

Footnotes

This study was supported by a faculty research grant from Yonsei University College of Medicine (6-2015-0051).

References

1.
2. Berkhemer OA,
3. Fransen PS,
4. Beumer D, et al
. A randomized trial of intraarterial treatment for acute ischemic stroke. N Engl J Med 2015;372:11–20 doi:10.1056/NEJMoa1411587 pmid:25517348
2.
2. Campbell BC,
3. Mitchell PJ,
4. Kleinig TJ, et al
; EXTEND-IA Investigators. Endovascular therapy for ischemic stroke with perfusion-imaging selection. N Engl J Med 2015;372:1009–18 doi:10.1056/NEJMoa1414792 pmid:25671797
3.
2. Goyal M,
3. Demchuk AM,
4. Menon BK, et al
; ESCAPE Trial Investigators. Randomized assessment of rapid endovascular treatment of ischemic stroke. N Engl J Med 2015;372:1019–30 doi:10.1056/NEJMoa1414905 pmid:25671798
4.
2. Gaha M,
3. Roy C,
4. Estrade L, et al
. Inter- and intraobserver agreement in scoring angiographic results of intra-arterial stroke therapy. AJNR Am J Neuroradiol 2014;35:1163–69 doi:10.3174/ajnr.A3828 pmid:24481332
5.
2. Suh SH,
3. Cloft HJ,
4. Fugate JE, et al
. Clarifying differences among Thrombolysis in Cerebral Infarction scale variants: is the artery half open or half closed? Stroke 2013;44:1166–68 doi:10.1161/STROKEAHA.111.000399 pmid:23412375
6.
2. Bar M,
3. Mikulik R,
4. Jonszta T, et al
. Diagnosis of recanalization of the intracranial artery has poor inter-rater reliability. AJNR Am J Neuroradiol 2012;33:972–74 doi:10.3174/ajnr.A2896 pmid:22241381
7.
2. Tomsick T
. TIMI, TIBI, TICI: I came, I saw, I got confused. AJNR Am J Neuroradiol 2007;28:382–84 pmid:17297017
8.
2. Kallmes DF
. TICI: if you are not confused, then you are not paying attention. AJNR Am J Neuroradiol 2012;33:975–76 doi:10.3174/ajnr.A2905 pmid:22241379
9.
2. Zaidat OO,
3. Yoo AJ,
4. Khatri P, et al
; Cerebral Angiographic Revascularization Grading (CARG) Collaborators, STIR Revascularization working group, STIR Thrombolysis in Cerebral Infarction (TICI) Task Force. Recommendations on angiographic revascularization grading standards for acute ischemic stroke: a consensus statement. Stroke 2013;44:2650–63 doi:10.1161/STROKEAHA.113.001972 pmid:23920012
10.
2. Gerber JC,
3. Miaux YJ,
4. von Kummer R
. Scoring flow restoration in cerebral angiograms after endovascular revascularization in acute ischemic stroke patients. Neuroradiology 2015;57:227–40 doi:10.1007/s00234-014-1460-x pmid:25407716
11.
2. Khatri P,
3. Neff J,
4. Broderick JP, et al
; IMS-I Investigators. Revascularization end points in stroke interventional trials: recanalization versus reperfusion in IMS-I. Stroke 2005;36:2400–03 doi:10.1161/01.STR.0000185698.45720.58 pmid:16224088
12.
2. Landis JR,
3. Koch GG
. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74 doi:10.2307/2529310 pmid:843571
13.
2. Becker KJ,
3. Monsein LH,
4. Ulatowski J, et al
. Intraarterial thrombolysis in vertebrobasilar occlusion. AJNR Am J Neuroradiol 1996;17:255–62 pmid:8938295
14.
2. Egan R,
3. Clark W,
4. Lutsep H, et al
. Efficacy of intraarterial thrombolysis of basilar artery stroke. J Stroke Cerebrovasc Dis 1999;8:22–27 doi:10.1016/S1052-3057(99)80035-4 pmid:17895133
15.
2. Lin DD,
3. Gailloud P,
4. Beauchamp NJ, et al
. Combined stent placement and thrombolysis in acute vertebrobasilar ischemic stroke. AJNR Am J Neuroradiol 2003;24:1827–33 pmid:14561611
16.
2. Macleod MR,
3. Davis SM,
4. Mitchell PJ, et al
. Results of a multicentre, randomised controlled trial of intra-arterial urokinase in the treatment of acute posterior circulation ischaemic stroke. Cerebrovasc Dis 2005;20:12–17 doi:10.1159/000086121 pmid:15925877
17.
2. Bergui M,
3. Stura G,
4. Daniele D, et al
. Mechanical thrombolysis in ischemic stroke attributable to basilar artery occlusion as first-line treatment. Stroke 2006;37:145–50 doi:10.1161/01.STR.0000195178.20019.dc pmid:16339473
18.
2. Schulte-Altedorneburg G,
3. Hamann GF,
4. Mull M, et al
. Outcome of acute vertebrobasilar occlusions treated with intra-arterial fibrinolysis in 180 patients. AJNR Am J Neuroradiol 2006;27:2042–47 pmid:17110663
19.
2. Lutsep HL,
3. Rymer MM,
4. Nesbit GM
. Vertebrobasilar revascularization rates and outcomes in the MERCI and multi-MERCI trials. J Stroke Cerebrovasc Dis 2008;17:55–57 doi:10.1016/j.jstrokecerebrovasdis.2007.11.003 pmid:18346645
20.
2. Pfefferkorn T,
3. Mayer TE,
4. Opherk C, et al
. Staged escalation therapy in acute basilar artery occlusion: intravenous thrombolysis and on-demand consecutive endovascular mechanical thrombectomy: preliminary experience in 16 patients. Stroke 2008;39:1496–500 doi:10.1161/STROKEAHA.107.505123 pmid:18323481
21.
2. Nagel S,
3. Schellinger PD,
4. Hartmann M, et al
. Therapy of acute basilar artery occlusion: intraarterial thrombolysis alone vs bridging therapy. Stroke 2009;40:140–46 doi:10.1161/STROKEAHA.108.526566 pmid:18927446
22.
2. Schonewille WJ,
3. Wijman CA,
4. Michel P, et al
; BASICS study group. Treatment and outcomes of acute basilar artery occlusion in the Basilar Artery International Cooperation Study (BASICS): a prospective registry study. Lancet Neurol 2009;8:724–30 doi:10.1016/S1474-4422(09)70173-5 pmid:19577962
23.
2. Yu YY,
3. Niu L,
4. Gao L, et al
. Intraarterial thrombolysis and stent placement for acute basilar artery occlusion. J Vasc Interv Radiol 2010;21:1359–63 doi:10.1016/j.jvir.2010.05.017 pmid:20688533
24.
2. Roth C,
3. Mielke A,
4. Siekmann R, et al
. First experiences with a new device for mechanical thrombectomy in acute basilar artery occlusion. Cerebrovasc Dis 2011;32:28–34 doi:10.1159/000324948 pmid:21576940
25.
2. Andersson T,
3. Kuntze Söderqvist Å,
4. Söderman M, et al
. Mechanical thrombectomy as the primary treatment for acute basilar artery occlusion: experience from 5 years of practice. J Neurointerv Surg 2013;5:221–25 doi:10.1136/neurintsurg-2011-010096 pmid:22434906
26.
2. Espinosa de Rueda M,
3. Parrilla G,
4. Zamarro J, et al
. Treatment of acute vertebrobasilar occlusion using thrombectomy with stent retrievers: initial experience with 18 patients. AJNR Am J Neuroradiol 2013;34:1044–48 doi:10.3174/ajnr.A3329 pmid:23124642
27.
2. Mordasini P,
3. Brekenfeld C,
4. Byrne JV, et al
. Technical feasibility and application of mechanical thrombectomy with the Solitaire FR Revascularization device in acute basilar artery occlusion. AJNR Am J Neuroradiol 2013;34:159–63 doi:10.3174/ajnr.A3168 pmid:22723058
28.
2. Möhlenbruch M,
3. Stampfl S,
4. Behrens L, et al
. Mechanical thrombectomy with stent retrievers in acute basilar artery occlusion. AJNR Am J Neuroradiol 2014;35:959–64 doi:10.3174/ajnr.A3796 pmid:24287087
29.
2. Mourand I,
3. Machi P,
4. Milhaud D, et al
. Mechanical thrombectomy with the Solitaire device in acute basilar artery occlusion. J Neurointerv Surg 2014;6:200–4 doi:10.1136/neurintsurg-2012-010629 pmid:23645572
30.
2. Fugate JE,
3. Klunder AM,
4. Kallmes DF
. What is meant by “TICI”? AJNR Am J Neuroradiol 2013;34:1792–97 doi:10.3174/ajnr.A3496 pmid:23578670
31.
2. Kundel HL,
3. Polansky M
. Measurement of observer agreement. Radiology 2003;228:303–08 doi:10.1148/radiol.2282011860 pmid:12819342
32.
2. Broderick JP,
3. Palesch YY,
4. Demchuk AM, et al
; Interventional Management of Stroke (IMS) III Investigators. Endovascular therapy after intravenous t-PA versus t-PA alone for stroke. N Engl J Med 2013;368:893–903 doi:10.1056/NEJMoa1214300 pmid:23390923
33.
2. Tomsick T,
3. Broderick J,
4. Carrozella J, et al
; Interventional Management of Stroke II Investigators. Revascularization results in the Interventional Management of Stroke II trial. AJNR Am J Neuroradiol 2008;29:582–87 doi:10.3174/ajnr.A0843 pmid:18337393
34.
2. Tomsick TA,
3. Yeatts SD,
4. Liebeskind DS, et al
; IMS III Investigators. Endovascular revascularization results in IMS III: intracranial ICA and M1 occlusions. J Neurointerv Surg 2014 Oct 23. [Epub ahead of print] doi:10.1136/neurintsurg-2014-011318 pmid:25342652
35.
2. Cho TH,
3. Nighoghossian N,
4. Mikkelsen IK, et al
. Reperfusion within 6 hours outperforms recanalization in predicting penumbra salvage, lesion growth, final infarct, and clinical outcome. Stroke 2015;46:1582–89 doi:10.1161/STROKEAHA.114.007964 pmid:25908463
36.
2. Singer OC,
3. Berkefeld J,
4. Nolte CH, et al
; ENDOSTROKE Study Group. Mechanical recanalization in basilar artery occlusion: the ENDOSTROKE study. Ann Neurol 2015;77:415–24 doi:10.1002/ana.24336 pmid:25516154
37.
2. Mourand I,
3. Machi P,
4. Nogue E, et al
. Diffusion-weighted imaging score of the brain stem: a predictor of outcome in acute basilar artery occlusion treated with the Solitaire FR device. AJNR Am J Neuroradiol 2014;35:1117–23 doi:10.3174/ajnr.A3870 pmid:24524920

Received June 3, 2015.
Accepted after revision July 22, 2015.

Main menu

User menu

Search

American Journal of Neuroradiology

The Revascularization Scales Dilemma: Is It Right to Apply the Treatment in Cerebral Ischemia Scale in Posterior Circulation Stroke?

Abstract

ABBREVIATIONS:

Materials and Methods

Patients

Image Acquisition

Image Interpretation

Statistical Analysis

Results

Discussion

Conclusions

Acknowledgments

Footnotes

References

News and Updates

Resources

Opportunities

American Society of Neuroradiology