Publicly Available Metrics Underestimate AJNR Twitter Impact and Follower Engagement

C.M. Tomblinson; V. Wadhwa; E. Latimer; C.H. Gauss; J.L. McCarty

doi:10.3174/ajnr.A6299

Abstract

SUMMARY: Social media use by professional organizations has increased as a platform to disseminate information, affording an alternative avenue to engage membership and the public. The American Journal of Neuroradiology (AJNR) posts cases and articles, hosts Tweet chats, advertises podcasts, and more on its Twitter account (@TheAJNR). The objective of this study was to determine whether user engagement is underestimated on the basis of publicly available metrics and to assess the engagement rate. This study demonstrated that engagement extends beyond visible metrics, suggesting an AJNR “silent” following beyond what is readily apparent. Median engagement rates from the @TheAJNR account from 2017 to 2019 appear stable since last reported in 2016 and are comparable with those reported in other professional medical journals.

ABBREVIATION:

ER: engagement rate

Twitter continues to be an active social media platform for radiologists, trainees, and academic organizations. Although Twitter was originally a personal communication platform, its dynamic and rapidly changing information stream encourages propagation of topics of interest in the medical community. Currently, this platform is routinely used to disseminate information for large medical societies, including the American Society of Neuroradiology.

Twitter has become a catalyst for real-life meet-ups by conference attendees and allows virtual participation for those unable to attend. In the field of neuroradiology, Radmanesh et al¹ first described social media incorporation and use patterns by analyzing data from the 2014 Annual Meeting of the American Society of Neuroradiology (ASNR). When tweets from the Journal of the American College of Radiology targeted topics of member interest, Web site traffic increased 31%, and unique Web site visitors increased 20%.² Studies by Koontz et al³ and Patel et al⁴ found that social media use among radiologists and radiology trainees ranged from 59% to 91%, with usage rates varying by generation. Furthermore, Patel et al noted that 60% of radiologists in their study used Twitter for professional purposes. These high rates of use demonstrate a new digital era in which information is being consumed on-line.

While many of the interactions are visible to the public (retweets, likes, and replies), some of the more important metrics for content engagement are not visible (impressions, engagements, engagement rate, URL clicks) to the public but can be accessed by the account owner (Fig 1). The American Journal of Neuroradiology (AJNR) Twitter account (@TheAJNR) has been in existence since 2013, now garnering more than 6400 followers and 7300 tweets at the time of publication. Each @TheAJNR posted case and published article is paired with a unique URL link to route users to the corresponding AJNR Web page.

Fig 1.

Twitter Analytics. The free Twitter Analytics page provides data for the account and for each individual tweet from the account over designated time periods that can be viewed on-line or exported to Excel (Microsoft, Redmond, Washington). Only the account user has access to this page. The exported data include impressions, engagements, URL clicks, retweets, likes, and all other metrics collected for this project.

All social media activity is designed to increase engagement with users. However, the type of engagement important for an academic journal such as AJNR includes URL clicks, in addition to likes and retweets. One objective of this study was to assess whether case views and article readership resulting from Twitter posts (indicated by the number of URL clicks) exceed the number of visible engagements (particularly, retweets and likes), implying a “silent” Twitter following and educational outreach beyond what is readily apparent. Another objective of the study was to assess the engagement rate of the @TheAJNR account since the most recent reporting of AJNR engagement rate data from 2015 to 2016.⁵

MATERIALS AND METHODS

Data Source

Twitter Analytics data were obtained from the AJNR Twitter account for tweets published from January 2017 to June 2019. Available Twitter Analytics data include impressions, engagements, and engagement rate for each tweet.

“Impressions” refer to the number of times a tweet passes through user timelines or is included in a search.

“Engagements” are defined by the total number of times a user interacted with a tweet through a click for any purpose (retweet, reply, follow, like, URL link, hashtag, photo, or tweet expansion). The engagements of interest for our study were retweets, likes, and URL clicks.

“Engagement rate” (ER) is defined by total number of engagements divided by total number of impressions.

Likes and retweets can be seen by any Twitter user and were considered publicly visible engagements. The number of URL clicks data is only accessible by the owner of the Twitter account, thus considered a form of private engagement only visible to the account owner.

Tweet types were classified into 3 categories: cases, articles, and other. The “other” category included Tweet chats, blog posts, news announcements, advertisements for podcasts, upcoming meeting information, and deadline reminders.

Statistical Analysis

Summary statistics were obtained for URL clicks, retweets, likes, and replies for each of the 3 tweet types from January to December 2017. Both the difference between the number of URL clicks and the number of retweets and the difference between the number of URL clicks and the number of likes were computed for each tweet of each type (AJNR case, article, and other). Both difference response variables were analyzed separately using a 1-way analysis of variance with unequal variances to test for a difference among the 3 tweet-type population mean differences. Subsequently, for both the URL-click-retweet differences and the URL-click-like differences, case and article tweets were compared using a statistical contrast that used a t test. Also, subsequent to the 1-way analysis of variance, for each tweet type, a t test was performed to determine whether there was sufficient statistical evidence to suggest that the population mean number of URL clicks was greater than that of retweets, and a similar t test was performed for URL clicks and likes. To assess the level of engagement based on URL clicks, we obtained an estimate of the percentage of combined case and article tweets that had at least 25 URL clicks, and similar percentages were obtained for that same set of combined tweets that had at least 50, at least 75, and at least 100 URL clicks. These statistical analyses were conducted using SAS 9.4 (SAS Institute, Cary, North Carolina).

Using Twitter Analytics data available from January 2017 through June 2019, we calculated descriptive statistics for engagement rate, which included medians along with 25th and 75th percentiles by year and overall. These statistics were obtained using SPSS, Version 23 (IBM, Armonk, New York).

RESULTS

Nine hundred one tweets were analyzed, including 173 cases, 366 articles, and 362 others for the first objective of the study. There was evidence to suggest that at least 2 of the population mean differences among the 3 tweet types were significantly different for the URL click-retweet differences (P < .001) and the URL clicklike differences (P < .001).

Of particular interest was the comparison between tweets for AJNR articles and AJNR cases. For the difference between URL clicks and retweets, the mean difference was statistically significantly greater for cases than articles (50.1 versus 13.0, P < .001). The same was true for the difference between URL clicks and likes (49.8 versus 11.8, P < .001).

For all 3 tweet types, the mean number of URL clicks was significantly greater than the mean number of retweets (cases, 54.6 versus 4.6, P < .001; articles, 18.1 versus 5.0, P < .001; other, 7.3 versus 1.5, P < .001) and the mean number of likes (case, 54.6 versus 4.9, P < .001; articles, 18.1 versus 6.3, P < .001; other, 7.3 versus 3.0, P < .001) (Table 1).

View this table:

Table 1:

Tweet metrics by type^{^a}

The second objective of the study entailing an assessment of the engagement rate of the @TheAJNR account used 2187 tweets (893 from 2017, 911 from 2018, and 383 from January to June 2019). The median ER for 2017 was 4.1% (25th percentile = 2.3%, 75th percentile =7.3%); for 2018 was 3.0% (1.7%, 5.2%), and was 3.5% (2.0%, 6.0%) for the first 6 months of 2019. The overall median ER was 3.5% (1.9%, 6.2%).

DISCUSSION

Free to users and available only to the owner of an account, the Twitter Analytics page offers data to the user that extend far beyond publicly visible metrics. In addition to likes, retweets, and replies that are available to the public, the account owner can evaluate trends in the engagement of his or her tweets based on other private metrics such as impressions, detail expands, URL clicks, and engagement rate.

Our URL click-retweet and URL click-like differences highlight this engagement of URL clicks visible only to the account owner that far outnumber the popular publicly visible metrics of retweets and likes. This feature is of particular importance because URL clicks are the best indicator of AJNR readership. A user who clicks the URL link is interested in reading more than the small snippet of information previewed in the Twitter post (limited to 280 characters and 4 pictures). Such a user is actively engaged in the AJNR content, but that interaction will leave no publicly visible footprint unless that same user likes, retweets, or comments.

When evaluating the frequency of URL clicks, we found that almost 40% of case and article tweets combined from the AJNR account get at least 25 URL clicks. A little over 4% reach at least 100 URL clicks, with the most popular case during our study receiving 205 URL clicks (Tables 1 and 2).

View this table:

Table 2:

Frequency of engagement (URL clicks) with cases and articles

Each week, the AJNR posts a “Classic Case” submitted by readers (Fig 2). On May 7, 2018, a case of CLIPPERS received only 9 retweets and likes each but had 134 URL clicks, which channel the Twitter user directly to the AJNR site for full viewing.⁶ Thirty percent of the engagement for this particular tweet was through a link click. For each aforementioned case, AJNR posts a quiz where users can vote on a diagnosis. The corresponding case quiz for that week received 1033 votes, at least 42 of which came from Twitter but possibly more through direct funneling to the Web site, equaling about 4% of total quiz votes.

Fig 2.

Link clicks far exceed publicly visible metrics. For this AJNR Classic Case top tweet, there were only 9 retweets and 9 likes (A, black arrow and box), 2 publicly visible metrics. B, Twitter Analytics revealed that the link was clicked 134 times (black arrow and box), a metric only visible to the @TheAJNR Twitter account user and indicating a large silent-but-active following.

One social media primer for radiologists states that “engagements track the deepest level of interaction with social media” and are “the most powerful metric, one not available with print media.”⁷ When evaluating article engagement, we found that a recent review article posted on May 16, 2017, had 25 likes and 15 retweets; however, the article link was clicked 85 times, taking viewers directly to the article abstract.⁸ Web site analytics show that this article has been accessed 1433 times and downloaded 773 times. Here, a full 62% of the tweet engagement was in the form of a link click. Similar to a previous study by Hoang et al, ⁹ this study supports that posting AJNR content on social media may expose viewers to the society’s content and funnel traffic to the Web site, increasing member and nonmember engagement with existing materials diligently curated by AJNR. Certainly, most of these accessions and downloads are independent of Twitter, however the fact remains that accessions through URL clicks are higher than what meets the eye visually.

Our review of the literature revealed a paucity of studies that serve as benchmarks for end-user engagement on social media in the scientific community, but the most commonly used metric has been ER. Leary et al¹⁰ examined the use of a social media campaign at the Resuscitation Science Symposium at the American Heart Association’s Scientific Sessions using Twitter Analytics data of 8 professional bloggers who live-tweeted during the sessions and used dedicated hashtags. The median ER for these bloggers was 2.4%. In another study analyzing the engagement data for the Journal of Vascular and Interventional Radiology (JVIR), Wadhwa et al¹¹ showed that the median ER for @JVIRmedia steadily increased with time from 1.96% in 2014 to 4.47% in 2017. Another similar study by Wadhwa et al⁵ analyzing Twitter Analytics data for AJNR from August 2015 to July 2016 showed a median ER of 3.4%. When we expanded on this prior study on AJNR Twitter, further analysis showed that the median ER for @TheAJNR in the years 2017 and 2018 and the first 6 months of 2019 (January to June) was 4.1%, 3.0%, and 3.5%, respectively. The median ERs from our study are similar to the median ERs from other studies related to medical journals and use of social media described above. Further studies by other professional societies will be needed to more accurately define end-user engagement benchmarks for our industry, which include ER.

It is known from the prior work of Wadhwa et al⁵ that tweets containing an image or a hashtag or published in the morning hours have a higher ER. Their study also found that AJNR cases were the most popular style of tweet, concordant with the current study.⁵

There are several important limitations of this study. For instance, the presence of a URL click does not imply that a viewer is reading the article in full or looking through an entire case, and this has been reported previously by Hoang et al.⁹ Next, it cannot be ascertained what proportion of AJNR Web site activity is driven by Twitter at the current time, but this represents an area for future growth if the hope is to direct traffic to the AJNR Web site through social media. Last, the study was performed during a relatively short time frame since one of the authors (J.L.M.) assumed Twitter editorship in January 2017.

CONCLUSIONS

The AJNR Twitter account has an ever-present silent following with a far broader audience reach than previously recognized. This study suggests that the extent of viewership and on-line engagement should not be measured by likes and retweets of AJNR cases and articles alone. Rather, the number of URL clicks is more representative of the viewership of AJNR on-line content and outnumbers both retweets and likes. Also, median engagement rates from the @TheAJNR account from 2017 to 2019 appear stable since last reported in 2016 and are comparable with those reported in other professional medical journals, helping to inform the end-user engagement benchmarks for this industry.

Footnotes

Paper previously presented at: Annual Meeting of the American Society of Neuroradiology, June 2–7, 2018; Vancouver, British Columbia, Canada.

References

1.↵
1. Radmanesh A,
2. Kotsenas AL.
Social media and scientific meetings: an analysis of Twitter use at the annual meeting of the American Society of Neuroradiology. AJNR Am J Neuroradiol 2016;37:25–27 doi:10.3174/ajnr.A4168 pmid:25430857
FREE Full Text
2.↵
1. Hawkins CM,
2. Hillman BJ,
3. Carlos RC, et al
. The impact of social media on readership of a peer-reviewed medical journal. J Am Coll Radiol 2014;11:1038–43 doi:10.1016/j.jacr.2014.07.029 pmid:25439618
CrossRef PubMed
3.↵
1. Koontz NA,
2. Kamer AP,
3. Dodson SC, et al
. Social media utilization at an academic radiology practice. Acad Radiol 2018;25:111–17 doi:10.1016/j.acra.2017.08.012 pmid:29056400
CrossRef PubMed
4.↵
1. Patel SS,
2. Hawkins CM,
3. Rawson JV, et al
. Professional social networking in radiology: who is there and what are they doing? Acad Radiol 2017;24:574–79 doi:10.1016/j.acra.2016.09.026 pmid:28153576
CrossRef PubMed
5.↵
1. Wadhwa V,
2. Latimer E,
3. Chatterjee K, et al
. Maximizing the tweet engagement rate in academia: analysis of the AJNR Twitter feed. AJNR Am J Neuroradiol 2017;38:1866–68 doi:10.3174/ajnr.A5283 pmid:28663265
Abstract/FREE Full Text
6.↵
Classic Case. A 70-year-old man with gait ataxia, diplopia, and dysphonia. http://bit.ly/CC-2018May7. #NeuroRad. Twitter. Accessed May 31, 2018
7.↵
1. Kohli MD,
2. Daye D,
3. Towbin AJ, et al
. Social media tools for department and practice communication and branding in the digital age. Radiographics 2018;38:1773–85 doi:10.1148/rg.2018180090 pmid:30303796
CrossRef PubMed
8.↵
@TheAJNR. #ReviewArticle: Vertebroplasty and kyphoplasty for osteoporotic vertebral fractures: what are the latest data? http://bit.ly.2IIYSiN @JoshuaAHirsch #NeuroRad #Spine #NeuroIR. Twitter. Accessed May 31, 2017
9.↵
1. Hoang JK,
2. McCall J,
3. Dixon AF, et al
. Using social media to share your radiology research: how effective is a blog post? J Am Coll Radiol 2015;12:760–65 doi:10.1016/j.jacr.2015.03.048 pmid:25959491
CrossRef PubMed
10.↵
1. Leary M,
2. McGovern S,
3. Dainty KN, et al
. Examining the use of a social media campaign to increase engagement for the American Heart Association 2017 Resuscitation Science Symposium. J Am Heart Assoc 2018;7 doi:10.1161/JAHA.118.008710 pmid:29654198
Abstract/FREE Full Text
11.↵
1. Wadhwa V,
2. Devgan A,
3. Patel MV, et al
. Factors increasing tweet engagement rate for the Journal of Vascular and Interventional Radiology Twitter feed. J Vasc Interv Radiol 2018;29:1057–59 doi:10.1016/j.jvir.2018.03.004 pmid:29935786
CrossRef PubMed

Received July 31, 2019.
Accepted after revision September 16, 2019.

View Abstract

In this issue

Download PDF

Email Article

Citation Tools

Purchase

Cited By...

[1] 1.↵
Radmanesh A,
Kotsenas AL.
Social media and scientific meetings: an analysis of Twitter use at the annual meeting of the American Society of Neuroradiology. AJNR Am J Neuroradiol 2016;37:25–27 doi:10.3174/ajnr.A4168 pmid:25430857
FREE Full Text

[2] Radmanesh A,

[3] Kotsenas AL.

[4] 2.↵
Hawkins CM,
Hillman BJ,
Carlos RC, et al
. The impact of social media on readership of a peer-reviewed medical journal. J Am Coll Radiol 2014;11:1038–43 doi:10.1016/j.jacr.2014.07.029 pmid:25439618
CrossRef PubMed

[5] Hawkins CM,

[6] Hillman BJ,

[7] Carlos RC, et al

[8] 3.↵
Koontz NA,
Kamer AP,
Dodson SC, et al
. Social media utilization at an academic radiology practice. Acad Radiol 2018;25:111–17 doi:10.1016/j.acra.2017.08.012 pmid:29056400
CrossRef PubMed

[9] Koontz NA,

[10] Kamer AP,

[11] Dodson SC, et al

[12] 4.↵
Patel SS,
Hawkins CM,
Rawson JV, et al
. Professional social networking in radiology: who is there and what are they doing? Acad Radiol 2017;24:574–79 doi:10.1016/j.acra.2016.09.026 pmid:28153576
CrossRef PubMed

[13] Patel SS,

[14] Hawkins CM,

[15] Rawson JV, et al

[16] 5.↵
Wadhwa V,
Latimer E,
Chatterjee K, et al
. Maximizing the tweet engagement rate in academia: analysis of the AJNR Twitter feed. AJNR Am J Neuroradiol 2017;38:1866–68 doi:10.3174/ajnr.A5283 pmid:28663265
Abstract/FREE Full Text

[17] Wadhwa V,

[18] Latimer E,

[19] Chatterjee K, et al

[20] 6.↵
Classic Case. A 70-year-old man with gait ataxia, diplopia, and dysphonia. http://bit.ly/CC-2018May7. #NeuroRad. Twitter. Accessed May 31, 2018

[21] 7.↵
Kohli MD,
Daye D,
Towbin AJ, et al
. Social media tools for department and practice communication and branding in the digital age. Radiographics 2018;38:1773–85 doi:10.1148/rg.2018180090 pmid:30303796
CrossRef PubMed

[22] Kohli MD,

[23] Daye D,

[24] Towbin AJ, et al

[25] 8.↵
@TheAJNR. #ReviewArticle: Vertebroplasty and kyphoplasty for osteoporotic vertebral fractures: what are the latest data? http://bit.ly.2IIYSiN @JoshuaAHirsch #NeuroRad #Spine #NeuroIR. Twitter. Accessed May 31, 2017

[26] 9.↵
Hoang JK,
McCall J,
Dixon AF, et al
. Using social media to share your radiology research: how effective is a blog post? J Am Coll Radiol 2015;12:760–65 doi:10.1016/j.jacr.2015.03.048 pmid:25959491
CrossRef PubMed

[27] Hoang JK,

[28] McCall J,

[29] Dixon AF, et al

[30] 10.↵
Leary M,
McGovern S,
Dainty KN, et al
. Examining the use of a social media campaign to increase engagement for the American Heart Association 2017 Resuscitation Science Symposium. J Am Heart Assoc 2018;7 doi:10.1161/JAHA.118.008710 pmid:29654198
Abstract/FREE Full Text

[31] Leary M,

[32] McGovern S,

[33] Dainty KN, et al

[34] 11.↵
Wadhwa V,
Devgan A,
Patel MV, et al
. Factors increasing tweet engagement rate for the Journal of Vascular and Interventional Radiology Twitter feed. J Vasc Interv Radiol 2018;29:1057–59 doi:10.1016/j.jvir.2018.03.004 pmid:29935786
CrossRef PubMed

[35] Wadhwa V,

[36] Devgan A,

[37] Patel MV, et al

Main menu

User menu

Search

American Journal of Neuroradiology

Publicly Available Metrics Underestimate AJNR Twitter Impact and Follower Engagement

Abstract

ABBREVIATION: