If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Results of randomized clinical trials are often first presented as conference abstracts but these abstracts may be difficult to find, and trial results included in the abstract may not be followed by subsequent journal publications. In a review of abstracts submitted to 8 major medical and surgical conferences in 2017, we identified 237 abstracts reporting primary results of randomized clinical trials accepted for presentation at 3 major gastroenterology and hepatology conferences. The aims of this new analysis were to determine the publication rate for these abstracts and the proportion of publications that included trial registration numbers in the publication abstract.
Methods
Clinical trial registries, PubMed, Europe PMC, and Google Scholar were searched through November 1, 2021 for publications reporting trial results for the selected abstracts. Publications were reviewed to determine if they included a trial registration number and if the registration number was in the abstract.
Results
Publications were found for 157 abstracts (66%) within 4 years of the conference. Publications were found more frequently for the 194 abstracts reporting results of registered trials (144, 74%) than for the 43 abstracts reporting unregistered trials (13, 30%), but only 67% of these 144 publications included the registration number in the publication abstract. Ten unpublished trials had summary results posted on ClinicalTrials.gov.
Conclusion
Clinical trial results could be more accessible if all trials were registered, authors included registration numbers in both conference and journal abstracts, and journal editors required the inclusion of registration numbers in publication abstracts for registered clinical trials.
Results of randomized clinical trials are often first presented as conference abstracts but these abstracts may be difficult to find, and trial results included in the abstract may not be followed by subsequent journal publications. A recent systematic review of 425 publications examining the proportions of conference abstracts that were subsequently published in full found that only 37% of 307,028 abstracts were published.
For 181 publications with data on time to publication, 58% of abstracts presenting results of controlled clinical trials were published within 48 months (18 publications) compared to 39% of other abstracts (163 publications). At 120 months, these percentages increased to 69% and 45%, respectively. Analyses of publication rates for abstracts presented at gastroenterology and hepatology conferences have found similar low publication rates.
Research outcomes in British gastroenterology: an audit of the subsequent full publication of abstracts presented at the British Society of Gastroenterology.
Is gastroenterology research in decline? A comparison of abstract publication rates from The British Society of Gastroenterology meetings between 1995 and 2005.
This analysis included abstracts reporting primary results of randomized clinical trials accepted for presentation at 3 major gastroenterology and hepatology conferences: The Liver Meeting of the American Association for the Study of Liver Diseases (AASLD), The American Gastroenterological Association Digestive Disease Week (DDW), and The Annual Meeting of the European Association for the Study of the Liver (EASL).
The primary aim of this new analysis is to determine the time from abstract presentation to full journal publication. A secondary aim was to determine the frequency with which trial registration numbers were included in abstracts of the identified publications as recommended by both the International Committee of Medical Journal Editors (ICMJE)
Submission of clinical trial data to a conference should be thought of as an intermediate part of a process that begins with trial registration at the design stage and culminates in the publication of the data in a peer-reviewed journal. Our results show that there is still a need for improvement in the percent of abstracts that are published and the percent of publications that include trial registration numbers in abstracts. If trial results cannot be published or publication is delayed, investigators should post summary trial results on ClinicalTrials.gov even if posting is not required by law.
Methods
Conferences
The 3 conferences were The Liver Meeting of the AASLD, October 20–24, 2017, in Washington, DC; The American Gastroenterological Association DDW, May 6–9, 2017, in Chicago, Illinois; and The Annual Meeting of the EASL, April 19–23, 2017, in Amsterdam, the Netherlands.
Abstract Selection
Abstracts from AASLD, DDW, and EASL were downloaded for review.
Abstracts submitted to these conferences were identified as potentially eligible for inclusion by searching for the word “randomized” (or “randomised”). These abstracts were then examined to determine whether they reported the results of one or more randomized clinical trials. Abstracts that used the word “randomized” (or “randomised”) only in background or conclusion sections or that reported nonhuman randomized studies or meta-analyses were excluded. Abstracts reporting results of more than 3 clinical trials were also excluded from further review. Selected abstracts were then classified as “primary,” “preprimary,” or “secondary.” A primary abstract was one that appeared to report the primary results of a trial. Abstracts that reported baseline data, mechanistic results, secondary outcomes, ancillary studies, subgroup analyses, or extended follow-up were classified as preprimary or secondary depending on their timing, before or after the report of primary outcomes. Only abstracts reporting primary results are included in this report.
Data Abstraction
An Excel spreadsheet of reviewed abstracts was created that included the abstract number, title, type of abstract (oral, poster, “late breaking”), reason for exclusion or inclusion, number of randomized participants, single site or multicenter, country of first author, disease category, type of intervention, and whether the results were described as interim or the trial as ongoing. Author affiliations were used to determine whether the trial had a site in the United States and whether the trial was a multicenter study. As reported previously,
: citation of a trial registration number, use of the word “randomized” (or “randomised”) in the title, a clear statement of the primary outcome, the number randomized in each group, the number analyzed in each group, and the dates of recruitment and follow-up. These 6 reporting items were then included in a summary score that ranged from 0 to 6. All abstracts were coded by one investigator and reviewed by a second.
Trial Registration
Trial registration numbers were searched for in the World Health Organization International Clinical Trials Registry Platform search portal
For trials registered in ClinicalTrials.gov, pipe-delimited flat files with data as of May 2, 2017; October 17, 2017; and November 1, 2021, were downloaded from the Clinical Trials Transformation Initiative Aggregate Analysis of ClinicalTrials.gov.
Downloaded information included trial status, date of registration, trial start and completion dates, date of last update, dates of results submission and posting, the number and location of trial sites, trial sponsor and collaborators, and patient enrollment. Following previous definitions, we classified the probable source of funding as “industry” if the lead sponsor or a collaborator was from industry and the National Institutes of Health (NIH) was not identified as a sponsor or collaborator. We classified the probable funding source as “NIH” if NIH was either the sponsor or a collaborator and the lead sponsor was not from industry. Otherwise, the funding source was classified as “other,” as done in prior analyses of data from ClinicalTrials.gov.
For trials registered in other registries, prospective registration and industry sponsorship were determined by reviewing the individual trial registrations on the registration websites.
Journal Publications
Publications of results for trials presented at these conferences were identified by searching in PubMed, PMC Europe, Google, Google Scholar, and trial registries. The last search was performed on November 1, 2021. The PubMed ID was used to download data from PubMed including the publication title, authors, journal, volume, pages, and the date that the publication became available on PubMed. If both a trial registration number and a journal publication were found, the presence or absence of the registration number in the publication and the abstract were coded.
Statistical Methods
The number of days from 7 days after the start of the conference (“time zero”) to the date that the publication was added to PubMed was used as a measure of time to publication. Trials that had not been published were censored at 4 years after “time zero” so that the follow-up time was the same for all 3 conferences. Given the uniform maximum time of 4 years, we report percent published, restricted mean survival times, and show plots of time to publication. The numbers of trial participants were grouped as 4–50, 51–100, 101–200, and ≥ 200 and registration data were used for abstracts with missing or unclear number of participants.
Survival curves were calculated using SAS proc Lifetest and figures were created using the R package ‘survival’. Restricted mean survival time (RMST) and 95% confidence intervals were calculated using the R package ‘survRM2’. Cox regression analysis was used to evaluate predictors of time to publication including oral or poster presentation, type of intervention evaluated in the trial, trial registration (no or yes), and number of trial participants (SAS Proc PHREG). Results are presented as hazard ratios and 95% Wald confidence intervals (CIs). Analyses were stratified by conference to adjust for differences among the conferences by allowing distinct time-dependent ‘baseline’ hazard functions for models not directly comparing the conferences. Assessments of proportional hazards assumptions were conducted via inspection of appropriate log-log plots. Abstract coding details are provided in Table A1. Data and programs have been posted on Open Science Framework (https://osf.io/u2b9k/).
Results
Abstract Selection
Our review of abstracts presented at these 3 conferences identified 437 abstracts that included the word “randomized” (or “randomised”), representing 4.6% of 9461 abstracts accepted for oral or poster presentations at these conferences (6.5% of 2268 for AASLD, 2.9% of 5455 for DDW, and 7.6% of 1738 for EASL). Only 237 of the 437 abstracts (54%) reported primary results of a randomized clinical trial, 8 (2%) reported preprimary results, and 90 (21%) reported secondary results of randomized clinical trials (Table 1). The remaining 102 (23%) abstracts were excluded from further review, most frequently because they reported a meta-analysis or systematic review, used the word “randomized” (or “randomised”) only in background or conclusion sections, or were nonhuman studies.
Table 1Selection of Abstracts From Those Including the Words ‘Randomized’ or ‘Randomised’
Two DDW abstracts originally classified as reporting primary results were subsequently determined to be reporting secondary results and have been excluded from these analyses.
62 (42)
105 (67)
70 (53)
237 (54)
Data are n (%).
AASLD, American Association for the Study of Liver Disease; DDW, Digestive Disease Week; EASL, European Association for the Study of the Liver; RCT, randomized clinical trial.
a Two DDW abstracts originally classified as reporting primary results were subsequently determined to be reporting secondary results and have been excluded from these analyses.
Overall, 43% of the 237 abstracts reporting primary results of randomized clinical trials were oral presentations, 12% were late breaking, and 12% stated that the results included in the abstract were interim (Table 2). Seventy 4 percent included a drug or biologic intervention, 30% had an author located in the United States, and 48% were multicenter trials (Table 2). As reported previously (18), 18% of abstracts included registration numbers in the abstract, 65% identified the trial as randomized in the title, and 52% clearly specified a primary outcome (Table 2).
The 6 CONSORT, criteria included citation of a trial registration number, use of the word “randomized” (or “randomised”) in the title, a clear statement of the primary outcome, the number randomized in each group, the number analyzed in each group, and the dates of recruitment and follow-up.
30 (48)
69 (66)
25 (36)
124 (52)
Data are n (%) or mean (IQR).
AASLD, American Association for the Study of Liver Disease; ALD, alcoholic liver disease; CONSORT, Consolidated Standards of Reporting Trials; DDW, Digestive Disease Week; EASL, European Association for the Study of the Liver; NAFLD, nonalcoholic fatty liver disease; NASH, nonalcoholic steatohepatitis; US, United States.
a Abstracts can be counted in more than one intervention category.
b The 6 CONSORT, criteria included citation of a trial registration number, use of the word “randomized” (or “randomised”) in the title, a clear statement of the primary outcome, the number randomized in each group, the number analyzed in each group, and the dates of recruitment and follow-up.
Registration, Journal Publication, and Results Posting
We were able to find trial registration numbers for 194 (82%) of the 237 abstracts, but 9 of these trials were registered after the conference (Table 3). Publications were found for 157 of the 237 abstracts (66%) (Table 3). The 4-year publication rate was higher for DDW (70%) and EASL (69%) than for AASLD (58%), but this difference was not statistically significant (P = .28 by the Pearson’s chi-squared test). Twenty seven percent of trials had results posted on ClinicalTrials.gov within 4 years and 70% were either published or had results posted on ClinicalTrials.gov within 4 years (68% AASLD, 72% DDW, and 70% EASL) (Table 3).
Table 3Trial Registration, Journal Publication, and Posting of Results
AASLD, American Association for the Study of Liver Disease; DDW, Digestive Disease Week; EASL, European Association for the Study of the Liver; ANZCTR, Australia New Zealand Clinical Trials Registry; ChiCTR, Chinese Clinical Trial Register; CTRI, Clinical Trials Registry–India; EU-CTR, European Union Clinical Trials Register; ISRCTN, International Standard Randomised Controlled Trial Number; JPRN, Japan Primary Registries Network; KCT, Korean Clinical Trial Registry; NTR, Netherlands Registry.
a Registered before conference abstract submission deadline.
b Registered after conference abstract submission deadline.
Publications were found for only 30% of the 43 abstracts reporting results of unregistered trials compared to 74% of 194 registered trials (Figure 1). The RMST was 1.91 years (95% CI 1.71–2.12) for registered trials and 3.22 (2.84–3.61) for unregistered trials, hazard ratio 3.87 (95% CI 2.19–6.84). This difference was consistent when the 3 conferences were analyzed separately (Figure 2). The RMSTs for registered trials compared to unregistered trials were 2.41 (2.01–2.80) and 3.33 (2.60–4.05) for AASLD, 1.68 (1.38–1.97) and 3.07 (2.47–3.67) for DDW, and 1.82 (1.46–2.19) and 3.41 (2.77–4.05) for EASL. The hazard ratios for registered versus not registered were 3.17 (0.97–10.37) for AASLD, 3.79 (1.73–8.30) for DDW, and 4.73 (1.46–15.31) for EASL.
Figure 1Time to publication by registration status. Publication rates for registered trials were 74% (144/194) compared to 30% for unregistered trials (13/43). The restricted mean time to publication (RMST) was 1.91 years (95% confidence interval 1.71–2.12) for registered trials and 3.22 (2.84–3.61) for unregistered trials, hazard ratio 3.87 (95% confidence interval 2.19–6.84).
Figure 2Time to publication by registration status and conference. Publication rates for registered trials were 65% (33/51) for AASLD, 79% for DDW (66/84), and 76% (45/59) for EASL. For unregistered trials the percentages were 27% (3/11), 33% (7/21), and 27% (3/11), respectively. The RMSTs for registered trials compared to unregistered trials were 2.41 (2.01–2.80) and 3.33 (2.60–4.05) for AASLD, 1.68 (1.38–1.97) and 3.07 (2.47–3.67) for DDW, and 1.82 (1.46–2.19) and 3.41 (2.77–4.05) for EASL. The hazard ratios for registered versus not registered were 3.17 (0.97–10.37) for AASLD, 3.79 (1.73–8.30) for DDW, and 4.73 (1.46–15.31) for EASL.
We looked for other predictors of publication within 4 years both for all abstracts (Table A2) and abstracts reporting the results of registered trials (Table A3). For all abstracts, results of trials were more likely to be published if the abstract was selected for oral presentation (hazard ratio 1.70, 95% CI 1.24–2.33), the trial had 50 or more participants (2.10, 1.38–3.19), the abstract had at least one author in the United States (1.41, 1.01–1.98), the trial was multicenter (1.76, 1.27–2.42), the word “randomized” or “randomised” was in the abstract title (1.53, 1.06–2.21), the primary outcome was specified (1.60, 1.16–2.21), and the number analyzed in each group was in the abstract (1.46, 1.06–2.03) (Table A2). In multivariable analyses, only trials with > 50 participants (1.87, 1.23–2.85) and trial registration (3.59, 2.03–6.37) were significant predictors of publication within 4 years.
For abstracts reporting registered trials, results of trials were more likely to be published if the abstract was selected for oral presentation (1.43, 1.03–2.00), the trial included a drug intervention (1.50, 1.01–2.25), the trial had 50 or more participants (2.01, 1.27–3.17), the number analyzed in each group was in the abstract (1.48, 1.06–2.08), and the trial was registered within 21 days of the start of recruitment (1.57, 1.07–2.29) (Table A3). In multivariable analyses, the only significant predictors were trials with > 50 participants (2.05, 1.29–3.24) and registration within 21 days (1.60, 1.09–2.35).
The 157 publications were published in 71 different journals (Table A4). Only 67% of 144 publications reporting results of registered clinical trials included the trial registration number in the publication abstract. For registered trials, registration numbers were included in the publication abstract for 64% of the 94 reports of registered clinical trials published in gastroenterology or hepatology journals but only 10 of the 39 journals always included the registration number in the publication abstract (Table 4).
Table 4Registration Numbers in Publication Abstracts Published in Gastroenterology or Hepatology Journals
We found that 66% of 237 abstracts reporting primary results of randomized clinical trials at AASLD, DDW, and EASL in 2017 were published within 4 years. Publications were found for only 30% of the 43 abstracts reporting results of unregistered trials compared to 74% of 194 abstracts reporting results of registered trials (hazard ratio 3.87, 95% CI 2.19–6.84). Some of this difference might have been due to the difficulty in finding publications for unregistered trials but it was consistent across the conferences. Abstracts that were selected for oral presentation and abstracts that reported results for larger trials were more likely to be published within 4 years than those that did not.
Our publication rate compares favorably to that of publications reporting time to publication for other conference abstracts. A Cochrane review of 425 reports of publication rates for more than 300,000 abstracts found that 37% (95% CI, 35%–39%) were published over varying lengths of follow-up.
Only 181 of the reports used survival analysis to estimate the cumulative publication rate, which was 46% after 10 years. From their figures, we estimate that only 40% of all abstracts and 58% of abstracts reporting results of clinical trials were published within 4 years.
Sixteen publications reporting time to publication for abstracts accepted at gastroenterology or hepatology conferences are summarized in Table 5. Abstracts reporting results for randomized clinical trials were only a small proportion of the number of abstracts for most of these reviews, making comparison with our study more difficult. In addition, the follow-up time is variable and only 3 reports used life table analysis methods to report the percent published at specific years after the conference.
Many reports included tables or graphs with time to publication but based these analyses only on published abstracts, making comparison between reports impossible. Only 2 reports had publication rates more than our rate of 66%.
Research outcomes in British gastroenterology: an audit of the subsequent full publication of abstracts presented at the British Society of Gastroenterology.
Research outcomes in British gastroenterology: an audit of the subsequent full publication of abstracts presented at the British Society of Gastroenterology.
Is gastroenterology research in decline? A comparison of abstract publication rates from The British Society of Gastroenterology meetings between 1995 and 2005.
ACBHT, Association of Hepato-biliary Surgery and Transplantation; ACG, American College of Gastroenterology; APA, American Pancreatic Association; ASGE, American Society for Gastrointestinal Endoscopy; SSG, British Society of Gastroenterology; DDW, Digestive Disease Week; HBP, Americas Hepato-Pancreato-Biliary Congress (HBP); ILTS, International Liver Transplantation Society; NA, Not available; PC, Pancreas Club; RCT, Randomized clinical trial; RSGH, Romanian Society of Gastroenterology and Hepatology; SFCE, French Society of Digestive Surgery; UEGW, United European Gastroenterology Week.
a Percent of abstracts reporting results of a randomized clinical trial.
Identifying publications that followed conference presentations was difficult. Although 194 trials were registered, only 42 (22%) of the conference abstracts for registered trials included the trial registration number in the conference abstract. Of 144 publications that reported results of a registered clinical trial, 5 (3%) did not include the registration number in the publication and 43 (30%) did not include this number in the abstract (Table A4). This finding is consistent with results of prior reviews of the inclusion of registration numbers in journal abstracts and a systematic review of the search methods used to link clinical trial registrations to publications.
Abstracts reporting of HIV/AIDS randomized controlled trials in general medicine and infectious diseases journals: completeness to date and improvement in the quality since CONSORT extension for abstracts.
Most of these methods depend heavily on publication titles and abstracts, and trial registration numbers are frequently not cited in either. Although several approaches have been proposed for automated searches of ClinicalTrials.gov and PubMed to address this issue,
Unreported links between trial registrations and published articles were identified using document similarity measures in a cross-sectional analysis of ClinicalTrials.gov.
Also, ClinicalTrials.gov adds publication links to the ClinicalTrials.gov record if the registration number is included in the publication abstract. None of the reports in Table 5 or in the Cochrane review reported on the use of registration numbers to identify publications but many of these reviews were conducted before trial registration became common and they were not limited to clinical trials.
In summary, clinical trial results would be more accessible if all trials were registered, authors included registration numbers in both conference and journal abstracts, and journal editors required the inclusion of registration numbers in publication abstracts for registered clinical trials and also verified the accuracy of the registration numbers. The increasing use of trial registration and results reporting should increase the contribution of these trials to science and ease practicing clinicians’ access to trial findings through multiple channels, all as originally intended by ICMJE.
We therefore recommend that trial investigators who submit abstracts to conferences and present their data at these conferences plan for the full publication of the results, include registration numbers so that conference abstracts and publications can be linked, and if data cannot be published, submit summary results to ClinicalTrials.gov.
Author's Contributions:
Elizabeth C Wright: Conception and design of the study, analysis and/or interpretation of data, collection of data, drafting or revision of the manuscript, and approval of the final version of the manuscript. Devika Kapuria: Conception and design of the study, collection of data, drafting or revision of the manuscript, and approval of the final version of the manuscript. Gil Ben-Yakov: Conception and design of the study, collection of data, drafting or revision of the manuscript, and approval of the final version of the manuscript. Disha Sharma: Conception and design of the study, collection of data, drafting or revision of the manuscript, and approval of the final version of the manuscript. Dev Basu: Conception and design of the study, collection of data, drafting or revision of the manuscript, and approval of the final version of the manuscript. Min Ho Cho: Conception and design of the study, collection of data, drafting or revision of the manuscript, and approval of the final version of the manuscript. Tomilowo Abijo: Conception and design of the study, analysis and/or interpretation of data, collection of data, drafting or revision of the manuscript, and approval of the final version of the manuscript. Kenneth J Wilkins: Conception and design of the study, analysis and/or interpretation of data, drafting or revision of the manuscript, and approval of the final version of the manuscript.
Research outcomes in British gastroenterology: an audit of the subsequent full publication of abstracts presented at the British Society of Gastroenterology.
Is gastroenterology research in decline? A comparison of abstract publication rates from The British Society of Gastroenterology meetings between 1995 and 2005.
Abstracts reporting of HIV/AIDS randomized controlled trials in general medicine and infectious diseases journals: completeness to date and improvement in the quality since CONSORT extension for abstracts.
Unreported links between trial registrations and published articles were identified using document similarity measures in a cross-sectional analysis of ClinicalTrials.gov.
Conflicts of Interest: The authors disclose no conflicts.
Funding: Support was provided by the National Institute of Diabetes and Digestive and Kidney Diseases, National Institutes of Health.
Ethical Statement: The corresponding author, on behalf of all authors, jointly and severally, certifies that their institution has approved the protocol for any investigation involving humans or animals and that all experimentation was conducted in conformity with ethical and humane principles of research.
Data Transparency Statement: Data and programs have been posted on Open Science Framework (https://osf.io/u2b9k/).
00205-9&id=fx1.jpg){kind=link}
00205-9&id=gr1.jpg){kind=link}
00205-9&id=gr2.jpg){kind=link}