Unpublished Papers Perish

At the 1999 meeting of the American Society of Neuroradiology (ASNR), 293 papers, excluding posters and case reports, were presented. This represented 55% of submitted abstracts that were evaluated by the Program Committee. One may assume that the majority of such papers, having undergone this type of review, would find their way into peer-reviewed, indexed journals. It is, however, likely that such will not be the case. The data presented by Marx et al (page 1173), regarding papers presented in 1993 at the annual meetings of the ASNR and RSNA, suggest that only approximately 100 will find their way into leading journals in medical imaging such as the American Journal of Neuroradiology, Radiology, or the American Journal of Roentgenology. This is a surprisingly low number, and deserves further comment and analysis.

With only one third of presented papers making their way into peer-reviewed journals, the question is, what was the fate of the other two thirds? Did the authors simply not take the steps necessary to put their papers into publishable form (ie, never submitted for publication), or was the work submitted for publication, but subsequently rejected? Marx et al were not able to obtain the data necessary to separate these two possibilities. Nonetheless, if the former were the situation, one could hypothesize that either the authors never got around to formalizing their work into a full-length paper, or, upon deeper inspection, they simply did not have the appropriate data to construct a valid scientific article. If the latter were the situation, this has implications for the quality of submissions and the data submitted to a program committee. When evaluating submitted abstracts for a meeting, members of a program committee often only can determine if the ideas presented are new, potentially significant, interesting, and plausible, and using these criteria, such submissions are often accepted for presentation. But frequently, because of insufficient data, incomplete description of methods, absent images, or other pertinent information, the true scientific validity of the abstract is difficult to judge. Later, after the manuscript is submitted for potential publication, a deeper analysis is possible; then the paper may falter when a detailed analysis by journal reviewers and editors is undertaken. The paper then may never be published and perishes, suffering an academic death. With this in mind, one quickly can come to accept the reason many journals, including the AJNR, discourage citation of abstracts from various meetings. These may never have met the rigors of standard peer review and referencing. Such presentations could propagate invalid and erroneous conclusions.

To address this problem, a program committee could require a greatly expanded ASNR abstract to enable better evaluation of the scientific value of a submitted investigation. A glance back through the years (1984 to the present) of the “Proceedings” of the ASNR shows a remarkable improvement in this regard. Nevertheless, additional information could be requested such as more complete data, critical images, and a deeper analysis of the results. Also, decreasing the lag between the required date for abstract submission and the annual meeting could result in the inclusion of more hard data in the abstract. When a long time elapses between abstract submission and presentation, authors may be inclined to write abstracts that read like promissory notes rather than fully developed scientific investigations. Were these the requirements, a paper accepted for the annual meeting of the ASNR might then have a greater probability of acceptance into the AJNR. This greater probability would be attributable to the rigors of two meaningful peer-review processes (ie, one by the ASNR Program Committee; the other by the AJNR when it is subsequently submitted for publication). By raising the “abstract bar,” the rate of publication of these submissions would inevitably increase. In addition, the attendees would have in their possession a more valuable document concerning the proceedings of the meeting, and the authors of such submissions would be a step closer to having an investigation that would stand up to a journal's critical peer review. We suspect that if such requirements were in place, a far higher percentage of presented papers would eventually be published and would not in fact “perish.”

Although the low rate of publication of papers presented at scientific meetings is the most significant point of Marx's paper, there are two other items that deserve brief comment; namely, clinical versus experimental and prospective versus retrospective studies. Because many factors important in a scientific investigation are uncontrolled in a retrospective study, it is no surprise that prospective studies are published at a higher rate than retrospective studies. Thus, recommending well-designed prospective studies that have statistical power should, of course, be encouraged. In addition, the value of well-designed experimental studies is crucial to the continued development of the specialty of neuroradiology, a fact that may not be fully appreciated by some AJNR and Radiology subscribers concerned mainly with clinical applications of imaging techniques. Nonetheless, we must encourage such experimental investigations if neuroradiologists are to remain leaders in the field of neuroimaging.

With Marx's paper as a starting point, the journal encourages others to become involved in similar types of publication analysis. With appropriate data saved and accumulated over the next few years, we can, by a prospective analysis, determine many of the issues left unanswered by Marx's paper. A few examples of potential avenues for investigation are the effects of expanded abstract requirements on eventual publication, determination of the impact factor (citation of a paper in subsequent peer-reviewed manuscripts) of published articles, and the categories of presented papers most likely to be accepted for eventual publication. The Society and the Journal will become stronger through such efforts.