Surgical research remains comic opera and maybe for good reasons

Twenty-five years ago, Lancet Editor Richard Horton, prepared an interesting commentary entitled “Surgical research or comic opera: questions, but few answers” [1]. Here, he rightfully criticised the field of surgical research for publishing very few randomised controlled trials (RCTs) and pointed out that half of the articles published in nine surgical journals in the January issue of 1996 were case series. Now, 25 years have passed, and it would seem appropriate to revise the current status.

METHODS

The January issues of the surgery journals (category “surgery” in Journal Citation Reports) and the ten general medicine journals (category “medicine, general & internal” in Journal Citation Reports) with the highest impact factors (IF) for 2020 were analysed. The surgical journals were: JAMA Surg (IF = 14.8), Ann Surg (IF = 13.0), J Heart Lung Transplant (IF = 10.2), J Neurol Neurosurg Psychiatry (IF = 10.2), Endoscopy (IF = 10.1), Am J Transplant (IF = 8.1), Dig Endosc (IF = 7.6), Hepatobiliary Surg Nutr (IF = 7.3), Eur J Vasc Endovasc Surg (IF = 7.1), J Hepatobiliary Pancreat Sci (IF = 7.0). The medical journals were: N Engl J Med (IF = 91.2), Lancet (IF = 79.3), JAMA (IF = 56.3), BMJ (IF = 39.9), Ann Intern Med (IF = 25.4), Lancet Digit Health (IF = 24.5), JAMA Intern Med (IF = 21.9), J Cachexia Sarcopenia Muscle (IF = 12.9), PLoS Med (IF = 11.1), J Intern Med (IF = 9.0).

The focus of the present paper was on various types of original research. Therefore, guideline articles, systematic reviews, narrative reviews and papers without an abstract (letters, editorials, etc.) were excluded. Funding was defined as any external funding, and the number of papers receiving specific funding from industry (pharmaceutical or device companies) was registered. Studies were categorised by study design. Thus, data were divided into RCTs, prospective studies (case series, case-control studies, observational studies, Delphi design studies, cohort studies and surveys) and retrospective studies (database studies and retrospective case series).

Impact factors for medical versus surgical journals were compared using the Mann Whitney U test. Data from medical versus surgical papers were compared using the χ²- or Fisher’s exact test, as appropriate. The study was not registered in a public database since it was not a trial, and under Danish law approval from an ethics committee was not required.

RESULTS

The ten most prestigious journals in general medicine had significantly higher impact factors than their surgical counterparts. Thus, the median (range) of the impact factors was 9.1 (7.0-14.8) for the surgical journals and 25.0 (9.0-91.2) for the medicine journals (p = 0.004, Mann Whitney U test). This difference was not due to fewer citable items in the medicine journals, as the total number was 123 in medicine versus 101 in surgery. Thus, the difference in impact factor must be the result of more citations per article.

More RCTs and fewer retrospective studies were registered in medicine than in surgery, and more prospective studies were registered in the surgical than in the medicine journals (Table 1). Furthermore, more “other” funding (e.g. from governmental funding agencies, etc.) and more funding from industry was recorded in medicine than in surgery (Table 1).

DISCUSSION

In the analysis conducted 25 years ago, 7% of the surgical papers were RCTs [1] compared with 10% in the present analysis. In contrast, among papers in medicine journals today, 40% were RCTs. Prospective studies including case series comprised 46% of the surgical papers 25 years ago [1] compared with 54% studies with a prospective study design in surgery and 41% in medicine today. Richard Horton criticised surgical research for relying excessively on non-RCTs such as case series but also noted that case series are inexpensive, quick and easy to perform [1]. Therefore, the present analysis also comprised study funding, and the findings were striking, showing significantly more funding in papers in medicine than in surgery journals. Thus, funding opportunities are much better in medicine research, and this may, at least partly, explain why more RCTs are published in medicine than in surgery.

Surgical research has traditionally used simple study designs such as the prospective cohort or occasionally a case-control design. For some research questions, this will be inadequate, and we do need to be able to conduct large clinical RCTs in surgery. Unfortunately, surgical RCTs are often discontinued, and the most common reasons seem to be poor recruitment and a lack of continued funding [2]. Thus, in surgery, some obstacles need to be addressed. Firstly, the traditional blinded study design may be very difficult or impossible for certain interventions, and the traditional placebo-controlled design carries various special problems in surgery. In some cases, this issue may be overcome by innovative research methods [3], but it cannot always be solved. However, increasing efforts seem to be devoted to conducting blinded placebo-controlled trials in surgery as 66 such trials were published during 2000-2015 period compared with only 19 in the 1950-2000 period [4]. Nevertheless, placebo surgery trials are still characterised by many difficulties as shown recently in a study demonstrating that only few of these trials are completed and published [5]. Secondly, funding is often difficult since traditional surgical research does not involve industry to the same degree as do drug trials. Thirdly, surgical trials often require hundreds or even thousands of patients because of variability in trauma response and ensuing effects on patient outcomes.

Blinding of patients and staff is difficult in surgery. However, years ago, in Denmark we succeeded in conducting the only effectively blinded study of laparoscopic versus open colon resection [6]. The study in question was conducted with a very large postoperative bandage covering the entire abdomen, and therefore neither the patient nor the nursing staff or the inpatient doctors could see what had occurred. The surgical description in the medical record was hidden, and the daily wound inspections were performed by specially dedicated project staff without any involvement with the regular clinical team and also blinded to the patient. The organisation and implementation of this study was complicated, but not undoable. A similar concept was implemented successfully with cholecystectomy [7], but such a design will be extremely difficult to implement in large international multicentre studies comprising thousands of patients. Therefore, the COLOR [8] and COLOR II [9] studies chose not to implement a blinded design. Funding, however, remains a considerable obstacle for many surgical interventional trials, and national or other types of public funding will most probably be necessary to increase overall research quality in surgery in the future.

Research traditions have varied between surgery and medicine, with more focus on (and knowledge about) epidemiology and study design issues in medicine than in surgery. There seems, however, to be a positive trend in this respect as trial registration, disclosure of funding and investigator conflicts have improved markedly over the past ten years in surgical RCTs [10]. Various initiatives are underway, e.g., numerous educational efforts are being made locally in many countries, the Cochrane Collaboration has established review groups focusing on surgical issues [11] and large research collaborations in surgery have been established [12]. The future for research quality in surgery may therefore be bright, provided non-industry funding opportunities may be secured for these studies.

CONCLUSION

In conclusion, surgical research is, indeed, still comic opera. RCTs remain rare, and we rely excessively on retrospective study designs compared with the medical specialties. Reasons may be found in the research traditions, but another significant factor may very well be different funding opportunities, where the medical studies are able to obtain better funding, especially from industry. Furthermore, in surgery, it may be difficult or impossible to blind the patient and investigator to the given treatment. To increase the quality of evidence in surgical research where industry sponsorship may not always be relevant, it will be important to focus on funding for surgery in future national/governmental funding programmes. There should be no doubt that both the desire and the energy needed to raise the quality of research are present within the surgical disciplines.

Correspondence Jacob Rosenberg. E-mail: jacob.rosenberg@regionh.dk
Accepted 12 November 2021
Conflicts of interest none. Disclosure form provided by the author is available with the article at ugeskriftet.dk/dmj
References can be found with the article at ugeskriftet.dk/dmj
Cite this as Dan Med J 2021;68(12):A10210792