Publication rate and PhD enrolment following a medical pre-graduate research programme

At Aarhus University (AU), a formalised and optional one-year pre-graduate research programme (PGRP) in medicine has been in place for more than two decades [1]. The aim and intention of the PGRP at AU is to facilitate development of research talent and recruitment into a research career [2]. Increasing popularity has led to the initiation of similar programmes at Danish medical schools at the University of Southern Denmark, the University of Copenhagen and Aalborg University [3]. Despite broad national and international [3, 4-6] recognition of the importance of research engagement at the undergraduate level in ensuring high-quality medical research [7-10], the Danish PGRP is now challenged by efforts made to shorten the average time spent in medical school [11]. An evaluation of the outcomes of the Danish PGRP and a debate regarding the future of medical pre-graduate research is therefore warranted.

The Danish PGRP was last evaluated in 1996 in its former version coined Diploma in Medical Research [1] in which an average of 35 students were enrolled per year, 54% published and 33% were recruited into PhD programmes. The present study aimed to describe the number of scientific publications and PhD enrolments associated with the current pre-graduate medical research programme at Health, AU, and to identify any effect modifiers.

We hypothesised that the rate of publications per student and PhD enrolment would decrease compared with the 1996 evaluation due to the increased recruitment of students and supervisors.

METHODS

Participants and characteristics

We conducted a cohort study based on databases and a literature search including all medical students enrolled in the PGRP at Health, AU, between 1 January 2003 and 1 July 2012. The period was chosen to ensure access to the electronic database while achieving a minimum 18-month follow-up period.

The number of new medical students enrolled annually at AU was collected from a public database made available by The Ministry of Science, Innovation and Higher Education [12].

The research student database included student names, enrolment and completion dates of the PGRP (i.e., the date signed on the diploma), student ID card numbers and civil registration numbers from which age, gender and advancement in their educational pro-
gramme could be obtained. Furthermore, the database included the position, title and affiliation of supervisors. The research students were affiliated with the same university section or clinical department as their main supervisor. The above characteristics were studied to identify any effect modifiers.

Outcomes

The primary endpoints were publication rate with a PGRP supervisor, and PhD recruitment. PhD recruitment was assessed for all students and was defined as enrolment in the PhD programme at Health, AU, before 2 August 2013. Linkage of the research student and the PhD student databases was accomplished using each student’s unique civil registration number. From the PhD student database, only names and enrolment date were used.

The publication rate was studied for research students enrolled between 1 July 2006 and 31 December 2009. These dates were chosen to ensure that we would be able to study the approximately 200 most recently enrolled PGRP students and to achieve a three and a half-year follow-up period. A sample size of 200 students was needed to detect a change in the publication rate of five percentage points (two-sided alpha = 0.05 and
power = 0.9) compared with the 1996 evaluation.

The publication number was defined as the number of scientific publications per research student. Publications were identified by a systematic literature search in PubMed, Embase, Scopus and Web of Science carried out by two librarians who were not involved in the interpretation of the results. The name of the research student was used as the search query for the publication search and included all possible combinations of first, middle, and last names – in full or as initials. Before a student was assigned “0 papers”, we cross-checked with the publication list of the main supervisor. For each paper, we noted the publication date and authors. The publications were categorised as either original articles or other. A sub-analysis compared publication numbers between research students who did and did not enrol for a PhD at Health, AU, within three and a half years.

Secondarily, we studied the effects of time and gender on recruitment into the PGRP.

Statistical methods

For the analysis of the effect of time, the study period was divided into three periods: January 2003-June 2006; July 2006-Deccember 2009; and January 2010-July 2012; the first period was used as reference. In the analysis of effect modification, “0 papers” and “PhD: No” were used as the reference groups. The characteristics studied were the same in all three analyses (see variables in Table 1), and their distributions were compared by calculating prevalence proportions, medians with interquartile ranges and prevalence proportion ratios with 95% confidence intervals. All calculations were tested using the chi-squared test. p < 0.05 was considered statistically significant.

The data were analysed using Stata software
(version 11.1; StataCorp, College Station, TX, USA).

Trial registration: not relevant.

RESULTS

Characteristics and pre-graduate research programme recruitment

From 2003 to 2012, 695 research students out of 3,523 eligible medical students were identified from the research student database at Health, AU. Eight students who applied for a PGRP but withdrew their application before the start of the PGRP were excluded, which resulted in 687 research students with 240, 227 and 220 students in the three time periods (i.e., January 2003-June 2006; July 2006-December 2009; and January 2010-July 2012, respectively). The characteristics of these students are listed in Table 1. From the first to the second and third time period, there was an increase in the number of supervisors per research student (≥ 3 supervisors: 36% versus 53% versus 55%; both p < 0.001 with the first period as reference group); and in more cases, supervisors had different affiliations (23% versus 41% versus 45%; both p < 0.001). In the middle period, more students were > 24 years old (81% versus 88% versus 85%; p = 0.03 and p = 0.20, respectively) and completed the PGRP in < 18 months (38% versus 48% versus 45%; p = 0.03 and p = 0.12, respectively). In the last period, fewer had at least one member of the supervisor team with an academic degree of Doctor of Medical Science (88% versus 93% versus 73%; p = 0.07 and p < 0.001, respectively). Other variables were similar between groups (see variables in Table 1).

The percentage of a medical class that attended a PGRP increased from 20% (68/348) to 33% (123/370) over the study period (p < 0.001). In these calculations, research students were compared with medical students enrolled five years previously because research students had completed a median of five years of medical school before enrolling for the PGRP (Table 1). Overall, 28% of male and 19% of female medical students enrolled into pre-graduate research (p < 0.001). There was a significant change in the fraction of main supervisors who were female from the first to the last period (15% versus 18% versus 24%; p = 0.37 and p = 0.02, respectively).

Publication rate

Between July 2006 and December 2009, 227 students were enrolled for the PGRP; however, eight did not complete it. Within three and a half years from enrolment into the PGRP, 52% (117/227) of research students (males, 54%; females, 47%) had published at least one paper together with one or more of their supervisors; in total 224 articles. As several students published > 1 paper (> 1, one, and no papers = 62, 61 and 96 students, respectively) and some students published without any of their supervisors, a total of 263 publications were published by 123 students during the period. The research student was the first author on 43% (97/224) of the papers published in relation to the PGRP. The vast majority of the publications were original papers (90% (236/263)).

Using “0 papers” as a reference, students who published more than one paper were more likely to embark on a PhD (11% versus 25%; p = 0.03) and to complete their PGRP faster than scheduled (36% versus 64%; p < 0.01). Their main supervisors were more likely to be male (76% versus 92%; p = 0.01). No other variables (Table 1) differed. A sub-analysis showed that the fraction of research students who published ≥ 1 paper with ≥ 1 supervisor was the same among those who enrolled in a PhD at Health, AU within three and a half years and those who did not (62% versus 50%; p = 0.22).

PhD recruitment

Overall, 179 of the 687 (26%) research students started a PhD at Health, AU within a follow-up period of 1-10.5 years. Within a fixed period of three and a half years after enrolment into the PGRP, 13% (63/467) of the research students who began their PGRP in 2003-2009 enrolled for a PhD. However, with a follow-up period of six years, 48% (144/303) of research students embarked on a PhD. Students who embarked on a PhD programme within three and a half years from enrolment into the PGRP were more likely to have started their PGRP before completing four years of medical school (5% versus 17%; p = 0.02). Other variables (Table 1) did not differ.

DISCUSSION

This cohort study reports outcomes regarding the PGRP at Health, AU, in the 2003-2012-period. Despite a significant increase in PGRP recruitment, i.e., from a fifth to a third of a medical student class, more than half of the research students had authored or co-authored a scientific publication, and 13% had enrolled in a PhD programme at Health, AU within three and a half years after enrolment in the PGRP. No decrease in publication rate was observed compared with the 1996 evaluation [1]. The only statistically significant factor for PhD recruitment was early research enrolment during medical school. Enrolment for a PhD within three and a half years and faster-than-scheduled completion of the PGRP, both of which indicate determination, were positively associated with more scientific publications. Student gender was not associated with more scientific publications. However, we found that a gender gap exists in research recruitment.

This is the first study to evaluate quantitative outcomes of the on-going Danish pre-graduate research programme. The strengths of our study include its
sample size, covering all medical students enrolled in pre-graduate research throughout a decade at Health, AU; full follow-up of all research students and lack of recall bias or subjective overestimation because a systematic literature search and register-based approaches were used.

Limitations of our publication include any variation in estimates caused by misspellings, typing errors, delayed registration of articles in the databases and multiple persons with the same name. Name changes related to marriage may have caused a minor underestimation, particularly for female authorship. Our estimate is conservative because in case of doubt, a publication was not counted. The three and a half-year cut-off after enrolment is the result of a trade-off between capturing as many publications resulting from the PGRP as possible and not counting those resulting from an ensuing PhD project or non-formalised research projects with the same supervisor.

The PhD recruitment estimate was influenced by both the number of research students recruited and the overall number of PhD students recruited, which again depends on financing opportunities, political initiatives to increase the number of PhDs, the students’ motivation to enrol in research and the portion of enrolled PhD students at Health, AU who do not have a medical background. Both the number of research students and the number of PhD students [13] have risen throughout our study period, increasing the competition for funding.
The PhD recruitment estimate is also limited by lack of access to PhD data from other universities in Denmark or abroad. The results would be stronger if a control group existed, and if we had included research students from 1996 and onwards rather than only end-period data.

Despite the limitations mentioned, our estimates for the publication rate and PhD recruitment demonstrate that the fraction of students who publish appears to be stable over time, although more research students enrolled in our evaluation than were included in the 1996 evaluation [1]. The publication rate in our study is also comparable to American results [5], while British students publish markedly less [4], and almost 100% publish in Norway [6] where publishing is mandatory. The percentage of a class conducting a PGRP ranges from 10% in Norway to over 50% in Great Britain and the US [4-6].

PhD recruitment is lower in our study than in the 1996 evaluation [1] and the Norwegian study [6]; in both, a third were recruited for PhDs within 1-6 years. Part of the explanation may be that our study only considers PhD enrolment at Health, AU. Another possible explanation may be the smaller scale of the other programmes [1, 6], which more likely attract a select group of especially dedicated students who are more likely to continue onto a research career.

It can be debated whether the PGRP should move towards either the American or the Norwegian approach; however, the stable results of the PGRP strategy chosen do not call for changes. The PGRP is not just a pre-PhD; it is justified in and of itself by its scientific contribution in terms of publications. This is further supported by the lack of a difference in the publication rate between research students who are and are not recruited to a PhD, respectively. Although only students enrolled at AU were studied, the results are likely to be generalisable to other Danish medical schools that have almost identical programmes.

CONCLUSIONS

Our study demonstrates that even though the number of research students has increased over the past decades, more than half of the students still publish scientific articles after their PGRP, and a significant portion continue their research career in the context of a PhD. Factors predicting publications and PhD recruitment were fast completion and early enrolment in the PGRP. Gender-specific issues already play a role in recruitment at the pre-graduate level.

Correspondence: Marlene Skovgaard, Society for Medical Student Research,Vennelyst Boulevard 9, 8000 Aarhus C, 8000 Aarhus C, Denmark.
E-mail: marlene.skovgaard@gmail.com

Accepted: 18 June 2015

Conflicts of interest:Disclosure forms provided by the authors are available with the full text of this article at www.danmedj.dk

Acknowledgements: The authors would like to extend their gratitude to Jens Peter SejerChristensen and Rasmus Søndergaard Nielsen, AU Library, for systematic literature search and to Hanne Munch Kristiansen and Iben Brøndum, AU Library, for suggestions in the development of the search strategy.
Julie Kolding Olsen, Graduate School of Health, is thanked for management of the research student and PhD databases and help in gathering the data.
Lise Wogensen Bach, Vice-Dean Graduate School of Health, is thanked for comments on the manuscript. The Graduate School of Health is thanked for financial support of the first author.