Introduction: The CanMEDS framework describes seven roles in postgraduate training, but training and courses relevant to these roles can be limited. Massive open online courses (MOOCs) – free online courses in which anyone can participate, anywhere – may improve course participation. This study investigates the relevance of MOOCs for postgraduate medical training within the CanMEDS framework.
Material and methods: We extracted a list of all courses posted by the two largest MOOC providers, Coursera and EdX, and reviewed all course descriptions and categorised each course into one of three categories – “relevant,” “possibly relevant” or “not relevant” – reflecting the degree of relevance to each of the seven CanMEDS roles. We also noted course workload, duration and the name of the educational institution.
Results: We agreed the most on the role of health advocate (Cronbach’s α = 0.85) and the least on the role of collaborator (Cronbach’s α = 0.46). After a consensus-building process, 165 courses were found to be relevant or possibly relevant, mostly to the roles as scholar (n = 75) and medical expert (n = 57). The courses had a median duration of seven weeks and a median weekly workload of 4.5 hours, and were predominantly from North American universities.
Conclusion: A large number of MOOCs are relevant for postgraduate medical training. A weekly workload of 4.5 hours may enable course participation even for busy clinicians. Physicians should consider these free and universally available courses as relevant and potentially effective means of education.
Funding: not relevant.
Trial registration: not relevant.
The CanMEDS framework was implemented to describe the areas of competence that physicians need in order to produce better patient outcomes . These competences are described as seven roles (medical expert, communicator, collaborator, manager, health advocate, scholar, and professional) that constitute a diverse range of necessary abilities. The seven roles are used in education planning and play a key role in career planning, particularly because they are employed in the application process for specialist training positions .
Identifying key components of medical competence makes it possible to tailor medical education to instill these key components in students. A questionnaire study among Danish junior physicians found an improvement in the overall education after the implementation of CanMEDS . However, adoption of the CanMEDS framework alone is not sufficient ; it must be accompanied by relevant training and course initiatives . The supply and demand of courses may be challenged by geographic limitations , low economic prioritisation, lack of time [7-9], or lack of initiatives. As a result, training is insufficient ; and to overcome these challenges, courses should ideally be universally available, affordable, flexible in terms of scheduling, and independent of local preferences and initiatives.
Massive open online courses (MOOCs) are free online classes in which anyone, anywhere, can participate . MOOCs are typically based on weekly video-based lectures composed of several shorter video sequences, each 5-10 minutes in length . Video sequences are typically accompanied by questions; and completion of homework, assignments, or participation in case discussions may be required on a weekly basis to provide an interactive learning experience. One unique feature is the diversity of discussion that may arise when engaging thousands of students. Feedback on homework and assignments is provided by peers, aided by instructions on how to provide feedback and what kind of feedback to give. Currently, two major MOOC providers – Coursera and EdX (Table 1) – offer hundreds of freely available courses. This service is considered a potential game changer in continuous medical education and postgraduate training [6, 11]. However, there has been no systematic investigation into the relevance of the great pool of MOOCs for postgraduate medical training.
This study reviews and evaluates all available courses offered by the largest MOOC providers, and the relevance of those courses to the seven roles identified in the CanMEDS framework.
MATERIAL AND METHODS
One researcher extracted a list of links for all courses offered by Coursera and EdX as of 19 December 2013, excluding all non-English courses, and the research team held an introductory session for discussion of the review processes with co-analysis of 60 courses. Two researchers then independently reviewed course web pages within two months. Expired links were investigated using the Internet Archive (Internet Archive, San Francisco, CA) and excluded if unavailable.
For each course, two researchers worked separately using the course descriptions to categorise each course into one of three categories – “relevant,” “possibly relevant,” or “not relevant” – for each of the seven roles of the CanMEDS framework, using the explanatory description provided by the Danish Health and Medicines Authority, which is based on the original CanMEDS framework [1, 12]. The researchers noted weekly workload, course duration, and the name of the educational institution. After rating the courses independently, the two researchers met to perform a joint review and discussion until consensus was reached.
We used SPSS 20 for Mac (IBM, Chicago, IL, USA) for data analysis. The inter-rater agreement between the two researchers was calculated as Cronbach’s α for each of the seven roles using the intraclass correlation coefficient (two-way mixed average measures). The 60 co- analysed courses were removed from this analysis. Due to skewed distributions, data on weekly workload and course duration are reported in median and interquartile range (IQR).
Trial registration: not relevant.
We found 668 courses, 556 of which were from Coursera and 112 from EdX. We excluded 74, as 73 were non-English and one was an expired link. The remaining 594 courses were evaluated and rated. The inter-rater agreement between the two raters was α = 0.83 for medical expert, α =0.73 for communicator, α = 0.46 for collaborator, α = 0.75 for manager, α = 0.85 for health advocate, α = 0.81 for scholar, and α = 0.62 for professional.
In total, 165 courses were found relevant or possibly relevant to one or more of the seven roles. Of these, 100 (39%) were rated as being relevant or possibly relevant for more than one role: 82 (50%) covered two roles, 15 (9%) covered three roles, and three (2%) covered four roles. For each of the seven roles, at least one course was relevant or possibly relevant. Two roles had a markedly higher number of relevant or possibly relevant courses – medical expert (57 courses) and scholar (75 courses) – compared with the other roles (Figure 1). All relevant and possibly relevant courses are summarised in Additional file 1. Examples are listed in Table 2.
Relevant and possibly relevant courses had a median weekly workload of 4.5 hours (IQR: 3-6 hours), and their median duration was seven weeks (IQR: 6-9 weeks). These courses were predominantly provided from North American universities (n = 138), and a smaller number of courses were provided from European (n = 15) and Australian universities (n = 6). Eight universities each provided five or more courses, which collectively comprised almost half (n = 73) of all the courses collected: Johns Hopkins University (n=20), University of Pennsylvania (n = 12), Duke University (n = 9), University of California, San Francisco (n = 9), Harvard University (n = 6), Stanford University (n = 6), University of California, Berkeley (n = 6), and University of Toronto (n = 5).
This study shows that a large number of freely available MOOCs are relevant for physicians and that they have a reasonable workload. With MOOCs, course participation is no longer limited by time of day or geographic location. For example, a 20-minute window of time freed up by a cancelled appointment may be sufficient to watch a lecture or follow up on homework.
The two-step evaluation process in this study included a consensus-building process to account for individual perceptions of the seven roles. Our inter-rater analysis found that the relevance of courses to the roles as medical expert, health advocate and scholar was rated fairly similarly, whereas the relevance to the other roles was subject to more marked differences. This variation is not surprising; studies indicate that some of the seven roles may be difficult to interpret and that their definitions may be subject to interpersonal variance [2, 13].
MOOCs provide a large number of relevant courses on a diverse range of topics. Most were categorised as relevant to the roles as medical expert or scholar. This may be owed to the current strategy of MOOCs that may be to target a broad audience, i.e. disease pathways and basic scientific methods. Another explanation may be that training in some CanMEDS are more suitable in clinical environments. MOOCs have a specified start date; therefore, the numbers do not reflect the number of courses that are available at any given time, but rather the total number of courses available in general. Considering that postgraduate medical training may be limited by an insufficient number and variety of relevant courses, especially in rural areas, MOOCs may be a game changer in medical education, and attention should be given to these freely available opportunities [5, 6, 11].
Courses rated as “relevant” and “possibly relevant” had a median duration of seven weeks, and a median weekly workload of 4.5 hours that can be completed in smaller bits throughout the week . Considering the work schedule of the busy physician, these numbers suggest that MOOCs may enable a higher degree of course participation.
We found that North American universities developed the vast majority of the identified MOOCs, and eight North American universities collectively offered almost half of these courses. This concentration of courses in a limited number of universities representing a small geographic area can be particularly problematic for international course participants as the course contents may be focused on local circumstances, which is a problem in fields like health-care regulations, ethics law and practice. For example, a North American course on how to increase the clinical trial participation rate among Asian minorities may not be applicable to countries in which participation discrepancies may differ due to demographic and societal differences. This distinction is not a critique, as some courses only make sense in a local context, but it illustrates that given the likely increased future participation from universities outside of North America, it may be important to ensure emphasis on a diverse range of issues.
In a systematic review and meta-analysis by Cook and colleagues, internet-based learning (IBL) interventions were compared to two other scenarios: no intervention and non-internet interventions . Compared to a “no intervention” scenario, IBL was associated with a higher degree of clinical knowledge and skills, as well as with better patient outcomes . This is important when considering the role of MOOCs in postgraduate medical training as MOOCs may enable course activity that would otherwise not be possible at all due to a busy work schedule. When IBL is compared to other instructional methods, the effect seems dependent on intervention characteristics . Duration > 1 week and a high degree of discussion were associated with better knowledge outcomes in IBL interventions . Clinical outcomes were more favourable in IBL interventions with higher amounts of interactivity, practice exercises, discussions, repetitions and instruction through real-time online interaction . Discussion and repetition in IBL were also associated with favourable behaviours in practice and patient outcomes . We found MOOCs rated as “relevant” and “possibly relevant” to be of long duration. MOOCs generally include homework, exercises, repetitions and discussions with peers, all of which are associated with better outcomes. Students provide feedback to their peers – a powerful method, as teaching peers is an effective mode of learning, and learning from peers is facilitated by social and cognitive congruence [15, 16]. Additionally, asynchronous video-based teaching can be incorporated to increase intrinsic cognitive load and decrease extraneous cognitive load by video editing or re-shooting the video, if necessary [17, 18]. Therefore, we believe that MOOCs are appropriately designed for postgraduate medical training.
The availability of a course does not equal student participation, and it is arguable whether it is, indeed, possible to follow these courses. Student demographics, as reported earlier by Coursera, show that the median age of students is 35 years, 73.3% have a full-time job and 95% have a university degree (3% have a professional degree such as MD or JD) . However, only 7% of the thousands of students that participate in each course complete the entire course . One reason for this may be that virtually no barriers exist for course enrollment, and students can sign up without actually committing to the courses. As such, if MOOCs are to be used in postgraduate training, attention should be given to ensuring proper documentation of course participation. Another reason for course dropout may be that students are unable to keep up with the academic level, although both Coursera and EdX describe the academic requirements in their course descriptions.
Courses are generally of a high academic standard, and several have been approved for continuing medical education [6, 11]. A considerable limitation, however, is that the education is streamlined for all participants in a MOOC. In usual classroom teaching, the teacher can customise the education according to the knowledge and the interest of the students. When thousands of students take a streamlined MOOC, these finer aspects of teaching are practically impossible to maintain.
The results of this study should be interpreted acknowledging its strengths and limitations. We evaluated all courses from the largest and most relevant MOOC providers. The two-step evaluation process revealed a difference between the two researchers, and it is likely that whether a course is relevant for an individual may vary based on what the individual perceives as relevant in general. Additionally, each course was assessed and categorised based on the course description, which may not always accurately reflect the course contents compared to actual participation in the course, which we did not do in this study as we found it to be practically impossible to assess 600+ courses. In the United States, the Accreditation Council of Graduate Medical Education competency framework is used which includes six core competencies that overlap with the seven roles of the CanMEDS framework . Our results may therefore also indicate that the courses assessed in this study could be relevant for physicians planning a career in the United States, although it should be noted that this was not investigated specifically.
We identified a large number of MOOCs that are free, universally available and relevant for postgraduate medical training. A median weekly workload of 4.5 hours allows busy clinicians to fit the coursework into their schedules. Most courses originate from North America and target the CanMEDS roles as medical expert or scholar. A broader representation of suppliers and more courses targeting all seven roles would be preferable. MOOCs should be considered a viable method of postgraduate medical training as they seem feasible, and courses exist within all of the seven CanMEDS roles.
Correspondence: Yousif Subhi, Center for Klinisk Uddannelse, 5404, Rigshospitalet, Blegdamsvej 9, 2100 Copenhagen Ø, Denmark. E-mail: firstname.lastname@example.org
Accepted: 8 August 2014
Conflicts of interest:Disclosure forms provided by the authors are available with the full text of this article at www.danmedj.dk
Royal College of Physicians and Surgeons of Canada. The CanMEDS Framework. www.royalcollege.ca/canmeds/framework (5 May 2014).
Lundh A, Skjelsager K, Wildgaard K. Use of professional profiles in applications for specialist training positions. Dan Med J 2013;60(7):A4648.
Kodal T, Kjær NK, Qvesel D. Improved education after implementation of the Danish postgraduate medical training reform. Dan Med J 2012;59(3): A4392.
Berkenbosch L, Schoenmaker SG, Ahern S et al. Medical residents’ perceptions of their competencies and training needs in health care management: an international comparison. BMC Med Educ 2013;13:25.
Mortensen L, Malling B, Ringsted C et al. What is the impact of a national postgraduate medical specialist education reform on the daily clinical training 3.5 years after implementation? A questionnaire survey. BMC Med Educ 2010;10:46.
Murphy K, Munk PL. Continuing medical education: MOOCs (Massive Open Online Courses) and their implications for radiology learning. Can Assoc Radiol J 2013;64:165.
Rusius V. Using the European working time directive for surgical trainees: how to make it work. Med Educ 2010;44:632-3.
Andresen K, Achiam MP, Rosenberg J. Increase in working hours is an option for improving surgical education in Denmark. Ugeskr Læger 2013;175:1328-31.
Deshpande GA, Soejima K, Ishida Y et al. A global template for reforming residency without work-hours restrictions: decrease caseloads, increase education. Findings of the Japan Resident Workload Study Group. Med Teach 2012;34:232-9.
Hoy MB. MOOCs 101: An introduction to massive open online courses. Med Ref Serv Q 2014;33:85-91.
Harder B. Are MOOCs the future of medical education? BMJ 2013;346:f2666.
Danish Health and Medicines Authority. The seven roles. http://sundhedsstyrelsen.dk/publ/Publ2013/05maj/De7laegeroller2udg.pdf (5 May 2014).
Ringsted C, Hansen TL, Davis D et al. Are some of the challenging aspects of the CanMEDS roles valid outside Canada? Med Educ 2006;40:807-15.
Cook DA, Levinson AJ, Garside S et al. Internet-based learning in the health professions: a meta-analysis. JAMA 2008;300:1181-96.
Tolsgaard MG. Clinical skills training in undergraduate medical education using a student-centered approach. Dan Med J 2013;60(8):B4690.
Ten Cate O, Durning S. Dimensions and psychology of peer teaching in medical education. Med Teach 2007;29:546-52.
Mayer RE, Moreno R. Nine ways to reduce cognitive load in multimedia learning. Educ Psychol 2003;38:43-52.
Subhi Y, Bitsch M. The use of video as preparation for the objective structured clinical examination in medicine and surgery. Ugeskr Læger 2014;176:44-6.
TED Blog. MOOCs by the numbers: Where are we now? http://blog.ted.com/2014/01/29/moocs-by-the-numbers-where-are-we-now/ (5 May 2014).
Mickelson JJ, Macneily AE. Translational education: tools for implementing the CanMEDS competencies in Canadian urology residency training. Can Urol Assoc J 2008;2:395-404.