AI in Medical Education: A Theory-Based Approach

Artificial intelligence is making waves in education, and medical education is along for the ride. From automating tedious tasks to creating adaptive learning environments, AI is becoming a game-changer.

But let's pause for a moment. While it's tempting to fully embrace the AI revolution, we need to do so thoughtfully. This is where a theory-based approach comes into play.

In this comprehensive discussion, we'll examine the essential role of theory in the deployment of artificial intelligence within medical education. We'll delve into potential issues, explore how theory can provide solutions, and share practical insights supported by academic research.

Artificial intelligence is fundamentally altering the landscape of education at large. Whether it's automating administrative tasks, personalizing learning experiences, or even assisting in research, AI is revolutionizing how we think about education (Mollick and Mollick, 2023a; 2023b).

And medical education is no exception. AI offers the potential for innovative teaching methods, enhanced student assessment, and streamlined administrative operations, marking a departure from traditional approaches (Wartman and Combs, 2019).

While the excitement around AI is understandable, we should tread carefully. The technology holds great promise but is not without its perils. As we'll explore in the sections to follow, blindly employing AI can lead to numerous issues, making the case for a more rigorous, theory-based approach.

Let's take a practical example to illustrate the point. Imagine you're a program director for a medical residency, and you want to use AI to sort through applicants. At face value, it might seem logical to use past data to train the AI. For instance, you could use a dataset of prior accepted applicants and another dataset of prior rejected applicants, and then teach the AI to differentiate between them.

Seems simple enough, right?

But there's a significant issue with this seemingly straightforward approach: bias. When you train an AI model based on historical data, you risk perpetuating pre-existing biases, whether they relate to race, gender, sex, location, or disability. This problem arises because the AI might not comprehend the biases implicit in the data (Masters, 2023).

Artificial intelligence, by its nature, is incredibly complex. This complexity, coupled with the fact that the underlying logic of AI can be opaque, leaves ample room for bias to sneak in. Because we don't have full visibility into how the AI is making its decisions, we run the risk of unknowingly allowing biases to emerge (Masters, 2023). Despite this being an area of active research, some wonder if we can ever achieve full transparency in AI.

In the realm of medical education, the appeal of a tool that "just works" can be compelling, especially when resources are limited. However, merely demonstrating that "AI works" in classifying applicants leaves much to be desired. It's crucial to also consider when, why, and for whom artificial intelligence works in this context (Tolsgaard et al., 2023).

The problem with an "it works" mindset is that it neglects the nuanced understanding necessary for truly effective educational interventions. The focus is often solely on the outcomes, like the rate of accurate applicant sorting, rather than the mechanism by which these outcomes are achieved.

As we touched upon previously (Beyond Basic Science: Why Medical Education Theory Matters) we should aspire for more than surface-level understanding. We ought to dig deeper into the factors that contribute to AI's success or failure in different educational contexts (Tolsgaard et al., 2020).

To improve the effectiveness of artificial intelligence in medical education, what we need is a theory-based approach (Tolsgaard et al., 2020). Theory acts as the bedrock upon which we can build more equitable and effective AI applications.

Before implementing AI, we should employ education theory to dissect the underlying issues. For example, if we're using AI to assist with the selection process for residency applicants, theories related to program applicant selection offer a structured way to investigate the task at hand (Caretta-Weyer et al., 2023).

Using the theoretical lens of 360-Degree Evaluation as an example, we might first choose a more holistic evaluation method. And then after we have our theory-based approach, we can train separate AIs to evaluate different aspects of an applicant—such as skills, knowledge, and abilities—and integrate these assessments into a unified evaluation.

When we explicitly identify our theoretical framework, we have a more transparent system that allows for rigorous examination of bias. For example, if we note that the specific AI trained to evaluate skills is showing biases, we can more easily trace it back and work on correcting it.

By aligning our AI approach with established theories, we tap into a rich tapestry of academic research. This allows us to stand on the shoulders of giants, leveraging existing literature to make our AI applications more robust and grounded.

Lastly, our efforts become a part of the broader knowledge base. When we construct our AI systems around shared frameworks like 360-Degree Evaluation, we add to our collective understanding of these theoretical constructs and their applications in medical education.

When we incorporate a theory-based approach into our AI systems for medical education, we're contributing to a shared understanding within our field. This isn't just about improving our own programs or projects; it's about advancing the field of medical education as a whole.

By structuring our AI initiatives around shared mental models like 360-Degree Evaluation, we allow for a more communal approach to tackling the complex issues in medical education. These frameworks become common ground where educators and AI specialists can collaborate more effectively.

Moreover, the lessons we learn and the improvements we make can be integrated into the broader body of knowledge. This collaborative benefit means we're not just enhancing our own understanding but also contributing to the community's collective wisdom.