Theory-Based AI in Medical Education: Chain of Transformation

Artificial intelligence has immense potential to transform medical education. However, as we previously covered, we must lean on theory to guide these applications meaningfully. (AI in Medical Education: A Theory-Based Approach)

Today, we're diving into a case study that exemplifies this fusion of theory and technology. We'll start by examining how the Chain of Density ChatGPT prompt uses the theoretical lens of information density to improve AI text summarization.

However, we don't stop there. We experiment with this concept by merging it with Transformative Learning Theory, an educational theory aimed at facilitating deep, structural shifts in the basic premises of thought, feelings, and actions.

Our goal is to provide a proof-of-concept for how fusing theory and AI will allow educators to re-examine old problems with fresh perspectives, and also to explore completely new innovations.

We'll even give you an experimental prompt so you can try "transformative summarization" yourself!

Text summarization through artificial intelligence has been an area of ongoing research and development. Summarization in general works by taking an input source, and then producing an output that is shorter. Various methods have been proposed, the simplest of which is probably just asking ChatGPT to "summarize the below."

What sets the Chain of Density GPT-4 Prompt apart is its re-examination of the text summarization task through the lens of information density. We’ll see how the application of this theoretical lens led to easy identification of next steps, and how it led to a measurable improvement beyond simple prompts like "summarize the below."

This prompt is a result of collaborative research between Salesforce AI, MIT, and Columbia University (From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting). It offers a focused improvement in AI-driven text summarization by applying a specific theoretical lens—information density.

To illustrate the concept of information density, the two examples below are similar in length but differ in their information density:

1. "The surgery went well."
2. "The appendectomy was uncomplicated."

Both examples contain the same number of words (four), making them equal in length. However, they vary in the amount and specificity of the information they provide, thus making them differ in their information density.

Now back to Chain of Density.
The paper discusses the concept of summarization, and then applies the theoretical lens of information density. When thinking about summarization, the authors viewed summarization as changing information density (there are other ways we can think about summarization... foreshadowing). Since the summarization process takes a source material and makes it shorter while trying to maintain the same information, the summary has a higher information density.

So! Viewing summarization from this lens, the next steps probably now make a lot of sense. The researchers first measured the information density most preferred by humans, and then they measured the information density produced by standard summarization techniques.

They found that the default summaries had lower information density than human preference, and so they crafted a prompt to increase the information density. For more about how they did that and the prompt itself, visit the paper link above!

Importantly, they showed that based on their theoretical foundation, they could guide AI to produce better (i.e. more human-preferred) summaries.

The Chain of Density prompt demonstrates one of MedEdMENTOR's central claims.

The above is an example of how applying a specific theoretical lens—in this case, information density—can lead to incremental but significant improvements in the field of AI-based text summarization.

Now, let's get a bit more... experimental.

As we say here on MedEdMENTOR, theories are a succession of lenses to view the world.

So let's try using theories as a succession of lenses to view something like AI summarization. And then let's try fusing it with some AI to create a new hypothetical innovation.

Just as Chain of Density used information density as their lens, let's see what happens if we try out some other lenses.

Particularly in the domain of medical education, several theories can be applied to reframe the AI summarization problem and offer novel solutions.

Looking through the Theory Database, we come across a few.

1. Transformative Learning: This theory focuses on deep, structural shifts in the basic premises of thought, feelings, and actions. It is especially relevant in medical education, where ongoing learning and adaptability are crucial. Summaries generated through this lens could aim to not just inform but transform, providing insights that challenge existing assumptions or encourage new ways of thinking.
2. Experiential Learning: Rooted in the idea that learning is a process that takes place through experience, this theory could guide the development of summaries that are rich in case studies, real-world examples, or actionable insights, thereby enriching the learning experience for healthcare professionals.
3. Socio-Cultural Theory: This theory emphasizes the role of social interactions and cultural context in learning and behavior. In the context of summarization, a socio-cultural lens could inform the creation of summaries that are sensitive to the collaborative and interpersonal aspects of healthcare settings, perhaps even facilitating teamwork and shared understanding.

The chain of density prompt showcased how a focus on information density could improve text summarization. For the sake of exploration, let's try applying another theoretical framework—Transformative Learning Theory—in the specific context of health professions education.

You can read more in our Theory Database, but Transformative Learning Theory centers on facilitating deep-seated changes in how learners perceive the world and their place within it. One of the critical steps in this form of learning is "critical reflection," the process of examining and questioning one's beliefs, assumptions, and values to gain a deeper understanding of one's experiences.

Instead of focusing on 'information density,' could we look at summaries through the lens of 'transformation density'? What if instead of focusing on compacting down the information in the source material, we compacted down the transformative potential of the source material. That is, we took the source material and we enhanced its ability to change how we process our experiences and see the world.

(Please note that the term 'transformation density' is something we made up just now!)

In the realm of medical education, this could manifest as an AI-enhanced journaling session for medical students. (If someone would make this I think it'd be very interesting!)

Imagine developing an initiative where medical students journal their experiences as they go on clinical rotations (patient interactions, ethical dilemmas, and so on).

• Level 1 - The curriculum designer has medical students keep a journal, and they review it after each rotation.
• Level 2 - The curriculum designer goes on MedEdMENTOR, learns about the importance of theory, and incorporates themes of Transformative Learning, so that students are guided through critical reflection of their journals.

Now, we could stop here and be pretty happy with ourselves. But students are busy, and how likely are they going to put in full effort at the end of a rotation?

What if we could give them a nudge?

• Level 3 - The curriculum designer goes on MedEdMENTOR, and attempts to fuse AI and theory. Medical students keep a journal, but now at the end of the rotation, an AI kick-starts the process of their Transformative Learning.

An AI tool could take journal entries and produce summaries not aimed at compacting information, but instead at highlighting points for critical reflection. In other words, the summary would emphasize those instances that most strongly invite the student to question or re-evaluate their existing beliefs and practices.

But wait, there's more!

In addition to summarizing, the AI could pinpoint moments or decisions that warrant further reflection, and generate discussion points. Or maybe students could choose the 'density' of transformational content they wish to see in their summary, thus tailoring the output to their current educational or emotional state.

By using the lens of Transformative Learning, we're opening up new avenues for using AI to facilitate not just the absorption of information but the evolution of understanding and professional identity formation within the medical education sphere.

A simple Chain of Transformation prompt. (If anyone wants to send us a tweaked one, we'll put it here!)

It mostly works! Note that our summaries get progressively shorter, while technically in the original Chain of Density paper, it should start short and stay the same length. The original Chain of Density prompt is also much longer.

One of the fascinating aspects of fusing theory and artificial intelligence is that it can bridge the gap between abstract thought and real-world utility. This approach is particularly pertinent in medical education, where pedagogical theories can be directly implemented through interactive and automated platforms for immediate student benefit.

Another advantage of this theoretical approach is that it invites collaboration between AI developers, educational theorists, and medical educators. The result could be tools and resources that are not just technically sophisticated but pedagogically sound and contextually appropriate for the medical education field.

• Validity: How would we measure the effectiveness of summaries focused on 'transformation density'?
• Ethical Considerations: While this section is not focused on ethical aspects, they are implicitly ever-present, particularly when dealing with sensitive student reflections.