Background
PAL3 is an embodied pedagogical AI framework for personalized adaptive learning on mobile devices, which provides on-the-job training, ongoing assessment and supports lifelong learning. PAL3 tracks where learners are (knowledge, past training and experience), where they want to go (career and learning goals), and uses that information to give personalized, adaptive coaching and resource recommendations to build a more resilient workforce. PAL3 runs on smartphones (iOS and Android) making it available to learners wherever they are, whenever they need it. The framework has been adapted to address domains such as electronics training (ONR), leadership and resilience strategies (MOM RP/N1/N17), suicide prevention training (N1/N17), and upskilling AI competencies (Army DEVCOM).
Objectives
The goal of the PAL3 project is to support learners throughout their careers and help them navigate career transitions successfully. Typically gaps in training lead to skill decay, due to lack of structure and/or motivation to continue studying. To address these issues, PAL3 uses a combination of learning science techniques for engagement (open learner models, self-regulated learning support), game-like mechanisms (team leaderboards, scenario-based training) and a personalized recommender system that draws from an extensive library of curated training resources that combine pre-existing resources (guides, tutorial videos) and customizable interactive content (dialog-based tutoring, quizzes, simulations). Using techniques borrowed from the entertainment industry, PAL3 has been designed to be sufficiently engaging that learners will use it voluntarily. PAL3 is also able to leverage its AI recommender and built-in learning technologies even when offline, enabling usage where internet connectivity is unreliable or unavailable.
Results
The PAL3 platform has produced strong learning and engagement across multiple domains. In a study at Naval Station Great Lakes, PAL3 prevented skill decay in Electronics for 70 sailors who used the system voluntarily, versus sailors who were not given the system. Moreover, sailors’ skills were aligned to their effort using the system, with more frequent users improving their skills rather than merely maintaining them. Follow-up research on leadership transitions and strategies, demonstrated a 16% increase in learning gains for leadership and resilience. Ongoing research has leveraged the PAL3 framework to AI Upskilling (AI-UP) and toward suicide prevention general military training (SAFER).
Next Steps
PAL3 is currently delivered as a smartphone or tablet app (iOS or Android). In future work, PAL3 will be extended to include a web-based delivery, and a prototype version which integrates immersive technologies, such as Augmented Reality training scenarios and activities. The primary goal for PAL3 is to make apps based on the PAL3 framework available more widely. Versions of PAL3 for resilience and suicide prevention are being evaluated for transition to DoD service members. Upcoming versions of PAL3 for AI Upskilling are being developed with the goal to release them more broadly to the public app store. As a long term goal, PAL3 also seeks to encourage open ecosystems of intelligent content and tools, with its capabilities to deliver a wide range of different content types.
Published academic research papers are available here. For more information Contact Us
Publications
Personal Assistant for Lifelong Learning (PAL3)
William Swartout, Benjamin Nye, Albert (Skip) Rizzo. (2023). Intelligent Tutoring System for Sensitive Topics: Adapting the PAL3 Framework for Suicide Prevention Training. In Sinatra, A.M., Graesser, A.C., Hu, X., Townsend, L.N., Rus, V. (Ed.), Design Recommendations for Intelligent Tutoring Systems: Volume 11 – Professional Career Education (Vol. 11, pp. 75–89).
View PDF
This paper describes how PAL3 was modified to support suicide prevention training. Key additions included 1) an adaptive survey at the beginning which was used to prioritize training and screen for at-risk users who were directed to seek immediate help, 2) a subsystem to support Safety Plans, and 3) provisions for the protection of sensitive user data.
Hampton, A. J., Nye, B. D., Pavlik, P. I., Swartout, W. R., Graesser, A. C., & Gunderson, J. Mitigating Knowledge Decay from Instruction with Voluntary Use of an Adaptive Learning System. Paper presented at the Artificial Intelligence in Education (AIED-18), London, 2018
View PDF
Describes a controlled study done with over 107 sailors (70 in treatment group, 37 control) that showed that PAL3, when used on a voluntary basis, could eliminate in aggregate the knowledge decay that occurs during the gap that occurs between training assignments.
Swartout, W., Nye, B.D., Hartholt, A., Reilly, A., Graesser, A.C., VanLehn, K., Wetzel, J., Liewer, M., Morbini, F., Morgan, B., Wang, L., Benn, G., Rosenberg, M. Designing a Personal Assistant for Life-Long Learning (PAL3) in Proceedings of the 29th International FLAIRS Conference, May 16-18, 2016.
View PDF
An overview of the core design of the PAL3 system.
MentorPal/MentorPanel
Okado, Y., Nye, B. D., Aguirre, A., & Swartout, W. (2023). Can virtual agents scale up mentoring?: Insights from college students’ experiences using the CareerFair.AI platform at an American Hispanic-serving institution. International Conference on Artificial Intelligence in Education (AIED-23), 189–201. Nominated for Best Paper.
View PDF
Mentoring promotes underserved students’ persistence in STEM but is difficult to scale up. Conversational virtual agents can help address this problem by conveying a mentor’s experiences to larger audiences. The present study examined college students’ (N = 138) utilization of CareerFair.ai, an online platform featuring virtual agent-mentors that were self-recorded by sixteen real-life mentors and builtusing principles from the earlier MentorPal framework. Findings included positive pre/post changes in intent to pursue STEM coursework and high user acceptance ratings (e.g., expected benefit, ease of use), with under-represented minority (URM) students giving significantly higher ratings on average than non-URM students. Self-reported learning gains of interest, actual content viewed on the CareerFair.ai platform, and actual learning gains were associated with one another, suggesting that the platform may be a useful resource in meeting a wide range of career exploration needs.
Nye, B. D., Davis, D. M., Rizvi, S. Z., Carr, K., Swartout, W., Thacker, R., & Shaw, K. (2020). Feasibility and usability of MentorPal, a framework for rapid development of virtual mentors. Journal of Research on Technology in Education, 1-23
View PDF
One-on-one mentoring is effective for helping novices with career development. However, traditional mentoring scales poorly. To address this problem, MentorPal emulates conversations with a panel of virtual mentors based on recordings of real STEM professionals. Students freely ask questions as they might in a career fair, while machine learning algorithmsrespond with best-match answers.
Nye, B., Anderson, C., Campbell, J., Krishnamachari, M., Kaimakis, N., and Swartout, W. MentorPal: Interactive Virtual Mentors Based on Real-life STEM Professionals, I/ITSEC 2017, November 27th – December 1st, 2017.
View PDF
In an ideal world, all students could meet STEM role models as they explore different careers. However, events such as career fairs do not scale well: professionals have limited time and effective mentors are not readily available in all fields. This paper describes the initial design of MentorPal, a tablet-based app designed to address this problem, that gives students the opportunity to converse with interactive recordings of real-life STEM professionals.
OpenTutor
Nye, B. D., Sanghrajka, R., Bodhwani, V., Acob, M., Budziwojski, D., Carr, K., Kirshner, L., & Swartout, W. R. (2021, 2021-04-18). OpenTutor: Designing a Rapid-Authored Tutor that Learns as you Grade. The International FLAIRS Conference Proceedings, 34(1). https://doi.org/10.32473/flairs.v34i1.128576
View PDF
Despite strong evidence that dialog-based intelligent tutoring systems (ITS) can increase learning gains, few courses include these tutors. In this research, we posit that existing dialog-based tutoring systems are not widely used because they are too complex and unfamiliar for typical teacher to adapt or augment. OpenTutor is an open-source research project intended to scale up dialog-based tutoring by enabling ordinary teachers to rapidly author and improve dialog-based ITS, where authoring is presented through familiar tasks such as assessment item creation and grading.