04-24, 14:45–15:15 (Europe/Berlin), A1
This study explores the efficacy of chatbots as dialogical argumentation systems for behaviour change, focusing on vaccine hesitancy during the COVID-19 pandemic. A Python-based chatbot, developed in 2021, engaged in argumentative dialogues with users reluctant to get vaccinated, resulting in a 20% positive change in participants' stances. As natural language processing technologies, like ChatGPT, advance, it is crucial to compare them to traditional expert systems. Prior studies have shown ChatGPT's reliability in addressing vaccine hesitancy. This research compares our chatbot with ChatGPT, evaluating persuasiveness through crowdsourced participants. The findings inform resource allocation decisions, guiding the choice between domain-specific expert systems and enhancing versatile models like ChatGPT. Understanding comparative strengths aids in preventing the dissemination of misinformation in behaviour change contexts.
Chatbots have the potential of being used as dialogical argumentation systems for behaviour change applications. They thereby offer a cost-effective and scalable alternative to in-person consultations with health professionals that users could engage in from the comfort of their own home. During events like the global COVID-19 pandemic, it is even more important than usual that people are well informed and make conscious decisions that benefit themselves. Getting a COVID-19 vaccine is a prime example of a behaviour that benefits the individual, as well as society as a whole. In 2021, prior to the release of ChatGPT, we presented a chatbot (developed in Python using scikit learn and flask) that engaged in dialogues with users who did not want to get vaccinated, with the goal to persuade them to change their stance and get a vaccine. The chatbot was equipped with a small repository of arguments that it used to counter user arguments which were presented in free-text by the user on why they were reluctant to get a vaccine. We evaluated our chatbot in a study with participants and found that 20% of the participants had a positive change in stance (e.g. changing their stance from "unlikely to get a vaccine" to "neutral" or "likely to get a vaccine" after chatting with the chatbot).
The rapid advancements in natural language processing and the release of technologies such as ChatGPT raises the need to compare them to traditional expert systems in order to (1) identify potential problems in the new technologies and (2) assess whether they can replace traditional expert systems. Several studies have already used ChatGPT to address vaccine hesitancy and to tackle vaccine myths and concluded that ChatGPT is indeed a reliable source of non-technical information to the public. We were, therefore, interested to compare our system to ChatGPT and simulate the conversations participants had with our chatbot using ChatGPT and evaluate which conversations were considered more convincing by crowdsourced participants who are not domain experts.
Research like this helps us understand whether we need to continue investing resources into domain specific expert systems or rather invest them into improving ChatGPT and make it more reliable and credible to avoid spreading misinformation.
None
Expected audience expertise: Python –None
Abstract as a tweet (X) or toot (Mastodon) –Comparison between ChatGPT and Domain-Specific Expert System - which is more convincing in getting people to vaccinate against COVID-19?
Public link to supporting material, e.g. videos, Github, etc. –https://lisanka93.github.io/img/paper8.pdf, https://www.youtube.com/watch?v=Hh944jtZoHE
Meet Lisa, a dynamic force at the intersection of computer science, education, and global exploration. Armed with a Ph.D. from University College London in computer science, Lisa's academic journey was a quest to imbue chatbots with the art of engaging in compelling, argumentative dialogues using authentic language.
During her time at UCL, Lisa discovered her passion for teaching, dedicating herself to shaping the minds of future technologists. Her expertise and enthusiasm led her to positions as an associate lecturer at UCL and a guest teacher at the prestigious Oxford University.
Eager for new horizons, Lisa decided to take her skills beyond the academic realm. Breaking away from London and the confines of traditional employment, she embraced the life of a digital nomad. Lisa now crisscrosses the globe, sharing her knowledge on Python and machine learning with corporate and private clients. Her expertise doesn't stop there; Lisa also delves into data science projects in the intriguing realm of legal tech.
In the digital realm, Lisa is a prolific creator of online teaching content, fluent in English, German, and Russian. When she's not immersed in the world of technology, Lisa finds balance through her love for yoga. As a certified Ashtanga and Rocket Vinyasa Yoga teacher, she brings mindfulness and physical well-being to her nomadic lifestyle. Additionally, Lisa is a skilled Thai Massage Therapist, adding a therapeutic touch to her repertoire.
In the quieter moments of her globetrotting life, Lisa unwinds with a good book, illustrating that even in the fast-paced world of technology and travel, there's always time for the simple pleasure of a captivating story. Lisa's journey exemplifies the harmonious blend of intellect, wanderlust, and a passion for sharing knowledge that defines her unique and inspiring presence in both the tech and wellness spheres.