Evaluation of a Chatbot Powered by ChatGPT for the Preliminary Diagnosis of Dengue

Abstract

Dengue poses a public-health challenge in several regions. Early detection is essential to reduce the impact of a condition, and artificial intelligence (AI) and chatbots provide new opportunities to enhance diagnosis. This study evaluates the effectiveness of integrating a chatbot with ChatGPT version GPT-3.5 for the preliminary diagnosis of dengue and its contribution to timely detection. To evaluate this, two types of tests were conducted using a dataset of 30 dengue cases. In the first test, the chatbot was evaluated without being trained with dengue symptoms. In the second test, however, the model was trained using dengue symptoms obtained from official websites such as the World Health Organization (WHO) and the Pan American Health Organization (PAHO). The performance of the chatbot was evaluated using the confusion matrix, performance metrics, and user satisfaction. The results of the second test showed impressive performance, with accuracy, sensitivity, and specificity of 100%. This surpassed the first test, which achieved accuracy, sensitivity, and specificity of 83%, 80%, and 90%, respectively. In addition, 15 users reported positive satisfaction, with an overall average rating of 4.25 out of 5. In conclusion, these results highlight the effectiveness of the chatbot as a valuable public health tool for the early detection and management of dengue. It is important to note that, despite the remarkable diagnostic results of the chatbot integrated with ChatGPT, it does not replace medical judgment. © 2024 Elsevier B.V., All rights reserved.