Deep Learning-Based Amharic Chatbot for University FAQs

# Deep Learning-based Amharic Chatbot for FAQs in Universities
Source: https://arxiv.org/html/2402.01720
###### Abstract

University students often spend a considerable amount of time seeking answers to common questions from administrators or teachers. This can become tedious for both parties, leading to a need for a solution. In response, this paper proposes a chatbot model that utilizes natural language processing and deep learning techniques to answer frequently asked questions (FAQs) in the Amharic language. Chatbots are computer programs that simulate human conversation through the use of artificial intelligence (AI), acting as a virtual assistant to handle questions and other tasks. The proposed chatbot program employs tokenization, normalization, stop word removal, and stemming to analyze and categorize Amharic input sentences. Three machine learning model algorithms were used to classify tokens and retrieve appropriate responses: Support Vector Machine (SVM), Multinomial Naïve Bayes, and deep neural networks implemented through TensorFlow, Keras, and NLTK. The deep learning model achieved the best results with 91.55% accuracy and a validation loss of 0.3548 using an Adam optimizer and SoftMax activation function. The chatbot model was integrated with Facebook Messenger and deployed on a Heroku server for 24-hour accessibility. The experimental results demonstrate that the chatbot framework achieved its objectives and effectively addressed challenges such as Amharic Fidel variation, morphological variation, and lexical gaps. Future research could explore the integration of Amharic WordNet to narrow the lexical gap and support more complex questions.

## IIntroduction

Artificial Intelligence (AI) has been a hot topic since its inception in 1956. Its ultimate goal is to create intelligent machines that can think and act like humans. AI can be implemented in almost every sphere of work, and intelligent agents can do many tasks, from labor work to sophisticated operations[2 (https://arxiv.org/html/2402.01720#bib.bib1),5 (https://arxiv.org/html/2402.01720#bib.bib2)]. Natural Language Processing (NLP) is a field of study that focuses on the interaction between computers and humans in a natural way, which includes many natural language-related topics such as sentiment analysis, text similarity, question answering, and text summarization[6 (https://arxiv.org/html/2402.01720#bib.bib3)].

One of the applications of AI that has gained momentum in recent years is chatbots. Chatbots seek to mimic human conversation, and their architecture combines a language model and machine learning algorithms to provide an informal communication channel between a human user and a machine. Chatbots are being used in many fields, from customer service and knowledge collection to entertainment, and they have the potential to enhance user experience and communication[12 (https://arxiv.org/html/2402.01720#bib.bib4)].

However, despite the popularity of chatbots, there is no research on Amharic chatbots. The Amharic language poses challenges due to its morphological richness and the shortage of resources. Therefore, this article proposes a study to develop an Amharic chatbot using deep learning techniques to facilitate instant information retrieval for users. The study aims to promote better interactivity, sociability, and knowledge acquisition in higher education, and to provide a wide range of services for mobile users who spend most of their time on email and messaging platforms such as Telegram and Messenger.

### I-AProblem Background

Nowadays, universities handle a large number of regular requests from students and others seeking information and answers to commonly asked questions. This often requires a dedicated support staff and can be time-consuming for students who need to physically visit various offices to seek information. In addition, administrators and teachers are burdened with the tiresome obligation of answering the same questions repeatedly and conducting numerous meetings with students to address their concerns. While email is an effective medium for providing information to a large number of students, it can be slow and ineffective for handling single requests or specific problems. Students also waste time searching for frequently asked questions on different websites and web pages.

To address these challenges, artificial intelligence in the form of chatbots can provide a solution. Chatbots can simulate a conversation with users in natural language and provide immediate and up-to-date information. However, developing chatbots for languages such as Amharic poses a challenge as the language has a different form of grammatical structure, representation of characters, and formation of statements. While there are chatbots developed for other languages, to the best of our knowledge, there is no Amharic chatbot for FAQs developed so far. Therefore, building a chatbot that can translate and understand multiple languages, including Amharic, requires additional time and effort during the designing and development phase. In addition, it is sometimes challenging for translator services to differentiate between languages with the same script, leading to difficulties in detecting the right language when users use both languages in a phrase. Additionally, chatbots must be aware of the end-user’s culture, able to understand regional tones, and the ability to understand conversations and different regional accents or linguistic varieties (Dialects).

### I-BRelated Work

In this section, we review the existing research on chatbots and question answering systems, with a focus on the Amharic language.

One notable related work is the FAQchat system, which retrained the ALICE system using the frequently asked questions (FAQs) from the School of Computing at the University of Leeds. FAQchat employed keyword-based retrieval without linguistic tools or meaning analysis. Users found it preferable over Google due to its direct answers and fewer links[11 (https://arxiv.org/html/2402.01720#bib.bib5)].

Another study described the design and development of a chatbot for university FAQs[10 (https://arxiv.org/html/2402.01720#bib.bib6)]. It utilized Artificial Intelligence Markup Language (AIML) and Latent Semantic Analysis (LSA). AIML handled template-based and general questions, while LSA addressed service-based questions. The chatbot operated in three steps: user query processing, predefined format matching, and pattern-based answer presentation. However, this rule-based chatbot lacked the ability to learn new input data.

A conversational chatbot model was proposed as a substitution for industry FAQ pages[8 (https://arxiv.org/html/2402.01720#bib.bib7)]. It controlled the conversation flow based on user requests and provided natural language responses, including direct answers, requests for additional information, or recommended actions. The model employed deep learning techniques for intent and entity recognition.

In another study, a framework was developed for a chatbot that could be used by independent companies as a customer support replacement[14 (https://arxiv.org/html/2402.01720#bib.bib8)]. The framework incorporated AI at its core, utilizing TensorFlow to create a neural network trained with a design document for response generation. The system consisted of a user interface, neural network model, NLP unit, and feedback system.

Researchers have also explored Arabic chatbots that employ AIML pattern matching for FAQ answering[13 (https://arxiv.org/html/2402.01720#bib.bib9)]. While achieving a high correctness rate for Arabic questions, these systems faced challenges when dealing with different Arabic forms.

The first Arabic chatbot, BOTTA, was developed using AIML on the Pandorabots platform[3 (https://arxiv.org/html/2402.01720#bib.bib10)]. BOTTA aimed to simulate conversation and engage with Arabic language users.

Additionally, there have been efforts to develop Amharic factoid and non-factoid question answering systems[15 (https://arxiv.org/html/2402.01720#bib.bib11),16 (https://arxiv.org/html/2402.01720#bib.bib12),1 (https://arxiv.org/html/2402.01720#bib.bib13)]. These systems employed various techniques such as preprocessing, question analysis, document retrieval, and response extraction. However, no comprehensive chatbot capable of answering Amharic FAQs for students has been developed.

In summary, the related works have explored different aspects of chatbot and question answering systems. However, there is still a need for a robust chatbot capable of addressing FAQs in the Amharic language. In comparison to existing chatbot systems, the proposed Amharic chatbot offers several distinct advantages. While previous systems relied on keyword-based retrieval and lacked linguistic tools or meaning analysis, our chatbot leverages deep learning techniques to provide more accurate and contextually meaningful responses. This study aims to bridge this gap by utilizing deep learning techniques to develop a comprehensive chatbot solution that supports Amharic language. Moreover, unlike other chatbots that are primarily designed for widely spoken languages, our Amharic chatbot addresses the specific challenges of an under-resourced language, making it a valuable tool for the Amharic-speaking community. The user feedback and comparative evaluations have shown that our Amharic chatbot outperforms general-purpose chatbots in terms of providing direct answers, reducing reliance on external links, and delivering a more personalized user experience.

## IIMethodology

The research methodology used in this study was Design Science Research (DSR)[4 (https://arxiv.org/html/2402.01720#bib.bib14)], which focuses on creating artifacts that can solve practical business problems. DSR aims to generate scientific knowledge while developing technology-based solutions. It consists of three cycles: the Relevance Cycle, the Design Science Research Cycle, and the Rigor Cycle. The Relevance Cycle connects the research context with the design science activities, defining the problem space and acceptance criteria. The Design Science Research Cycle involves building and evaluating artifacts using computational and mathematical methods. The Rigor Cycle connects design science activities with existing knowledge and foundations. The research also follows a six-step design science research methodology process model for information systems formulated by Peffers et al.[9 (https://arxiv.org/html/2402.01720#bib.bib15)]. These six-step process model are: Problem Identification and Motivation, Objectives of the Study, Design and Development, Demonstration, Evaluation, and Communication. The problem is identified as the lack of an Amharic chatbot for FAQs in universities. The objective is to design and implement a deep learning-based Amharic chatbot. The design and development activity focus on creating the chatbot framework using different tools and techniques.

### II-AData Collection

To develop a domain-specific chatbot, a dataset is required to test and validate the chatbot’s performance. The researcher chose engineering students as the target audience because they have multiple questions related to their department, selecting engineering departments, and other related questions that ensure the breadth of the dataset.

A purposive sampling method was used to collect the dataset for the chatbot model. Purposive sampling is a sampling technique that relies on the researcher’s judgment when selecting the units to be studied. A questionnaire was distributed to engineering students in Mekelle University and Aksum University. The questionnaire was completed by 80 students, 38 males and 12 females from Mekelle University and 13 males and 17 females from Aksum University.

The questionnaire aimed to identify frequently asked questions by students in engineering departments during their studies. The collected data was used as a basis for determining the Amharic training data. The collected data was translated, preprocessed, and corrected to Amharic language using Google Translator and some Amharic language experts.

After collecting the questionnaires, the dataset information required for the chatbot was extracted, and the dataset was narrowed down to cover 60 topics. The collected datasets were structured into a JSON file. JavaScript Object Notation (JSON) uses human-readable text to store and transmit data objects consisting of attribute-value pairs and array data types. With the JSON package in Python, the JSON file was able to be read and be prepared to be processed and used for training. The series of intents consist of tags, patterns, responses, context set and context filter.

Each intent entry consists of atag(a unique name for each of the 60 topics),patterns(sample queries for each topic),responses(candidate answers from which one is randomly selected after identifying the topic),context set(which changes the conversation state if needed), andcontext filter(which filters results based on the current context).

This JSON structure model chat data has two main advantages. First, if someone has a chat dataset, every query can be marked for a tag. The training dataset can thus be attached to existing query data and answers. Secondly, if there is no previous dataset, anyone can be able to create and add data in that format. The conversation’s flow is no problem to determine. If anyone wants to mark a dialogue about other data, the ‘Context filter’ tag will be followed.

### II-BChatbot Framework Design

The conceptual chatbot framework consists of three main parts: the user, the user interface, and the chatbot model as shown in Fig.1 (https://arxiv.org/html/2402.01720#S2.F1).

Refer to caption

Figure 1:Conceptual framework of FAQ chatbot.The user part defines the end-users or students who need answers and information about their studying and other related questions in the university. The user interface (UI) part describes how a user communicates and interacts with the chatbot. It is a series of elements of natural languages that allow for interaction between user-chatbot models. This means users can communicate on their terms, not the computers. However, a chatbot’s communication skill can vary depending on the interface created. A chatbot interface that uses default answers, like button options, limits the question the user can ask and understands the chatbot. But the chatbot that is built in this paper was designed to understand and respond to a variety of Amharic text inputs through a Facebook Messenger which acts as an interface and interacts to the chatbot model using a Flask webhook; and deployed it in Heroku web server to have a real-time conversation.

The chatbot model part designed in this paper, as shown in Fig.2 (https://arxiv.org/html/2402.01720#S2.F2), consisted of three main parts: intent classification, training, and response generation.

Refer to caption

Figure 2:Amharic chatbot model.In the classification part, preprocessing techniques such as tokenization, normalization, stop word removal, and stemming were applied to the user questions. This preprocessing helped in identifying the intent of the questions and retrieving appropriate responses. The dataset underwent preprocessing and stemming, and a classification model was trained to assign class labels to new queries and provide suitable responses.

The training part involved preprocessing and feeding the collected dataset to the classification model for training and feature extraction.

The response generation part focused on providing appropriate responses based on the data provided in the classification part. The response generation part involves selecting an appropriate response b