Original title: Developing a Cybersecurity Domain Chatbot based on an Open Source Large Language Model
Authors: AHSAN, Shahrukh Azhar
Document type: Master’s theses
Year: 2024
Language: eng
Abstract: The objective of this research is to determine the effectiveness of fine-tuned open-source LLMs in the domain of cybersecurity. Specifically, the study evaluates how effective fine-tuning is for LLMs to learn and provide accurate information about recently reported software vulnerabilities. The LLMs used in this study were Falcon-7B and Llama-2-7b-chat-hf. A custom dataset of 19,135 question-answer pairs referencing publicly reported software vulnerabilities in 2023, was used. These vulnerabilities were sourced from the NVD (National Vulnerability Database) and OWASP (The Open Web Application Security Project). A total of six fine-tuned models were obtained. The similarity between their generated answers and true answers was determined with gpt-3.5 for evaluation. The best fine-tuned models showed quite significant improvements in validation accuracy and code generation capabilities w.r.t their base variants.
Keywords: cybersecurity; fine-tuning; large language models; LoRA; machine learning; natural language processing; open-source LLMs; QLoRA; transformers
Citation: AHSAN, Shahrukh Azhar. Developing a Cybersecurity Domain Chatbot based on an Open Source Large Language Model. České Budějovice, 2024. diplomová práce (Mgr.). JIHOČESKÁ UNIVERZITA V ČESKÝCH BUDĚJOVICÍCH. Přírodovědecká fakulta

Institution: University of South Bohemia in České Budějovice (web)
Document availability information: Fulltext is available in the Digital Repository of University of South Bohemia.
Original record: http://www.jcu.cz/vskp/76793

Permalink: http://www.nusl.cz/ntk/nusl-695376


The record appears in these collections:
Universities and colleges > Public universities > University of South Bohemia in České Budějovice
Academic theses (ETDs) > Master’s theses