|
|
Impact of AI Tools on Code Quality and Security
Vinarčík, Peter ; Holop, Patrik (oponent) ; Malinka, Kamil (vedoucí práce)
This work presents a newly developed application that is able to perform fully automated large-scale research for evaluating the safety and quality of AI-generated code. Also, a new way of evaluating the safety of AI-generated code has been presented, utilizing MITRE's methodology in combination with SAST tools performing static analysis on the code. In addition, the application is enhanced with an improved AI chatbot whose output is enhanced with the results of static analysis at generation time. The user inputs a dataset of prompts into the application, and if code with a vulnerability has been generated for a particular prompt, that vulnerability is scored by the established methodology, and the user is informed not only that the code is vulnerable, but how vulnerable it is. As part of the solution, large-scale, pilot testing of popular AIs, such as ChatGPT-4 or Gemini, is performed over a dataset of prompts using the new application, in contrast to existing studies. The results showed the dominance of ChatGPT-4 running on the GPT-4 model, over the other AIs tested.
|
host ::
RSS.