BCS Member Groups

Code Smells Human-Written vs LLM Generated Programs | BCS Oxfordshire branch

Synopsis Code smells are indications of quality issues that are present in program code in the form of certain syntactic structures and patterns of code metrics. Although they are not bugs that cause software fail, they result in difficulty and cost in software maintenance, evolution, and reuse. Since Beck and Fowler introduced the term “code smells” in 2018, intensive research has been reported in the literature on understanding the impact of code smells on various software quality attributes in the revision and transition categories, developing techniques and tools for detecting code smells and refactoring program code to reduce code smells. It is now widely applied in software engineering practice. In recent years, with the rapid advance of machine learning (ML) in natural and programming language processing, large language models (LLMs) have become increasingly used in software development practice. LLM’s performance in code generation is a key element of the leader boards of LLM competition. However, existing works have mostly focused on the functional correctness of generated code, while research found that the major driving force for the adoption of generated code is not its correctness but whether it is useful as a starting point for further development. A question is how to evaluate the usefulness of generated code for further development. In this talk, we argue that code smell detection and assessment could offer a good answer to this question. The talk will consist of two parts. The first introduce the notion of code smells, and briefly reviews the techniques and automated tools for detecting code smells. The second reports our experiments with on detecting the smells of LLM generated code and comparing them with code written by professional programmers. About the speaker Dr Hong Zhu is a Professor of Computer Science at the Oxford Brookes University, Oxford, UK, where he was the Chair of Applied Formal Methods Research Group and now chairs the Cloud Computing And Cybersecurity Research Group. He obtained his BSc, MSc and PhD degrees in Computer Science from Nanjing University, China, in 1982, 1984 and 1987, respectively. He was a faculty member of Nanjing University from 1987 to 1998 as a Lecturer, Associate Professor and Full Professor and PhD Student Supervisor. He joined Oxford Brookes University in November 1998. His research interests are in software development methodologies, including software engineering of cloud-native applications, software engineering of AI and machine learning applications, multi-agent systems, formal methods, software design, software testing, programming languages, software modelling, and automated software engineering tools and environments, etc. He has published 2 books and more than 200 research papers in journals and international conferences. He is a senior member of IEEE, a member of BCS, The Chartered Institute for IT, and ACM. He is a co-founder of the IEEE/ACM International Workshop/ Conference series on Automation of Software Test (AST) and a co-founder of the IEEE International Conference series on Artificial Intelligence Test (AITest). He is a member of the Editorial Board of the Software Quality Journal and a member of the Steering Committee of the IEEE International Congress on Intelligent Service Oriented Systems Engineering (IEEE CISOSE). He has also served numerous international conferences as a General Chair, PC chair and PC members and a reviewer for numerous journals and research funding bodies.