SC workshop Non-functional quality of LLM-generated code

Dear Software Center members,

We invite you to a follow-up workshop on quality assurance of LLM-generated code, with a particular focus on Non-Functional Quality Characteristics (NFQCs).

Agenda:

9:00 – 9:15       Welcome and Introduction

9:15 – 9:45       Xin Sun presents the results of the non-functional quality characteristics in generated code

9:45 – 9:55       Vasili Mankevich gives an overview of project 69: Work Activities leveraging AI in Software Engineering

9:55 – 10:00      Short break

10:00 – 10:30     Discussion, and feedback on practical experience, challenges, and opportunities

10:30 – 10:50     Future roadmap and Open discussion

10:50 – 11:00     Summary and closing

In last year’s workshop, we discussed concerns and experiences about NFQC issues in LLM-generated code, explored practical challenges observed in real development settings, and considered possible solutions. Questions such as which NFQC to prioritize, how to evaluate them in practice, and the possible trade-offs among these NFQCs were central to that discussion. Building on this, we conducted an empirical study of Security, Maintainability, and Performance (time and memory) using three common LLMs.

During the workshop, we will:

1. Present findings from our empirical study on NFQC trade-offs in LLM-generated code, highlight patterns across models and tasks, and introduce new ideas for agent-based automated refactoring.
2. Discuss how our observations relate to your systems and workflows, and how similar issues have been addressed.
3. Identify key opportunities, open challenges and practical insights that can guide future research and prototype tool development.

We would be delighted if you could join us and contribute to the discussion. Feel free to invite more colleagues; participating in the first workshop is not required.

This is a Software Center internal event and the common NDA applies. Project number: 61.

The workshop will be recorded for use by the undersigned researchers only.

Welcome!

Kristian Sandahl, Xin Sun, Daniel Ståhl, and Christoph Kessler

For a calendar invite with a MS Teams link, send a mail to kristian.sandahl@liu.se