Prof. Dr. Alexander Pretschner (TUM): Why LLMs Won't Replace Software Engineers
When: Monday, 7 October 2024, 10 - 11:30 am
Access: Via Teams
Abstract: Will LLMs replace software engineers? Of course not! Yet, some seem to think so; others blog about where LLMs helpfully (will) act as assistants; researchers do empirical studies in domains with (too) many parameters to control; and esteemed consultancies seriously tell us exactly which range of productivity gains to expect.
What we rarely read about is *why* we think or do not think LLMs will be useful in or across different software engineering activities. In this talk, I will revisit more or less successful software engineering trends from the Seventies (stepwise refinement: less successful), the Eighties (Computer-Aided Software Engineering: less successful), and the Nineties (agility: more successful). I'll relate them by showing how they methodologically make use of different flavors of abstractions. One flavor is abstraction in the sense of an actual loss of information--that we use to capture underspecifiction when we simply don't know enough. The other flavor is abstraction in the sense of lambda abstraction--that we use to denote complex phenomena with simple labels, but where the information that was abstracted away was never lost and can always be filled in. Our first central argument is that LLMs likely will be helpful with the second flavor, but less helpful with the first flavor.
Our second central argument is that experience shows how bad we are at getting specifications right. We seem to need an iterative process that interleaves gathering external information, thinking, writing down, verifying, and validating. While LLMs likely can be helpful in this process, we still need a human to guide the process - and this human needs all the typical qualifications that software engineers possess!
These perspectives will be key to understanding why and when LLMs can be more or less cost-effective in software engineering. We argue that the usefulness and cost effectiveness of LLMs in Software Engineering depends on the level of understanding your context, your use of abstractions, your lack of knowledge, your quality requirements, and your level of being standard in terms of business and technology domains.