What skills do people need to successfully program with AI

• Artificial intelligence
• Research

In brief

• A new ETH study investigates which skills are needed to create simple apps and computer programs using AI agents.
• According to the study, knowledge of computer science had the biggest impact on participants' performance at "vibe coding".
• Participants' writing skills also influenced how successful they were at vibe coding. On the other hand, those who frequently use large language models performed worse.

For a long time, the coding of apps or computer programs was reserved for experts with a command of complex programming languages such as Python or Java. Thanks to AI tools such as Claude Code, Cursor or Loveable, however, this is no longer the case. Non-experts can now also become software developers by using natural language to describe how an app or program should work. An AI interprets these instructions - also known as prompts - and generates the program code in the background. The term "vibe coding" has caught on to describe this new approach to programming.

But does that mean that programming with AI agents is now something anyone can do? In a new study, ETH researchers Sverrir Thorgeirsson, Theo Weidmann and Professor Zhendong Su investigated which skills affect people's success at vibe coding. As well as the ability to express themselves clearly in writing, people also need a basic knowledge of computer science when it comes to using AI to develop apps or programs that actually work.

People who understand how apps work are at an advantage

For the study, the researchers recruited 100 Zurich students who had completed at least an introductory course in computer science and already had some experience with AI-assisted programming. The students were tasked with using an AI agent to recreate an existing app for planning meals, to add new functions to an app for organising their own university courses, and to replicate an abstract application with no discernible purpose. They also had to write a short essay on a specialist topic that was familiar to them, as well as completing tests to examine their computer science knowledge and general cognitive ability.

The three researchers demonstrated that the participants' knowledge of computer science had the greatest impact on how well they completed the tasks. This effect remained intact when the researchers controlled for differences in the students' general cognitive ability - although, as the study only investigates correlations, it wasn't possible to determine exactly why this was the case.

The researchers suspect, however, that people with a better understanding of how programs work can provide more efficient instructions to an AI - even without seeing the code itself. "Our understanding is that good computer scientists can plan an app's structure more precisely and debug potential errors faster. They're also more likely to know relevant technical terms in order to direct the AI agent more precisely," explains Theo Weidmann, a doctoral student of computer science at the Advanced Software Technologies Lab of ETH Zurich.

Better results with clear and structured prompts

In the study, the authors also found a significant correlation between success at vibe coding and the students' general writing skills. Weidmann attributes this to the fact that, in vibe coding, writing the prompts becomes a form of coding in itself. "People who formulate clear and structured prompts achieve better results, while unclear or imprecise wording is more likely to lead to defective software."

The three researchers were surprised to find that students who are particularly frequent users of large language models in their everyday lives fared worse not only at writing essays, but also at vibe coding. The reasons for this couldn't be conclusively clarified in the correlation study. However, the study authors believe that frequent use of large language models may weaken people's ability to express themselves. Conversely, it could also be the case that students who are less proficient at writing are more likely to use AI tools.

AI corrects already-correct code

Coding with AI was also the subject of another recently published study by ETH researchers. ETH Professor Martin Vechev and his team investigated how good common AI agents are at correcting code that is actually already correct. Fixing code is one of the key potential applications of AI in software development.

The results are sobering: in more than 70 percent of cases, the AI agents corrected code even though it contained no errors. "Common AI agents suggest fixes to what is already correct code, which means that we still need improvements in AI technology. This is also a reminder that human experts must continue to check AI-generated code rather than relying on AI alone," explains Vechev, adding that there's still work to do before some aspects of software development can be fully automated with AI.

Reference

Thorgeirsson S, Weidmann T, Su Z, Computer Science Achievement and Writing Skills Predict Vibe Coding Proficiency, Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI '26), 13 - 17 April 2026, Barcelona, DOI: external page 10.1145/3772318.3791666

Newsletter subscription