Undermind - AI-Powered Scientific Research Assistant

Research topic

I want to find recent papers on creating agent-like AI systems, especially in the context of coding assistants that can manage or interact with an entire codebase. I am interested in things like the architectural design, training methodologies, real-world applications, and performance metrics.

Report

Recent literature shows significant advancements in creating agent-like AI systems for coding assistants that manage and interact with entire codebases, with frameworks like MAGIS and CodeAgent demonstrating improved performance and integration capabilities [1, 2].

Details:

Architectural Design:

Multi-agent Frameworks: Several papers discuss multi-agent systems tailored for software development tasks. MAGIS introduces four distinct agents — Manager, Custodian, Developer, and QA — to collaborate in resolving GitHub issues [1], while CodeAgent integrates five programming tools for repo-level interactions [2].
Collaborative Architectures: MetaGPT adopts a meta-programming approach, employing Standardized Operating Procedures (SOPs) for LLM-based agent collaborations to avoid logic inconsistencies [3].
Dynamic Roles: AgileCoder integrates Agile Methodology with roles like Product Manager, Developer, and Tester assigned to agents, enhancing real-time project management and codebase understanding through a Dynamic Code Graph Generator [7].

Training Methodologies and Performance Metrics:

Supervised Learning and Reinforcement Learning: Works like CodeRL leverage reinforcement learning alongside pre-trained models to refine code generation through iterative feedback mechanisms [35].
Benchmark Comparisons: Papers like AutoDev and SWE-agent provide comparative performance metrics against benchmarks like SWE-bench and HumanEval, showcasing enhancements in issue resolution and coding efficiency [4, 9].
Task-Specific Evaluation: AgentCoder focuses on iterative testing and optimization using a multi-agent framework, demonstrating superior performance on practical benchmarks when compared to single-agent systems [8].

Real-World Applications:

Practical Integration: AutoDev highlights an AI-driven framework capable of planning and executing diverse codebase operations, ranging from file editing to testing and version control [4].
IDE and Tool Integration: GitAgent and similar systems emphasize the integration of external tools and environments like GitHub for real-time extension and task execution, significantly enhancing agent adaptability and tool coverage [12].
Advanced IDE Concepts: A proposal for transforming traditional IDEs into Intelligent Development Environments (IDEs) suggests the role of these environments in facilitating interaction between human programmers and AI agents for project management and development tasks [20].

Performance Evaluation:

Efficacy Metrics: Studies such as those on AutoDev and CodeAgent report substantial improvements, with AutoDev showing enhanced issue resolution capabilities and CodeAgent demonstrating adaptability across various task scenarios [4, 2].
Comparative Analysis: SWE-agent achieves state-of-the-art performance on multiple benchmarks, highlighting the impact of custom agent-computer interfaces [9].

Conclusion:

The recent advancements in agent-like AI systems for coding assistants emphasize robust multi-agent frameworks, innovative training methodologies, and real-world integration, demonstrating improved efficiency and effectiveness in managing and interacting with entire codebases. Key papers such as MAGIS, CodeAgent, AutoDev, and SWE-agent provide comprehensive insights into these developments, underscoring significant performance improvements and practical applications [1, 2, 4, 9].

References

Last 5 years

Last 2 years

> 1 citation per year

> 5 citations per year

Topic Match

Cit./Year

Year

Paper

Paper Relevance Summary

96.7%

9.2

2024

[1] MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution Wei Tao, ..., and Yu-Xi Cheng ArXiv 2024 - 7 citations - Show abstract - Cite - PDF 96.7% topic match

Provides an LLM-based Multi-Agent framework for GitHub issue resolution. Proposes four customized agents (Manager, Custodian, Developer, QA) to address software evolution tasks. Demonstrates a significant performance improvement over baseline LLMs in resolving issues at the repository level.

95.6%

13.5

2024

[2] CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges Kechi Zhang, ..., and Zhi Jin ArXiv 2024 - 13 citations - Show abstract - Cite - PDF 95.6% topic match

Presents CodeAgent, an LLM-based agent framework. Integrates five programming tools for repo-level interactions like information retrieval and code testing. Shows significant LLM performance improvements, addressing real-world repo-level coding challenges and adaptability across tasks.

95.6%

0.0

2023

[3] MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework Sirui Hong, ..., and Jürgen Schmidhuber Journal Not Provided 2023 - 0 citations - Show abstract - Cite 95.6% topic match

Demonstrates multi-agent collaborative framework for software engineering. Introduces MetaGPT, encoding Standardized Operating Procedures (SOPs) into LLM-based agents to solve complex tasks through collaboration. Highlights improvement in solution coherence on collaborative software engineering benchmarks, relevant to AI systems managing entire codebases.

95.4%

1.3

2024

[4] AutoDev: Automated AI-Driven Development Michele Tufano, ..., and Neel Sundaresan ArXiv 2024 - 1 citations - Show abstract - Cite - PDF 95.4% topic match

Presents an AI-driven software development framework. AutoDev's AI agents autonomously plan and execute intricate software engineering tasks across the entire codebase. Enables complex objectives like file editing, build processes, testing, and git operations, accessing various development and analysis tools.

95.4%

3.5

2024

[5] CodeR: Issue Resolving with Multi-Agent and Task Graphs Dong Chen, ..., and Qianxiang Wang ArXiv 2024 - 2 citations - Show abstract - Cite - PDF 95.4% topic match

Proposes a multi-agent framework for issue resolving in code repositories. Utilizes task graphs to repair and resolve bugs and add features. Evaluates performance on SWE-bench lite, solving 28.33% of issues on the first attempt.

94.2%

0.0

2023

[6] Towards Trustworthy AI Software Development Assistance Daniel Maninger, ..., and Mira Mezini Proceedings of the 2024 ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results 2023 - 0 citations - Show abstract - Cite - PDF 94.2% topic match

Proposes a holistic architecture for trustworthy AI software development assistants. Utilizes a foundational LLM with graph-based code representations and a knowledge graph for semantic comprehension and quality assurance. Addresses multi-tasking, code quality, and safety, suggesting relevance to agent-like systems managing entire codebases.

93.8%

0.0

2024

[7] AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology Minh Huynh Nguyen, ..., and Nghi D. Q. Bui ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 93.8% topic match

Proposes AgileCoder, a multi-agent system for software development. Integrates Agile Methodology with agents acting as Product Manager, Developer, and Tester. Incorporates a Dynamic Code Graph Generator for real-time codebase comprehension and management.

93.1%

15.5

2023

[8] AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation Dong Huang, ..., and Heming Cui ArXiv 2023 - 31 citations - Show abstract - Cite 93.1% topic match

Provides a multi-agent system for code generation and optimization. Shows interactions among programmer, test designer, and test executor agents to improve code quality iteratively. Extensive experiments demonstrate improved performance over single-agent models and prompt engineering techniques.

92.6%

36.8

2024

[9] SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering John Yang, ..., and Ofir Press ArXiv 2024 - 24 citations - Show abstract - Cite - PDF 92.6% topic match

Introduces SWE-agent for autonomously solving software engineering tasks. Details custom interfaces that enhance LM agent capabilities in coding, navigating repositories, and executing programs. Shows state-of-the-art performance on benchmarks, indicating relevance to coding assistant research with holistic codebase management.

92.1%

8.8

2024

[10] CodePori: Large Scale Model for Autonomous Software Development by Using Multi-Agents Zeeshan Rasheed, ..., and P. Abrahamsson ArXiv 2024 - 8 citations - Show abstract - Cite - PDF 92.1% topic match

Introduces CodePori for automated code generation using multi-agent systems. Utilizes LLM-based agents for tasks like design, development, review, verification. Demonstrates efficiency in generating code for large-scale projects autonomously.

90.1%

5.4

2024

[11] Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization Yoichi Ishibashi and Yoshimasa Nishimura ArXiv 2024 - 4 citations - Show abstract - Cite - PDF 90.1% topic match

Provides a multi-agent framework for large-scale code generation and optimization. Details self-organized agents that independently generate and modify code components, dynamically scalable based on problem complexity. Demonstrates improved performance over single-agent systems in terms of code generation and handling of complex codebases (evaluated with HumanEval benchmark).

85.5%

4.0

2023

[12] GitAgent: Facilitating Autonomous Agent with GitHub by Tool Extension Bohan Lyu, ..., and Maosong Sun ArXiv 2023 - 4 citations - Show abstract - Cite - PDF 85.5% topic match

Provides an LLM-based agent, GitAgent, for autonomous tool extension from GitHub. Describes a four-phase procedure for integrating GitHub repositories based on user queries. Demonstrates the agent's effectiveness with a 69.4% success rate in experimental evaluations involving 30 user queries.

84.1%

0.0

2023

[13] AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation Dong Huang, ..., and Heming Cui Journal Not Provided 2023 - 0 citations - Show abstract - Cite - PDF 84.1% topic match

Introduces a multi-agent framework for code generation and testing. Describes specialized agents: programmer agent, test designer agent, test executor agent interacting iteratively. Relevant for architectural design and multi-agent methodologies in coding assistants; lacks detailed codebase management focus.

84.0%

1.6

2024

[14] MapCoder: Multi-Agent Code Generation for Competitive Problem Solving Md. Ashraful Islam, ..., and Md. Rizwan Parvez ArXiv 2024 - 1 citations - Show abstract - Cite - PDF 84.0% topic match

Introduces a multi-agent approach to code generation. Utilizes multi-agent prompting to handle natural language problem descriptions, code instructions, and execution of unit tests. Focuses on competitive problem solving, which may overlap but not fully cover codebase management and interaction.

81.9%

1.1

2024

[15] CodeAgent: Collaborative Agents for Software Engineering Daniel Tang, ..., and Tégawendé F. Bissyandé Journal Not Provided 2024 - 1 citations - Show abstract - Cite - PDF 81.9% topic match

Introduces a multi-agent system for automated code review. Describes CodeAgent's architecture with agents for different code review tasks and a QA-Checker supervisory agent. Focuses on code review, potentially relevant for broader codebase management and interaction tasks.

80.9%

1.4

2024

[16] LLM-Based Multi-Agent Systems for Software Engineering: Vision and the Road Ahead Junda He, ..., and David Lo ArXiv 2024 - 1 citations - Show abstract - Cite - PDF 80.9% topic match

Provides a vision for LLM-based Multi-Agent Systems in software engineering. Discusses cognitive abilities, autonomous problem-solving, and scalable solutions for complex software projects. Highlights potential applications, collaborative cross-examination, and research opportunities but lacks specific focus on managing an entire codebase.

74.3%

1.7

2024

[17] How to Understand Whole Software Repository? Yingwei Ma, ..., and Yongbin Li ArXiv 2024 - 1 citations - Show abstract - Cite - PDF 74.3% topic match

Highlights the importance of understanding entire repositories in ASE. Critiques existing methods for focusing only on local code information. Emphasizes challenges such as long input lengths and complex dependencies.

73.8%

44.5

2023

[18] Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models Andy Zhou, ..., and Yu-Xiong Wang ArXiv 2023 - 55 citations - Show abstract - Cite - PDF 73.8% topic match

Integrates LMs with a framework for decision-making. Introduces Language Agent Tree Search (LATS) unifying reasoning, acting, and planning in language models. Covers programming and managing code tasks, showing state-of-the-art accuracy, aligning with coding assistant goals.

73.2%

0.0

2024

[19] EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms Siyu Yuan, ..., and Deqing Yang ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 73.2% topic match

Introduces the EvoAgent method for automatic multi-agent generation. Uses evolutionary algorithms to extend LLM-based agent frameworks into scalable multi-agent systems autonomously. Relevant for architectural design and multi-agent systems, but lacks a direct focus on coding assistant applications.

67.6%

0.0

2024

[20] A New Generation of Intelligent Development Environments Mark Marron ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 67.6% topic match

Shows a vision for transforming traditional IDEs into Intelligent Development Environments (IDEs). IDE facilitates human-AI interaction and organizes workflow from requirements to deployment using AI agents. Focuses on architecture and integration but may lack specifics on multi-task learning or performance metrics.

66.4%

0.0

2024

[21] From Language Models to Practical Self-Improving Computer Agents Alex Sheng ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 66.4% topic match

Shows self-improving AI agents for diverse computer tasks. LLM agent generates and uses software augmentations to extend capabilities. Includes automated software development and real-world problem solving, potentially relevant for coding assistants managing codebases.

65.1%

0.4

2022

[22] Self-Programming Artificial Intelligence Using Code-Generating Language Models Alex Sheng and Shankar Padmanabhan Journal Not Provided 2022 - 1 citations - Show abstract - Cite - PDF 65.1% topic match

Shows a practical implementation of self-programming AI using code generation language models. Illustrates how a self-programming AI can modify its own source code to improve performance and create auxiliary sub-models. Relevant as it includes AI-driven code modifications, though it emphasizes self-improvement over codebase management.

64.5%

17.9

2023

[23] Experiential Co-Learning of Software-Developing Agents Cheng Qian, ..., and Maosong Sun ArXiv 2023 - 18 citations - Show abstract - Cite - PDF 64.5% topic match

Introduces Experiential Co-Learning for LLM-driven software-developing agents. Describes a framework where instructor and assistant agents leverage past experiences for improved collaborative task performance. Focuses on reducing manual involvement and improving task efficiency, but may lack details on codebase management or integration.

63.9%

6.8

2022

[24] Software assistants in software engineering: A systematic mapping study Maxime Savary-Leblanc, ..., and S. Gérard Software: Practice and Experience 2022 - 14 citations - Show abstract - Cite - PDF 63.9% topic match

Investigates research efforts on software assistants in software engineering. Focuses on the creation and interaction of software assistants for design, construction, and maintenance. Emphasizes user-assistant interactions, potentially relevant to understanding real-world application and performance of coding assistants.

63.8%

0.0

2024

[25] Envisioning the Next-Generation AI Coding Assistants: Insights & Proposals K. Nghiem, ..., and Nghi D. Q. Bui ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 63.8% topic match

Provides insights and proposals for next-generation AI coding assistants. Focuses on IDE integration, backend design, and data collection methodologies. Lacks specific details on agent-like architectures or managing entire codebases, but relevant for discussions on integration and design.

63.4%

31.2

2022

[26] Assessing the quality of GitHub copilot’s code generation Burak Yetistiren, ..., and Eray Tüzün Proceedings of the 18th International Conference on Predictive Models and Data Analytics in Software Engineering 2022 - 67 citations - Show abstract - Cite - PDF 63.4% topic match

Provides an evaluation of GitHub Copilot's code generation quality. Details an assessment of the accuracy and performance of suggestions by GitHub Copilot. Focuses on code generation quality rather than architectural design or holistic codebase management.

63.1%

20.9

2023

[27] Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review M. Wong, ..., and C. Tan Entropy 2023 - 33 citations - Show abstract - Cite - PDF 63.1% topic match

Provides a comprehensive review of NLP techniques for AI-assisted programming. Focuses on transformer-based LLMs for tasks like code generation, completion, and defect detection. Covers challenges and opportunities but does not specifically address agent-like systems for codebase management.

62.6%

52.2

2023

[28] Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agents Yashar Talebirad and Amirhossein Nadiri ArXiv 2023 - 82 citations - Show abstract - Cite - PDF 62.6% topic match

Provides a framework for enhancing LLM capabilities through multi-agent collaboration. Uses multi-agent systems to improve task handling in scenarios like software development. Focuses more broadly on multi-agent collaboration rather than solely on coding assistants managing entire codebases.

61.1%

1.4

2024

[29] How far are AI-powered programming assistants from meeting developers' needs? Xin Tan, ..., and Li Zhang ArXiv 2024 - 1 citations - Show abstract - Cite - PDF Abstract: Recent In-IDE AI coding assistant tools (ACATs) like GitHub Copilot have significantly impacted developers' coding habits. While some studies have examined their effectiveness, there lacks in-depth investigation into the actual assistance process. To bridge this gap, we simulate real development scenarios encompassing three typical types of software development tasks and recruit 27 computer science students to investigate their behavior with three popular ACATs. Our goal is to comprehensively assess ACATs' effectiveness, explore characteristics of recommended code, identify reasons for modifications, and understand users' challenges and expectations. To facilitate the study, we develop an experimental platform that includes a data collection plugin for VSCode IDE and provides functions for screen recording, code evaluation, and automatic generation of personalized interview and survey questions. Through analysis of the collected data, we find that ACATs generally enhance task completion rates, reduce time, improve code quality, and increase self-perceived productivity. However, the improvement is influenced by both the nature of coding tasks and users' experience level. Notably, for experienced participants, the use of ACATs may even increase completion time. We observe that"edited line completion"is the most frequently recommended way, while"comments completion"and"string completion"have the lowest acceptance rates. The primary reasons for modifying recommended code are disparities between output formats and requirements, flawed logic, and inconsistent code styles. In terms of challenges and expectations, optimization of service access and help documentation is also concerned by participants except for functionality and performance. Our study provides valuable insights into the effectiveness and usability of ACATs, informing further improvements in their design and implementation. 61.1% topic match

Evaluates: `Evaluates effectiveness of current AI coding assistants (ACATs) in real developer scenarios.` Details: `Assesses performance through simulated tasks, user behavior, and challenges using ACATs like GitHub Copilot.` Relevance: `Focuses on practical usability rather than architectural or training methodologies; may lack in-depth discussion on multi-tasking or codebase management.`

57.1%

2.3

2024

[30] Generating Java Methods: An Empirical Assessment of Four AI-Based Code Assistants Vincenzo Corso, ..., and O. Riganelli 2024 IEEE/ACM 32nd International Conference on Program Comprehension (ICPC) 2024 - 2 citations - Show abstract - Cite - PDF 57.1% topic match

Provides an empirical assessment of four popular AI-based code assistants. Evaluates GitHub Copilot, Tabnine, ChatGPT, and Google Bard using 100 real-life Java methods. Reveals effectiveness decreases with dependencies outside single classes, hinting at limitations in holistic codebase management.

56.4%

0.9

2023

[31] Developer Experiences with a Contextualized AI Coding Assistant: Usability, Expectations, and Outcomes Gustavo Pinto, ..., and Edward Monteiro 2024 IEEE/ACM 3rd International Conference on AI Engineering – Software Engineering for AI (CAIN) 2023 - 1 citations - Show abstract - Cite - PDF 56.4% topic match

Evaluates the usability of a contextualized AI coding assistant. Focuses on user experiences, time savings, and accuracy for domain-specific tasks. Limited discussion on architectural design or holistic codebase management may reduce relevance.

53.8%

1.4

2024

[32] “You’re on a bicycle with a little motor”: Benefits and Challenges of Using AI Code Assistants Wendy Mendes, ..., and Cleidson De Souza 2024 IEEE/ACM 17th International Conference on Cooperative and Human Aspects of Software Engineering (CHASE) 2024 - 1 citations - Show abstract - Cite - PDF 53.8% topic match

Describes developers’ experiences with AI code assistants Based on interviews, presents benefits, challenges, and strategies for using AI tools like Tabnine and GitHub Copilot Focuses on usability and practical experience rather than architectural design or full codebase management capabilities

52.8%

18.4

2022

[33] CodexDB: Synthesizing Code for Query Processing from Natural Language Instructions using GPT-3 Codex Immanuel Trummer Proc. VLDB Endow. 2022 - 46 citations - Show abstract - Cite 52.8% topic match

Shows AI generating query processing code from natural language. Utilizes GPT-3 Codex to decompose and translate instructions into SQL processing steps. Does not address agent-like systems for entire codebase management; limited to SQL code synthesis from instructions.

50.4%

6.9

2024

[34] AutoCodeRover: Autonomous Program Improvement Yuntong Zhang, ..., and Abhik Roychoudhury ArXiv 2024 - 5 citations - Show abstract - Cite - PDF Abstract: Researchers have made significant progress in automating the software development process in the past decades. Recent progress in Large Language Models (LLMs) has significantly impacted the development process, where developers can use LLM-based programming assistants to achieve automated coding. Nevertheless software engineering involves the process of program improvement apart from coding, specifically to enable software maintenance (e.g. bug fixing) and software evolution (e.g. feature additions). In this paper, we propose an automated approach for solving GitHub issues to autonomously achieve program improvement. In our approach called AutoCodeRover, LLMs are combined with sophisticated code search capabilities, ultimately leading to a program modification or patch. In contrast to recent LLM agent approaches from AI researchers and practitioners, our outlook is more software engineering oriented. We work on a program representation (abstract syntax tree) as opposed to viewing a software project as a mere collection of files. Our code search exploits the program structure in the form of classes/methods to enhance LLM's understanding of the issue's root cause, and effectively retrieve a context via iterative search. The use of spectrum based fault localization using tests, further sharpens the context, as long as a test-suite is available. Experiments on SWE-bench-lite which consists of 300 real-life GitHub issues show increased efficacy in solving GitHub issues (22-23% on SWE-bench-lite). On the full SWE-bench consisting of 2294 GitHub issues, AutoCodeRover solved around 16% of issues, which is higher than the efficacy of the recently reported AI software engineer Devin from Cognition Labs, while taking time comparable to Devin. We posit that our workflow enables autonomous software engineering, where, in future, auto-generated code from LLMs can be autonomously improved. 50.4% topic match

Proposes an automated approach for program improvement. Utilizes LLMs and code search for fixing bugs and adding features. Focuses on abstract syntax trees rather than as a file collection; emphasizes software engineering perspective.

50.4%

61.5

2022

[35] CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning Hung Le, ..., and S. Hoi ArXiv 2022 - 153 citations - Show abstract - Cite - PDF 50.4% topic match

Proposes a new framework for program synthesis using pretrained models and RL. Combines code-generating language models with a critic network for functional correctness feedback. Focuses on code generation rather than comprehensive codebase management or multi-tasking capabilities.

49.3%

0.0

2024

[36] Multi-Agent Software Development through Cross-Team Collaboration Zhuoyun Du, ..., and Cheng Yang ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 49.3% topic match

Shows multi-agent collaboration for autonomous software development using large language models. Details phases like requirements analysis, development, review, and testing in a waterfall model. Lacks focus on managing or interacting with the entire codebase and multiple decision paths.

48.5%

10.0

2023

[37] TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usage Jingqing Ruan, ..., and Rui Zhao Journal Not Provided 2023 - 14 citations - Show abstract - Cite - PDF 48.5% topic match

Paper Overview: Proposes a framework for LLM-based AI agents for task planning. Details: Designs two agent types to handle complex tasks combining planning and tool usage. Relevance: Does not specifically address coding assistants or codebase management; focuses on general task planning and tool usage.

48.4%

31.1

2023

[38] TPTU: Task Planning and Tool Usage of Large Language Model-based AI Agents Jingqing Ruan, ..., and Rui Zhao ArXiv 2023 - 62 citations - Show abstract - Cite - PDF 48.4% topic match

Proposes a structured framework for LLM-based AI agents Describes two agents (one-step and sequential) for task planning and tool usage Focuses on task planning and tool integration rather than specific coding assistant applications

47.5%

1.4

2019

[39] Towards concept based software engineering for intelligent agents Ole Meyer and V. Gruhn https://doi.org/10.1109/RAISE.2019.00015 2019 - 8 citations - Show abstract - Cite 47.5% topic match

Presents concept-based software engineering for intelligent agents. Focuses on productivity and quality goals using hierarchical reinforcement learning. Discusses integration of software engineering principles with intelligent agent development, but lacks specific focus on managing entire codebases.

46.5%

4.6

2024

[40] Unsupervised Evaluation of Code LLMs with Round-Trip Correctness Miltiadis Allamanis, ..., and Pengcheng Yin ArXiv 2024 - 4 citations - Show abstract - Cite - PDF 46.5% topic match

Introduces round-trip correctness (RTC) for evaluating code LLMs. RTC checks if synthesized code remains semantically equivalent after round-trip prediction and evaluation. Focuses on evaluation method rather than architectural design or codebase management capabilities.

44.8%

10.2

2019

[41] Towards an Autonomous Bot for Automatic Source Code Refactoring Marvin Wyrich and J. Bogner 2019 IEEE/ACM 1st International Workshop on Bots in Software Engineering (BotSE) 2019 - 57 citations - Show abstract - Cite 44.8% topic match

Presents an autonomous bot for automatic source code refactoring. Integrates refactoring actions into version control, automating and proposing changes for developer review. Focuses on refactoring specifically, lacks broader codebase management or multi-task capabilities for complete coding assistance.

44.7%

0.0

2024

[42] AutoManual: Generating Instruction Manuals by LLM Agents via Interactive Environmental Learning Minghao Chen, ..., and Xiaofei He ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 44.7% topic match

Introduces a framework for LLM agents to autonomously learn and adapt. Describes AutoManual where agents categorize environmental knowledge into rules and optimize them online through interaction. Primarily focuses on environmental learning and interaction, potentially relevant for understanding agent adaptability but lacks specifics on coding assistants and codebase management.

40.9%

0.5

2021

[43] Teaming up with information agents J. Diggelen, ..., and B. Vecht ArXiv 2021 - 2 citations - Show abstract - Cite - PDF 40.9% topic match

Provides insight into AI agents teaming up with humans. Highlights the importance of human-AI collaboration for practical applications, emphasizing that AI should possess teaming intelligence. Does not specifically address coding assistants or entire codebase management, focusing instead on general human-AI interactions.

39.9%

0.0

2024

[44] Agent-Driven Automatic Software Improvement Fernando Vallecillos Ruiz Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering 2024 - 0 citations - Show abstract - Cite - PDF 39.9% topic match

39.1%

0.0

2019

[45] Towards Concept Based Software Engineering for Intelligent Agents M. Ole and Gruhn Volker 2019 IEEE/ACM 7th International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE) 2019 - 0 citations - Show abstract - Cite 39.1% topic match

Introduces a concept-based software engineering approach for intelligent agents. Highlights the use of hierarchical reinforcement learning to improve productivity and quality in agent development. Focuses broadly on intelligent agents and software engineering; may not address coding assistants interacting with entire codebases specifically.

37.9%

4.0

2023

[46] Chatbots As Fluent Polyglots: Revisiting Breakthrough Code Snippets David A. Noever and Kevin Williams ArXiv 2023 - 8 citations - Show abstract - Cite - PDF 37.9% topic match

Examines AI-driven code assistants for analyzing influential code. Provides insights and improvements in clarity or performance of significant historical code. Includes future work on multi-tasking, refactoring legacy code, and automated documentation.

36.9%

10.6

2023

[47] Fully Autonomous Programming with Large Language Models Vadim Liventsev, ..., and L. Moonen Proceedings of the Genetic and Evolutionary Computation Conference 2023 - 18 citations - Show abstract - Cite - PDF 36.9% topic match

Shows a SED approach for program synthesis with LLMs. Examines replace, repair, and hybrid debug strategies for improving code correctness. Focuses on the SED framework rather than holistic codebase management.

35.3%

0.0

2024

[48] Using AI-Based Coding Assistants in Practice: State of Affairs, Perceptions, and Ways Forward Agnia Sergeyuk, ..., and Iftekhar Ahmed ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 35.3% topic match

Provides insights from a large-scale survey on the use of AI coding assistants. Examines developer usage and perceptions across five software development activities and stages. While it focuses on user perceptions, it may not cover agent architecture or codebase management directly.

35.0%

0.0

2021

[49] Autonomous Learning with Automatically Created Models and a Novel Model Selection Harshal V Bharatia 2021 IEEE Symposium Series on Computational Intelligence (SSCI) 2021 - 0 citations - Show abstract - Cite 35.0% topic match

Provides: a method for autonomous learning and model selection. This method automates model creation, refinement, and selection using reinforcement learning. While it focuses on adaptive learning and dynamic model ensembles, it does not address coding assistants or codebase management specifically.

33.8%

3.3

2023

[50] CoLadder: Supporting Programmers with Hierarchical Code Generation in Multi-Level Abstraction Ryan Yen, ..., and Jian Zhao ArXiv 2023 - 4 citations - Show abstract - Cite - PDF 33.8% topic match

Provides hierarchical code generation support with LLMs. Facilitates task decomposition, direct code manipulation, and result evaluation at multiple abstraction levels. Focuses on user interaction with code rather than full codebase management or multi-agent systems.

32.8%

0.0

2024

[51] From Today's Code to Tomorrow's Symphony: The AI Transformation of Developer's Routine by 2030 Matteo Ciniselli, ..., and L. Grazia ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 32.8% topic match

Provides a comparative analysis of AI-assisted programming from 2024 to 2030. Details the transformation of developers' roles and the envisioned HyperAssistant tool for comprehensive support. Focuses on broader AI impacts in software development rather than specific architectures or multi-tasking AI for codebase management.

31.9%

8.8

2024

[52] StepCoder: Improve Code Generation with Reinforcement Learning from Compiler Feedback Shihan Dou, ..., and Tao Gui ArXiv 2024 - 8 citations - Show abstract - Cite - PDF 31.9% topic match

30.1%

29.6

2024

[53] Executable Code Actions Elicit Better LLM Agents Xingyao Wang, ..., and Heng Ji ArXiv 2024 - 27 citations - Show abstract - Cite - PDF 30.1% topic match

27.4%

29.5

2018

[54] Leveraging Grammar and Reinforcement Learning for Neural Program Synthesis Rudy Bunel, ..., and Pushmeet Kohli ArXiv 2018 - 203 citations - Show abstract - Cite - PDF 27.4% topic match

24.5%

0.7

2023

[55] Chat2Code: A Chatbot for Model Specification and Code Generation, The Case of Smart Contracts I. Qasse, ..., and Mohammad Hamdaqa 2023 IEEE International Conference on Software Services Engineering (SSE) 2023 - 1 citations - Show abstract - Cite 24.5% topic match

23.4%

0.2

2015

[56] Creating Complex Applications via Self-Adapting Autonomous Agents in an Intelligent System Framework Tammy R. Fuller and Gerald E. Deane 2015 IEEE 9th International Conference on Self-Adaptive and Self-Organizing Systems 2015 - 2 citations - Show abstract - Cite Abstract: In this paper, we present a process, developed over years of practical commercial use, where applications accomplishing a wide variety of complex tasks are created from a common framework, through the use, recombination and iterative refinement of autonomous agents in a Multi-Agent Intelligent System. Driven by a need to solve real-world problems, our focus is to make businesses run more efficiently in an increasingly complex world of systems and software that must work together seamlessly. By listening closely to our customers' problems, we discovered points of commonality, as well as patterns of anomalies related to the flow of data through communication channels and data processing systems, include accounting, inventory, customer relationship management, scheduling systems and many more. We solved their problems through the creation of an Intelligent System, where we defined and implemented software agents that were highly configurable, responsive in real-time and useable in various settings. Autonomous agents adhere to a standard format of three major components: the goal or triggering criteria, the action, and the adaptation response. Agents run within a common Intelligent System framework and agent libraries provide a vast set of component behaviors to build applications from. Agents have one or more of the following component behaviors: sensory aware, geo-position aware, temporally aware, API aware, device aware, and many more. Additionally, there are manager-level agents whose goal is to keep the overall system in balance, through dynamic resource allocation on a system level. To prove the viability of this process, we present a variety of applications representing wide ranging behaviors, many with overlapping agents, created via this approach, all of which are in active commercial use. Finally, we discuss future enhancements toward self-organization, where end users express their requirements declaratively to solve larger business needs, resulting in the automatic instantiation of a solution specific intelligent system. 23.4% topic match

18.8%

0.0

2024

[57] Microservices and API Deployment Optimization using AI Nilesh Charankar International Journal on Recent and Innovation Trends in Computing and Communication 2024 - 0 citations - Show abstract - Cite - PDF 18.8% topic match

18.1%

0.8

2023

[58] B-Coder: Value-Based Deep Reinforcement Learning for Program Synthesis Zishun Yu, ..., and Hongxia Yang ArXiv 2023 - 1 citations - Show abstract - Cite - PDF 18.1% topic match

16.0%

1.1

2024

[59] Assessing AI-Based Code Assistants in Method Generation Tasks Vincenzo Corso, ..., and O. Riganelli 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) 2024 - 1 citations - Show abstract - Cite - PDF 16.0% topic match

Compares AI-based code assistants in method generation tasks. Evaluates GitHub Copilot, Tabnine, ChatGPT, and Google Bard on accuracy and efficiency. Focuses on specific coding tasks, but lacks emphasis on multi-agent systems managing entire codebases.

15.8%

10.7

2024

[60] RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation Qinyu Luo, ..., and Maosong Sun ArXiv 2024 - 9 citations - Show abstract - Cite - PDF 15.8% topic match

15.4%

2.1

2023

[61] ChatDev: Communicative Agents for Software Development Cheng Qian, ..., and Maosong Sun Journal Not Provided 2023 - 3 citations - Show abstract - Cite - PDF 15.4% topic match

15.2%

0.0

2024

[62] Impact of Artificial Intelligence in Education Sector Ms. Shilpa Sandhu International Journal for Research in Applied Science and Engineering Technology 2024 - 0 citations - Show abstract - Cite 15.2% topic match

14.0%

2.9

2024

[63] A Unified Debugging Approach via LLM-Based Multi-Agent Synergy Cheryl Lee, ..., and Michael R. Lyu ArXiv 2024 - 2 citations - Show abstract - Cite - PDF 14.0% topic match

13.9%

0.8

2022

[64] A Unified Code Review Automation for Large-scale Industry with Diverse Development Environments Hyungjin Kim, ..., and Chul-Joo Kim 2022 IEEE/ACM 44th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) 2022 - 2 citations - Show abstract - Cite 13.9% topic match

13.6%

3.4

2023

[65] Exploring and Characterizing Large Language Models for Embedded System Development and Debugging Zachary Englhardt, ..., and Vikram Iyer Extended Abstracts of the CHI Conference on Human Factors in Computing Systems 2023 - 5 citations - Show abstract - Cite - PDF 13.6% topic match

13.1%

5.6

2019

[66] RefBot: Intelligent Software Refactoring Bot Vahid Alizadeh, ..., and Meriem Chater 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE) 2019 - 29 citations - Show abstract - Cite 13.1% topic match

13.0%

1.5

2023

[67] Software Engineering Using Autonomous Agents: Are We There Yet? Samdyuti Suri, ..., and Vikrant S. Kaulgud 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) 2023 - 2 citations - Show abstract - Cite 13.0% topic match

12.4%

0.0

2024

[68] AICoderEval: Improving AI Domain Code Generation of Large Language Models Yinghui Xia, ..., and Jinsong Yang ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 12.4% topic match

12.3%

0.0

2016

[69] Artificial intelligence at the gates of dawn? Thomas Eiter Journal Not Provided 2016 - 0 citations - Show abstract - Cite Abstract: In recent years, there have been significant achievements in different areas of Artificial Intelligence (AI) which made it to the headlines. After Deep Blue was the first computer program winning against a ruling world champion of chess in 1997, which was a milestone in the history of AI, the Mars Rover missions started in the early 2000s for which intelligent autonomous planning and control is instrumental. The DARPA challenges in 2004 and 2005 have stimulated intensive research on autonomous vehicles, and due to efforts by global players like Apple and Google in cooperation with car industry, encountering such vehicles on the road appears to be realistic in the near future. On another end, IBM's Watson victory over Jeopardy champions in 2011, and the most recent defeat of top Go players in 2015 and 2016 by Alpha-Go, a computer program by Google's DeepMind company, are further milestones in the history of AI, which are based on novel use of search and deep learning techniques. Fueled by this and many success stories in other application areas, e.g. in speech recognition and robotics, there is a hype about Artificial Intelligence these days. A lot is written in the press, and concerns about emerging super-intelligence have been uttered by prominent people, in fear of the "technological singularity" that the intellectual power of machines will surpass the one of humans, which they see as a threat to humanity. Less apocalyptic yet still serious are worries about the effects of AI on employment. Whether AI will end jobs has been raised in a number of articles and has been discussed by leading economists and AI researchers at the World Economic Forum. This talk reviews some of the issues above and emphasizes that despite the recent achievements and advances in the state of the art, there is still a number of issues that, from a cognitive perspective, are not resolved yet, among them understanding of the mind, deeper real world reasoning, natural language understanding, philosophical aspects, and ethical or emotional aspects, to mention a few. Specifically, we will look at recent results, trends and issues in knowledge representation and reasoning, with a focus on declarative problem solving. Arguably deep and complex reasoning is a key feature of intelligence, but current capabilities, let alone for superintelligence, are still limited. Nonetheless, advances in computational logic have led to technology such as description logics, constraint programming, SAT/SMT and Answer Set Programming that offers tools for solving important problems in a range of 1 Technische Universität Wien, Institute of Information Systems, eiter@kr.tuwien.ac.at 12.3% topic match

11.8%

0.0

2024

[70] Test-Driven Development for Code Generation N. Mathews and Mei Nagappan ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 11.8% topic match

11.1%

202.1

2023

[71] MetaGPT: Meta Programming for Multi-Agent Collaborative Framework Sirui Hong, ..., and Chenglin Wu ArXiv 2023 - 286 citations - Show abstract - Cite - PDF 11.1% topic match

10.9%

0.9

2023

[72] LLM4TDD: Best Practices for Test Driven Development Using Large Language Models Sanyogita Piya and Allison Sullivan ArXiv 2023 - 1 citations - Show abstract - Cite - PDF 10.9% topic match

10.2%

5.6

2024

[73] S-Agents: Self-organizing Agents in Open-ended Environments Jia-Qing Chen, ..., and Li Zhang ArXiv 2024 - 5 citations - Show abstract - Cite - PDF 10.2% topic match

9.5%

23.1

2021

[74] Towards Automating Code Review Activities Rosalia Tufano, ..., and G. Bavota 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) 2021 - 92 citations - Show abstract - Cite - PDF 9.5% topic match

9.4%

0.5

2023

[75] Accelerating software development with AI: exploring the impact of ChatGPT and GitHub Copilot Illia Solohubov, ..., and Stepan Skrupsky CTE 2023 - 1 citations - Show abstract - Cite 9.4% topic match

9.3%

None

[76] B UILDING THE F UTURE OF R ESPONSIBLE AI: A P ATTERN -O RIENTED R EFERENCE A RCHITECTURE FOR D ESIGNING L ARGE L ANGUAGE M ODEL BASED A GENTS Qinghua Lu, ..., and Australia Journal Not Provided None - 0 citations - Show abstract - Cite 9.3% topic match

9.1%

37.2

2024

[77] Large Language Model based Multi-Agents: A Survey of Progress and Challenges Taicheng Guo, ..., and Xiangliang Zhang ArXiv 2024 - 35 citations - Show abstract - Cite - PDF 9.1% topic match

8.8%

0.0

2024

[78] Automating Patch Set Generation from Code Review Comments Using Large Language Models Md Tajmilur Rahman, ..., and Mir Yousuf Sultan 2024 IEEE/ACM 3rd International Conference on AI Engineering – Software Engineering for AI (CAIN) 2024 - 0 citations - Show abstract - Cite - PDF 8.8% topic match

8.7%

14.2

2017

[79] How to Design a Program Repair Bot? Insights from the Repairnator Project Simon Urli, ..., and Monperrus Martin 2018 IEEE/ACM 40th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP) 2017 - 109 citations - Show abstract - Cite - PDF 8.7% topic match

8.6%

0.0

2019

[80] Interactive Software Refactoring Bot Vahid Alizadeh, ..., and Meriem Chater Journal Not Provided 2019 - 0 citations - Show abstract - Cite 8.6% topic match

8.5%

71.9

2023

[81] SWE-bench: Can Language Models Resolve Real-World GitHub Issues? Carlos E. Jimenez, ..., and Karthik Narasimhan ArXiv 2023 - 88 citations - Show abstract - Cite - PDF 8.5% topic match

8.4%

25.2

2022

[82] What is it like to program with artificial intelligence? Advait Sarkar, ..., and B. Zorn Annual Workshop of the Psychology of Programming Interest Group 2022 - 60 citations - Show abstract - Cite - PDF 8.4% topic match

Provides insights into LLM-assisted programming. Examines how large language models offer new paradigms compared to traditional programming aids. Focuses on usability studies and practical experiences, less on architectural design or managing entire codebases.

8.4%

2.3

2024

[83] AI-Assisted Programming Tasks Using Code Embeddings and Transformers S. Kotsiantis, ..., and M. Tzagarakis Electronics 2024 - 2 citations - Show abstract - Cite - PDF 8.4% topic match

8.3%

26.0

2023

[84] Self-planning Code Generation with Large Language Models Xue Jiang, ..., and Ge Li ACM Transactions on Software Engineering and Methodology 2023 - 47 citations - Show abstract - Cite - PDF 8.3% topic match

8.3%

0.8

2023

[85] On Automated Assistants for Software Development: The Role of LLMs Mira Leung and Gail Murphy 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) 2023 - 1 citations - Show abstract - Cite - PDF 8.3% topic match

8.2%

78.3

2023

[86] Self-collaboration Code Generation via ChatGPT Yihong Dong, ..., and Ge Li ACM Transactions on Software Engineering and Methodology 2023 - 134 citations - Show abstract - Cite - PDF 8.2% topic match

8.1%

4.9

2023

[87] Formally Specifying the High-Level Behavior of LLM-Based Agents M. Crouse, ..., and Luis A. Lastras ArXiv 2023 - 6 citations - Show abstract - Cite - PDF 8.1% topic match

8.0%

4.2

1998

[88] Co-ordination in Artificial Agent Societies: Social Structures and Its Implications for Autonomous Problem-Solving Agents Sascha Ossowski Journal Not Provided 1998 - 109 citations - Show abstract - Cite 8.0% topic match

7.9%

0.6

2023

[89] Coarse-Tuning Models of Code with Reinforcement Learning Feedback Abhinav C. P. Jain, ..., and U. Wisconsin Journal Not Provided 2023 - 1 citations - Show abstract - Cite - PDF 7.9% topic match

7.8%

0.5

2008

[90] Towards Mining for Influence in a Multi Agent Environment R. Logie, ..., and K. Waugh Journal Not Provided 2008 - 8 citations - Show abstract - Cite 7.8% topic match

7.7%

0.0

2023

[91] AI Pair Programming Tool Prof. Anand Magar International Journal for Research in Applied Science and Engineering Technology 2023 - 0 citations - Show abstract - Cite 7.7% topic match

7.6%

0.0

2024

[92] The Future of Software Engineering in an AI-Driven World Valerio Terragni, ..., and Kelly Blincoe ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 7.6% topic match

7.6%

40.5

2023

[93] InferFix: End-to-End Program Repair with LLMs Ma Jin, ..., and Alexey Svyatkovskiy Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering 2023 - 73 citations - Show abstract - Cite - PDF Abstract: Software development life cycle is profoundly influenced by bugs; their introduction, identification, and eventual resolution account for a significant portion of software development cost. This has motivated software engineering researchers and practitioners to propose different approaches for automating the identification and repair of software defects. Large Language Models (LLMs) have been adapted to the program repair task through few-shot demonstration learning and instruction prompting, treating this as an infilling task. However, these models have only focused on learning general bug-fixing patterns for uncategorized bugs mined from public repositories. In this paper, we propose : a transformer-based program repair framework paired with a state-of-the-art static analyzer to fix critical security and performance bugs. combines a Retriever – transformer encoder model pretrained via contrastive learning objective, which aims at searching for semantically equivalent bugs and corresponding fixes; and a Generator – an LLM (12 billion parameter Codex Cushman model) finetuned on supervised bug-fix data with prompts augmented via adding bug type annotations and semantically similar fixes retrieved from an external non-parametric memory. To train and evaluate our approach, we curated , a novel, metadata-rich dataset of bugs extracted by executing the Infer static analyzer on the change histories of thousands of Java and C# repositories. Our evaluation demonstrates that outperforms strong LLM baselines, with a top-1 accuracy of 65.6% for generating fixes in C# and 76.8% in Java. We discuss the deployment of alongside Infer at Microsoft which offers an end-to-end solution for detection, classification, and localization of bugs, as well as fixing and validation of candidate patches, integrated in the continuous integration (CI) pipeline to automate the software development workflow. 7.6% topic match

7.5%

492.4

2022

[94] ReAct: Synergizing Reasoning and Acting in Language Models Shunyu Yao, ..., and Yuan Cao ArXiv 2022 - 1100 citations - Show abstract - Cite - PDF 7.5% topic match

7.2%

0.0

2024

[95] Open-Source AI-based SE Tools: Opportunities and Challenges of Collaborative Software Learning Zhihao Lin, ..., and Li Li ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 7.2% topic match

7.2%

0.0

2024

[96] Unlock the Correlation between Supervised Fine-Tuning and Reinforcement Learning in Training Code Large Language Models Jie Chen, ..., and Liang Xiang ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 7.2% topic match

7.2%

7.3

2024

[97] ContrastRepair: Enhancing Conversation-Based Automated Program Repair via Contrastive Test Case Pairs Jiaolong Kong, ..., and Qi Guo ArXiv 2024 - 6 citations - Show abstract - Cite - PDF 7.2% topic match

7.1%

0.0

2024

[98] In-IDE Human-AI Experience in the Era of Large Language Models; A Literature Review Agnia Sergeyuk, ..., and M. Izadi ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 7.1% topic match

7.0%

0.0

2024

[99] AI-powered Code Review with LLMs: Early Results Zeeshan Rasheed, ..., and Pekka Abrahamsson ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 7.0% topic match

7.0%

6.3

2022

[100] Codex Hacks HackerRank: Memorization Issues and a Framework for Code Synthesis Evaluation Anjan Karmakar, ..., and R. Robbes ArXiv 2022 - 13 citations - Show abstract - Cite - PDF 7.0% topic match

6.9%

2.7

2024

[101] Exploring Autonomous Agents through the Lens of Large Language Models: A Review Saikat Barua ArXiv 2024 - 2 citations - Show abstract - Cite - PDF 6.9% topic match

6.9%

9.8

2023

[102] A Survey on Large Language Models for Software Engineering Quanjun Zhang, ..., and Zhenyu Chen ArXiv 2023 - 10 citations - Show abstract - Cite - PDF 6.9% topic match

6.9%

5.2

2024

[103] When LLM-based Code Generation Meets the Software Development Process Feng Lin, ..., and Tse-Husn Chen ArXiv 2024 - 4 citations - Show abstract - Cite - PDF 6.9% topic match

6.8%

9.8

2023

[104] GameGPT: Multi-agent Collaborative Framework for Game Development Dake Chen, ..., and Haoyang Zhang ArXiv 2023 - 12 citations - Show abstract - Cite - PDF 6.8% topic match

6.7%

None

[105] OPTIMIZATION ERROR DETECTION AND GENERATION OF CODE USING ARTIFICIAL INTELLIGENCE Mrs.Abhilasha Kore, ..., and Rugved Kamble Journal Not Provided None - 0 citations - Show abstract - Cite 6.7% topic match

6.7%

1.4

2023

[106] Copilot for Xcode: Exploring AI-Assisted Programming by Prompting Cloud-based Large Language Models C. Tan, ..., and Ching Nam Hang ArXiv 2023 - 2 citations - Show abstract - Cite - PDF 6.7% topic match

6.5%

None

[107] Large Language Models in Software Engineering: A Critical Review and Future Research Directions Ali Bektas Journal Not Provided None - 0 citations - Show abstract - Cite 6.5% topic match

6.4%

178.6

2023

[108] Improving Factuality and Reasoning in Language Models through Multiagent Debate Yilun Du, ..., and Igor Mordatch ArXiv 2023 - 287 citations - Show abstract - Cite - PDF 6.4% topic match

6.4%

328.4

2023

[109] A Survey on Large Language Model based Autonomous Agents Lei Wang, ..., and Ji-rong Wen ArXiv 2023 - 446 citations - Show abstract - Cite - PDF 6.4% topic match

6.3%

8.9

2023

[110] Balancing Autonomy and Alignment: A Multi-Dimensional Taxonomy for Autonomous LLM-powered Multi-Agent Architectures Thorsten Händler ArXiv 2023 - 11 citations - Show abstract - Cite - PDF Abstract: Large language models (LLMs) have revolutionized the field of artificial intelligence, endowing it with sophisticated language understanding and generation capabilities. However, when faced with more complex and interconnected tasks that demand a profound and iterative thought process, LLMs reveal their inherent limitations. Autonomous LLM-powered multi-agent systems represent a strategic response to these challenges. Such systems strive for autonomously tackling user-prompted goals by decomposing them into manageable tasks and orchestrating their execution and result synthesis through a collective of specialized intelligent agents. Equipped with LLM-powered reasoning capabilities, these agents harness the cognitive synergy of collaborating with their peers, enhanced by leveraging contextual resources such as tools and datasets. While these architectures hold promising potential in amplifying AI capabilities, striking the right balance between different levels of autonomy and alignment remains the crucial challenge for their effective operation. This paper proposes a comprehensive multi-dimensional taxonomy, engineered to analyze how autonomous LLM-powered multi-agent systems balance the dynamic interplay between autonomy and alignment across various aspects inherent to architectural viewpoints such as goal-driven task management, agent composition, multi-agent collaboration, and context interaction. It also includes a domain-ontology model specifying fundamental architectural concepts. Our taxonomy aims to empower researchers, engineers, and AI practitioners to systematically analyze the architectural dynamics and balancing strategies employed by these increasingly prevalent AI systems. The exploratory taxonomic classification of selected representative LLM-powered multi-agent systems illustrates its practical utility and reveals potential for future research and development. 6.3% topic match

6.3%

1.1

2024

[111] Improving Automated Code Reviews: Learning from Experience Hong Yi Lin, ..., and Wachiraphan Charoenwet 2024 IEEE/ACM 21st International Conference on Mining Software Repositories (MSR) 2024 - 1 citations - Show abstract - Cite - PDF 6.3% topic match

6.1%

6.6

2024

[112] CYCLE: Learning to Self-Refine the Code Generation Yangruibo Ding, ..., and Baishakhi Ray Proceedings of the ACM on Programming Languages 2024 - 5 citations - Show abstract - Cite - PDF 6.1% topic match

6.1%

18.3

2015

[113] Repairing Programs with Semantic Code Search (T) Yalin Ke, ..., and Yuriy Brun 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) 2015 - 167 citations - Show abstract - Cite 6.1% topic match

6.0%

2.4

2024

[114] A Comparative Review of AI Techniques for Automated Code Generation in Software Development: Advancements, Challenges, and Future Directions A. Odeh, ..., and Abdul Salam Mohammed TEM Journal 2024 - 2 citations - Show abstract - Cite - PDF 6.0% topic match

5.9%

1.4

2024

[115] Rethinking Software Engineering in the Foundation Model Era: From Task-Driven AI Copilots to Goal-Driven AI Pair Programmers Ahmed E. Hassan, ..., and Zhen Ming Jiang ArXiv 2024 - 1 citations - Show abstract - Cite - PDF 5.9% topic match

5.8%

0.2

2020

[116] Making robots useful Gemma K. Alderton Science 2020 - 1 citations - Show abstract - Cite 5.8% topic match

5.8%

8.4

2024

[117] Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering T. Ridnik, ..., and Itamar Friedman ArXiv 2024 - 8 citations - Show abstract - Cite - PDF 5.8% topic match

5.8%

3.3

1987

[118] An architecture for intelligent assistance in software development G. Kaiser and P. Feiler International Conference on Software Engineering 1987 - 126 citations - Show abstract - Cite 5.8% topic match

5.8%

318.1

2023

[119] The Rise and Potential of Large Language Model Based Agents: A Survey Zhiheng Xi, ..., and Tao Gui ArXiv 2023 - 412 citations - Show abstract - Cite - PDF Abstract: For a long time, humanity has pursued artificial intelligence (AI) equivalent to or surpassing the human level, with AI agents considered a promising vehicle for this pursuit. AI agents are artificial entities that sense their environment, make decisions, and take actions. Many efforts have been made to develop intelligent agents, but they mainly focus on advancement in algorithms or training strategies to enhance specific capabilities or performance on particular tasks. Actually, what the community lacks is a general and powerful model to serve as a starting point for designing AI agents that can adapt to diverse scenarios. Due to the versatile capabilities they demonstrate, large language models (LLMs) are regarded as potential sparks for Artificial General Intelligence (AGI), offering hope for building general AI agents. Many researchers have leveraged LLMs as the foundation to build AI agents and have achieved significant progress. In this paper, we perform a comprehensive survey on LLM-based agents. We start by tracing the concept of agents from its philosophical origins to its development in AI, and explain why LLMs are suitable foundations for agents. Building upon this, we present a general framework for LLM-based agents, comprising three main components: brain, perception, and action, and the framework can be tailored for different applications. Subsequently, we explore the extensive applications of LLM-based agents in three aspects: single-agent scenarios, multi-agent scenarios, and human-agent cooperation. Following this, we delve into agent societies, exploring the behavior and personality of LLM-based agents, the social phenomena that emerge from an agent society, and the insights they offer for human society. Finally, we discuss several key topics and open problems within the field. A repository for the related papers at https://github.com/WooooDyy/LLM-Agent-Paper-List. 5.8% topic match

5.7%

10.7

2023

[120] CodeTF: One-stop Transformer Library for State-of-the-art Code LLM Nghi D. Q. Bui, ..., and Steven C. H. Hoi ArXiv 2023 - 17 citations - Show abstract - Cite - PDF 5.7% topic match

5.5%

0.0

2024

[121] Significant Productivity Gains through Programming with Large Language Models Thomas Weber, ..., and Sven Mayer Proceedings of the ACM on Human-Computer Interaction 2024 - 0 citations - Show abstract - Cite - PDF 5.5% topic match

5.5%

0.5

2016

[122] A Runtime Framework for Machine-Augmented Software Design Using Unsupervised Self-Learning Roberto Rodrigues Filho and Barry Porter 2016 IEEE International Conference on Autonomic Computing (ICAC) 2016 - 4 citations - Show abstract - Cite 5.5% topic match

5.4%

1.4

2024

[123] Is Attention All You Need? Toward a Conceptual Model for Social Awareness in Large Language Models Gianmario Voria, ..., and Fabio Palomba Proceedings of the 2024 IEEE/ACM First International Conference on AI Foundation Models and Software Engineering 2024 - 1 citations - Show abstract - Cite - PDF 5.4% topic match

5.4%

15.2

2021

[124] Automatic Program Repair with OpenAI's Codex: Evaluating QuixBugs Julian Aron Prenner and R. Robbes ArXiv 2021 - 48 citations - Show abstract - Cite - PDF 5.4% topic match

5.3%

0.6

2023

[125] LMs: Understanding Code Syntax and Semantics for Code Analysis Wei Ma, ..., and Yang Liu Journal Not Provided 2023 - 1 citations - Show abstract - Cite - PDF 5.3% topic match

5.3%

20.5

2022

[126] Taking Flight with Copilot C. Bird, ..., and Idan Gazit Queue 2022 - 41 citations - Show abstract - Cite - PDF 5.3% topic match

5.2%

19.2

2023

[127] BiasAsker: Measuring the Bias in Conversational AI System Yuxuan Wan, ..., and Michael R. Lyu Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering 2023 - 31 citations - Show abstract - Cite - PDF 5.2% topic match

5.1%

1.0

2023

[128] An Infinity of Pong: A Raspberry Pi Pico W handheld writes its own games Jose Antonio Garcia Peiro IEEE Spectrum 2023 - 2 citations - Show abstract - Cite 5.1% topic match

5.0%

8.0

2023

[129] AceCoder: Utilizing Existing Code to Enhance Code Generation Jia Li, ..., and Zhi Jin Journal Not Provided 2023 - 14 citations - Show abstract - Cite - PDF 5.0% topic match

4.9%

10.2

2023

[130] RLTF: Reinforcement Learning from Unit Test Feedback Jiate Liu, ..., and Deheng Ye ArXiv 2023 - 15 citations - Show abstract - Cite - PDF 4.9% topic match

4.9%

1.5

2023

[131] Large Language Models Should Ask Clarifying Questions to Increase Confidence in Generated Code Jiexi Wu Journal Not Provided 2023 - 2 citations - Show abstract - Cite - PDF 4.9% topic match

4.7%

4.1

2024

[132] On Evaluating the Efficiency of Source Code Generated by LLMs Changan Niu, ..., and Vincent Ng Proceedings of the 2024 IEEE/ACM First International Conference on AI Foundation Models and Software Engineering 2024 - 3 citations - Show abstract - Cite - PDF 4.7% topic match

4.7%

1.3

2024

[133] Enhancing the General Agent Capabilities of Low-Parameter LLMs through Tuning and Multi-Branch Reasoning Qinhao Zhou, ..., and Yongbin Li ArXiv 2024 - 1 citations - Show abstract - Cite - PDF 4.7% topic match

4.6%

0.0

2024

[134] MaPPing Your Model: Assessing the Impact of Adversarial Attacks on LLM-based Programming Assistants John Heibel and Daniel Lowd Journal Not Provided 2024 - 0 citations - Show abstract - Cite - PDF 4.6% topic match

4.5%

0.0

2024

[135] Agentless: Demystifying LLM-based Software Engineering Agents Chun Xia, ..., and Lingming Zhang Journal Not Provided 2024 - 0 citations - Show abstract - Cite - PDF 4.5% topic match

4.4%

0.0

2024

[136] Assessing the Effectiveness and Security Implications of AI Code Generators Maryam Taeb, ..., and Shonda Bernadin Journal of The Colloquium for Information Systems Security Education 2024 - 0 citations - Show abstract - Cite - PDF 4.4% topic match

4.3%

1.3

2009

[137] Evaluating User-Adaptive Systems: Lessons from Experiences with a Personalized Meeting Scheduling Assistant P. Berry, ..., and N. Yorke-Smith Conference on Innovative Applications of Artificial Intelligence 2009 - 20 citations - Show abstract - Cite 4.3% topic match

4.1%

5.4

2024

[138] An Empirical Study on Usage and Perceptions of LLMs in a Software Engineering Project Sanka Rasnayaka, ..., and Ganesh Neelakanta Iyer ArXiv 2024 - 5 citations - Show abstract - Cite - PDF 4.1% topic match

4.1%

None

[139] A Good Novelist Should be a Good Coder: From Language Critics to Automatic Code Generation Brian Munoz Calonge and Mu-sheng Lin Journal Not Provided None - 0 citations - Show abstract - Cite 4.1% topic match

4.0%

0.8

2023

[140] Automatic Unit Test Data Generation and Actor-Critic Reinforcement Learning for Code Synthesis P. Gorinski, ..., and Ignacio Iacobacci ArXiv 2023 - 1 citations - Show abstract - Cite - PDF 4.0% topic match

3.9%

5.2

2000

[141] ADVISOR: A Machine Learning Architecture for Intelligent Tutor Construction J. Beck, ..., and C. Beal Journal Not Provided 2000 - 126 citations - Show abstract - Cite 3.9% topic match

3.8%

80.2

2023

[142] Communicative Agents for Software Development Chen Qian, ..., and Wei Liu ArXiv 2023 - 160 citations - Show abstract - Cite - PDF 3.8% topic match

3.8%

13.5

2021

[143] Trust Enhancement Issues in Program Repair Yannic Noller, ..., and Abhik Roychoudhury 2022 IEEE/ACM 44th International Conference on Software Engineering (ICSE) 2021 - 45 citations - Show abstract - Cite - PDF 3.8% topic match

3.7%

0.0

2023

[144] Harnessing Predictive Modeling and Software Analytics in the Age of LLM-Powered Software Development (Invited Talk) Foutse Khomh Proceedings of the 19th International Conference on Predictive Models and Data Analytics in Software Engineering 2023 - 0 citations - Show abstract - Cite 3.7% topic match

3.7%

44.3

2022

[145] Do Users Write More Insecure Code with AI Assistants? Neil Perry, ..., and D. Boneh Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security 2022 - 95 citations - Show abstract - Cite - PDF 3.7% topic match

3.5%

5.8

2018

[146] Autonomously Reusing Knowledge in Multiagent Reinforcement Learning Felipe Leno da Silva, ..., and Anna Helena Reali Costa International Joint Conference on Artificial Intelligence 2018 - 38 citations - Show abstract - Cite - PDF 3.5% topic match

3.3%

0.9

2023

[147] Evaluating the Usability and Functionality of Intelligent Source Code Completion Assistants: A Comprehensive Review Tilen Hliš, ..., and Luka Pavlič Applied Sciences 2023 - 1 citations - Show abstract - Cite - PDF Abstract: As artificial intelligence advances, source code completion assistants are becoming more advanced and powerful. Existing traditional assistants are no longer up to all the developers’ challenges. Traditional assistants usually present proposals in alphabetically sorted lists, which does not make a developer’s tasks any easier (i.e., they still have to search and filter an appropriate proposal manually). As a possible solution to the presented issue, intelligent assistants that can classify suggestions according to relevance in particular contexts have emerged. Artificial intelligence methods have proven to be successful in solving such problems. Advanced intelligent assistants not only take into account the context of a particular source code but also, more importantly, examine other available projects in detail to extract possible patterns related to particular source code intentions. This is how intelligent assistants try to provide developers with relevant suggestions. By conducting a systematic literature review, we examined the current intelligent assistant landscape. Based on our review, we tested four intelligent assistants and compared them according to their functionality. GitHub Copilot, which stood out, allows suggestions in the form of complete source code sections. One would expect that intelligent assistants, with their outstanding functionalities, would be one of the most popular helpers in a developer’s toolbox. However, through a survey we conducted among practitioners, the results, surprisingly, contradicted this idea. Although intelligent assistants promise high usability, our questionnaires indicate that usability improvements are still needed. However, our research data show that experienced developers value intelligent assistants highly, highlighting their significant utility for the experienced developers group when compared to less experienced individuals. The unexpectedly low net promoter score (NPS) for intelligent code assistants in our study was quite surprising, highlighting a stark contrast between the anticipated impact of these advanced tools and their actual reception among developers. 3.3% topic match

3.2%

15.6

2023

[148] CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules Hung Le, ..., and Shafiq R. Joty ArXiv 2023 - 19 citations - Show abstract - Cite - PDF 3.2% topic match

3.1%

1.3

2006

[149] Taming Heterogeneous Agent Architectures with Aspects Alessandro F. Garcia and C. Lucena Journal Not Provided 2006 - 24 citations - Show abstract - Cite 3.1% topic match

3.0%

9.1

2020

[150] The Interplay of Sampling and Machine Learning for Software Performance Prediction Christian Kaltenecker, ..., and S. Apel IEEE Software 2020 - 43 citations - Show abstract - Cite 3.0% topic match

3.0%

0.0

2024

[151] Code Agents are State of the Art Software Testers Niels Mündler, ..., and Martin T. Vechev ArXiv 2024 - 0 citations - Show abstract - Cite - PDF 3.0% topic match

3.0%

0.0

2023

[152] AI-driven software engineering Josh Mahmood Ali Advances in Engineering Innovation 2023 - 0 citations - Show abstract - Cite - PDF 3.0% topic match

2.9%

10.9

2015

[153] Repairing Programs with Semantic Code Search Yalin Ke and Kathryn T. Stolee Journal Not Provided 2015 - 109 citations - Show abstract - Cite 2.9% topic match

2.7%

24.3

2022

[154] Discovering the Syntax and Strategies of Natural Language Programming with Generative Language Models Ellen Jiang, ..., and Michael Terry Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems 2022 - 65 citations - Show abstract - Cite 2.7% topic match

2.7%

8.9

2024

[155] Embodied LLM Agents Learn to Cooperate in Organized Teams Xudong Guo, ..., and Mengdi Wang ArXiv 2024 - 7 citations - Show abstract - Cite - PDF 2.7% topic match

2.6%

0.0

2017

[156] An Accountability-Driven Organization Programming Technique Stefano Tedeschi Journal Not Provided 2017 - 0 citations - Show abstract - Cite 2.6% topic match

2.6%

34.3

2023

[157] Large Language Models for Software Engineering: Survey and Open Problems Angela Fan, ..., and Jie M. Zhang 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) 2023 - 56 citations - Show abstract - Cite - PDF 2.6% topic match

2.5%

15.5

2023

[158] Understanding the Usability of AI Programming Assistants Jenny T. Liang, ..., and Brad A. Myers ArXiv 2023 - 31 citations - Show abstract - Cite - PDF 2.5% topic match

2.5%

1.5

2023

[159] A Taxonomy for Autonomous LLM-Powered Multi-Agent Architectures Thorsten Händler International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management 2023 - 3 citations - Show abstract - Cite 2.5% topic match

2.4%

1.7

2024

[160] Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration Junyang Wang, ..., and Jitao Sang ArXiv 2024 - 1 citations - Show abstract - Cite - PDF 2.4% topic match

2.3%

4.2

2024

[161] Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence Weize Chen, ..., and Maosong Sun Journal Not Provided 2024 - 2 citations - Show abstract - Cite - PDF 2.3% topic match

2.2%

27.9

2023

[162] AutoAgents: A Framework for Automatic Agent Generation Guangyao Chen, ..., and Yemin Shi ArXiv 2023 - 35 citations - Show abstract - Cite - PDF 2.2% topic match

2.0%

7.7

2016

[163] A Literature Review of Research in Bug Resolution: Tasks, Challenges and Future Directions Zhang Tao, ..., and A. Chan Comput. J. 2016 - 67 citations - Show abstract - Cite 2.0% topic match

2.0%

2.6

2023

[164] Exploring the Problems, their Causes and Solutions of AI Pair Programming: A Study with Practitioners of GitHub Copilot Xiyu Zhou, ..., and Muhammad Waseem Journal Not Provided 2023 - 3 citations - Show abstract - Cite - PDF 2.0% topic match

1.9%

0.0

2000

[165] Intelligent Agents: Software Technology for the new Millennium B. Faltings Journal Not Provided 2000 - 1 citations - Show abstract - Cite Abstract: Why agents? Computing has evolved from the age of calculating machines (1950–60) through information processing (1970–80) to information environments (1990–2000). Information environments, such as the world wide web, provide the basis for autonomous software systems. Examples of such systems are systems for mail and news delivery, document indexing (web crawlers), and consistency maintenance. Autonomous systems have also appeared in robotics, in particular for mobile robots. At the same time, software is becoming complex to the point of being unmanageable. For example, the Microsoft Windows 2000 system contains 35 million lines of code with many bugs. Its release date has been pushed back several times and it is not clear whether it will ever become sufficiently reliable for commercial success. To avoid the difficulties of maintaining consistency in such large software projects, the tendency is to decompose systems into small components which can be understood independently of each other. However, component paradigms are still lacking intuitive metaphors which make component behaviour easy to understand. Both developments – autonomous software and robotic agents as well as software components – are now converging into a single technology: agents . Agents turn software components into proactive processes that are: • autonomous in that they react themselves to observations of their environment without requiring explicit commands, • proactive in that they recognize and react to changes in the environment which present opportunities, • embedded in that their actions respect the real-time constraints imposed by the environment, • heterogeneous in that many different kinds of agents can work together in the same system, and be added or removed without interrupting it. According to the agent vision, complex homogeneous software systems will be replaced by networks of communicating agents. Agents can be written independently as long as they conform to a standard communication language, and they can be integrated or modified even in a running system. Infrastructures for agent-based computing are being developed with very heavy investment from large companies such as SUN and IBM and are beginning to appear on the market (for example, JINI from SUN). 1.9% topic match

1.9%

908.8

2021

[166] Evaluating Large Language Models Trained on Code Mark Chen, ..., and Wojciech Zaremba ArXiv 2021 - 3165 citations - Show abstract - Cite - PDF 1.9% topic match

1.9%

1.0

2024

[167] The Rise of Thinking Machines: A Review of Artificial Intelligence in Contemporary Communication Mohammad Javad Gholami and Taqi Al Abdwani Journal of Business, Communication & Technology 2024 - 1 citations - Show abstract - Cite - PDF 1.9% topic match

1.9%

0.0

2023

[168] Automatic Robotic Development through Collaborative Framework by Large Language Models Zhirong Luan, ..., and Badong Chen 2023 China Automation Congress (CAC) 2023 - 0 citations - Show abstract - Cite - PDF 1.9% topic match

1.8%

3.2

2010

[169] Adinda: a knowledgeable, browser-based IDE A. Deursen, ..., and Anja Guzzi 2010 ACM/IEEE 32nd International Conference on Software Engineering 2010 - 47 citations - Show abstract - Cite - PDF 1.8% topic match

1.8%

0.0

2024

[170] Iterative Student Program Planning using Transformer-Driven Feedback Elijah Rivera, ..., and S. Krishnamurthi Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 2024 - 0 citations - Show abstract - Cite 1.8% topic match

1.8%

26.1

2023

[171] Towards A Unified Agent with Foundation Models Norman Di Palo, ..., and Martin A. Riedmiller ArXiv 2023 - 38 citations - Show abstract - Cite - PDF 1.8% topic match

1.6%

9.0

2022

[172] Security Implications of Large Language Model Code Assistants: A User Study Gustavo Sandoval, ..., and S. Garg ArXiv 2022 - 27 citations - Show abstract - Cite - PDF 1.6% topic match

1.5%

102.7

2023

[173] Reflexion: an autonomous agent with dynamic memory and self-reflection Noah Shinn, ..., and A. Gopinath ArXiv 2023 - 205 citations - Show abstract - Cite - PDF 1.5% topic match

1.4%

9.5

2023

[174] Generate and Pray: Using SALLMS to Evaluate the Security of LLM Generated Code Mohammed Latif Siddiq and Joanna C. S. Santos ArXiv 2023 - 11 citations - Show abstract - Cite - PDF 1.4% topic match

1.4%

0.0

2023

[175] What Writing Assistants Can Learn from Programming IDEs Sergey Titov, ..., and T. Bryksin ArXiv 2023 - 0 citations - Show abstract - Cite - PDF 1.4% topic match

1.3%

9.5

2020

[176] Applications of AI in classical software engineering Marco Barenkamp, ..., and Oliver Thomas AI Perspectives 2020 - 42 citations - Show abstract - Cite - PDF 1.3% topic match

1.3%

0.5

2017

[177] Reinforcement Learning for Argumentation: Describing a PhD Research Sultan Alahmari, ..., and D. Kudenko Journal Not Provided 2017 - 4 citations - Show abstract - Cite 1.3% topic match

1.3%

5.6

2024

[178] Understanding the Weakness of Large Language Model Agents within a Complex Android Environment Mingzhe Xing, ..., and Zhengjin Xiao ArXiv 2024 - 5 citations - Show abstract - Cite - PDF 1.3% topic match

1.3%

0.0

2004

[179] Development of Methodology for Programming Autonomous Agents K. Erol, ..., and Lun Lang Journal Not Provided 2004 - 0 citations - Show abstract - Cite 1.3% topic match

1.2%

0.5

2021

[180] How to trust auto-generated code patches? A developer survey and empirical assessment of existing program repair tools Yannic Noller, ..., and Abhik Roychoudhury ArXiv 2021 - 2 citations - Show abstract - Cite 1.2% topic match

1.1%

0.0

2024

[181] Beyond Accuracy and Robustness Metrics for Large Language Models for Code Daniel Rodríguez-Cárdenas 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) 2024 - 0 citations - Show abstract - Cite - PDF 1.1% topic match

1.1%

1.4

2022

[182] AI-assisted programming: applications, user experiences, and neuro-symbolic techniques (keynote) Sumit Gulwani Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering 2022 - 3 citations - Show abstract - Cite 1.1% topic match

1.0%

0.0

2002

[183] A Multi-Agent Based Framework to Build Intelligent Tutoring Systems E. Costa, ..., and A. Perkusich Journal Not Provided 2002 - 0 citations - Show abstract - Cite 1.0% topic match

0.9%

0.0

2024

[184] Empowering Agile-Based Generative Software Development through Human-AI Teamwork Sai Zhang, ..., and Zhiqiang Zhuang Journal Not Provided 2024 - 0 citations - Show abstract - Cite - PDF 0.9% topic match

0.9%

0.9

2023

[185] Function-constrained Program Synthesis Patrick Hajali and Ignas Budvytis ArXiv 2023 - 1 citations - Show abstract - Cite - PDF Abstract: This work introduces (1) a technique that allows large language models (LLMs) to leverage user-provided code when solving programming tasks and (2) a method to iteratively generate modular sub-functions that can aid future code generation attempts when the initial code generated by the LLM is inadequate. Generating computer programs in general-purpose programming languages like Python poses a challenge for LLMs when instructed to use code provided in the prompt. Code-specific LLMs (e.g., GitHub Copilot, CodeLlama2) can generate code completions in real-time by drawing on all code available in a development environment. However, restricting code-specific LLMs to use only in-context code is not straightforward, as the model is not explicitly instructed to use the user-provided code and users cannot highlight precisely which snippets of code the model should incorporate into its context. Moreover, current systems lack effective recovery methods, forcing users to iteratively re-prompt the model with modified prompts until a sufficient solution is reached. Our method differs from traditional LLM-powered code-generation by constraining code-generation to an explicit function set and enabling recovery from failed attempts through automatically generated sub-functions. When the LLM cannot produce working code, we generate modular sub-functions to aid subsequent attempts at generating functional code. A by-product of our method is a library of reusable sub-functions that can solve related tasks, imitating a software team where efficiency scales with experience. We also introduce a new"half-shot"evaluation paradigm that provides tighter estimates of LLMs' coding abilities compared to traditional zero-shot evaluation. Our proposed evaluation method encourages models to output solutions in a structured format, decreasing syntax errors that can be mistaken for poor coding ability. 0.9% topic match

0.9%

0.6

2017

[186] Towards an IDE to Support Programming as Problem-Solving Nicholas Nelson, ..., and A. Hoek Annual Workshop of the Psychology of Programming Interest Group 2017 - 5 citations - Show abstract - Cite 0.9% topic match

0.9%

0.0

2023

[187] Programming Languages for AI Programing Agents (Invited Talk) Mark Marron Proceedings of the 19th ACM SIGPLAN International Symposium on Dynamic Languages 2023 - 0 citations - Show abstract - Cite Abstract: Over the past decade software development has shifted from a process centered around writing code to a process that increasingly involves composition of external packages and managing the integration of code from other team members. The next decade-plus will be defined by the shift from a process where humans are the central developers of code into one where AI agents, likely Large Language Model (LLM) based, will be the major creators of code and humans will shift to a supervisory role as curators, integrating rich framework-functionality and code developed by AI programming agents. In this new world we must ask ourselves – are programming languages as they exist today fit for purpose and how do they evolve to meet the needs of this future programming model. This talk represents an opinionated take on the question and attempts to outline specific areas of investigation that need to be addressed by the PL community as part of this journey including: What programming language features help/hinder AI agents when understanding and generating code? What programming language features help/hinder human agents when working with an AI Copilot? What programming language tools are needed to empower AI agents in creating grounded and reliable outputs? How can intents be expressed as part of the program representation – examples, constraints, natural language, external documents? How do we empower end-users as part of this transformation? What programming language features are needed to support new AI driven workflows – live coding, interactive requirement gathering, AI TDD? Effectively answering these questions plays a key role in determining if AI driven programming represents a revolution in how software is developed or is limited to being a programming productivity aid for existing development workflows. As such our community should play a central role in understanding this space and leading in the development of this technological transformation! 0.9% topic match

0.8%

13.0

2023

[188] Comparing Software Developers with ChatGPT: An Empirical Investigation N. Nascimento, ..., and Donald D. Cowan ArXiv 2023 - 21 citations - Show abstract - Cite - PDF 0.8% topic match

0.7%

0.5

1992

[189] A Real Time Blackboard Based Architecture P. Lalanda, ..., and J. Haton European Conference on Artificial Intelligence 1992 - 16 citations - Show abstract - Cite 0.7% topic match

0.7%

293.0

2006

[190] google,我,萨娜 方华 https://doi.org/10.1201/b18055-8 2006 - 5565 citations - Show abstract - Cite 0.7% topic match

0.7%

302.0

2022

[191] Competition-level code generation with AlphaCode Yujia Li, ..., and O. Vinyals Science 2022 - 873 citations - Show abstract - Cite - PDF Abstract: Programming is a powerful and ubiquitous problem-solving tool. Systems that can assist programmers or even generate programs themselves could make programming more productive and accessible. Recent transformer-based neural network models show impressive code generation abilities yet still perform poorly on more complex tasks requiring problem-solving skills, such as competitive programming problems. Here, we introduce AlphaCode, a system for code generation that achieved an average ranking in the top 54.3% in simulated evaluations on recent programming competitions on the Codeforces platform. AlphaCode solves problems by generating millions of diverse programs using specially trained transformer-based networks and then filtering and clustering those programs to a maximum of just 10 submissions. This result marks the first time an artificial intelligence system has performed competitively in programming competitions. Description Machine learning systems can program too Computer programming competitions are popular tests among programmers that require critical thinking informed by experience and creating solutions to unforeseen problems, both of which are key aspects of human intelligence but challenging to mimic by machine learning models. Using self-supervised learning and an encoder-decoder transformer architecture, Li et al. developed AlphaCode, a deep-learning model that can achieve approximately human-level performance on the Codeforces platform, which regularly hosts these competitions and attracts numerous participants worldwide (see the Perspective by Kolter). The development of such coding platforms could have a huge impact on programmers’ productivity. It may even change the culture of programming by shifting human work to formulating problems, with machine learning being the main one responsible for generating and executing codes. —YS Modern machine learning systems can achieve average human-level performance in popular competitive programming contests. 0.7% topic match

0.7%

1.8

2007

[192] Automated Continuous Testing of Multi-Agent Systems C. Nguyen, ..., and P. Tonella Journal Not Provided 2007 - 32 citations - Show abstract - Cite 0.7% topic match

0.6%

26.2

2022

[193] Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants Gustavo Sandoval, ..., and Brendan Dolan-Gavitt USENIX Security Symposium 2022 - 62 citations - Show abstract - Cite - PDF 0.6% topic match

0.6%

105.3

2017

[194] A Survey of Machine Learning for Big Code and Naturalness Miltiadis Allamanis, ..., and Charles Sutton ACM Computing Surveys (CSUR) 2017 - 767 citations - Show abstract - Cite - PDF 0.6% topic match

0.6%

3.6

2024

[195] DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning Siyuan Guo, ..., and Jun Wang ArXiv 2024 - 3 citations - Show abstract - Cite - PDF 0.6% topic match

0.5%

0.0

1998

[196] TrIAs { An Architecture for Trainable Information AssistantsMathias Bauer, ..., and denglerg Germanyfbauer Journal Not Provided 1998 - 0 citations - Show abstract - Cite Abstract: Mathias Bauer & Dietmar Dengler DFKI Stuhlsatzenhausweg 3 66123 Saarbr ucken, Germany fbauer,denglerg@dfki.de Abstract Software agents are intended to perform certain tasks on behalf of their users. In many cases, however, the agent's competence is not su cient to produce the desired outcome. This paper presents an approach to cooperative problem solving in which a software agent and its user try to support each other in the achievement of a particular goal. As a side effect the user can extend the agent's capabilities in a programming-by-demonstration dialog, thus enabling it to autonomously perform similar tasks in the future. Introduction Software agents are intended to autonomously perform certain tasks on behalf of their users. In many cases, however, the agent's competence might not be su cient to produce the desired outcome. Instead of simply giving up and leaving the whole task to the user, a much better alternative would be to precisely identify what the cause of the current problem is, communicate it to another agent who can be expected to be able (and willing) to help, and use the results to carry on with achieving the original goal. The user of a system can certainly be expected to have some interest in obtaining a useful response, even at the cost of having to intervene from time to time. As a consequence it seems rational to ask him for help whenever the system gets into trouble. By enhancing the communication between user and agent by a training phase, the agent's capabilities can be extended such that similar problems in the future can be prevented and the problem-solving process becomes more e cient. This paper presents an approach to cooperative problem solving in which a software agent and its user try to support each other in the achievement of a particular goal. By demonstrating the solution of a subproblem, the user extends the agent's capabilities and thus enhances its usefulness for all potential users. As this training mode based on the paradigm of programming Browser 0.5% topic match

0.4%

6172.9

2020

[197] Language Models are Few-Shot Learners Tom B. Brown, ..., and Dario Amodei ArXiv 2020 - 28342 citations - Show abstract - Cite - PDF 0.4% topic match

0.4%

0.8

2017

[198] Advanced techniques for search-based program repair C. Timperley Journal Not Provided 2017 - 6 citations - Show abstract - Cite Abstract: Debugging and repairing software defects costs the global economy hundreds of billions of dollars annually, and accounts for as much as 50% of programmers' time. To tackle the burgeoning expense of repair, researchers have proposed the use of novel techniques to automatically localise and repair such defects. Collectively, these techniques are referred to as automated program repair. Despite promising, early results, recent studies have demonstrated that existing automated program repair techniques are considerably less effective than previously believed. Current approaches are limited either in terms of the number and kinds of bugs they can fix, the size of patches they can produce, or the programs to which they can be applied. To become economically viable, automated program repair needs to overcome all of these limitations. Search-based repair is the only approach to program repair which may be applied to any bug or program, without assuming the existence of formal specifications. Despite its generality, current search-based techniques are restricted; they are either efficient, or capable of fixing multiple-line bugs---no existing technique is both. Furthermore, most techniques rely on the assumption that the material necessary to craft a repair already exists within the faulty program. By using existing code to craft repairs, the size of the search space is vastly reduced, compared to generating code from scratch. However, recent results, which show that almost all repairs generated by a number of search-based techniques can be explained as deletion, lead us to question whether this assumption is valid. In this thesis, we identify the challenges facing search-based program repair, and demonstrate ways of tackling them. We explore if and how the knowledge of candidate patch evaluations can be used to locate the source of bugs. We use software repository mining techniques to discover the form of a better repair model capable of addressing a greater number of bugs. We conduct a theoretical and empirical analysis of existing search algorithms for repair, before demonstrating a more effective alternative, inspired by greedy algorithms. To ensure reproducibility, we propose and use a methodology for conducting high-quality automated program research. Finally, we assess our progress towards solving the challenges of search-based program repair, and reflect on the future of the field. 0.4% topic match

0.4%

1.1

2024

[199] Interactions with Prompt Problems: A New Way to Teach Programming with Large Language Models J. Prather, ..., and Bailey Kimmel ArXiv 2024 - 1 citations - Show abstract - Cite - PDF 0.4% topic match

0.4%

60.7

2014

[200] Code completion with statistical language models Veselin Raychev, ..., and Eran Yahav Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation 2014 - 641 citations - Show abstract - Cite 0.4% topic match

0.4%

1.0

2024

[201] State-Based Dynamic Graph with Breadth First Progression For Autonomous Robots Tushar Chugh, ..., and Jeshwanth Challagundla 2024 IEEE 14th Annual Computing and Communication Workshop and Conference (CCWC) 2024 - 1 citations - Show abstract - Cite Abstract: This paper introduces a novel method for enhancing robotic systems using Large Language Models (LLMs). We focus on leveraging LLMs to substantially improve robots’ decision-making and interaction with their environment. Our proposed framework employs an agent-based approach, where robots utilize LLMs for sophisticated pattern recognition, environmental understanding, and autonomous decision-making. The main innovation of this research is the integration of LLMs into a robotic system, enabling robots to process large volumes of unstructured data, recognize complex patterns, and make informed decisions with increased precision and efficiency. This integration marks a significant leap in robotic cognitive abilities, surpassing the constraints of traditional programming. Our methodology transforms LLMs into dynamic elements within robotic systems, fostering enhanced and responsive interactions with the environment. Robots equipped with LLMs are thus capable of advanced autonomous operations, including navigating complex environments, solving intricate problems, and interacting more naturally with humans. The primary contribution of this work is the creation of an agent-based graph framework, designed to facilitate collaborative problem-solving in robotic systems. This framework consists of multiple agents working collaboratively, spanning from data ingestion to comprehensive world understanding and decision-making. These agents include modules responsible for various operational aspects, such as environmental analysis, data processing, and specialized LLMs for data interpretation and summarization. Positioning LLMs as proactive, inquisitive agents, our approach enables them to actively seek information and efficiently collaborate with other agents to complete tasks. The dynamic nature of this graph search and inter-agent communication model is a considerable innovation in robotics, offering a more integrated and effective approach for robots to tackle complex tasks, thereby enhancing their ability to operate autonomously and intelligently in diverse environments. 0.4% topic match

0.2%

0.3

2010

[202] A decision support system utilizing a semantic agent Radha Shankarmani, ..., and Poorva Kaushil 2010 IEEE International Conference on Software Engineering and Service Sciences 2010 - 4 citations - Show abstract - Cite 0.2% topic match

0.2%

109.4

2022

[203] Expectation vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models Priyan Vaithilingam, ..., and Elena L. Glassman CHI Conference on Human Factors in Computing Systems Extended Abstracts 2022 - 293 citations - Show abstract - Cite 0.2% topic match

0.2%

56.6

2023

[204] AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors in Agents Weize Chen, ..., and Jie Zhou ArXiv 2023 - 113 citations - Show abstract - Cite - PDF 0.2% topic match

0.1%

7.8

2023

[205] How Do Data Analysts Respond to AI Assistance? A Wizard-of-Oz Study Ken Gu, ..., and Tim Althoff Proceedings of the CHI Conference on Human Factors in Computing Systems 2023 - 10 citations - Show abstract - Cite - PDF 0.1% topic match

0.1%

0.0

2017

[206] Problem-Solving Applications in Developer Environments Nicholas Nelson Annual Workshop of the Psychology of Programming Interest Group 2017 - 0 citations - Show abstract - Cite 0.1% topic match

0.1%

82.8

2022

[207] The Robots Are Coming: Exploring the Implications of OpenAI Codex on Introductory Programming James Finnie-Ansley, ..., and J. Prather Proceedings of the 24th Australasian Computing Education Conference 2022 - 238 citations - Show abstract - Cite - PDF 0.1% topic match

0.1%

3.7

2023

[208] How Practitioners Expect Code Completion? Chaozheng Wang, ..., and Yuetang Deng Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering 2023 - 4 citations - Show abstract - Cite 0.1% topic match

0.1%

19.4

2021

[209] Few-Shot Semantic Parsing with Language Models Trained on Code Richard Shin and Benjamin Van Durme North American Chapter of the Association for Computational Linguistics 2021 - 59 citations - Show abstract - Cite - PDF 0.1% topic match

0.0%

1.1

2019

[210] A Case for Backward Compatibility for Human-AI Teams Gagan Bansal, ..., and E. Horvitz ArXiv 2019 - 6 citations - Show abstract - Cite - PDF 0.0% topic match

0.0%

11.8

2021

[211] Can OpenAI Codex and Other Large Language Models Help Us Fix Security Bugs? H. Pearce, ..., and Brendan Dolan-Gavitt ArXiv 2021 - 47 citations - Show abstract - Cite 0.0% topic match

0.0%

2.2

2024

[212] Advancements in software engineering using AI Hazem W. Marar Computer Software and Media Applications 2024 - 2 citations - Show abstract - Cite - PDF 0.0% topic match

0.0%

0.1

1998

[213] Software agents and intelligent object fusion C. Nourani ACM SIGSOFT Softw. Eng. Notes 1998 - 3 citations - Show abstract - Cite - PDF 0.0% topic match

0.0%

1313.5

2018

[214] Improving Language Understanding by Generative Pre-Training Alec Radford and Karthik Narasimhan Journal Not Provided 2018 - 9188 citations - Show abstract - Cite 0.0% topic match

0.0%

7.6

2023

[215] CoAIcoder: Examining the Effectiveness of AI-assisted Human-to-Human Collaboration in Qualitative Analysis Jie Gao, ..., and S. Perrault ACM Transactions on Computer-Human Interaction 2023 - 13 citations - Show abstract - Cite - PDF 0.0% topic match

0.0%

3.1

2022

[216] The Potential of Artificial Intelligence as a Method of Software Developer's Productivity Improvement Ekaterina A. Moroz, ..., and I. M. Novozhilov 2022 Conference of Russian Young Researchers in Electrical and Electronic Engineering (ElConRus) 2022 - 9 citations - Show abstract - Cite 0.0% topic match

0.0%

12.0

2022

[217] AI-Driven Development Is Here: Should You Worry? Neil A. Ernst, ..., and T. Menzies IEEE Software 2022 - 34 citations - Show abstract - Cite - PDF 0.0% topic match

0.0%

3.0

2024

[218] Automatic Programming: Large Language Models and Beyond Michael R. Lyu, ..., and Patanamon Thongtanunam ArXiv 2024 - 2 citations - Show abstract - Cite - PDF 0.0% topic match

0.0%

0.0

2024

[219] Automated Code Review Tool Mangave V. V. INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT 2024 - 0 citations - Show abstract - Cite - PDF 0.0% topic match

0.0%

0.2

2012

[220] Automatic code generation using artificial intelligence. Yuri Danilchenko Journal Not Provided 2012 - 2 citations - Show abstract - Cite 0.0% topic match

Research topic

Report

References

Select the format you want to download:

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS

Download Citations

Citation Text

BibTeX

RIS