Paper page - Empirical Study on the Characteristics and Evolution of AI-usage in GitHub Repositories: Evidence from Code Comments

Source: https://huggingface.co/papers/2606.06843

Abstract

AI tools are increasingly integrated into software development workflows, with developers primarily using LLMs for code implementation and enhancement while maintaining ongoing oversight through refactoring and bug fixes, showing a shift from direct code generation to conceptual support over time.

Developers increasingly useAI toolssuch as ChatGPT, Copilot, and Claude in everyday software workflows, but prior studies often evaluate LLM outputs in isolation rather than examining how developers adapt them in real projects. We analyze 35,361 GitHub code comments that explicitly reference AI use and their associated code blocks. We first open-code 500 unique comments and code blocks to derive a taxonomy of AI-assisted development activities, then annotate the full dataset using two LLM-based classifiers and aggregate predictions with Dawid-Skene expectation-maximization. We also analyze 12,996 subsequent commit messages to study how AI-assisted code evolves after introduction, and examinetemporal trendsfrom December 2022 to March 2026. Our results show that developers primarily useLLMsforcode implementation, followed bycode enhancement,debugging,documentation, andtesting. Subsequent commits frequently involverefactoringand cleanup, feature integration and extension, andbug fixing, indicating sustainedhuman oversightin adapting AI-assisted code. Over time, AI-referencing comments shift from directcode generationtoward knowledge and conceptual support andcode enhancement. These findings suggest thatAI toolsare becoming embedded not only as code-generation aids, but also ascollaborative support mechanismswhose outputs are refined, extended, and corrected by developers over time.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2606\.06843

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2606.06843 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2606.06843 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2606.06843 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.