AI-Assisted Programming

← All chapters

Chapter 15 of 17

AI-Assisted Programming

Covers AI coding assistants for code generation, debugging, testing, documentation and review. It weighs the productivity evidence against the security research, explains the pass@k metric, and builds a responsible developer workflow.

In this chapter

1. 15.1 How AI coding assistants work
2. 15.2 Code generation
3. 15.3 Auto-debugging and code explanation
4. 15.4 Testing, documentation, review and refactoring
5. 15.5 Risks and how to manage them
6. 15.6 Designing an AI-assisted developer workflow (CO5)
7. 15.7 Worked numericals
8. 15.8 Industry use cases
9. 15.9 Review questions
10. Knowledge sources

15.1 How AI coding assistants work

Code is text with strict structure, and public code repositories provide huge amounts of training data. Code models are large language models trained or fine-tuned on code. OpenAI's Codex (2021), which powered the first GitHub Copilot, was a GPT model fine-tuned on public code from GitHub.

Modes of use

Mode	How it works	Example tools
Inline completion	Suggests the next lines as you type; accept with Tab	GitHub Copilot in VS Code, JetBrains
Chat in the editor	Ask questions about selected code or the project	Copilot Chat, Gemini Code Assist
General chat assistant	Paste code into a chat model	ChatGPT, Claude, Gemini
Agent mode	The assistant plans and edits several files, runs tests and iterates	Copilot agent mode, Claude Code and similar tools
Code review	AI comments on a pull request	Copilot code review and others

15.2 Code generation

The developer's "prompt" can be a comment, a function name, a docstring or a chat instruction.

Example: comment-driven generation

# Calculate monthly EMI for a loan using the reducing-balance formula.
# Inputs: principal in rupees, annual interest rate in percent, tenure in years.
# Handle zero interest.
def calculate_emi(principal, annual_rate, years):
    n = years * 12
    if annual_rate == 0:
        return principal / n
    r = annual_rate / 12 / 100
    return principal * r * (1 + r) ** n / ((1 + r) ** n - 1)

print(round(calculate_emi(500000, 9, 5), 2))   # 10379.18

The developer must still check the formula and test it. For ₹5,00,000 at 9% for 5 years the EMI is ₹10,379.18, which matches standard bank calculators.

Tips for better code generation

• Give the language, version and libraries: "Python 3.11, pandas only."
• Describe inputs, outputs and edge cases.
• Ask for small functions, not whole applications at once.
• Ask for tests along with the code.
• Keep related files open, since editor assistants use them as context.

15.3 Auto-debugging and code explanation

Debugging with AI

def average(marks):
    return sum(marks) / len(marks)

average([])   # ZeroDivisionError: division by zero

Prompt: "This function crashes with ZeroDivisionError when the list is empty. Explain why and fix it so it returns None for an empty list."

Typical fix:

def average(marks):
    if not marks:
        return None
    return sum(marks) / len(marks)

A good debugging prompt includes: the full error message and stack trace, the relevant code, sample input, expected versus actual output, and what you have already tried.

Explaining code: "Explain this regular expression piece by piece," or "Explain what this legacy COBOL routine does, then suggest an equivalent in Java." Explanation is valuable for onboarding new team members and for maintaining old systems.

15.4 Testing, documentation, review and refactoring

Task	Example prompt
Unit tests	"Write pytest tests for calculate_emi covering normal values, zero interest, one-year tenure and invalid negative principal."
Documentation	"Write docstrings in Google style for all functions in this file, and a README section on usage."
Code review	"Review this function for bugs, security issues, performance and readability. List issues by severity."
Refactoring	"Refactor into smaller functions without changing behaviour. Keep all existing tests passing."
Translation	"Convert this Python data-cleaning script to SQL."

15.5 Risks and how to manage them

Risk	What can go wrong	Control
Incorrect code	Looks plausible but fails on edge cases	Tests, code review, run before merging
Insecure code	SQL injection, weak cryptography, hard-coded secrets	Security scanning tools; secure coding review
Invented APIs	Calls to functions or packages that do not exist	Check documentation; dependency review
Licensing	Suggestions that closely match licensed public code	Duplicate-detection filters; legal policy
Data leakage	Proprietary code sent to an external service	Enterprise plans with data protection; company policy
Skill erosion	Beginners accept code they do not understand	Require explanation; pair programming; manual practice

Research on security. Pearce and colleagues ("Asleep at the Keyboard?", 2021) generated 1,689 programs with early Copilot across scenarios chosen to involve common weaknesses, and found about 40% contained vulnerabilities. Perry and colleagues at Stanford (2022) found that participants with an AI assistant wrote less secure code, yet were more likely to believe their code was secure. Both studies used earlier tools, but the lesson stands: AI-written code needs the same review as human code.

15.6 Designing an AI-assisted developer workflow (CO5)

1. Plan: write the requirement and acceptance criteria yourself; ask AI to point out missing cases.
2. Scaffold: let AI generate boilerplate such as project structure, configuration and data classes.
3. Implement: write core logic with inline suggestions; accept only what you understand.
4. Test: ask AI to generate tests, then add your own for edge cases.
5. Review: run linters and security scanners; ask AI for a review; then a human reviewer approves.
6. Document: generate docstrings and README; edit for accuracy.
7. Govern: use enterprise accounts, do not paste secrets, and record where AI was used if the organisation requires it.

15.7 Worked numericals

Numerical 15.1: Developer time saved.

In the GitHub Copilot experiment by Peng and colleagues (2023), developers with Copilot completed the task 55.8% faster. Suppose a team's comparable tasks take 6 hours each without AI and it does 40 such tasks a month.

Real gains are usually smaller across a whole project, because a controlled task is more self-contained than daily work with meetings, reviews and unclear requirements.

Numerical 15.2: pass@k, the standard code-generation metric.

The HumanEval benchmark (Chen et al., 2021) gives a model programming problems with unit tests. pass@k is the probability that at least one of k generated solutions passes. If n samples are generated and c are correct:

For n = 10 samples with c = 3 correct:

If a developer looks at five suggestions, there is a 91.7% chance at least one works, but the developer must be able to recognise which one.

Numerical 15.3: Suggestion acceptance rate.

In a week Copilot shows a developer 1,200 suggestions; 360 are accepted, and 45 of those are later changed because of bugs.

Teams track both: a high acceptance rate with high rework means developers are accepting code too easily.

15.8 Industry use cases

• GitHub Copilot productivity study. In a controlled experiment, 95 professional developers were asked to write an HTTP server in JavaScript; those with Copilot finished 55.8% faster than the control group (Peng et al., 2023).
• Codex and HumanEval. OpenAI's Codex paper introduced HumanEval, 164 hand-written programming problems with tests. Codex solved 28.8% at pass@1 and 70.2% when 100 samples were drawn per problem. Today's models score far higher, and harder benchmarks such as SWE-bench, built from real GitHub issues, are now used (Chen et al., 2021).
• Security research. The Pearce et al. and Perry et al. studies above are widely cited by companies when setting policies that require security review of AI-generated code.

15.9 Review questions

1. Compare inline completion, chat and agent mode in AI coding assistants.
2. Write a comment-style prompt to generate a function that validates an Indian PAN number format, and list the test cases you would use.
3. Evaluate the risks of AI-generated code. Which controls would you insist on for a banking application?
4. Design a seven-step AI-assisted workflow for building a student attendance web app.
5. With n = 10 samples and c = 2 correct, compute pass@3. (Answer: 1 − 56/120 ≈ 0.533.)
6. A developer accepts 250 of 1,000 suggestions and later reworks 50. Compute the acceptance and rework rates. (Answer: 25%, 20%.)

Knowledge sources

• Peng, S., Kalliamvakou, E., Cihon, P. and Demirer, M., The Impact of AI on Developer Productivity: Evidence from GitHub Copilot, 2023.
• Chen, M. et al., Evaluating Large Language Models Trained on Code (Codex and HumanEval), 2021.
• Pearce, H. et al., Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions, IEEE S&P 2022.
• Perry, N. et al., Do Users Write More Insecure Code with AI Assistants?, ACM CCS 2023.
• Jimenez, C. E. et al., SWE-bench: Can Language Models Resolve Real-World GitHub Issues?, ICLR 2024.
• GitHub, GitHub Copilot documentation.

← Chapter 14☰ All chaptersChapter 16 →

Independent study notes for learning purposes. Not affiliated with any university or course provider. Facts and case-study figures are linked to their original sources. ChatGPT, Gemini, Copilot, Jasper, Synthesia, Runway, Lumen5 and other product names are trademarks of their respective owners.