Prompting LLMs for Code: Proven Patterns for Unit Tests and Refactors

You’ve probably been there. You paste a function into your AI assistant, ask it to write unit tests, and get back something that looks right but fails half the time. Or you ask for a refactor, and the model breaks a dependency you didn’t even know existed. It’s frustrating because the model is smart enough to understand Python or JavaScript, yet it keeps missing the specific constraints of your project.

The problem isn’t usually the model’s intelligence; it’s how we talk to it. Most developers treat Large Language Models (LLMs) like magic boxes, hoping a vague request yields a perfect result. But recent research shows that structured prompting drastically improves reliability. Specifically, when it comes to generating unit tests and performing refactors, certain patterns consistently outperform others. If you want fewer iterations and more passing tests, you need to stop guessing and start using proven prompt architectures.

Why Generic Prompts Fail on Code Tasks

LLMs don’t have common sense in the way humans do. They predict the next token based on patterns learned from billions of lines of code. When you ask, "Write tests for this," the model guesses what a typical test looks like. It doesn’t know your edge cases, your mocking strategy, or your assertion library unless you tell it.

This gap leads to hallucinated APIs or incorrect logic. A study involving professional programmers revealed that while many use iterative chatting to fix errors, they often lack a structured starting point. This results in high token usage and increased latency because you’re essentially teaching the model from scratch every time. By shifting from conversational guessing to structured instruction, you can reduce the number of back-and-forth interactions significantly.

The Core Pattern: Context and Instruction

The most effective pattern identified in recent DevGPT dataset analyses is the "Context and Instruction" framework. This isn’t just about adding more words; it’s about providing the right kind of information. Think of it as setting the stage before asking for the performance.

For unit tests, this means explicitly defining the input/output specifications. Don’t just paste the function. Provide the signature, the docstring, and crucially, the pre-conditions and post-conditions. If a function expects a non-null string, say so. If it throws a specific exception on empty input, mention it. This reduces ambiguity, which is the primary cause of failed generations.

Here’s a breakdown of what to include in your context block:

• Language and Framework: Specify Python 3.11 with pytest, not just "Python."
• Dependencies: List external libraries (e.g., NumPy, Pandas) and their versions if critical.
• Existing Code: Paste the target function clearly separated by delimiters.
• Constraints: State rules like "Use parameterized tests" or "Mock all database calls."

Pattern for Unit Tests: The Recipe Approach

When writing unit tests, the "Recipe" pattern proves exceptionally effective. Instead of asking for tests generally, provide a concrete example of what a good test looks like in your codebase. This is known as few-shot prompting, where one or two examples guide the model’s style and structure.

Imagine you have a utility function for calculating tax. A generic prompt might yield tests that miss boundary conditions. A recipe-style prompt would look like this:

By anchoring the model to a specific format, you ensure consistency across your test suite. Research indicates that including implementation details-like specific assertion methods-helps the model avoid inventing its own syntax that might not match your testing framework.

Pattern for Refactoring: Clarify Intent and Scope

Refactoring is riskier than test generation because it changes existing behavior. A bad refactor can break production silently. The key here is clarity on scope and intent. Vague prompts like "Clean up this code" are dangerous. The model might optimize for speed when you wanted readability, or vice versa.

Use the "Goal-Oriented" pattern. Explicitly state what should change and what must remain constant. For example:

1. Objective: Extract the validation logic into a separate class.
2. Constraint: Do not change the public API signature of the main service.
3. Style: Maintain existing variable naming conventions.
4. Output: Provide the new class definition and the modified service method.

This structure prevents the model from making sweeping changes that aren’t asked for. It also helps in debugging if the output is wrong, because you can trace which constraint was violated.

Handling Ambiguity with Pre- and Post-Conditions

One of the top ten guidelines derived from automated prompt optimization studies is the explicit definition of pre-conditions and post-conditions. In code, these are the contracts between functions. LLMs struggle to infer these implicitly.

For instance, if a function modifies a list in place versus returning a new list, the difference matters for testing. If you don’t specify, the model might assume immutability and write tests that fail against mutable code. Always define the state before and after execution in your prompt.

Consider this comparison of prompt quality:

The data suggests that moving from vague to structured prompts reduces the failure rate of generated code significantly. The criterion for success in rigorous testing was that at least one of ten runs produced passing tests. Structured prompts hit this mark far more reliably than conversational ones.

Avoiding Chain-of-Thought Pitfalls

You might have heard of Chain-of-Thought (CoT) prompting, where you ask the model to think step-by-step. While useful for complex math, CoT can be inefficient for straightforward code tasks. It increases token usage and inference latency without always improving accuracy for simple refactors or standard unit tests.

Instead of long reasoning chains, focus on precise specification. A single, well-crafted prompt that includes context, examples, and constraints is often superior to a multi-turn dialogue. This approach lowers computational costs and speeds up your workflow. Save CoT for algorithmic problems where the logic itself is the hurdle, not the syntax.

Practical Checklist for Your Next Prompt

Before hitting send, run through this quick check. It covers the essential elements that researchers found most impactful for code generation reliability.

• Is the language version specified? (e.g., Java 17 vs. Java 8)
• Are dependencies listed? (e.g., JUnit 5, Mockito)
• Did you provide an example? (A snippet of desired output style)
• Are edge cases defined? (Nulls, empties, large numbers)
• Is the output format clear? (Single code block, separate files, etc.)

If you skip any of these, expect to spend extra time editing the result. The goal is to make the first draft usable, not just plausible.

Frequently Asked Questions