Coding · Read + Write

Test case generation

Augment Rep · High Reason · High

The problem

Engineers write code; tests come last (if at all). Coverage gaps live for months. QA writes manual tests reactively, after a bug ships.

The AI approach

LLM reads the function spec + signature, generates unit tests covering happy path + edge cases, including property-based tests where applicable.

The outcome

~80% coverage

Coverage from ~50% → ~80% on new functions. Engineers review and adopt; tests catch regressions before they ship.

Try itInput → Process → Output

Input — function spec
apply_promocheckout/promo.py
def apply_promo(code: str, cart: Cart) -> Cart: """ Look up promo code (case-insensitive); apply discount. Returns cart unchanged if code is invalid or expired. Discount.amount is non-negative. """ discount = lookup(code.upper()) if discount is None: return cart cart.total -= discount.amount return cart
Process — AI pipeline
1Parse function signature + specReadGenerative
2Identify edge casesReadPredictive
3Generate test casesWriteGenerative
4Run + report failing assertionsRulesSymbolic
Output — generated test suite
Click Run demo to generate happy-path and edge-case tests.
9 tests generated happy · edge · property
happytest_apply_valid_promo
assert apply_promo("SAVE10", cart_100).total == 90
happytest_lowercase_input_normalised
assert apply_promo("save10", cart_100).total == 90
edgetest_invalid_code_returns_unchanged
assert apply_promo("BOGUS", cart_100).total == 100
edgetest_expired_promo
with patched_now(after_expiry): assert apply_promo("Q1ONLY", c).total == c.total
edgetest_zero_amount_discount
assert apply_promo("ZERO", c100).total == 100
propertytest_total_never_negative (Hypothesis)
@given(st.text(), carts()) — assert result.total >= 0
Coverage: +28% on this module. 4 edge cases not in current suite. Engineer reviews + commits.

Three AI types in this use case

SymbolicTest runner (pytest / jest); coverage tool; CI gate that requires generated tests to pass.
PredictiveEdge-case predictor (off-by-one boundaries, type-mismatch, null inputs) trained on bug history.
GenerativeLLM reads signature + docstring, generates pytest cases including property-based tests via Hypothesis.

The stack

  • AST parser · Tree-sitter / ast
  • LLM · Claude / GPT-4
  • Runner · pytest / jest
  • Property · Hypothesis

When this works

  • Functions have clear interfaces (typed signatures, docstrings)
  • Existing test suite to pattern-match against
  • Engineers will review — not blind-trust

When it fails

  • Implicit dependencies (DB, network) — generated tests are over-mocked
  • Tests test the implementation not the spec — refactor breaks them
  • Quantity over quality — 200 trivial tests, 0 useful
  • Auto-merge: AI tests pass, but the test was wrong