Mocking external calls

Why a real LLM API call has no place in a test

An agent backend’s route often calls something genuinely external — an LLM API, another service. Testing that route by letting it make a real call has three real problems: it’s slow (a network round-trip on every single test run, for every test that touches this route), it costs real money (an actual LLM API call, every time the test suite runs — including in CI, potentially dozens of times a day), and it’s not deterministic (a real LLM’s response can vary, and the network itself can simply fail, unrelated to whether your code is actually correct).

unittest.mock — replacing the real call with a controlled fake

unittest.mock.patch temporarily replaces a function (or object) with a fake one, for the duration of a test, then automatically restores the real one afterward:

# main.py
async def call_llm_api(prompt: str) -> str:
    # a real network call, in production
    ...

@app.post("/generate")
async def generate(prompt: str):
    result = await call_llm_api(prompt)
    return {"result": result}

# test_main.py
from unittest.mock import patch, AsyncMock

@patch("main.call_llm_api", new_callable=AsyncMock)
def test_generate_returns_llm_result(mock_call_llm_api, client):
    mock_call_llm_api.return_value = "mocked response"

    response = client.post("/generate", json={"prompt": "hello"})

    assert response.status_code == 200
    assert response.json() == {"result": "mocked response"}
    mock_call_llm_api.assert_called_once_with("hello")

pytest

click Run to see this pane's output

Not a single real network request happened — call_llm_api never ran its real body at all, yet generate still executed for real, including the part that turns whatever call_llm_api produces into the actual response.

@patch("main.call_llm_api", new_callable=AsyncMock) replaces call_llm_api, specifically as it’s referenced inside main.py, with a mock for the duration of this test — AsyncMock specifically, since the real function is async def, and the mock needs to be await-able the same way. Setting mock_call_llm_api.return_value controls exactly what the mock produces when awaited, with no real network call happening at all. mock_call_llm_api.assert_called_once_with("hello") additionally confirms the route actually called it correctly — with the right argument, exactly once — which is a check a real network call couldn’t give you nearly as precisely.

Mocking a failure, not just a success

Mocking is just as useful for testing how a route handles the external call going wrong — something genuinely hard to trigger reliably against a real API on demand, but trivial against a mock:

@patch("main.call_llm_api", new_callable=AsyncMock)
def test_generate_handles_llm_failure(mock_call_llm_api, client):
    mock_call_llm_api.side_effect = ConnectionError("LLM API unreachable")

    response = client.post("/generate", json={"prompt": "hello"})

    assert response.status_code == 503

pytest

click Run to see this pane's output

Triggering this reliably against a real LLM API would mean actually taking the network down, or hoping for a flaky moment — against a mock, it’s one line (side_effect = ConnectionError(...)), on demand, every single run.

side_effect (rather than return_value) makes the mock raise that exception when called, standing in for a real network failure — assuming generate itself catches ConnectionError and translates it into a proper 503 Service Unavailable response, exactly the kind of error-path behavior Concept 5’s discipline says deserves its own explicit test.

This is what keeps a test a unit test

Recall the unit-vs-integration distinction from earlier in this lesson: mocking is the specific tool that keeps a test from silently becoming something else. Without mocking call_llm_api, test_generate_returns_llm_result would actually be an integration test against a real external service — slow, costly, and non-deterministic, exactly the three problems named at the top of this concept. Mocking cuts out the one genuinely external dependency, leaving a test that still exercises the route’s own logic (does it call the LLM function correctly, does it shape the response correctly, does it handle a failure correctly) without depending on anything outside your own code actually running.

Check your understanding

1/5

What are the three concrete problems with a test that makes a real LLM API call on every run?

Exercise · Graded (real unittest.mock, in-browser)

Given the route below (already written, in main.py -- read-only), write test_main.py with two test functions: test_generate_success, mocking call_llm_api to return "mocked response" and asserting a 200 with the correct body; and test_generate_llm_failure, mocking call_llm_api to raise a ConnectionError and asserting a 503. Both mocked calls actually run against the real route above, so a correct mock plus a correct assertion is what makes each test pass.