The LLM layer, in three objects

Earlier in this series we put provider differences behind one adapter, so switching model vendors is a config change. This post solves a different problem, and it is worth being clear about the difference up front.

Part 8 was about portability: the outside world of providers, hidden behind one function. This post is about a boundary inside your own agent: keeping "what to send", "how to send it" and "what came back" as three separate things, so the agent loop never touches an API format directly.

Why the seam is worth building

Without it, the agent's loop reaches straight into the provider's client. Message formatting, tool schema conversion, response parsing and error handling all end up mixed into the loop, and the loop that should read as reason-act-observe reads as plumbing instead.

Three small objects pull that plumbing out.

• LlmRequest packages what to send.
• LlmClient makes the call.
• LlmResponse standardises what comes back.

Each does one job and knows nothing about the others' jobs. The loop deals in these three, never in an API.

Three objects between the loop and the API. The loop deals in these, never in raw formats.

LlmRequest: what to send

LlmRequest is the outbound gate. It holds everything the model needs for one call and nothing else.

class LlmRequest(BaseModel):
    instructions: str                 # the system prompt
    contents: list[ContentItem]       # the conversation so far, flattened
    tools: list[BaseTool]             # the tools available this call
    output_type: type | None = None   # a schema, when structured output is wanted

Two details matter.

tools holds BaseTool instances, the uniform interface from the last post. The client turns them into whatever schema the provider expects, so the request stays provider-agnostic.

Notice what is absent: the ExecutionContext stays out of the request. The request carries a flat list of contents rather than the whole run. Something upstream selects which events become contents and packages them, which keeps LlmRequest a simple data object and keeps context-selection logic in one place. For now that selection flattens every event into the list; a later post on memory makes it smarter, and only that one place changes.

LlmResponse: what came back

LlmResponse is the inbound gate. Whatever the provider returned, the rest of the agent sees this shape.

class LlmResponse(BaseModel):
    content: str = ""                          # the text answer, if any
    tool_calls: list[ToolCall] = []            # the tools it wants, if any
    error_message: str | None = None           # set when the call failed

The value of standardising here is that the agent loop reads response.tool_calls and response.content without caring which provider produced them, and a failed call arrives as a filled error_message rather than a raised exception the loop has to wrap.

LlmClient: how to send it

LlmClient is the only object that touches the API. It takes an LlmRequest, makes the call, and returns an LlmResponse.

class LlmClient:
    def __init__(self, model: str):
        self.model = model

    async def generate(self, request: LlmRequest) -> LlmResponse:
        try:
            messages = self._build_messages(request)          # contents -> API format
            tools = [to_schema(t) for t in request.tools]     # BaseTool -> API schema
            raw = await call_provider(self.model, messages, tools)
            return self._parse_response(raw)                  # API reply -> LlmResponse
        except Exception as e:
            return LlmResponse(error_message=str(e))          # never crash the loop

The generate method does three things in order: build the provider's message format from the request's contents, convert the tools to the provider's schema, make the call, and parse the reply back into an LlmResponse. A failure becomes a response with an error set, so the loop upstream stays simple.

Part 8's portability lives right here: _build_messages, to_schema and _parse_response are the provider-specific corners, all sealed inside this one class.

Putting it together

From the caller's side the three objects read as one clean exchange.

client = LlmClient(model="<your-model>")

request = LlmRequest(
    instructions="You are a support analysis assistant.",
    contents=[Message(role="user", content="How many Q2 tickets were password resets?")],
    tools=[read_tickets, calculate],
)

response = await client.generate(request)
response.tool_calls        # what the model wants to run next

Package, send, receive. Each object does one of the three, which is exactly what lets the agent loop in the next post stay short.

What to take from this

• Part 8 hid provider differences for portability. This layer separates package, send and receive inside your agent, which is a different concern and a different payoff.
• The request carries a flat contents list, not the whole ExecutionContext, so selecting what the model sees stays in one upstream place, ready to get smarter later.
• Failures come back as an LlmResponse with an error set, never as an exception the loop has to catch. The loop reads three clean objects and nothing else.

State, tools and the LLM layer are all in place. The next post assembles them into the agent itself.