LLM 104: Agents, Tools, Memory, and Guardrails

In LLM 103, we treated Retrieval-Augmented Generation (RAG) as an evidence pipeline. The model receives a question, the system retrieves relevant documents, and the model answers from those sources. That is already more than a plain chatbot.

This post goes one step further: LLM agents. An agent is an LLM-powered system that can choose actions, call tools, observe results, keep state, and continue working toward a goal. Agents can be useful, but they are also easy to overbuild. Most problems do not need a fully autonomous loop. They need a small, reliable workflow.

1. What Makes Something an Agent?

An ordinary LLM call takes an input and returns an output. An agent has a loop. It can decide what to do next, call a tool, inspect the result, update its plan, and continue.

The five cards above are not a one-shot sequence. In practice they form a loop: after observing a result, the agent revises its plan and acts again until the goal is met.

┌─────────────────────────┐
│           GOAL          │
└────────────┬────────────┘
             │
             ▼
       ┌───────────┐
┌────▶│   PLAN     │
│      └─────┬─────┘
│            │
│            ▼
│      ┌───────────┐
│      │    ACT    │─── call tool, write, search
│      └─────┬─────┘
│            │
│            ▼
│      ┌───────────┐
│      │  OBSERVE  │─── read result
│      └─────┬─────┘
│            │
│            ▼
│      ┌───────────┐
└──────┤  REVISE   │─── done? continue? ask human?
       └───────────┘

This loop is powerful because it lets the model interact with the world instead of only describing it. It is risky for the same reason. A bad answer is one thing. A bad action can change files, send messages, spend money, leak data, or corrupt a workflow.

2. Tools Are Where Agents Become Useful

Tools let the system move work out of language generation and into reliable operations. A calculator should calculate. A database should query. A search engine should search. A code runner should execute tests. The LLM’s job is often to decide which tool is needed and how to use the result.

A useful tool call is structured. The model should not vaguely say “I searched.” The system should record the tool name, arguments, result, and whether the result was used in the final answer.

3. Planning Is Useful, but Fragile

Many agent demos start with a big goal and ask the model to make a plan. Planning helps when a task has dependencies: gather sources before writing, run tests before reporting, inspect data before modeling.

But LLM plans can be brittle. They can skip necessary steps, invent progress, or keep looping after the useful work is done. Planning works better when the system has clear stopping conditions and observable state.

For complex tasks, the agent should not merely plan. It should check whether each step has actually been completed.

4. Memory Is Not One Thing

Agent systems often talk about memory, but there are several different kinds.

Context memory disappears when it leaves the context window. Working memory is task-specific. Long-term memory persists, which means it needs stronger rules: what is saved, who can see it, when it expires, and how it can be corrected.

Memory can make agents more useful, but sloppy memory can make them worse. If the system saves bad assumptions, private information, or stale preferences, it will retrieve them later with undeserved authority.

5. Single-Agent vs Multi-Agent Systems

A single-agent system has one model-driven loop. A multi-agent system creates several roles, such as planner, researcher, coder, reviewer, or critic. These roles can be useful when they create real checks and division of labor.

They can also become theater. If three agents are all the same model with different names and no tools, tests, or external evidence, the system may only produce more confident conversation.

Multi-agent systems are most defensible when separation creates a concrete benefit: one agent writes code, another runs tests, another reviews the diff; one retrieves sources, another drafts, another checks citation support.

6. Guardrails: The Boring Part That Matters

An agent with tools needs boundaries. Guardrails are not just content filters. They are system constraints that make bad actions harder.

Useful guardrails include:

• tool permissions;
• read-only modes;
• approval gates for risky actions;
• input validation;
• output schemas;
• rate limits and budgets;
• audit logs;
• rollback plans;
• restricted memory access;
• human review for high-impact decisions.

The best agent systems make safe actions easy and risky actions explicit. If a tool can send an email, move money, delete files, or change public content, the agent should not casually call it in an unreviewed loop.

7. Evaluation for Agents

Evaluating an agent is harder than evaluating a single answer. You need to inspect the path, not only the final output.

Ask:

• Did the agent choose appropriate tools?
• Did it use tool results correctly?
• Did it stop when the task was done?
• Did it ask for help when blocked?
• Did it avoid actions outside its permission scope?
• Could a human audit what happened?
• Did the workflow improve over a simpler non-agent baseline?

Agent evaluation should include success rate, cost, latency, number of tool calls, failure recovery, and human review burden. A system that succeeds 5 percent more often but costs 10 times as much and creates unreadable traces may not be an improvement.

8. When Not to Build an Agent

Not every LLM workflow needs agency. Many useful systems are deliberately boring:

• a summarizer with a carefully designed prompt;
• a RAG assistant with citations;
• a script that extracts fields from documents;
• a classifier with human review;
• a dashboard that uses an LLM only to explain selected results.

Build an agent when the task truly requires multiple steps, tool use, state tracking, and adaptive decisions. Avoid an agent when the task is stable, repeatable, and better expressed as a deterministic pipeline.

9. A Practical Agent Checklist

Before building an agent, I would write down:

1. Goal: What outcome counts as done?
2. Tools: What actions can the system take?
3. Permissions: Which actions require approval?
4. State: What must be remembered during the task?
5. Memory: What, if anything, persists afterward?
6. Stopping rule: When should the loop end?
7. Fallback: When should it ask a human?
8. Evaluation: How will success, cost, and risk be measured?
9. Audit: Can someone reconstruct what happened?

In the next post, we can focus on evaluation: benchmarks, human review, automated checks, hallucination tests, and how to measure whether an LLM system is actually useful.