An agent is an action system
The important difference between a chatbot and an agent is not the interface. It is authority. An agent can call tools, retrieve data, maintain state, and execute a sequence of decisions. That makes identity, permissions, failure handling, and observability part of the product—not infrastructure details to add later.
Microsoft Foundry Agent Service offers several runtime choices. The useful question is not which option appears most advanced. It is which one exposes enough control for the mission without creating unnecessary operating work.
Three deployment shapes
Prompt agents
Prompt agents are defined through instructions, a model, and tools, while Foundry operates the runtime. They are a strong fit for bounded assistants and workflows that do not require custom orchestration code.
The managed runtime reduces container and scaling work. It does not remove the need for versioned instructions, representative tests, permission design, and production ownership.
Hosted agents
Hosted agents package custom agent code for a Foundry-managed endpoint. They fit workflows that require custom orchestration, framework-specific behavior, multi-agent coordination, or nonstandard interaction protocols.
This flexibility creates more code and dependency responsibility. The organization still owns the logic even when Foundry operates the container environment.
Existing application runtimes
Teams can also keep agent code in an existing application and call Foundry’s Responses API for models and platform tools. This is useful when the application already has a mature deployment, identity, and observability model.
Do not migrate a stable runtime merely to make the architecture diagram look more unified.
Define the operating envelope
An instruction such as “be careful” is not a control. A production envelope must exist outside the model:
- expose only the tools needed for the current task;
- validate tool inputs and outputs with deterministic code;
- preserve the user or workload identity through downstream calls;
- require confirmation for consequential or irreversible actions;
- cap time, tool calls, spend, and the number of records affected;
- make every side effect idempotent or recoverable where possible;
- stop and escalate when confidence, permissions, or dependencies are insufficient.
The system should have less authority than the person it assists, not a shared credential with broader reach.
Design tools as contracts
Tool names and descriptions influence model behavior, but the receiving service remains the security boundary. Give each tool a narrow purpose and a typed schema. Reject ambiguous identifiers, unauthorized fields, and impossible state transitions.
Separate read tools from write tools. Make dry-run or preview operations available for high-impact changes. Return structured errors that allow the agent to recover without inventing a successful outcome.
For a customer-service case, “retrieve the approved order summary” and “request a cancellation for order X” are safer contracts than a generic “execute ERP operation” tool.
Trace decisions, not hidden reasoning
Useful observability captures the request, model and agent version, tool selection, validated arguments, tool outcome, latency, token use, policy decisions, and final result. It should not depend on collecting private chain-of-thought.
Review failed and near-failed traces as product evidence. A trace showing a correct answer after repeated unauthorized tool attempts is not a successful trace.
Human control must be operational
“Human in the loop” is often written without identifying the human, the trigger, or the response time. Define:
- which action requires approval;
- who receives the request;
- what evidence they see;
- how long the system waits;
- what happens on rejection or timeout;
- how an operator stops or rolls back the workflow.
The escalation route must work during an incident, not only in a demo.
The production test
Before granting more autonomy, prove that the agent can complete the bounded task, fail safely, preserve identity, respect limits, and leave an inspectable record. Start with advisory output, add tool use under confirmation, and expand authority only when evaluation and operational evidence justify it.
The best agent architecture is not the one with the most autonomy. It is the one with the minimum authority required to create the outcome.
Official reference
Microsoft describes prompt agents, hosted agents, direct Responses API use, tools, identity, and observability in the Foundry Agent Service overview.