What is a system prompt?
A system prompt is the set of instructions an application places at the start of every conversation with a language model, before any user message. It defines who the model is acting as, what it is allowed to do, how it should respond, which tools it may call, and what it must refuse. Users do not see it and cannot normally change it. Every reply the model gives is shaped by it.
In a customer-facing assistant, the system prompt might state the company name, the products covered, the tone, the languages supported, the rule to cite retrieved documents, and the instruction to hand off to a person for billing disputes. In an AI agent, it also describes the available tools, when to use each, and which actions require approval.
Why a system prompt matters
The system prompt is where business rules meet the model. It is the fastest place to change behavior and the first place to look when behavior is wrong. It also carries much of the burden of scope and safety, because a model with a clear, specific system prompt is harder to pull off task than one with a vague one. It remains only text, however. Prompt injection attempts aim at overriding it, so hard limits such as which tools exist and which records a user may read must be enforced in code and access control, with the system prompt as a second layer.
In FlatClaw, the open-source private AI platform from Kirk Tech Solutions, each agent’s system prompt is managed as configuration, separately from the access controls that decide which tools and records the agent can reach.
System prompts in practice
A regional retailer runs an order-status assistant. Its system prompt sets the persona and tone, restricts answers to orders the authenticated customer owns, lists the order lookup and return-label tools, forbids discussing promotions not in the retrieved policy, and directs anything involving a chargeback to a human agent. When the returns window changed, the policy document was updated and the system prompt stayed the same, because the prompt said to answer returns questions from the policy rather than restating the policy itself.
What to watch for
- Keep facts that change, such as policies and prices, in retrieved documents rather than in the system prompt.
- Assume the system prompt can be extracted by a determined user and never put secrets or credentials in it.
- Test the system prompt against adversarial inputs, because instructions that read clearly to a person may be easy to argue the model out of.
- Version every change and re-run the evaluation set, since a one-line edit can alter behavior across every conversation.