AI Assistants

An assistant is a conversation loop wrapped around some combination of knowledge (RAG), actions (tools) and rules (system prompt plus guardrails). What separates a harmless FAQ bot from a coding agent that runs shell commands is not the model; it is where the product sits on four axes.

Fix the axes first and the architecture mostly follows. A read-only, low-latency assistant over a stable corpus is RAG plus one model call. A high-risk, tool-rich assistant on a slow task is an agent loop with approval gates and a trace for every run.

A general conversational assistant has broad knowledge, few or no tools, low risk per answer and strict latency. Architecture: system prompt, conversation history with a rolling summary, optional web search tool, output guardrails. The hard part is consistency at scale — the same question asked 10,000 ways should get the same policy-compliant answer — and knowing when to say "I don't know".

A customer-support assistant narrows knowledge to a product corpus and adds tools that touch real accounts. Knowledge: RAG over help articles and policies with metadata filters per product and plan. Tools: get_order, get_subscription (read), issue_refund, cancel_plan (write, gated). Risk is moderate to high because the bot speaks for the company. Latency: a few seconds is fine; users are used to waiting for support.

A research assistant gathers and synthesizes: search, fetch, read, compare, cite. Knowledge is live and untrusted (the open web); tools are read-only; risk is mostly fabrication; latency is relaxed, minutes are acceptable. This is a genuine agent: the number of searches depends on what the first ones return. Hard parts: deciding when enough evidence has been collected, deduplicating sources, and citations that really support the claim — see Citations and Citations in the challenges.

A coding assistant operates on a repository with tools like read_file, search, edit_file, run_tests, run_command. Knowledge is the codebase (often retrieved on demand rather than pre-indexed). Risk ranges from trivial (suggesting a diff) to severe (running rm -rf in a real environment). Latency tolerance is high for autonomous tasks, low for inline completion. Hard parts: sandboxing, keeping the context window from filling with file contents, and verifying work with tests rather than trusting the model's claim of success.

The "ask anything about our company" assistant combines RAG over wikis, tickets and docs with tools into HR, CRM and engineering systems. It looks like the support bot but the knowledge axis is dominated by permissions: an engineer must not retrieve the CEO's compensation memo because it embedded well. Every retrieval must be filtered by the caller's identity before ranking, not after — see Metadata Filtering and Permissions, Authentication and Authorisation.

Latency expectations are chat-like, corpora are messy (Confluence exports, PDFs, slide decks), and freshness matters: yesterday's org chart is wrong. Hard parts: ingestion quality, access control, and stale content that ranks well. A modest scope with clean data beats company-wide coverage with 40% stale hits.