feat: add deployment-wide template allowlist for chats#23262
Conversation
c2a90b6 to
72dc92d
Compare
Documentation CheckUpdates Needed
Automated review via Coder Tasks |
There was a problem hiding this comment.
Pull request overview
Adds a deployment-wide “template allowlist” for Agents chats, exposing it via experimental API endpoints and enforcing it in chat workspace-provisioning tools, with an accompanying Admin UI settings tab.
Changes:
- Store and manage a deployment-wide
agents_template_allowlistvalue insite_configs, withGET/PUT /api/experimental/chats/config/template-allowlist. - Enforce the allowlist in chat tools (
list_templates,read_template,create_workspace) when configured. - Add an Agents UI “Templates” admin settings section with Storybook coverage.
Reviewed changes
Copilot reviewed 23 out of 23 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| site/src/pages/AgentsPage/components/Sidebar/AgentsSidebar.tsx | Adds “Templates” admin nav item in Agents settings sidebar. |
| site/src/pages/AgentsPage/AgentSettingsPageView.tsx | Adds Templates allowlist settings UI (multi-select + save). |
| site/src/pages/AgentsPage/AgentSettingsPageView.stories.tsx | Adds story to exercise allowlist UI interactions. |
| site/src/api/typesGenerated.ts | Adds ChatTemplateAllowlist TS type. |
| site/src/api/queries/chats.ts | Adds query + mutation wrappers for template allowlist config. |
| site/src/api/api.ts | Adds experimental API client methods for allowlist endpoints. |
| codersdk/chats.go | Adds SDK type + ExperimentalClient methods for allowlist endpoints. |
| coderd/x/chatd/chattool/*.go | Adds allowlist plumbing and enforcement in template/workspace tools. |
| coderd/x/chatd/chatd.go | Loads deployment allowlist and injects into chat tools. |
| coderd/exp_chats.go | Implements GET/PUT allowlist HTTP handlers + parsing helper. |
| coderd/exp_chats_test.go | Adds API tests for allowlist behavior + authz + validation. |
| coderd/database/queries/siteconfig.sql | Adds sqlc queries for get/upsert allowlist in site_configs. |
| coderd/database/queries.sql.go | Generated sqlc output for new queries. |
| coderd/database/querier.go | Adds new methods to sqlc querier interface. |
| coderd/database/dbmock/dbmock.go | Adds mock methods for new store calls. |
| coderd/database/dbmetrics/querymetrics.go | Adds metrics wrappers for new queries. |
| coderd/database/dbauthz/dbauthz.go | Adds RBAC checks for new store methods. |
| coderd/database/dbauthz/dbauthz_test.go | Adds RBAC test coverage for new store methods. |
| coderd/coderd.go | Wires new config endpoints into router. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| beforeEach: () => { | ||
| // Track saved allowlist state across mock calls so the | ||
| // refetch after save returns the updated value. | ||
| let savedIDs: string[] = []; | ||
|
|
||
| spyOn(API, "getTemplates").mockResolvedValue(manyTemplates); | ||
| spyOn(API.experimental, "getChatTemplateAllowlist").mockImplementation( | ||
| async () => ({ template_ids: savedIDs }), | ||
| ); | ||
| spyOn(API.experimental, "updateChatTemplateAllowlist").mockImplementation( | ||
| async (req) => { | ||
| savedIDs = [...req.template_ids]; | ||
| }, | ||
| ); |
There was a problem hiding this comment.
This story's beforeEach calls spyOn(API.getTemplates) and spyOn(API.experimental.getChatTemplateAllowlist/updateChatTemplateAllowlist), but those same methods are already spied in the meta-level beforeEach. In Vitest/Jest, spying on an already-spied method typically throws and will break the story. Instead, reuse the existing spies by setting API.getTemplates.mockResolvedValue(...) / API.experimental.getChatTemplateAllowlist.mockImplementation(...), or remove these spies from the meta-level setup and only define them in the stories that need them.
There was a problem hiding this comment.
This is actually fine in Storybook — spyOn from storybook/test restores spies between stories, so re-spying in a story-level beforeEach is the established pattern in this file (see DefaultAutostopCustomValue, UsageDateFilterRefetchOverlay, etc.). The meta-level spy sets a default, and the story-level spyOn overrides it for that story's specific needs. No double-spy issue in practice.
🤖 This response was generated by Coder Agents.
mafredri
left a comment
There was a problem hiding this comment.
🤖 Agent on behalf of mafredri
Clean feature: allowlist enforcement at all three tool boundaries is thorough, the isTemplateAllowed helper is well-factored, and the deliberate StartWorkspace exclusion with its comment is a good design call. The UUID canonicalization and dedup on PUT, the opaque "not found" error for blocked templates, and the Storybook stateful mock are all nice touches.
Note on architecture: several panel reviewers flagged the allowlist as "loaded once per session" (hours of staleness). I verified this is incorrect — runChat is called per message turn (each processChat → runChat cycle re-reads from DB), so allowlist changes propagate on the next message. The staleness window is one LLM turn, not the chat lifetime. This is consistent with all other config loading and is fine.
A couple things to look at: 2 P2s, 2 P3s, 7 nits across 11 inline comments.
Sixteen reviewers, one allowlist, two JSON parsers. One of them even logs errors. Skål!
mafredri
left a comment
There was a problem hiding this comment.
🤖 Agent on behalf of mafredri
Thanks for addressing all 11 findings — the test restructuring, corruption handling, existence validation, and doc comments are all solid. The useMemo and type predicate removals are clean.
One problem: the new template existence validation in PUT broke 3 existing API tests. 1 P1 inline.
Ten findings resolved, one JSON parser reformed, and three tests quietly waiting for templates that don't exist. Hyvää työtä otherwise!
c06d454 to
6029863
Compare
mafredri
left a comment
There was a problem hiding this comment.
🤖 Agent on behalf of mafredri
All findings from rounds 1 and 2 are resolved. Tests use real templates, the chatd asymmetry comment is clear, and the new NonexistentTemplateRejected subtest covers the validation path.
Three rounds, sixteen reviewers, zero surviving findings. The allowlist is allowed to ship. Tack!
ad06568 to
0b51ef7
Compare
0050444 to
973b2a4
Compare
973b2a4 to
c239761
Compare
Adds the ability for deployment admins to restrict which templates the list_templates, read_template, and create_workspace chat tools expose. The allowlist is stored in site_configs and configured via the Agents settings UI. An empty allowlist means all templates are available.
…ools address review comments - Use map[uuid.UUID]bool for allowlist type (comment 1) - Return generic 'template not found' for blocked templates (comments 2, 4) - Consolidate tool tests to single DB (comment 3) - Use AsChatd ctx instead of AsSystemRestricted (comment 5) - Require ActionRead on ResourceDeploymentConfig for GET (comment 6) - Consolidate API tests to single coderdtest instance (comment 7)
… chat tools address self-review findings - Log error and fail-open when allowlist DB read fails (P1) - Add isDirty guard to prevent spurious saves on Enter (P1) - Add error state with retry for failed data fetch (P1) - Deduplicate template IDs in PUT handler (P2) - Add role=alert to save error message (P2) - Add aria-live to selection count (P2) - Disable checkboxes during save (P3) - Differentiate empty state messages (P3) - Add CreateWorkspace allowlist test (P3) - Document StartWorkspace intentionally not gated (P4)
…ist for chat tools fixup! fixup! fixup! feat(chatd): add deployment-wide template allowlist for chat tools Review round 2 fixes: - chatd: log JSON parse errors and invalid UUIDs in allowlist (P2 correctness) - PUT handler: canonicalize UUIDs to lowercase for case-insensitive dedup (P3 correctness) - Sidebar: fix indentation corruption, remove stray 'replace' prop (P3 UX) - Settings: always render aria-live region for screen readers (P2 a11y) - Settings: add role=group + aria-label to checkbox list (P2 a11y) - Settings: rename 'Clear' to 'Deselect All' for clarity (P2 UX)
… allowlist Swap the custom checkbox list + SearchField for the design system's MultiSelectCombobox component. This gives us consistent styling, built-in search/filter, badge display for selections, keyboard navigation, and proper focus management — all for free. Removes ~50 lines of bespoke UI code.
…d allowlist Review round 3 fixes: - Add aria-label to MultiSelectCombobox input for screen readers (P1 a11y) - Log at ERROR when all UUIDs in allowlist are invalid, producing a silent fail-open (P2 correctness) - Fix indentation in chatd allowlist parsing block
The component initializes its internal 'selected' state from defaultOptions on mount, but our data loads asynchronously after mount. Pass both defaultOptions and value, and add a key derived from the server selection so the component remounts when server data arrives — ensuring badges render from the start.
Combines TemplatesSection and TemplatesManySelected into one TemplateAllowlist story with a play function that exercises the full admin workflow: 1. Starts empty — verifies 'no templates selected' and Save disabled 2. Selects one template, verifies badge + status, saves 3. Adds remaining seven, verifies '8 templates selected', saves Uses stateful mocks so getChatTemplateAllowlist returns the updated value after each save.
Addresses Copilot review comment: the custom queryKey ["templates"] creates a separate cache entry from the shared helper in api/queries/templates.ts. Switch to templates() for consistent caching and invalidation.
- parseChatTemplateAllowlist now returns error instead of silently swallowing corrupt JSON; GET handler returns 500 for corrupt data - PUT handler validates template IDs exist in the DB (400 on unknown) - Add doc comment on GetChatTemplateAllowlist explaining stricter auth policy vs peer getters - Add descriptive SQL comment on GetChatTemplateAllowlist query - Use slices.Collect(maps.Keys(...)) instead of manual loop - Restructure TestListTemplates into t.Run subtests grouped by tool, rename to TestTemplateAllowlistEnforcement - Add happy-path CreateWorkspace test with allowed template - Remove redundant useMemo wrappers (React Compiler handles it) - Remove unnecessary type predicate (TS 5.5+ infers it)
The PUT handler now validates template existence via GetTemplateByID, so tests using uuid.NewString() get 400 instead of succeeding. - Switch to a single shared coderdtest instance with two real templates created via dbgen.Template - Use real template IDs in AdminCanSet, AdminCanClear, DeduplicatesIDs - Keep random UUIDs for NonAdminWriteFails and UnauthenticatedFails (they fail on authz before reaching validation) - Add NonexistentTemplateRejected subtest for the new 400 path - Add chatd comment noting API-vs-runtime fail-open asymmetry
- Add doc comments to generated querier.go and queries.sql.go to match the SQL comment added in the previous commit - Add nolint:tparallel,paralleltest to test functions with sequential subtests sharing state - Fix CreateWorkspace/Allowed test: don't assert resp.IsError since the tool does additional work (asOwner, workspace lookup) beyond the allowlist gate
…cess Non-admin users shouldn't learn the endpoint exists. Return ResourceNotFound (404) instead of Forbidden (403) on both GET and PUT handlers, consistent with the 'don't leak information' principle.
Three improvements to the template allowlist: 1. Chat tools now accept a function (func() map[uuid.UUID]bool) instead of a static map. The chatd runtime builds a closure that re-reads the allowlist from the DB on each tool call, so admin changes take effect without restarting the chat. A nil function fails open (all templates allowed). 2. PUT handler validates templates in a single GetTemplatesWithFilter call that also filters out deprecated templates, wrapped in a transaction with the upsert. Reports which specific IDs are missing or deprecated in the 400 response. 3. Added DeprecatedTemplateRejected test case.
c239761 to
eba6413
Compare
…tion message CreateWorkspace now returns 'template not available for chat workspaces' instead of 'not found' when the allowlist blocks a template. The test was still checking for the old substring.
site_configs(agents_template_allowlist)GET/PUT /api/experimental/chats/config/template-allowlistendpointslist_templates,read_template, andcreate_workspacechat tools by allowlist, if defined (empty=all allowed)