feat(dashboard): LLM stats answer who/what spends tokens — member scope chips and spend breakdowns
Nightly Build / build (push) Successful in 4m27s
Nightly Build / build (push) Successful in 4m27s
The stats section was designed single-user: four global charts, no attribution. Request metadata rows now carry the session's source and the frame's agent_id/depth (additive ensure_column), denormalized at log time by the LoggingModel from the owner's pool — off the turn's hot path, degrading to NULLs, never to a lost row. The dashboard gains member scope chips filtering every chart, and a breakdown row splitting the range's billed tokens by member, kind (chat / sub-agents / cron / system agents / channels), agent, model and provider — a sub-agent's spend is attributed to the sub-agent itself. Rows predating the columns group under 'older data'.
This commit is contained in:
+16
-6
@@ -15,17 +15,27 @@ The status reflects the **whole instance**, not one person's account: there is o
|
||||
|
||||
## LLM usage stats
|
||||
|
||||
Four charts with a range switch (last hour / 24 hours / 7 days / 30 days):
|
||||
The section answers two questions: *how much is the instance being used?* and *who or what is spending the tokens?* A range switch (last hour / 24 hours / 7 days / 30 days) applies to everything on it, and — on an instance with more than one member — a row of member chips filters **all** the charts to one person; **Everyone** is the whole instance, as before.
|
||||
|
||||
Three charts show the trend over the range:
|
||||
|
||||
- **Requests** — how many LLM calls per minute, hour or day.
|
||||
- **Tokens** — the metered volume, split into input (split again into cached and non-cached) and output. The tooltip shows the cache-hit percentage: repeated context that was *cached* costs less and answers faster, so a high hit rate is good news, not a sign something is stuck.
|
||||
- **Avg latency** — how long a model call took on average.
|
||||
- **Models** — the top models by requests in the range.
|
||||
|
||||
Below them, **How the spend splits** breaks the range's billed tokens down into bars:
|
||||
|
||||
- **By member** — who consumed what (hidden while a member chip is selected: it would be a single bar).
|
||||
- **By kind** — direct chats vs **sub-agents** (specialists the assistant delegates to) vs **scheduled tasks** and the other background system agents. Data from before this breakdown existed groups under *Older data*.
|
||||
- **By agent** — which assistant or specialist consumed the most.
|
||||
- **By model** and **By provider** — where the money actually goes.
|
||||
|
||||
A bar's tooltip shows the request count and the input/output/cached split.
|
||||
|
||||
Three honest answers to give with a straight face:
|
||||
|
||||
- **These numbers are everyone's, together.** The charts aggregate the whole instance; there is no per-person breakdown on this page.
|
||||
- **They record how much, when and which model — never what was said.** The content of a request lives in the requester's own encrypted space; the charts read only counters.
|
||||
- **They record how much, when, by whom and on which model — never what was said.** The content of a request lives in the requester's own encrypted space; the charts read only counters.
|
||||
- **The per-member chips are a consumption view, not a surveillance tool.** They show token counts, not conversations.
|
||||
- **Empty is normal on a new instance.** "No LLM requests in the selected range" means exactly that: nothing has run in that window.
|
||||
|
||||
## Pending
|
||||
@@ -41,12 +51,12 @@ The same cards as the [Inbox](inbox.md) — approvals, questions and sign-in pro
|
||||
|
||||
- **Not a monitor.** Nothing here alerts anyone; it shows the present state to whoever is looking.
|
||||
- **Not where models are configured.** Adding providers and models, and their priority order, is the admin's Models and Providers pages.
|
||||
- **Not per-person.** No page on the instance shows "who used how much" — deliberately; usage is shared, like the models.
|
||||
- **Not a message log.** The per-member and per-kind charts show volumes of tokens, never what anyone asked or was answered.
|
||||
|
||||
## Common questions
|
||||
|
||||
- *"It says Degraded — should I worry?"* — it means the model checks are not all passing. Individual chats may still work on a fallback model; if it persists, the admin checks the provider (its key, its quota) on the Models/Providers pages.
|
||||
- *"Why are the bars so high at odd hours?"* — scheduled background work (system agents, cron tasks) uses the same models. The Tasks page and the system-agents page show what ran when.
|
||||
- *"Why are the bars so high at odd hours?"* — scheduled background work (system agents, cron tasks) uses the same models. The **By kind** chart shows exactly how much of the spend is background work versus direct chats; the Tasks page and the system-agents page show what ran when.
|
||||
- *"What is a token?"* — the unit LLM providers meter and bill by, roughly a word fragment. Input is what was sent (long history = more input; caching repeats cheaply), output is what was written back.
|
||||
- *"Why doesn't my child see this page?"* — their role uses the simple interface: chat, inbox and projects only.
|
||||
- *"Does the dashboard show what people asked?"* — no. Only counts, timings and model names; never content.
|
||||
|
||||
+1
-1
@@ -12,7 +12,7 @@ This index will grow over time. Right now it covers the chat window, the inbox,
|
||||
| --- | --- |
|
||||
| [chat.md](chat.md) | The chat window: full-page vs docked, the tab bar and what lands where, the composer's controls, the slash commands, and what happens while an answer is being written |
|
||||
| [inbox.md](inbox.md) | The Inbox: the three kinds of pending request, why background work asks here rather than in the chat, answering one (and the time-limited approvals), and why an unanswered card means a stopped job |
|
||||
| [dashboard.md](dashboard.md) | The Dashboard: the instance status line, the LLM usage charts (everyone's together, counts never content), the pending-inbox section, and what the no-models banner means |
|
||||
| [dashboard.md](dashboard.md) | The Dashboard: the instance status line, the LLM usage charts (filterable per member, with spend broken down by member, kind, agent, model and provider — counts never content), the pending-inbox section, and what the no-models banner means |
|
||||
| [security-groups.md](security-groups.md) | Security groups: allow / ask / deny per tool, the shield in the chat, what the default group already permits, and how an admin edits the rules |
|
||||
| [memory.md](memory.md) | Private and shared memory: what goes where, the indexes and history log, why some shared facts can't be changed on request |
|
||||
| [agents.md](agents.md) | Agents: the three kinds (chat, task, system), which one you are talking to and why, the specialist agents the assistant delegates to, how the model is chosen, and adding a custom agent |
|
||||
|
||||
Reference in New Issue
Block a user