9.0.0 Released — Air-Gapped Developer Velocity, Code Origin Engine & Local DORA Metrics: Zero Cloud Data Leaks

Step-by-Step Feature Walkthrough & Documentation

Tokonomics is the local AI token optimizer, AST context compiler, prefix cache pinner, and air-gapped developer velocity/DORA engine for VS Code, Claude Code, and Cursor. Track active coding hours vs AI cost, classify human vs AI code origin, correlate flow state friction, and measure local lead time with zero network egress.

Explore Features Step-by-Step Launch Interactive Simulator

14 Languages

Tree-sitter WASM AST Slicing

3x – 4x

Prompt Headroom for Subscriptions

< 15 ms

Local Exact Chunk Retrieval

100% Private

Zero Code/Prompt Collection

GET STARTED

Up and Running in 60 Seconds

Install Tokonomics and perform your first local, zero-network context analysis.

Install Extension

Open VS Code, press Ctrl+P / Cmd+P, and enter:

ext install tokonomics.tokonomics-ai

Run First Context Check

Open any source file, press Ctrl+Shift+P, and run:

Tokonomics: Run First Context Check

Follow the 5-step local dashboard guide. Sends zero prompts to models.

Code with Lean Context

In VS Code Chat (Ask), query Tokonomics or route to your subscription CLI:

@tokonomics explain auth flow in this workspace

STEP-BY-STEP GUIDES

Deep Dive: How to Use Every Feature

Click through any workflow below to see exact Command Palette triggers, inputs, and outputs.

Guided Onboarding

1. First Context Check

The fastest way to experience Tokonomics without sending prompts to any external AI provider or altering your workspace.

Without Tokonomics Blind Prompting

Unmeasured context size sent blindly to external LLMs
No preview of raw token weight or token-to-dollar estimate
Accidental dispatch of oversized files and syntax bloat

⚠️ Unmeasured Token Spend

With First Context Check Guided & Local

100% local 5-stage preview of active buffer & UTF-8 byte weight
Calibrated BPE token count before sending any prompts
Zero-send guarantee with zero network telemetry

✓ 100% Local BPE Preview

Step 1

Open any text or source file in your editor

Either highlight a few functions you want to analyze, or leave the cursor anywhere in the file.

Step 2

Trigger the command

Open the Command Palette (Ctrl+Shift+P / Cmd+Shift+P) and select:

Tokonomics: Run First Context Check

Step 3

Review the interactive dashboard estimation

Tokonomics opens the Dashboard with a step-by-step 5-stage tracker displaying:

Active Scope: Shows whether your selection or entire editor buffer is targeted.
Character & Byte Count: Exact UTF-8 input weight.
Estimated Raw Tokens: Fast BPE token count using the calibrated tokenizer.
Zero-Send Guarantee: Explains that no request was transmitted to upstream LLMs.

💡

Safety Invariant: First Context Check stores a single boolean completion flag in local storage upon finish/dismiss. It reads zero sensitive content and performs zero background network requests.

Workspace Intelligence

2. Context X-Ray (Token Density Heatmap)

Instantly discover which files in your repository consume the most context and identify prime candidates for AST skeletonization.

Without Context X-Ray Repository Blindspot

No visibility into which repository files consume the most tokens
Monolithic modules (4k+ tokens) consume 80% of prompt windows
Manual guessing of prime candidates for AST skeletonization

⚠️ 4,850 tokens per unpruned file

With Context X-Ray Multi-Criteria Sorting

Instant token density heatmap of indexed repository files
Quick-filter sorting by Potential Savings, Raw Tokens, or Savings %
Identify files offering 70%–80% immediate context reduction

✓ 1,240 tokens (-74.4% reduction)

Step 1

Open a trusted workspace

Ensure your project folder is open in VS Code with Workspace Trust granted.

Step 2

Execute Context X-Ray

Tokonomics: Run Context X-Ray

Approve the one-time workspace snapshot when prompted.

Step 3

Explore ranked context breakdown with multi-criteria sorting

The report renders an interactive table of repository files with quick-filter buttons to sort by Raw tokens, Potential savings, or Savings %:

File Path	Raw Tokens	Skeleton Tokens	Potential Reduction
`src/compiler/orchestrator.ts`	4,850	1,240	-74.4%
`src/protocol/gateway.ts`	3,120	890	-71.5%
`src/types/schemas.ts`	2,600	1,950	-25.0%

Status Bar Telemetry

3. Live Context Meter

Monitor your active text selection or editor buffer size in real time directly from the VS Code status bar.

Without Context Meter Silent Token Inflation

No feedback on how selection size affects chat prompt length
Risk of selecting massive files causing editor lag & overages
No safety threshold before sending large buffers

⚠️ Zero real-time buffer feedback

With Live Context Meter Status Bar Pill

Real-time token counter in status bar for active selection or file
Zero-send local calculation with immediate update on cursor move
Strict 300 KiB safety limit protecting UI thread responsiveness

✓ ⚡ 842 tokens (Live Status Bar)

Step 1

Turn on Context Meter (Opt-in)

Context Meter is completely opt-in to avoid unnecessary background overhead. To start:

Tokonomics: Start Context Meter

Step 2

View live status bar pill

A new item appears in the bottom right status bar:

⚡ Tokonomics: 842 tokens (Selection)

Select different code blocks or switch files; the counter updates automatically.

Step 3

Turn off anytime

Click the status bar item directly, or run Tokonomics: Stop Context Meter. Disposing drops all listeners immediately.

🛡️

Safety Boundary: Strict 300 KiB input limit. Buffers larger than 300 KiB automatically show "Input too large" to prevent UI thread lag.

Anti-Drift Technology

4. Chat Handoff (/handoff)

Reset your context window from 80k+ tokens down to 2.5k tokens with zero requirement amnesia.

Turn 25 Without Handoff Context Amnesia & Bloat

84,200 tokens accumulated over long conversational turns
Severe context drift, hallucination risk, and slow model responses
~$1.26 per message re-feeding dead terminal and file dumps

⚠️ 84,200 tokens (~$1.26 / turn)

Turn 25 With /handoff Anti-Drift Checkpoint

2,410 tokens (-97.1% context reduction) in fresh chat session
Structured 4-part checkpoint: Objectives, Decisions, Files, Next Actions
Crisp, sub-second responses with zero loss of task requirements

✓ 2,410 tokens (~$0.03 / turn, -97%)

Step 1

Reach Turn 15–25 in a long conversation

As chats grow past 20 turns, context drift degrades model intelligence and wastes thousands of tokens per prompt re-feed.

Step 2

Trigger Chat Handoff

In Tokonomics Chat, type /handoff or run from the Command Palette:

Tokonomics: Create Chat Handoff

Step 3

Review the synthesized checkpoint

An unsaved Markdown document opens instantly containing a distilled blueprint:

# Session State Handover [tokonomics-handoff/v1]
## 1. Active Objectives & Completed Tasks
- [x] Implemented Tree-sitter WASM parser
- [/] In Progress: Fix edge case in decorator preservation
## 2. Key Architectural Decisions
- Preserved control-flow guard blocks across all AST reductions
## 3. Touched Files & Touchpoints
- `src/compiler/ast.ts` (lines 45-80)
## 4. Next Immediate Action
- Run `npm test tests/ast.test.ts`

Start a brand-new chat session, paste this 2k token summary, and enjoy clean 97% context savings with full task continuity.

Diagnostic Surgery

5. Session Health & Zombie Log Audit

Analyze Claude Code session logs to detect repetitive tool outputs, dead terminal dumps, and context bloat.

Unmonitored Agent Sessions Zombie Terminal Bloat

Repeated terminal output dumps re-sent on every conversation turn
10,000+ character compiler error traces bloating conversation state
No diagnostic visibility into why agent chats become sluggish

⚠️ 50k+ chars of repeated terminal outputs

With Session Health Audit Diagnostic Surgery

Read-only inspection of Claude Code JSONL logs without privacy leaks
Trajectory growth metrics flagging repetitive commands & compaction markers
Actionable remediation to truncate zombie tool results

✓ Trajectory & Bloat Diagnostics

Step 1

Run Session Log Health

Tokonomics: Analyze Session Log Health

Step 2

Select a supported .json or .jsonl session log

Choose an external agent log from your project or Claude Code directory.

Step 3

Inspect the diagnostic report on the Dashboard

Tokonomics inspects the log in read-only mode and reports:

Context Growth Trajectory: Starting tokens vs peak tokens and growth rate.
Large Tool Results: Terminal outputs or file reads exceeding 10,000 characters.
Repeated Tool Results: Identical commands re-feeding multiple times.
Compaction Markers: Boundary points where summaries were created.

Agent Engineering

6. Tool-Schema Overhead Audit

Audit your MCP server tools or OpenAI function schemas to catch parameter bloat, verbose prompts, and missing enums.

Unoptimized Tool Schemas Schema Token Drain

15,000+ tokens consumed on every turn just for tool definitions
Flat parameter explosion (>6 args) confusing model calling accuracy
Verbose multi-paragraph argument descriptions and deep nesting

⚠️ >40% of prompt window on tool JSON

With Tool-Schema Linter Lean Function Specs

Automated linting for OpenAI, MCP inputSchema, and Claude formats
Dominant schema warnings when one tool eats >40% of tool budget
Actionable suggestions to group flat parameters and trim descriptions

✓ -65% Tool Schema Token Overhead

Step 1

Select tool JSON or choose a file

Highlight JSON in your editor containing OpenAI functions, MCP inputSchema, or Claude input_schema definitions.

Step 2

Run Tool Schema Audit

Tokonomics: Audit Tool-Schema Overhead

Step 3

Review the linting report

An unsaved Markdown report flags architectural defects:

Dominant Schema: Identifies if one massive tool consumes >40% of the entire tool definition budget.
Verbose Descriptions: Flags parameter descriptions >512 characters.
Flat Parameter Explosion: Flags tools with >6 flat arguments that confuse models.
Deep Nesting: Flags schemas nested deeper than 8 levels.

UI Surface

7. Dockable Native Chat Panel

Use Tokonomics directly in your Secondary Side Bar or split Editor Tab with streamed Markdown and live token telemetry.

Standard AI Chat Panels Unassisted Routing

Overpaying by running simple unit tests on expensive frontier models
No prompt secret redaction before persisting conversation history
Locked to single sidebar view without split-editor docking

⚠️ Expensive frontier pricing on routine tasks

Tokonomics Native Chat Model Tier Guidance

Non-intrusive tips: "💡 Run on Claude 3.5 Haiku at ~90% lower cost"
Automatic secret redaction for persisted local conversation history
Flexible docking in Secondary Side Bar or dedicated split Editor Tab

✓ Model Tier Advice (~90% Cost Reduction)

Step 1

Open Chat

Tokonomics: Open Chat

Or click the Tokonomics icon in the Secondary Side Bar.

Step 2

Open in Split Editor Tab

For wide-screen coding, run Tokonomics: Open Chat in Editor to dock conversation beside your code.

Step 3

Automatic secret redaction & session retention

Stores up to 10 recent workspace chats locally. Secret credentials and API keys are automatically redacted before persistence.

Step 4

Contextual model tier guidance

Tokonomics automatically surfaces non-intrusive tips when lighter and more cost-effective model tiers (like Claude 3.5 Haiku) can handle routine tasks at significant token and cost savings.

Zero API Invoices

8. CLI Subscription Chat (/claude & /codex)

Route prompts through your existing $20/month flat-rate Claude Code or Codex CLI login while benefiting from Tokonomics context compilation.

Direct API Key Billing Metered API Invoices

Variable per-token bills scaling with prompt size and turn count
Surprise end-of-month cloud invoices for teams
Risk of accidental API key compromise in workspace files

⚠️ Variable pay-per-token API fees

Tokonomics /claude & /codex Flat $20/mo Routing

Route prompts through your existing flat-rate official CLI subscription
Pre-compiles AST context to maximize prompt capacity headroom
Subprocess isolation with zero API keys stored or transmitted

✓ Flat $20/mo Subscriptions + Lean AST

Step 1

Login to official provider CLIs

claude login # or codex auth

Step 2

Ask questions using CLI commands

In VS Code Chat (Ask), query:

@tokonomics /claude refactor this method to use async/await

Tokonomics optimizes the surrounding AST context, passes it to your authenticated CLI session, and streams the answer.

Financial Governance

9. FinOps Dashboard & Task Budgets

Track real token consumption, group prompts by task, set advisory spend budgets, and import Claude usage.

Without FinOps Governance No Cost Attribution

Zero visibility into token spend by feature or coding session
No budget thresholds to catch runaway agent loops
Unable to quantify ROI or token savings across development tasks

⚠️ Uncontrolled & Untracked Spend

With Tokonomics FinOps Budget & Spend Ledger

Overview, Context, and Activity ledger organized by task ID
Non-blocking advisory budget limits (task, daily, monthly)
Separates observed usage costs, input projections, and avoided cost

✓ 100% Local Spend & Budget Control

Step 1

Open the Dashboard

Tokonomics: Show Savings Dashboard

Step 2

Start a Spend Task

Group all subsequent coding turns under an explicit task (e.g. "OAuth Refactor") and track cost per feature.

Step 3

Set Advisory Budgets

Configure task, daily, or monthly limits. Tokonomics alerts you when limits are approached without abruptly aborting your session.

Privacy-First Context

10. Encrypted Project Memory

Store key architectural decisions, conventions, and constraints locally with AES-256 encryption.

Without Project Memory Repetitive Prompting

Re-typing coding constraints, rules, and architecture on every prompt
Model forgetting critical conventions between separate chat sessions
Cloud memory services storing proprietary codebase rules externally

⚠️ Re-prompting overhead & cloud lock-in

With Tokonomics Memory AES-256 Encrypted

Strictly opt-in per workspace with authenticated local encryption
Persistent project conventions automatically attached to relevant prompts
Inspect, export, or delete stored rules anytime with one click

✓ Local Encrypted Architecture Rules

Step 1

Manage Project Memory

Tokonomics: Manage Project Memory

Step 2

Explicit Per-Workspace Consent

Project Memory is strictly OFF by default. It activates only after explicit per-workspace opt-in.

Step 3

Inspect, Export, or Delete Anytime

Review every stored architectural rule. You can delete or export your memory snapshot at any time.

Next-Gen Architecture (v8.6.0)

11. Next-Gen KPI Cockpit & Finviz Market Treemap

Real-time FinOps command center with 4-column sticky KPIs, multi-phase execution pipeline stepper, Finviz-style squarified market treemap, privacy-preserving Aptabase telemetry, git branch drift tracking, model Pareto efficiency frontier, and budget burn-rate velocity forecasting.

Traditional Static Dashboards Disconnected Metrics

Static snapshot requiring manual page refreshes to see new turns
Opaque black-box compilation with zero visibility into transformation stages
Flat unorganized directory lists without visual volume or pruning efficiency hierarchy
No git branch context: unable to know if a PR branch is blowing up context costs
Ad-hoc model selection without Pareto-optimal latency vs cost trade-off insights
Surprise end-of-month budget exhaustion without burn velocity projections

⚠️ Opaque & Static Spend Monitoring

Tokonomics 8.6.0 Cockpit Suite Live Dynamic FinOps

Unified 4-column sticky KPI bar with live polling (1s/5s/30s/off) and instant event-bus auto-refresh
Interactive 10-stage nanobar pipeline stepper showing micro-timings and token drops
Finviz-Style Market Treemap: 2-level squarified layout with sector headers, language badges, tickers, and color legend
Privacy-Preserving Aggregate Telemetry: anonymous metrics via Aptabase with zero code/prompt collection and user opt-out
Git branch financial drift tracking with baseline comparison and colored severity pills
Model Efficiency Frontier: log-scale 2D Pareto scatter plot for smart model routing
Burn-rate velocity gauge (V_b = Spend / Days) with monthly exhaustion forecaster

✓ Real-Time Adaptive FinOps Command Center

Step 1

Open the Savings Dashboard

Tokonomics: Show Savings Dashboard

The sticky 4-column KPI cockpit anchors at the top with cumulative tokens, net cost avoided, session efficiency, and context density ratio.

Step 2

Configure Inline Tray: Live Polling & Budget Caps

Select your preferred live refresh cadence (1s live pulse for heavy pairing, 5s default, 30s eco, or off). Set advisory monthly budget limits and toggle smart model auto-routing.

Step 3

Inspect the Multi-Phase Pipeline Stepper

Hover over or click any nanobar segment to inspect transformation stages: raw ingest, comment pruning, import slicing, AST context slicing, and type skeletonization with elapsed microseconds.

Step 4

Track Git Branch Financial Drift & Model Pareto Frontier

Compare token spend on your active feature branch against repository baselines. Explore the log-scale Model Efficiency Frontier scatter plot to balance token costs and latency.

Flagship Innovation (v8.6.0)

12. Model Context Protocol (MCP) Tool Firewall & Schema Pruner

Stop wasting 40%–60% of your prompt window on idle MCP JSON tool definitions. Tokonomics dynamically classifies developer intent, injects top candidate tools with fallback catalog virtualization (tokonomics_request_tool_catalog), and emits compact TypeScript signatures—slashing tool schema overhead by up to 90.0% in 2.3ms with 100% parameter accuracy.

Standard MCP Tool Injection Crippling Schema Tax

7,796+ tokens consumed on every single turn across 55 community tools
Monolithic JSON Schemas with redundant properties, enums, and boilerplate descriptions
Model attention diluted across 50+ inactive tools, increasing calling hallucinations
Developers forced to manually disable tools in config files between tasks

⚠️ 7,796 tokens wasted per prompt turn

Tokonomics MCP Tool Firewall Dynamic Intent Pruning

Real-time intent classifier matches prompt keywords to the active server domain in 0.23ms
Injects Top-4 high-affinity tools + tokonomics_request_tool_catalog safety paging
Minifies schemas into concise TypeScript pseudo-signatures, saving 90.0% context
Zero misfires: 100% parameter parity across 7 canonical community servers (55 tools)

✓ 779 tokens (-90.0% reduction in 2.3ms)

Step 1

Automatic Zero-Config Discovery

Tokonomics automatically discovers connected MCP servers configured in your VS Code workspace (or Claude Desktop / Cursor settings) including GitHub, Postgres, Filesystem, Brave Search, Puppeteer, Slack, and Sentry.

Step 2

Dynamic Intent Scoring & Candidate Selection

When you ask a question in chat, the on-device intent classifier analyzes query semantics in <0.5ms without external API calls, selecting the 4 highest-affinity candidate tools.

Step 3

Catalog Virtualization via Fallback Paging

Tokonomics automatically appends the lightweight tokonomics_request_tool_catalog tool. If the model determines it needs a tool outside the top candidate set, it requests it dynamically—guaranteeing 100% tool availability and zero dropped tasks.

Step 4

Ultra-Compact TypeScript Pseudo-Signatures

Tokonomics: Audit Tool-Schema Overhead

Replaces repetitive JSON Schema boilerplate (type: "string", description: "...") with concise TypeScript signatures (e.g. write_file(path: string, content: string, overwrite?: boolean)), cutting schema length by an additional 31%.

Flagship Innovation (v8.8.0)

13. Universal Agent Layer (MCP) & Execution Optimizer

Empower autonomous agents—including Claude Code CLI, Cursor Agent, Codex, and Copilot Agent mode—with Tokonomics' AST context optimization and Execution Bloat Distillation via Model Context Protocol (MCP) JSON-RPC 2.0 over stdio. Solves the token-vs-quality dilemma with Two-Stage Progressive Disclosure (structural skeletons first, on-demand byte-accurate symbol hydration) and Response Bloat Distillation (tokonomics_distill_output) for test suites, JSON arrays, and diffs with 85.5% aggregate token reduction and 100% diagnostic fidelity.

Standard Agent Context Exhaustion Context Window Thrashing

Agents read entire multi-thousand line files greedily into working memory
Terminal outputs (Jest, Pytest, Go, Cargo) flood context with thousands of passing test lines
Mega JSON arrays (500+ rows) burn 14,000+ tokens on repetitive structural boilerplate
Context window fills in 3–5 turns, triggering lossy compaction or token eviction

⚠️ 17,800+ tokens burned per multi-file test run

Tokonomics Agent Layer & Execution Optimizer Intelligent Distillation

Stage 1: Lean AST skeletons via tokonomics_read_skeleton (-38% to -58% tokens)
Stage 2: Verbatim symbol logic on demand via tokonomics_expand_symbol (100% fidelity)
Stage 3: Distills execution bloat via tokonomics_distill_output (-85.5% token reduction)
Preserves 100% of failure stacks, error diagnostics, and assertions with secret redaction

✓ 100% Accuracy & Diagnostic Fidelity (-85.5% Bloat Reduction)

Step 1

Copy Agent Host Configuration

In VS Code, press Ctrl+Shift+P and run Tokonomics: Copy Agent MCP Configuration. Choose Claude Code CLI, Cursor, or Generic MCP.

claude mcp add tokonomics -- node ~/.vscode/extensions/tokonomics-ai/dist/mcp-server.js

Step 2

Stage 1: Structural Skeleton Ingest

When the external agent inspects code files, it invokes tokonomics_read_skeleton. All function bodies and routines are replaced with lightweight token-tagged expansion hints, saving thousands of tokens across initial repo scans.

Step 3

Stage 2: Byte-Accurate Symbol Hydration

If the agent needs to implement, patch, or review a specific method, it invokes tokonomics_expand_symbol. Tokonomics returns the exact verbatim code block with 100% boundary fidelity—preventing hallucinations and preserving full implementation detail.

Step 4

Autonomous Budget & Cache Supervision

Agents query tokonomics_budget_status to monitor cumulative token spend and prevent infinite loops. Tools query tokonomics_check_cache to ensure prompt cache prefix stability across multi-turn sessions.

Step 5

Stage 3: Response Bloat Distiller & Execution Optimizer

When tests or commands execute, agents invoke tokonomics_distill_output. Repetitive passing suites and boilerplate arrays are stripped, while 100% of failure stacks, assertion diffs, and error messages are preserved with automatic credential redaction.

Top Open-Source Repositories

Empirical Execution Optimizer Benchmarks

🏆 -85.5% Token Reduction Grade A+ Certified

Tested across canonical real-world test suites and databases. Verifiable with 100% diagnostic fidelity and sub-millisecond (0.44ms p95) processing latency.

Target Repository	Framework / Format	Raw Tokens	Distilled Tokens	Reduction	Diagnostic Fidelity	Latency
facebook/jest	Jest / TypeScript	937 tokens	465 tokens	-50.4%	100% Failure Preserved	0.21 ms
tiangolo/fastapi	Pytest / Python	809 tokens	544 tokens	-32.8%	100% Traceback Preserved	0.18 ms
kubernetes/kubernetes	Go test / Go	489 tokens	294 tokens	-39.9%	100% Fail Stack Preserved	0.16 ms
BurntSushi/ripgrep	Cargo test / Rust	478 tokens	261 tokens	-45.4%	100% Panic Preserved	0.19 ms
spring-projects/spring-boot	JUnit / Java	774 tokens	290 tokens	-62.5%	100% Assertion Preserved	0.22 ms
supabase/postgrest	PostgreSQL JSON (500 rows)	14,336 tokens	735 tokens	-94.9%	100% Schema & Bounds	0.35 ms
AGGREGATE BENCHMARK	All 6 Target Repositories	17,823 tokens	2,589 tokens	-85.5%	100% Parity (0 Losses)	0.44 ms (p95)

v8.9.0 Flagship

14. Semantic Graph Slicing & Prefix Cache Pinning

High-precision program slicing engine extracting exact call hierarchies and type contracts paired with deterministic 3-tier prompt envelopes and 1,024-token boundary padding for guaranteed KV cache hits.

Without Tokonomics Monolithic Stuffing

Full source files bundled into context (1,500 – 3,000+ tokens per query)
Orthogonal helper classes and dead routines crowd out attention
Shuffled tool declarations and timestamps bust prompt cache on every turn
Zero boundary alignment forces sub-block cache evictions on Anthropic/Gemini

⚠️ 75%+ Redundant Token Waste & Cache Misses

With Semantic Slicing & Pinning Deterministic & Lean

Transitive call-tree & type-contract extraction (-75.6% tokens)
100% causal completeness: all referenced symbols and types preserved
Deterministic 3-tier layout: Anchor ➔ Canonical Tools ➔ Pinned AST Slices
Exact 1,024-token boundary padding guarantees multi-turn KV cache hits

✅ 75.6% Token Reduction & 100% Cache Invariance

Step 1

Target Symbol Program Slicing

When an agent analyzes or patches code, it invokes tokonomics_slice_graph with a seed symbol and depth. The engine parses the AST in-memory (< 0.3ms) and traverses call dependencies backward and forward.

Step 2

Transitive Type & Interface Closure

All imported data structures, interface contracts, and subclass definitions are recursively extracted and ordered hierarchically. Orthogonal utilities and unrelated router verbs are pruned without losing implementation details.

Step 3

Deterministic Prefix Envelope Construction

Agents invoke tokonomics_pin_prefix to arrange prompt context into three canonical tiers: (1) Static System Anchors, (2) Canonically Sorted Tool Schemas, and (3) Pinned Codebase Slices. Volatile user history is strictly isolated to the dynamic suffix.

Step 4

Dynamic Cache-Buster Sanitization

Embedded timestamps, ISO dates, session UUIDs, and turn counters are automatically neutralized to stable placeholders, preventing accidental provider-side prefix cache invalidation.

Step 5

1,024-Token Boundary Padding & Provider Markers

Prompts are padded with filler comments to exact 1,024-token multiples (e.g. Anthropic/Gemini KV cache block size) and tagged with provider-native markers (e.g. cache_control: { type: "ephemeral" }).

Top Open-Source Repositories

Empirical Semantic Graph Slicing & Prefix Pinning Benchmarks

🏆 -75.6% Aggregate Savings 100% Cache Invariance

Tested across top open-source production codebases. Verified 100% causal completeness, sub-millisecond (0.29ms mean) traversal latency, and Grade A+ certification.

Target Repository	Language / Domain	Raw Bundle	Sliced Graph	Reduction	Causal Fidelity	Prefix Cache Hit	Latency
reduxjs/redux	TypeScript (State Architecture)	1,610 tokens	282 tokens	-82.5%	100% Complete	GUARANTEED	0.34 ms
expressjs/express	JavaScript (HTTP Routing)	1,613 tokens	468 tokens	-71.0%	100% Complete	GUARANTEED	0.24 ms
tiangolo/fastapi	Python (Dependency Injection)	1,943 tokens	515 tokens	-73.5%	100% Complete	GUARANTEED	0.40 ms
BurntSushi/ripgrep	Rust / Go Pattern (Search Engine)	2,662 tokens	760 tokens	-71.5%	100% Complete	GUARANTEED	0.36 ms
facebook/react	TypeScript / JS (Fiber Reconciler)	785 tokens	75 tokens	-90.4%	100% Complete	GUARANTEED	0.12 ms
AGGREGATE BENCHMARK	All 5 Canonical Repositories	8,613 tokens	2,100 tokens	-75.6%	100% Parity (0 Losses)	100% Invariant	0.29 ms (mean)

v9.0.0 Milestone

15. Developer Velocity, Code Origin & Air-Gapped Local DORA

Measures active coding time, classifies human typing vs AI completions, correlates flow state friction, and computes inner-loop DORA metrics directly from local Git history—with strict zero outbound network egress.

Unified Developer Cockpit Architecture

1. Activity Telemetry & Heartbeats

2-minute activity bucket aggregation tracking keystrokes and active typing seconds. Auto-pauses on 120s idle to guarantee zero false active time.

2. Code Origin Classifier

Distinguishes continuous human keystrokes from bulk AI streaming insertions and paste events with ≥95% accuracy and prompt proximity correlation.

3. Flow State & Friction Index

Clusters unbroken 20m+ coding blocks into deep work sessions. Identifies post-prompt rollbacks and computes the Flow Efficiency Score (0-100).

4. Air-Gapped Local DORA

Derives Lead Time for Changes (AI vs Manual delta), Change Failure Rate (CFR), pre-commit test pass rates, and MTTR from local Git metadata.

Real-World Metric Invariants & Privacy Guarantees

Capability Area	Reference Tools	Tokonomics Architecture	Privacy & Air-Gap Invariant
Active Dev Hours vs AI Cost	Simple Coding Insights	Combines 2-min activity heartbeats with Claude JSONL and model pricing ledgers.	100% Local (0 Egress)
Human vs AI Code Origin	WakaTime / Code Time	Heuristic typing dynamics vs bulk AI insertions with entropy checks.	0 Keystroke Retention
Flow State & Friction	Code Time (Software.com)	Unbroken 20m+ coding block clustering with post-prompt rollback detection.	In-Memory Analysis
Local DORA Metrics	QuantumEdge	Inner-loop Lead Time for Changes, blended CFR, and test pass rate from local Git.	SHA-256 Audit Stamp

INTERACTIVE STUDIO

Interactive Feature Visualizer & Simulators

Experience Tokonomics features live in your browser: inspect side-by-side AST diffs, explore Context X-Ray sorting, test native chat model tier guidance, and compare handoff anti-drift checkpoints.

Original Source Context (Raw) 842 tokens

Tokonomics Compiled AST Skeleton 215 tokens (-74.5%)

627 Tokens Saved

74.5% Context Reduction

~$0.019 Avoided Cost / Turn

100% Contracts Preserved

v8.9.0 Flagship

Semantic Graph Slicing & Prefix Cache Pinning Simulator

Test bounded AST call-tree slicing across top repositories and simulate deterministic 3-tier prefix envelope pinning with 1,024-token boundary alignment.

⚡ Cache Hit: GUARANTEED 0.29 ms mean

Target Repository:

Provider:

Slice Depth:

Align to 1,024-token Block Boundary

Auto-Sanitize Cache-Busters (Dates / UUIDs)

Raw Repo Bundle (Monolithic Files) 1,610 tokens

Tokonomics Sliced Graph & Pinned Prefix 282 tokens (-82.5%)

1,328 Tokens Saved

-82.5% Context Reduction

761cf678... Pinned Prefix SHA-256

100% Invariant Multi-Turn Cache Retention

v8.6.0 Flagship

Model Context Protocol (MCP) Tool Firewall & Schema Pruner

Simulate real-world dynamic intent classification and catalog virtualization across 7 connected community MCP servers (55 canonical tools).

Connected Servers: GitHub (20) Postgres (7) Filesystem (9) Brave Search (3) Puppeteer (6) Slack (5) Sentry (5)

Select Scenario:

Prompt:

All 55 Connected MCP Tools (Raw Monolithic Schema) 7,796 tokens

Tokonomics MCP Tool Firewall (Top Candidates + Virtual Catalog) 779 tokens (-90.0%)

7,017 Tokens Saved

-90.0% Payload Reduction

2.31 ms Local Overhead

100% Accuracy Parity

+$60.61 Saved / 1k Prompts (Claude 3.7)

File Path	Raw Tokens	Skeleton Tokens	Potential Reduction	Token Density

src/compiler/orchestrator.ts Raw: 4,850 tokens → Skeleton: 1,240 tokens | Potential Savings: 3,610 tokens (-74.4%)

✓ Prime Candidate for AST Skeletonization

Tokonomics Chat — Secondary Side Bar (v8.6.0) Model: Claude 3.7 Sonnet

Try Prompt:

Turn 25 Without Handoff Context Amnesia Risk

84,200 tokens context payload re-sent on every query
~$1.26 per message in repetitive context re-feed costs
High latency: 8.4s response wait time
High drift risk: Lost earlier edge case constraints and objectives

⚠️ 84,200 tokens ($1.26 / message)

Turn 25 With Tokonomics /handoff Clean 97% Reduction

2,410 tokens synthesized checkpoint across 4 structured pillars
~$0.03 per message in fresh, agile conversation context
Sub-second response: 0.9s response turnaround
Zero requirement amnesia: Active goals, decisions, and files preserved

✓ 2,410 tokens ($0.03 / message, -97.1%)

⚡ Polling Pulse:

🌿 Working Branch:

🎯 Monthly Budget: $100 / mo

Tokens Saved Live

1,428,950

+4,210 tokens in active session

Net Avoided Cost +$0.12/msg

$42.86

Calculated via blended model pricing

Compression Efficiency 74.2% Avg

74.2%

Preserving 100% syntactic contracts

Context Density Optimal

0.88

Tokens per compiled line of code

Live Multi-Stage Context Pipeline Stepper

Micro-stage execution timeline from raw editor buffer to pruned context dispatch. Click any stage to inspect transformation deltas.

✓ All Invariants Passed (0.008s)

Ingest 842 tok

Normalize 838 tok

Comments 612 tok

Imports 490 tok

AST Slice 260 tok

Skeleton 215 tok

Dispatch 215 tok

Stage 1: Raw Buffer Ingest 0.4ms

Initial UTF-8 byte stream read and character boundary validation.

Tokens in Stage: 842 Cumulative Savings: 0%

Spatial AST Pruning Market Treemap

Two-level squarified Finviz-style market map by token volume. Click any file ticker to inspect savings or drill into Context X-Ray.

COMPILER 42.5k tok • 78% sav

TS PRUNER.TS

-84.2% 26.8k

TS CACHE.TS

-72.0% 15.7k

FINOPS & TELEMETRY 28.1k tok • 64% sav

TS LEDGER.TS

-68.0% 16.2k

TS TELEMETRY.TS

-58.5% 11.9k

UI & DASHBOARD 18.4k tok • 52% sav

TS DASHBOARD.TS

-52.0% 18.4k

Efficiency Scale:

>70% (High Slicing) 40–70% (Balanced) <40% (Preserved Core)

Selected: src/compiler/pruner.ts (1 file, 26,800 tokens) 84% Slicing Potential

Branch Drift & Burn Forecaster

Financial drift vs main baseline and dynamic budget burn-rate exhaustion projection.

Active Branch Drift (vs main) ⚠️ Minor Drift (+18.4%)

+$4.12

Additional PR Spend

+138k

Token Delta on Merge

On Track

Merge Review Verdict

Monthly Burn Velocity & Horizon Safe Velocity

$3.42 / day

Current Spend Velocity

$85.50 / mo

Projected Month Total

29 Days (Safe)

Estimated Exhaustion

Model Efficiency Frontier (Log-Scale Latency vs Cost)

2D Pareto-optimal models identified on the frontier curve. Hover over any model to inspect price, latency, and routing recommendations.

Pareto Optimal Highlighted

✓ 2D Pareto Optimal

Claude 3.5 Haiku

Input Price: $0.80 / 1M

P95 Latency: 280ms

Top Pareto Choice: near-frontier reasoning at 90% discount.

💡 Tokonomics Routing: Compiles lean context to ensure high cache hit rates and sub-second generation.

v8.8.0 Flagship

Universal Agent Layer & Execution Optimizer

Simulate how external autonomous agents (Claude Code, Cursor, Codex) interact with Tokonomics' MCP server over stdio to read AST skeletons, hydrate symbols, and distill bloated test/JSON execution output in real time.

Compatible Agents: Claude Code CLI Cursor Agent Codex CLI Copilot Agent

Select Target Module / Target:

Agent Protocol Step:

Agent RPC: tokonomics_read_skeleton(filePath: "src/auth/AuthService.ts")

stdio JSON-RPC 2.0 Protocol Trace 0.42 ms

Agent Working Memory Context 261 tokens (-78.6%)

1,189 Tokens Saved

-78.6% Context Compression

100% Symbol Boundary Fidelity

100% Task Accuracy Parity

⚡ One-Click Agent Configuration

claude mcp add tokonomics -- node ~/.vscode/extensions/tokonomics-ai/dist/mcp-server.js

REFERENCE

Command Palette Cheatsheet

Quickly find any Tokonomics command for your workflow.

SECURITY CONTRACT

Privacy & Local-First Safety Invariants

Tokonomics is engineered from the ground up for strict confidentiality.

💻

Local Processing Only

Context compilation, AST pruning, and token counting run entirely on your local machine. No external servers or proxies sit between you and your AI model.

🔒

Privacy-Preserving Telemetry

Optional, strictly anonymous usage metrics (via Aptabase) measure aggregate token savings, compression percentages, and runtime error categories. Zero prompts, source code, diffs, filenames, or personal identifiers are ever collected. Fully respects VS Code's global telemetry settings and provides direct opt-out control via Tokonomics: Toggle Usage Analytics.

🛡️

Restricted Mode Fail-Closed

In VS Code Restricted Mode, all workspace scanning, file indexing, and automatic context attachment are disabled immediately.

🔑

Secret & Credential Redaction

Immediately prior to dispatch, all context undergoes credential scanning. Passwords, API tokens, and private keys are redacted.

EMPIRICAL PROOF & BENCHMARKS

Tested on World-Class Open-Source Codebases

Empirical, reproducible AST optimization evaluated against canonical production source code from Redux, Express, Axios, Fastify, React, and Flask. Zero synthetic cherry-picking, 100% verifiable on-device.

BALANCED AST (T1)

-58.7%

Average Context Reduction

Across 34,418 production tokens in canonical repositories with 100% type & contract preservation.

PEAK FRAMEWORK EFFICIENCY

-89.3%

Express.js Framework Pruning

3,322 tokens reduced to 356 tokens while maintaining the full application prototype & middleware lifecycle.

ON-DEVICE SPEED

7.08 ms

Tree-sitter WASM Latency

Sub-15ms local compilation introduces zero perceptible delay in active developer coding workflows.

AI FINOPS SAVINGS

+$60.61

Per 1k Prompts on Claude 3.7

Direct token cost reduction per 1,000 requests, plus up to 90% prompt cache discounts on multi-turn sessions.

Repository	Canonical File	Language	Raw Context	Tokonomics Balanced	Savings (%)	Max Skeleton	Compile Latency
reduxjs/redux	`src/createStore.ts`	TypeScript	4,186 tokens	1,289 tokens	-69.2%	918 (-78.1%)	12.1 ms
expressjs/express	`lib/application.js`	JavaScript	3,322 tokens	356 tokens	-89.3%	308 (-90.7%)	5.4 ms
axios/axios	`lib/core/Axios.js`	JavaScript	3,038 tokens	451 tokens	-85.2%	196 (-93.5%)	2.2 ms
fastify/fastify	`lib/route.js`	JavaScript	5,626 tokens	1,418 tokens	-74.8%	615 (-89.1%)	6.0 ms
facebook/react	`packages/react/src/ReactHooks.js`	JavaScript	2,699 tokens	416 tokens	-84.6%	416 (-84.6%)	4.1 ms
pallets/flask	`src/flask/app.py`	Python	15,547 tokens	10,284 tokens	-33.9%	8,922 (-42.6%)	12.7 ms
AGGREGATE / AVERAGE	6 World-Class Projects	Multi-Lang	34,418 tokens	14,214 tokens	-58.7%	11,375 (-67.0%)	7.08 ms

Response Bloat Distiller & Top Repositories Benchmark (v8.8.0)

Evaluated across canonical repositories (Jest, FastAPI, Kubernetes, Ripgrep, Spring Boot, Supabase PostgREST). Tokonomics distills execution output by 85.5% in 0.44ms (p95) with 100% diagnostic fidelity.

-85.5%

Token Reduction

0.44ms

p95 Latency

100%

Diagnostic Fidelity

Grade A+

ISO/IEC 25010

Top Open-Source Repo	Target Harness / Language	Bloat Category	Raw Context	Distilled Tokens	Reduction	Failure Fidelity	Latency (p95)
facebook/jest	`Jest / TypeScript`	Test Runner Suite Output	937 tokens	465 tokens	-50.4%	100% Assertions Intact	0.21 ms
tiangolo/fastapi	`Pytest / Python`	Async Route Exception Trace	809 tokens	544 tokens	-32.8%	100% Traceback Preserved	0.18 ms
kubernetes/kubernetes	`Go test / Go`	Controller Reconcile Failures	489 tokens	294 tokens	-39.9%	100% Stack Trace Intact	0.16 ms
BurntSushi/ripgrep	`Cargo test / Rust`	Regex Engine Panic & Backtrace	478 tokens	261 tokens	-45.4%	100% Panic Frame Intact	0.19 ms
spring-projects/spring-boot	`JUnit / Java`	AutoConfiguration Failure Trace	774 tokens	290 tokens	-62.5%	100% Assertion Error Intact	0.22 ms
supabase/postgrest	`PostgreSQL JSON / Database`	500-Row JSON Query Result	14,336 tokens	735 tokens	-94.9%	100% Bounds & Schema	0.35 ms
AGGREGATE BENCHMARK	All 6 Production Repositories	Combined Execution Bloat	17,823 tokens	2,589 tokens	-85.5%	100% Diagnostic Fidelity	0.44 ms (p95)

Semantic Graph Slicing & Prefix Cache Pinning (v8.9.0)

Evaluated against top open-source multi-file architectures (Redux, Express, FastAPI, Ripgrep, React). Prunes orthogonal files with 100% causal completeness in 0.25ms (mean) and aligns prompts to 1,024-token boundaries for guaranteed prefix cache hits.

-75.6%

Aggregate Reduction

0.29ms

p95 Latency

100%

Causal Completeness

93%-95%

Cost Saved (1k Turns)

Repository	Language / Architecture	Raw Context	Sliced Context	Token Reduction	p95 Latency	Causal Completeness	Prefix Invariance	1k Turns Cost Saved
reduxjs/redux	`TypeScript / State Flow`	1,610 tokens	282 tokens	-82.5%	0.29 ms	100% Complete	100% Byte Invariant	93% ($4.83 → $0.36)
expressjs/express	`JavaScript / Router Graph`	1,613 tokens	468 tokens	-71.0%	0.18 ms	100% Complete	100% Byte Invariant	93% ($4.84 → $0.36)
tiangolo/fastapi	`Python / Dependency Injection`	1,943 tokens	515 tokens	-73.5%	0.29 ms	100% Complete	100% Byte Invariant	94% ($5.83 → $0.36)
BurntSushi/ripgrep	`Rust / Go Architecture`	2,662 tokens	760 tokens	-71.5%	0.41 ms	100% Complete	100% Byte Invariant	95% ($7.99 → $0.36)
facebook/react	`TypeScript / Fiber Reconciler`	785 tokens	75 tokens	-90.4%	0.09 ms	100% Complete	100% Byte Invariant	85% ($2.36 → $0.36)
AGGREGATE BENCHMARK	All 5 Canonical Repositories	8,613 tokens	2,100 tokens	-75.6%	0.25 ms (mean)	100.0% Complete	100.0% Byte Invariant	93.2% Cost Saved

Community MCP Server	Description	Tools	Raw Context	Compact Schemas	TypeScript Signatures	Full Tokonomics Firewall	Overhead Latency
@modelcontextprotocol/server-github	Repositories, branches, commits, PRs, issues & code	20 tools	3,607 tokens	3,262 (-9.6%)	2,484 (-31.1%)	970 (-73.1%)	2.46 ms
@modelcontextprotocol/server-postgres	Read-only queries, DDL, table inspection & explain	7 tools	856 tokens	716 (-16.4%)	684 (-20.1%)	622 (-27.3%)	2.42 ms
@modelcontextprotocol/server-filesystem	Workspace files, directories, editing, search & read	9 tools	1,087 tokens	983 (-9.6%)	849 (-21.9%)	722 (-33.6%)	2.25 ms
@modelcontextprotocol/server-brave-search	Web search, local search & live HTML content fetch	3 tools	359 tokens	317 (-11.7%)	297 (-17.3%)	658 (Paging)	2.30 ms
@modelcontextprotocol/server-puppeteer	Chromium navigation, clicking, screenshots & DOM	6 tools	661 tokens	597 (-9.7%)	554 (-16.2%)	626 (Paging)	2.19 ms
@modelcontextprotocol/server-slack	Channels, threads, messaging & emoji reactions	5 tools	647 tokens	577 (-10.8%)	525 (-18.9%)	896 (Paging)	2.07 ms
@modelcontextprotocol/server-sentry	Error crashes, stack traces, Discover events & triage	5 tools	585 tokens	509 (-13.0%)	490 (-16.2%)	723 (Paging)	2.20 ms
CUMULATIVE MULTI-SERVER AGENT RUN	All 7 servers active simultaneously	55 tools	7,796 tokens	6,949 (-10.9%)	5,871 (-24.7%)	779 (-90.0%)	2.31 ms

Anthropic

Claude 3.7 Sonnet ($3.00 / M)

+$60.61 saved per 1k requests

Calculated on the 20,204 tokens removed in Balanced mode. Reduces typical monthly multi-turn agent spend by $300–$750/seat.

OpenAI

GPT-4o ($2.50 / M)

+$50.51 saved per 1k requests

Eliminates repetitive context inflation in Copilot Chat and Cursor Composer while ensuring full API compatibility.

Economy Tier

Claude 3.5 Haiku ($0.80 / M)

+$16.16 saved per 1k requests

Maximizes speed and throughput on lightweight code explanation and quick refactoring passes.

DeepSeek

DeepSeek V3 ($0.14 / M)

+$2.83 saved per 1k requests

Ultra-efficient local/cloud execution where prompt caching and AST pruning maximize context headroom.

THE PROMPT CACHING DIVIDEND

Deterministic Skeletons Unlock 90% Cache Discounts

Both Anthropic and OpenAI offer an 80%–90% discount on cache-read input tokens when prompt prefixes are stable. Because Tokonomics generates deterministic, secret-sanitized AST skeletons, conversational turns share identical prefixes—compounding token savings across long coding sessions.

✅

100% Exported Signature Integrity

Every exported class, interface, function declaration, and generic type argument is preserved verbatim. The model always knows exactly how to invoke your code.

🛡️

Control Flow & Guard Preservation

Balanced (T1) mode retains guard clauses, early returns, throw expressions, and catch block shapes, preventing the LLM from misunderstanding boundary constraints.

⚡

Sub-10ms Multi-Language AST Engine

Runs in-memory with dedicated parsers for C, C++, Python, TypeScript, JavaScript, Go, Rust, Java, C#, Ruby, Swift, and Kotlin. Zero network round-trips, zero external dependencies.

🔄

Deterministic Fail-Closed Fallback

If an unknown language or corrupted file is detected, Tokonomics fails closed to safe pass-through without dropping user instructions or breaking requests.