feat: web UI completion and sub-1s indexing with native TS parsing
4 months ago
Authored
JoeJoeflynWeb UI (10 phases):
- Complete management UI with all bug fixes
Indexing performance:
- Replace ts-morph with native TypeScript compiler APIs + regex parsing
- Use length/4 token approximation instead of gpt-tokenizer for counting
- Wrap indexing loop in a single SQLite transaction
- Lazy getText() — skip for interface/type-only symbols
- Skip import traversal for files without exports
- Batch FTS inserts + sync DB methods for hot path
Graph status fix:
- Create graph_runs records during indexing
- Update registry graphStatus to completed + lastGraphBuiltAt
Memories fix:
- Add memories_fts table for full-text search on memories
- memory_query now searches memories alongside chunks
- Add searchMemories method to repository
Cleanup:
- Remove unused files: vector.ts, sqlite-index.ts, language.ts, tooltip.tsx
- Remove duplicate UI components (ui/button, ui/input) and lib/cn.ts
- Remove unused deps: drizzle-orm, ollama, openai, zod, js-yaml, remark-*, unified, unist-util-visit, toml
- Clean up tsup noExternal list
Index time: 1.58s → 0.88s wall clock for 163 files. Bundle: 14.86 MB → 12.19 MB.
Parentd1d2f38
145 files changed
- .claude/CLAUDE.md+379−0
- .env.example+6−8
- .gitignore+3−0
- LICENSE+21−0
- README.md+112−36
- apps/cli/.gitignore+2−0
- apps/cli/LICENSE+21−0
- apps/cli/README.md+112−0
- apps/cli/package.json
+54
−4
apps/cli/src/cli.ts+63−11apps/cli/src/mcp-bridge.ts+1−1apps/cli/src/resolve-cli.ts+54−0apps/cli/src/setup-claude.ts+5−5apps/cli/src/setup-codex.ts+4−5apps/cli/src/setup-opencode.ts+5−5apps/cli/src/web-server.ts+4−0apps/cli/tsup.config.ts+51−0apps/mcp/package.json+3−2apps/mcp/src/mcp-core.ts+120−0apps/web/package.json+11−3apps/web/src/components/chunk-boundary-bar.tsx+118−0apps/web/src/components/chunk-content.tsx+119−0apps/web/src/components/documents/documents-columns.tsx+72−0apps/web/src/components/documents/documents-data-table.tsx+159−0apps/web/src/components/documents/documents-filters.tsx+81−0apps/web/src/components/graph/GraphClient.tsx+93−13apps/web/src/components/graph/WorkspaceGraph.tsx+203−123apps/web/src/components/indexing-progress.tsx+138−0apps/web/src/components/memory-detail-panel.tsx+118−0apps/web/src/components/sidebar.tsx+38−27apps/web/src/components/structured-content-viewer.tsx+133−0apps/web/src/components/theme-toggle.tsx+3−3apps/web/src/components/workspace-selector.tsx+71−0apps/web/src/globals.css+281−11apps/web/src/lib/api.ts+155−7apps/web/src/lib/constants.ts+114−0apps/web/src/lib/pagination.tsx+16−9apps/web/src/lib/queries.ts+126−23apps/web/src/lib/use-sse.ts+189−0apps/web/src/lib/utils.ts+8−0apps/web/src/main.tsx+5−2apps/web/src/routeTree.gen.ts+62−17apps/web/src/routes/__root.tsx+56−2apps/web/src/routes/documents.tsx+105−35apps/web/src/routes/index.tsx+27−18apps/web/src/routes/jobs.tsx+0−77apps/web/src/routes/memories.tsx+140−0apps/web/src/routes/query.tsx+408−97apps/web/src/routes/settings.tsx+50−39apps/web/src/routes/workspaces/$workspaceId/documents/$documentId/chunks.tsx+318−0apps/web/src/routes/workspaces/$workspaceId/graph.tsx+6−6apps/web/src/routes/workspaces/$workspaceId/index.tsx+419−76apps/web/src/routes/workspaces/$workspaceId/symbols.tsx+240−0apps/web/src/routes/workspaces/index.tsx+11−9apps/web/src/routes/workspaces/new.tsx+15−16apps/web/src/server/index.ts+571−59apps/web/src/server/run-tracker.ts+112−0apps/web/src/server/sqlite.ts+753−24apps/web/src/server/start.ts+10−0apps/web/vite.config.ts+3−2apps/worker/src/cli.ts+0−3install.ps1+74−0install.sh+94−0package.json+2−6packages/config/src/env.ts+7−7packages/config/src/load-brain-config.ts+4−14packages/core/package.json+1−5packages/core/src/embeddings.ts+0−87packages/core/src/index.ts+0−1packages/core/src/retrieval.ts+45−44packages/core/src/tokenizer.ts+13−5packages/core/src/types.ts+12−0packages/core/src/vector.ts+0−3packages/db/package.json+1−7packages/db/src/index.ts+0−3packages/db/src/sqlite-index.ts+0−12packages/db/src/sqlite/database-loader.ts+5−1packages/db/src/sqlite/index.ts+1−1packages/db/src/sqlite/repository.ts+380−77packages/db/src/sqlite/schema.ts+2−15packages/db/src/sqlite/types.ts+132−20packages/db/src/sqlite/workspace-db.ts+74−11packages/indexer/package.json+1−9packages/indexer/src/code.ts+218−106packages/indexer/src/index-workspace.ts+271−159packages/indexer/src/language.ts+0−1packages/indexer/src/languages.ts+68−15packages/indexer/src/markdown.ts+1−1packages/indexer/src/types.ts+0−9packages/queue/src/index.ts+0−1packages/ui/package.json+7−3packages/ui/src/components/badge.tsx+1−1packages/ui/src/components/button.tsx+2−1packages/ui/src/components/card.tsx+1−1packages/ui/src/components/code-block.tsx+1−1packages/ui/src/components/input.tsx+1−1packages/ui/src/components/label.tsx+1−1packages/ui/src/components/select.tsx+175−13packages/ui/src/components/table.tsx+1−1packages/ui/src/components/textarea.tsx+1−1packages/ui/src/components/tooltip.tsx+0−91packages/ui/src/components/ui/alert-dialog.tsx+84−0packages/ui/src/components/ui/button.tsx+0−67packages/ui/src/components/ui/input.tsx+0−19packages/ui/src/components/ui/sheet.tsx+1−1packages/ui/src/components/ui/sidebar.tsx+4−4packages/ui/src/components/ui/slider.tsx+69−0packages/ui/src/icons/animated-icon.tsx+51−0packages/ui/src/icons/arrow-back-icon.tsx+69−0packages/ui/src/icons/arrow-narrow-right-icon.tsx+71−0packages/ui/src/icons/book-icon.tsx+105−0packages/ui/src/icons/brain-circuit-icon.tsx+137−0packages/ui/src/icons/brightness-down-icon.tsx+87−0packages/ui/src/icons/bug-icon.tsx+85−0packages/ui/src/icons/checked-icon.tsx+105−0packages/ui/src/icons/clock-icon.tsx+83−0packages/ui/src/icons/code-icon.tsx+62−0packages/ui/src/icons/code-xml-icon.tsx+77−0packages/ui/src/icons/down-chevron.tsx+62−0packages/ui/src/icons/file-description-icon.tsx+83−0packages/ui/src/icons/gear-icon.tsx+85−0packages/ui/src/icons/globe-icon.tsx+96−0packages/ui/src/icons/history-circle-icon.tsx+95−0packages/ui/src/icons/home-icon.tsx+73−0packages/ui/src/icons/index.ts+67−0packages/ui/src/icons/layers-icon.tsx+75−0packages/ui/src/icons/layout-dashboard-icon.tsx+104−0packages/ui/src/icons/layout-sidebar-right-icon.tsx+94−0packages/ui/src/icons/library-icon.tsx+91−0packages/ui/src/icons/logout-icon.tsx+82−0packages/ui/src/icons/magnifier-icon.tsx+78−0packages/ui/src/icons/moon-icon.tsx+71−0packages/ui/src/icons/plus-icon.tsx+54−0packages/ui/src/icons/refresh-icon.tsx+68−0packages/ui/src/icons/right-chevron.tsx+59−0packages/ui/src/icons/send-icon.tsx+75−0packages/ui/src/icons/shield-check.tsx+74−0packages/ui/src/icons/sparkles-icon.tsx+105−0packages/ui/src/icons/trash-icon.tsx+145−0packages/ui/src/icons/types.ts+41−0packages/ui/src/icons/user-plus-icon.tsx+112−0packages/ui/src/icons/x-icon.tsx+107−0packages/ui/src/index.ts+4−1packages/ui/src/lib/cn.ts+0−6tsconfig.base.json+3−1