Merge main into fix/fast-indexing: combine perf optimizations with embeddings v2
3 months ago
Authored
JoeJoeflyn- Keep all perf work: lazy graph building, fastHash, batch inserts, bun:sqlite,
template copy-on-init, streaming writes, worker pool, resolveBundledFile
- Integrate main's embeddings v2: insertEmbeddings with input_hash dedup,
secure-storage settings, hybrid retrieval, index_runs with embeddings_written
- Add embeddings migration (input_hash column, dedup, indexes) to workspace-db
- Add settings table to registry-db
- Fix tsup config: parse-worker entry + bun:sqlite external
- Fix resolveWorkerPath: add bundled worker path candidate
Parents1d792326177bda
119 files changed
- .env.example+11−1
- .husky/pre-commit+1−0
- .prettierignore+11−0
- .prettierrc+10−0
- .repomixignore+0−14
- .windsurf/skills/release-cli/SKILL.md+8−5
- BENCHMARK.md+61−40
- CHANGELOG.md+43−0
- README.md
+59
−11
apps/cli/README.md+39−26apps/cli/package.json+6−2apps/cli/src/resolve-cli.ts+3−3apps/cli/src/setup-claude.ts+1−1apps/cli/src/setup-codex.ts+1−1apps/cli/src/setup-devin.ts+1−1apps/cli/src/setup-opencode.ts+1−1apps/cli/src/setup-windsurf.ts+1−1apps/cli/tsup.config.ts+7−35apps/mcp/src/mcp-core.ts+262−150apps/web/src/components/context-blocks.tsx+17−14apps/web/src/components/graph/GraphClient.tsx+19−53apps/web/src/components/graph/WorkspaceGraph.tsx+19−26apps/web/src/components/sidebar.tsx+1−5apps/web/src/components/status-badge.tsx+5−1apps/web/src/lib/api.ts+49−5apps/web/src/lib/pagination.tsx+1−4apps/web/src/lib/queries.ts+12−8apps/web/src/lib/theme.tsx+1−3apps/web/src/main.tsx+1−1apps/web/src/routes/benchmark.tsx+72−16apps/web/src/routes/changelog.tsx+25−6apps/web/src/routes/documents.tsx+6−12apps/web/src/routes/index.tsx+89−25apps/web/src/routes/memories.tsx+12−16apps/web/src/routes/query.tsx+94−27apps/web/src/routes/settings.tsx+243−18apps/web/src/routes/workspaces/$workspaceId/graph.tsx+5−21apps/web/src/routes/workspaces/$workspaceId/index.tsx+41−72apps/web/src/routes/workspaces/index.tsx+18−11apps/web/src/routes/workspaces/new.tsx+25−8apps/web/src/server/index.ts+111−59apps/web/src/server/sqlite.ts+160−72brain.config.ts+8−8documents/backlogs/TREE_SITTER_UPGRADE_GUIDE.md+5−9documents/backlogs/embedding-review.md+1−1documents/backlogs/sigma.md+10−0landing/next-env.d.ts+1−1landing/package.json+5−1landing/public/og-animated.gif——landing/scripts/render-og-animated.mjs+282−0landing/src/app/components/animated-counter.tsx+3−2landing/src/app/components/dashboard-preview.tsx+3−9landing/src/app/components/footer.tsx+3−4landing/src/app/components/github-stars.tsx+1−3landing/src/app/components/hero-graph.tsx+5−10landing/src/app/components/interactive-graph.tsx+27−18landing/src/app/components/particle-text.tsx+13−26landing/src/app/components/typewriter.tsx+1−5landing/src/app/layout.tsx+3−6landing/src/app/opengraph-image.tsx+350−371landing/src/app/page.tsx+131−105landing/tsconfig.json+4−17package.json+10−1packages/config/src/env.ts+1−1packages/config/src/load-brain-config.ts+14−8packages/config/src/types.ts+3−3packages/core/src/embeddings.ts+146−34packages/core/src/memory.ts+7−8packages/core/src/retrieval.ts+128−56packages/core/src/rrf.ts+6−4packages/db/src/sqlite-index.ts+8−2packages/db/src/sqlite/index.ts+9−2packages/db/src/sqlite/local-workspace.ts+13−9packages/db/src/sqlite/registry-db.ts+6−0packages/db/src/sqlite/repository.ts+57−2packages/db/src/sqlite/schema.ts+90−35packages/db/src/sqlite/secure-storage.ts+116−0packages/db/src/sqlite/types.ts+132−71packages/db/src/sqlite/workspace-db.ts+56−0packages/indexer/src/index-workspace.ts+175−53packages/indexer/src/markdown.ts+11−5packages/indexer/src/retrieval-eval.ts+23−10packages/indexer/src/types.ts+1−0packages/ui/package.json+1−1packages/ui/src/components/badge.tsx+7−6packages/ui/src/components/button.tsx+5−10packages/ui/src/components/card.tsx+5−2packages/ui/src/components/code-block.tsx+1−1packages/ui/src/components/input.tsx+14−16packages/ui/src/components/label.tsx+6−3packages/ui/src/components/select.tsx+2−2packages/ui/src/components/table.tsx+58−45packages/ui/src/components/textarea.tsx+2−2packages/ui/src/components/tooltip.tsx+13−3packages/ui/src/components/ui/button.tsx+10−10packages/ui/src/components/ui/input.tsx+5−5packages/ui/src/components/ui/separator.tsx+7−7packages/ui/src/components/ui/sheet.tsx+27−43packages/ui/src/components/ui/sidebar.tsx+135−172packages/ui/src/components/ui/skeleton.tsx+3−3packages/ui/src/components/ui/tooltip.tsx+12−16packages/ui/src/hooks/use-mobile.ts+11−11packages/ui/src/index.ts+1−1packages/ui/src/lib/utils.ts+4−4pnpm-lock.yaml+433−403tests/chunking.test.ts+2−2tests/config-indexer.test.ts+1−1tests/e2e-search.test.ts+20−12tests/embed-benchmark.test.ts+376−0tests/fixtures/retrieval-eval.json+56−14tests/go-rust-indexer.test.ts+18−12tests/hybrid-retrieval.test.ts+111−34tests/index-workspace.test.ts+89−28tests/mcp-tools.test.ts+74−33tests/python-indexer.test.ts+46−37tests/retrieval-eval.test.ts+6−2tests/vector-search.test.ts+157−22tests/web-index-endpoint.test.ts+3−1tests/workspace-db.test.ts+35−15