minne/CHANGELOG.md

Changelog

Unreleased

• Infra: CI workflow fixes. CI is now a nix flake check which includes compilation, caching and running tests, clippy, fmt, validation for ort version.
• Docker-compose: The example now references the ghcr image, this is so we can remove the Dockerfile and reducing maintenance scope.
• Refactor: web scraping now uses servo-fetch (pure-Rust Servo engine) and PDF rendering uses pdfium-render (direct PDFium bindings) — reduces Docker image size by ~300MB, improves startup latency by ~100× for PDF rendering, and provides more stable output
• Fix: added pkgs.libglvnd to LD_LIBRARY_PATH in devenv so Servo engine can find libEGL.so at runtime
• Fix: updated Dockerfile to add libegl1 libegl-mesa0 libgles2 libfontconfig1 libfreetype6 runtime dependencies for servo-fetch
• Docs: updated architecture, features, and installation docs to reflect the new web processing stack
• Fix: added pre-commit hooks to further maintain code consistency.
• Security: updated some deps because dependabot told me, good bot.
• Security: bump async-openai to 0.41.1 (feature-gated types, transcription API rename; removes backoff transitive dep)
• Refactor: deduplicated test database setup across common/src/storage/.
• Refactor: split knowledge-graph.js monolith into focused functions.
• Evaluations: simplified crate layout — linear pipeline, sharded-only converted store, in-memory ingestion, db/ and cli/ modules; namespace reuse state in corpus manifest (removed cache/snapshots/); no legacy JSON/history compatibility (re-run --warm after upgrade)
• Performance: ingestion skips per-task index rebuild; worker runs scheduled REBUILD INDEX (default every 24h via index_rebuild_interval_secs, 0 disables)
• Performance: ingestion persists all artifacts in a single SurrealDB transaction per task (atomic replace by task id)
• Performance: entity embeddings during ingestion use batched embed_batch, matching chunk embedding
• Fix: ingestion reclaims tasks after a successful persist without re-running the pipeline when mark_succeeded failed
• Fix: content deletion clears graph relationships via shared TextContent::clear_ingested_children
• Fix: regression re suggestion of relationships
• Internal: extracted duplicate entity+embedding patterns into HasEmbedding and EmbeddingRecord traits with generic store_with_embedding, delete_by_source_id, and vector_search on SurrealDbClient.
• Infra: ort-version file removed — version inlined in flake.nix and devenv.nix; release.yml reads it via nix eval .#lib.ortVersion from the plan job
• Infra: screenshot-graph.webp and .dockerignore deleted — stale artifacts from Dockerfile era

1.0.3 (2026-06-12)

• Search: filter results by type — knowledge entities, ingested content, or both
• Admin: choose the local FastEmbed model from the admin UI; changes save immediately and apply after restart (re-embeds when the vector dimension changes)
• Performance: pooled FastEmbed workers and batched embedding generation for faster ingestion and search
• Performance: lower search and chat latency from backend allocation and retrieval optimizations
• Fix: modal dialogs (scratchpad editor, admin prompts, entity creation) open and close more reliably
• Fix: improved knowledge-entity relationship suggestions when creating entities manually
• Fix: API key revocation now correctly clears the stored key

1.0.2 (2026-02-15)

• Fix: edge case where navigation back to a chat page could trigger a new response generation
• Fix: chat references now validate and render more reliably
• Fix: improved admin access checks for restricted routes
• Performance: faster chat sidebar loads from cached conversation archive data
• API: harmonized ingest endpoint naming and added configurable ingest safety limits
• Security: hardened query handling and ingestion logging to reduce injection and data exposure risk

1.0.1 (2026-02-11)

• Shipped an S3 storage backend so content can be stored in object storage instead of local disk, with configuration support for S3 deployments.
• Introduced user theme preferences with the new Obsidian Prism look and improved dark mode styling.
• Fixed edge cases, including content deletion behavior and compatibility for older user records.

1.0.0 (2026-01-02)

• Locally generated embeddings are now default. If you want to continue using API embeddings, set EMBEDDING_BACKEND to openai. This will download a ONNX model and recreate all embeddings. But in most instances it's very worth it. Removing the network bound call to create embeddings. Creating embeddings on my N100 device is extremely fast. Typically a search response is provided in less than 50ms.
• Added a benchmarks create for evaluating the retrieval process
• Added fastembed embedding support, enables the use of local CPU generated embeddings, greatly improved latency if machine can handle it. Quick search has vastly better accuracy and is much faster, 50ms latency when testing compared to minimum 300ms.
• Embeddings stored on own table.
• Refactored retrieval pipeline to use the new, faster and more accurate strategy. Read blog post for more details.

Version 0.2.7 (2025-12-04)

• Improved admin page, now only loads models when specifically requested. Groundwork for coming configuration features.
• Fix: timezone aware info in scratchpad

Version 0.2.6 (2025-10-29)

• Added an opt-in FastEmbed-based reranking stage behind reranking_enabled. It improves retrieval accuracy by re-scoring hybrid results.
• Fix: default name for relationships harmonized across application

Version 0.2.5 (2025-10-24)

• Added manual knowledge entity creation flows using a modal, with the option for suggested relationships
• Scratchpad feature, with the feature to convert scratchpads to content.
• Added knowledge entity search results to the global search
• Backend fixes for improved performance when ingesting and retrieval

Version 0.2.4 (2025-10-15)

• Improved retrieval performance. Ingestion and chat now utilizes full text search, vector comparison and graph traversal.
• Ingestion task archive

Version 0.2.3 (2025-10-12)

• Fix changing vector dimensions on a fresh database (#3)

Version 0.2.2 (2025-10-07)

• Support for ingestion of PDF files
• Improved ingestion speed
• Fix deletion of items work as expected
• Fix enabling GPT-5 use via OpenAI API

Version 0.2.1 (2025-09-24)

• Fixed API JSON responses so iOS Shortcuts integrations keep working.

Version 0.2.0 (2025-09-23)

• Revamped the UI with a neobrutalist theme, better dark mode, and a D3-based knowledge graph.
• Added pagination for entities and content plus new observability metrics on the dashboard.
• Enabled audio ingestion and merged the new storage backend.
• Improved performance, request filtering, and journalctl/systemd compatibility.

Version 0.1.4 (2025-07-01)

• Added image ingestion with configurable system settings and updated Docker Compose docs.
• Hardened admin flows by fixing concurrent API/database calls and normalizing task statuses.

Version 0.1.3 (2025-06-08)

• Added support for AI providers beyond OpenAI.
• Made the HTTP port configurable for deployments.
• Smoothed graph mapper failures, long content tiles, and refreshed project documentation.

Version 0.1.2 (2025-05-26)

• Introduced full-text search across indexed knowledge.
• Polished the UI with consistent titles, icon fallbacks, and improved markdown scrolling.
• Fixed search result links and SurrealDB vector formatting glitches.

Version 0.1.1 (2025-05-13)

• Added streaming feedback to ingestion tasks for clearer progress updates.
• Made the data storage path configurable.
• Improved release tooling with Chromium-enabled Nix flakes, Docker builds, and migration/template fixes.

Version 0.1.0 (2025-05-06)

• Initial release with a SurrealDB-backed ingestion pipeline, job queue, vector search, and knowledge graph storage.
• Delivered a chat experience featuring streaming responses, conversation history, markdown rendering, and customizable system prompts.
• Introduced an admin console with analytics, registration and timezone controls, and job monitoring.
• Shipped a Tailwind/daisyUI web UI with responsive layouts, modals, content viewers, and editing flows.
• Provided readability-based content ingestion, API/HTML ingress routes, and Docker/Docker Compose tooling.