starred/minne
1
0
Fork 0
mirror of https://github.com/perstarkse/minne.git synced 2026-09-09 19:41:46 +02:00
Code Issues Packages Projects Releases 10 Wiki Activity
423 Commits 4 Branches 19 Tags
5724f11dc1b25976e6456e5b6fe7340351733a50
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Per Stark 5724f11dc1 chore: harden knowledge graph storage and clear common clippy warnings 
Enforce stable 1:1 entity embeddings, relationship endpoint auth, and
user-scoped deletes; align schemas/migrations and resolve common crate
clippy findings.
 2026-05-29 14:44:23 +02:00
Per Stark 189adb1a5f chore: harden analytics, conversation access, and per-user file dedup 
Use UPSERT for analytics counters, enforce message ownership in SQL,
return
NotFound when patch_title updates nothing, scope file dedup by user_id
with a
composite unique index, and expand tests for auth, ordering, and edge
cases.
 2026-05-29 14:44:23 +02:00
Per Stark 97beb91710 chore: optimize ingest payloads and add parallel task batch store 
Parse content before building file payloads to move shared metadata when
possible, add create_all_and_add_to_db for concurrent stores, and extend
tests for batch persistence and payload edge cases.
 2026-05-29 14:44:23 +02:00
Per Stark 85336d77a3 chore: harden common errors, fastembed blocking, and ingest ownership 
Run FastEmbed inference on spawn_blocking, propagate Surreal take
failures,
add AppError::internal and typed ingest/embedding parse errors, and take
owned file lists in ingestion payload construction.
 2026-05-29 14:44:23 +02:00
Per Stark 9d5e7cd794 chore: improved error handling 2026-05-28 19:58:14 +02:00
Per Stark 30bb59f243 chore: rename get_id to id, add doc comments, pre-allocate format_history 2026-05-27 18:06:16 +02:00
Per Stark 224a7db451 chore: lowercase all error messages and add # Errors doc sections 
- Fix err-lowercase-msg: normalize all #[error(...)] display strings to
  lowercase (AppError, FileError, ApiErr) and update affected tests
- Fix err-doc-errors: add # Errors sections to 25+ fallible public
  functions across db.rs, store.rs, embedding.rs, indexes.rs,
  ingestion_task.rs, and ingest_limits.rs
 2026-05-27 14:59:48 +02:00
Per Stark 4579725130 chore: resolve remaining uninlined_format_args clippy warnings 2026-05-27 14:34:37 +02:00
Per Stark 0b08801c90 chore: fix and reduce clippy allows in knowledge_entity.rs 
- rm duplicate 'document' match arm (match_same_arms)
- .get(0) -> .first() (get_first)
- for entity in all_entities.iter() -> &all_entities (explicit_iter_loop)
- 2x error!("{}", err_msg) -> error!("{err_msg}") (uninlined_format_args)
- 2x test format!()/assert!() positional -> inlined (uninlined_format_args)
- removed 6 now-unnecessary allow attributes
 2026-05-27 14:28:08 +02:00
Per Stark 45d13230a6 chore: add must_use to 27 non-Result public functions 
- constructors: KnowledgeEntity, TextChunk, Scratchpad, IngestionTask,
  Conversation, KnowledgeRelationship, Message, TextContent,
  KnowledgeEntityEmbedding, TextChunkEmbedding
- accessors: Theme::as_str, Theme::initial_theme, TaskState::as_str,
  TaskState::display_label, StorageManager::backend_kind,
  StorageManager::local_base_path, EmbeddingProvider::backend_label,
  EmbeddingProvider::dimension, EmbeddingProvider::model_code
- queries: TaskState::is_terminal, IngestionTask::can_retry,
  KnowledgeEntityType::variants, StorageManager::resolve_local_path,
  resolve_base_dir, IngestionTask::lease_duration
- helpers: Message::format_history
- builders: StorageManager::with_backend
 2026-05-27 14:23:56 +02:00
Per Stark 0acdba4f54 fix: replace manual embedding serialization with serde_json 
- replaced write!() loops with serde_json::to_string in 4 re-embedding methods
- standardized SQL building to use write!() with proper error propagation
- eliminates manual f32 vector string building (memory waste + loop risk)
 2026-05-27 14:13:19 +02:00
Per Stark 9609880cff fix: revoke_api_key sets NONE, remove unused bind, lowercase error msgs 
- fix bug where revoke_api_key set literal 'test_string_nullish' instead of NONE
- remove unused table_name bind in update_timezone
- lowercase ~16 error messages across 4 crates
 2026-05-27 13:56:32 +02:00
Per Stark 31d585b59f chore: removed anyhow from apperror for improved error handling 2026-05-27 13:33:02 +02:00
Per Stark 890a4b381d chore: index slicing and lowercase errors 2026-05-27 12:41:26 +02:00
Per Stark 2d630e2af9 chore: tightening and removing super fn 2026-05-27 11:23:39 +02:00
Per Stark 9ec11e1f79 chore: clippy and nix fmt 2026-05-27 11:23:08 +02:00
Per Stark c60db0fb56 perf: avoid small own clones and intermediate Vec allocations 
- Derive Copy on 6 small enums (MessageRole, TaskState, StorageKind, EmbeddingBackend, PdfIngestMode, KnowledgeEntityType)
- Change create_ingestion_payload files param from Vec<FileInfo> to &[FileInfo]
- Remove 5 intermediate Vec allocations (4 embedding serialization + 1 format_history) using write! loop
- Remove 7 unnecessary .clone() calls exposed by Copy derive
 2026-05-27 10:28:08 +02:00
Per Stark f5f0454904 fix: html-router dependency of json-stream-parser 2026-05-27 09:59:26 +02:00
Per Stark 18aadab8ee refactor: json-stream-parser aligned to clippy standard 2026-05-27 09:07:38 +02:00
Per Stark 414d2f5b34 chore: additional clippy fixes after rebasing 2026-05-27 07:37:18 +02:00
Per Stark 293440b0ee fix: pin surrealdb 2026-05-26 20:21:40 +02:00
Per Stark 041d9bd81f clippy: evaluations crate 2026-05-26 20:21:25 +02:00
Per Stark b4383bb227 perf: pre-allocate collections with known capacity in hot paths 
- Use with_capacity for chunk_by_source, results, per_entity_traces,
  and selected_chunks in assemble() where bound is known
- Pre-allocate tokens/terms vectors in normalize_fts_query and
  extract_keywords based on input length
- Pre-allocate neighbor_ids, seen, and ordered in graph expansion
  based on relationship count
 2026-05-26 20:21:25 +02:00
Per Stark 6c7b586fc5 perf: offload blocking calls to spawn_blocking 
- Move headless_chrome PDF rasterization from async context to
  spawn_blocking, keeping tokio worker threads responsive.
- Switch RerankerPool from tokio::sync::Mutex to std::sync::Mutex
  and run TextRerank::rerank inside spawn_blocking, since the
  rerank call is CPU-bound with no .await points.
 2026-05-26 20:21:25 +02:00
Per Stark 1927149ce9 lint: inherit workspace clippy config in json-stream-parser and evaluations 
Both crates were missing the [lints] workspace = true directive,
bypassing workspace clippy rules (unwrap_used, expect_used, etc.).
 2026-05-26 20:21:25 +02:00
Per Stark a52dc802de refactor: simplify and improve testing for initialization 2026-05-26 20:21:24 +02:00
Per Stark 000852c94c clippy: adhere to pedantic clippy, uniform test error handling 2026-05-26 20:21:13 +02:00
Per Stark 6a5d631287 chore: remove unused clap dep and fix test_session_table name 
- Remove clap dependency from retrieval-pipeline (RetrievalStrategy
  already has FromStr/Display; evaluations uses clap directly)
- Rename session table from test_session_table to session
 2026-05-26 20:14:29 +02:00
Per Stark b965c5a2e6 refactor: replace Box<dyn Error> with anyhow::Result 
- ingestion_pipeline::run_worker_loop returns anyhow::Result<()>
- api_router::ApiState::new returns anyhow::Result<Self>
- html_router::HtmlState::new_with_resources is infallible, returns Self
- main/server/worker binary entry points return anyhow::Result<()>
 2026-05-26 20:14:11 +02:00
Per Stark 79e46e9c09 refactor: extract serde helpers from stored_object! macro 
Move FlexibleIdVisitor, deserialize_flexible_id, and four datetime serde helpers
from repeating inside every macro expansion into a shared
common/src/storage/types/serde_helpers.rs module.

14 macro invocations × 6 items = ~84 fewer redundant function definitions.
Fragile cross-module imports (file_info::deserialize_flexible_id etc.)
are updated to point to the canonical module.
 2026-05-26 20:12:54 +02:00
Per Stark f22a1e5ba4 chore: devenv inconsistency, spawn server manually in dev 2026-02-15 18:31:43 +01:00
Per Stark 4d237ff6d9 release: 1.0.2 2026-02-15 11:57:04 +01:00
Per Stark eb928cdb0e test: minio to devenv, improved testing s3 and relationships 2026-02-15 08:52:56 +01:00
Per Stark 1490852a09 chore: dep updates & kv-mem separation to test feature 
docker builder update
 2026-02-15 08:51:48 +01:00
Per Stark b0b01182d7 test: add admin auth integration coverage 2026-02-14 23:11:35 +01:00
Per Stark 679308aa1d feat: caching chat history & dto 2026-02-14 19:43:34 +01:00
Per Stark f93c06b347 fix: harden html responses and cache chat sidebar data 
Use strict template response handling and sanitized template user context, then add an in-process conversation archive cache with mutation-driven invalidation for chat sidebar renders.
 2026-02-14 17:47:14 +01:00
Per Stark a3f207beb1 fix: simplified admin checking 2026-02-13 23:04:01 +01:00
Per Stark e07199adfc fix: name harmonization of endpoints & ingestion security hardening 2026-02-13 22:36:00 +01:00
Per Stark f22cac891c fix: redact ingestion payload logs and update changelog 2026-02-13 12:06:18 +01:00
Per Stark b89171d934 fix: parameterize storage-layer queries and add injection tests 2026-02-12 21:42:46 +01:00
Per Stark 0133eead63 fix: border in navigation 2026-02-12 20:39:36 +01:00
Per Stark e5d2b6605f fix: browser back navigation from chat windows 
addenum
 2026-02-12 20:32:06 +01:00
Per Stark bbad91d55b fix: references bug 
fix
 2026-02-11 22:02:40 +01:00
Per Stark 96846ad664 release: 1.0.1 2026-02-11 15:39:28 +01:00
Per Stark 269bcec659 docs: updated domain name 2026-02-11 15:17:03 +01:00
Per Stark 7c738c4b30 fix: gracefully handle old users 2026-02-11 07:50:19 +01:00
Per Stark cb88127fcb docs: updated readme 2026-01-18 18:48:53 +01:00
Per Stark 49e1fbd985 dev: devenv processes 2026-01-18 18:45:30 +01:00
Per Stark f2fa5bbbcc fix: edge case when deleting content 
nit
 2026-01-18 18:45:21 +01:00
Per Stark a3bc6fba98 design: better dark mode 2026-01-17 23:31:05 +01:00
Per Stark ece744d5a0 refactor: additional responsibilities to middleware, simplified handlers 
fix
 2026-01-17 21:07:25 +01:00
Per Stark a9fda67209 theme: obsidian-prism 2026-01-17 08:45:47 +01:00
Per Stark fa7f407306 feat: s3 storage backend 2026-01-16 23:38:47 +01:00
Per Stark b25cfb4633 feat: add user theme preference 
- Add theme field to User model (common)
- Create migration for theme field
- Add theme selection to Account Settings (html-router)
- Implement server-side theme rendering in base template
- Update JS for system/preference theme handling
- Remove header theme toggle for authenticated users
 2026-01-16 13:54:07 +01:00
Per Stark 0df2b9810c docs: addenum 2026-01-14 22:24:23 +01:00
Per Stark 354dc727c1 refactor: extendable templates 
refactor: simplification

refactor: simplification
 2026-01-13 22:18:00 +01:00
Per Stark 037057d108 fix: allow for multiple templates directories 2026-01-12 21:25:12 +01:00
Per Stark 9f17c6c2b0 fix: updating models in admin view 2026-01-12 21:01:53 +01:00
Per Stark 17f252e630 release: 1.0.0 
fix: cargo dist
 2026-01-11 20:35:01 +01:00
Per Stark db43be1606 fix: schemafull and textcontent 2026-01-02 15:41:22 +01:00
Per Stark 8e8370b080 docs: more complete and correct 2025-12-24 23:36:58 +01:00
Per Stark 84695fa0cc chore: wording 2025-12-22 23:03:33 +01:00
Per Stark 654add98bc fix: never block fts, rely on rrf 2025-12-22 22:56:57 +01:00
Per Stark 244ec0ea25 fix: migrating embeddings to new dimensions 
changing order
 2025-12-22 22:39:14 +01:00
Per Stark d8416ac711 fix: ordering of index creation 2025-12-22 21:59:35 +01:00
Per Stark f9f48d1046 docs: evaluations instructions and readme refactoring 2025-12-22 18:55:47 +01:00
Per Stark 30b8a65377 fix: migrations 
schemafull
 2025-12-22 18:32:08 +01:00
Per Stark 04faa38ee6 fix: admin page sorted 2025-12-21 21:35:52 +01:00
Per Stark cdc62dda30 Merge branch 'main' into benchmarks 2025-12-20 23:09:16 +01:00
Per Stark ab8ff8b07a changelog 2025-12-20 23:03:06 +01:00
Per Stark 79ea007b0a tidying stuff up, dto for search 2025-12-20 22:30:31 +01:00
Per Stark a5bc72aedf passed wide smoke check 2025-12-10 13:54:08 +01:00
Per Stark 2e2ea0c4ff faster index creation 2025-12-09 21:32:23 +01:00
Per Stark a090a8c76e retrieval simplfied 2025-12-09 20:35:42 +01:00
Per Stark a8d10f265c benchmarks: fin 2025-12-08 21:57:53 +01:00
Per Stark 0cb1abc6db beir-rff 2025-12-08 20:39:12 +01:00
Per Stark d1a6d9abdf dataset: beir 2025-12-04 17:50:35 +01:00
Per Stark d3fa3be3e5 retrieval: hybrid search, linear fusion 2025-12-04 12:48:59 +01:00
Per Stark a2c9bb848d release: 0.2.7 2025-12-04 12:25:46 +01:00
Per Stark dd881efbf9 benchmarks: ready for hybrid revised 2025-12-03 11:38:07 +01:00
Per Stark 2939e4c2a4 fix: removed stale embeddings handler 2025-11-29 20:07:48 +01:00
Per Stark 1039ec32a4 fix: all tests now in sync 2025-11-29 18:59:08 +01:00
Per Stark cb906c5b53 ndcg fix 2025-11-29 16:24:09 +01:00
Per Stark 08b1612fcb refactored to clap, mrr and ndcg 2025-11-28 21:26:51 +01:00
Per Stark 67004c9646 fix: index creation at init 2025-11-26 21:49:20 +01:00
Per Stark 030f0fc17d evals: v3, ebeddings at the side 
additional indexes
 2025-11-26 15:15:10 +01:00
Per Stark 226b2db43a retrieval-pipeline: v1 2025-11-19 12:58:27 +01:00
Per Stark 6f88d87e74 fix: add dockerfile changes related to retrieval-pipeline 2025-11-18 22:51:48 +01:00
Per Stark bd519ab269 benchmarks: v2 
Minor refactor
 2025-11-18 22:51:06 +01:00
Per Stark f535df7e61 retrieval-pipeline: v0 2025-11-18 22:46:35 +01:00
Per Stark 6b7befbd04 upsert relationship and creation 2025-11-18 21:18:09 +01:00
Per Stark 0eda65b07e benchmarks: v1 
Benchmarking ingestion, retrieval precision and performance
 2025-11-18 11:50:15 +01:00
Per Stark 04ee225732 design: improved admin page, new structure 2025-11-04 20:42:24 +01:00
Per Stark 13b7ad6f3a fix: added cargo lock to crane build 2025-11-04 12:59:32 +01:00
Per Stark 112a6965a4 Merge branch 'main' into development 2025-11-03 12:48:04 +01:00
Per Stark 911e830be5 Merge branch 'development' of github.com:perstarkse/minne into development 2025-11-03 12:40:36 +01:00
Per Stark 3196e65172 fix: improved storage manager, prep for s3 2025-11-03 12:39:15 +01:00
Per Stark 380c900c86 release: 0.2.6 
dist update

fix new workflow

fix

mkdir

moved to dist

fix only dir

dont verify sha files

fix verify ci part

fix

no checking anymore
 2025-11-01 21:26:06 +01:00
Per Stark a99e5ada8b Merge pull request #5 from josephleee/patch-1 
Update README.md
 2025-10-31 13:40:06 +01:00