3b96368021
refactor: replace headless_chrome with lighter alternatives
2026-06-21 21:43:33 +02:00
8ce612d7fe
chore: dependabot deps update
...
chore: openai version bump
2026-06-20 11:16:43 +02:00
84eff3af3c
chore: git-hooks rustfmt and clippy
2026-06-20 10:10:29 +02:00
3083a42396
refactor: extract generic ensure_fts_index helper
2026-06-19 08:10:49 +02:00
b58691af6a
refactor: consolidate test database setup in common/src/storage
2026-06-19 08:10:38 +02:00
0e2122b8f6
refactor: split knowledge-graph monolith and extract rubberbanding logic
2026-06-19 08:10:38 +02:00
3b3c2ac55e
chore: technical maintenance, reduced duplication
2026-06-18 14:58:13 +02:00
dbf8c91b1f
evals: eval crate overhaul, simplification and performance improvements
2026-06-17 19:23:11 +02:00
3aca33569d
perf: batch entity embeddings during ingest and expand retry tests.
...
Entity enrichment now uses embed_batch like chunks; the unused entity_embedding_concurrency knob is removed and ingest retry paths gain test coverage.
2026-06-12 18:40:36 +02:00
189c7d7c31
fix: atomic ingestion persist with task reclaim and shared cleanup.
...
One transaction per task replaces prior artifact rows; workers skip the pipeline when content already exists, eval seeding reuses persist_artifacts, and deletes clear graph children via shared SQL.
2026-06-12 16:27:07 +02:00
4947d48ecf
fix: don't close modal on inner HTMX requests.
...
changelog
2026-06-12 15:09:59 +02:00
4cd428185f
fix: schedule nightly index rebuild on worker and skip per-ingest rebuild.
...
Ingest relies on SurrealDB incremental index maintenance; the worker runs native REBUILD INDEX on a configurable interval with lease state on system_settings.
2026-06-12 15:01:53 +02:00
81797504d4
fix: load embedding dimensions once per persist and trim vector search select.
2026-06-12 13:54:51 +02:00
919e699287
release: 1.0.3
...
fix: load ort-version via bash script on all release runners, toolchain
harmonization
2026-06-12 12:42:40 +02:00
e65515a71c
chore: bump to 1.0.3 and harmonize onnx runtime version across nix, ci, and docker.
2026-06-12 09:11:55 +02:00
4559ee0aa8
fix: arc-share retrieved chunks, centralize entity embeddings, and trim hot-path clones.
2026-06-06 23:05:53 +02:00
676fdbc132
fix: replaced several instances if cloning, reduced allocations
2026-06-06 19:45:18 +02:00
93c65970f1
fix: leaner error handling by boxing large variants
2026-06-06 07:59:57 +02:00
20de557294
feat: configure FastEmbed model in config and admin, with restart to apply
...
Expose fastembed_model in config and a model dropdown on Admin → Models.
Persist dimension from the chosen model, require restart to load it, and
align legacy OpenAI default settings so fresh local-embedding installs
start cleanly.
2026-06-04 21:51:57 +02:00
c3b68e8bd3
feat: pool fastembed, batch embeddings, and reconcile embedding config on startup
2026-06-04 21:51:57 +02:00
5cca8dee01
fix: html-router modals and add insta snapshot tests.
...
Avoid nested forms in the scratchpad editor, centralize modal lifecycle in modal.js, return HTMX partials from archive, and add template compile plus layout snapshots.
2026-06-03 20:20:43 +02:00
d2c1ea7d2a
feat: can now choose search result types
2026-06-01 14:37:19 +02:00
744482f2c8
fix: knowledge entity suggestions simplification
2026-05-31 20:23:40 +02:00
6c3475ca0e
chore: ingestion-pipeline refactor, sort technical debt, rustfmt
2026-05-31 19:48:41 +02:00
e9d8654324
chore: refactor retrieval pipeline to chunk-first RRF with derived entities and slimmer eval surface.
...
Collapse the multi-strategy entity engine into one benchmarked chunk retrieval path, derive entities from retrieved chunks, and update consumers, docs, and clippy fixes across the workspace.
2026-05-30 22:19:08 +02:00
a8e30192ba
chore: harden api-router errors and add router integration tests while slimming html handlers.
2026-05-30 15:18:12 +02:00
ec80a4e540
chore: improve html-router auth, caching, and analytics while centralizing search labels in common.
...
small fix
2026-05-29 15:03:55 +02:00
920d7b5efb
chore: centralize embedding errors, retrieval strategy, and test DB helpers.
...
Replace anyhow in embedding production code with EmbeddingError, move
RetrievalStrategy into common config, and deduplicate Surreal test setup
via common::test_utils.
2026-05-29 14:44:23 +02:00
d90319f3b0
chore: harden common storage bootstrap and slim embedded db assets
...
Unify embedding config, build providers from system settings, and fail
startup when index builds error or time out. Move Surreal assets under
common/db so embeds exclude crate source, and read storage via streams.
2026-05-29 14:44:23 +02:00
964d57ec97
test: cover system settings sync, validation, and ingestion prompts
...
Add tests for embedding provider sync, patch isolation, typed backend
serde, and DB-backed ingestion prompts.
2026-05-29 14:44:23 +02:00
544a790e34
chore: harden system settings and unify prompt usage
...
Validate settings updates, use typed embedding backends, and route
ingestion through DB-stored prompts so admin edits take effect.
2026-05-29 14:44:23 +02:00
f625a7e0a9
chore: move serde helpers to common utils
...
Relocate SurrealDB serde helpers out of storage types so they can be
reused broadly, and align retrieval-pipeline test setup with configured
embedding dimensions.
2026-05-29 14:44:23 +02:00
b80f942cb5
chore: harden text chunk embeddings and text content storage
...
Align text chunk embedding identity with knowledge entities (chunk id as record id, UNIQUE chunk_id index, dimension validation), make cascade deletes transactional, and improve text content patch/search reliability with tests.
2026-05-29 14:44:23 +02:00
2e2a26f5f1
chore: harden knowledge graph storage and clear common clippy warnings
...
Enforce stable 1:1 entity embeddings, relationship endpoint auth, and
user-scoped deletes; align schemas/migrations and resolve common crate
clippy findings.
2026-05-29 14:44:23 +02:00
4071314e6a
chore: harden analytics, conversation access, and per-user file dedup
...
Use UPSERT for analytics counters, enforce message ownership in SQL,
return
NotFound when patch_title updates nothing, scope file dedup by user_id
with a
composite unique index, and expand tests for auth, ordering, and edge
cases.
2026-05-29 14:44:23 +02:00
baf3e8b172
chore: optimize ingest payloads and add parallel task batch store
...
Parse content before building file payloads to move shared metadata when
possible, add create_all_and_add_to_db for concurrent stores, and extend
tests for batch persistence and payload edge cases.
2026-05-29 14:44:23 +02:00
1e0dba72c8
chore: harden common errors, fastembed blocking, and ingest ownership
...
Run FastEmbed inference on spawn_blocking, propagate Surreal take
failures,
add AppError::internal and typed ingest/embedding parse errors, and take
owned file lists in ingestion payload construction.
2026-05-29 14:44:23 +02:00
1e25705377
chore: improved error handling
2026-05-28 19:58:14 +02:00
4f02fcb853
chore: rename get_id to id, add doc comments, pre-allocate format_history
2026-05-27 18:06:16 +02:00
9ccf8dde25
chore: lowercase all error messages and add # Errors doc sections
...
- Fix err-lowercase-msg: normalize all #[error(...)] display strings to
lowercase (AppError, FileError, ApiErr) and update affected tests
- Fix err-doc-errors: add # Errors sections to 25+ fallible public
functions across db.rs, store.rs, embedding.rs, indexes.rs,
ingestion_task.rs, and ingest_limits.rs
2026-05-27 14:59:48 +02:00
e63498ecee
chore: resolve remaining uninlined_format_args clippy warnings
2026-05-27 14:34:37 +02:00
b03290abd5
chore: fix and reduce clippy allows in knowledge_entity.rs
...
- rm duplicate 'document' match arm (match_same_arms)
- .get(0) -> .first() (get_first)
- for entity in all_entities.iter() -> &all_entities (explicit_iter_loop)
- 2x error!("{}", err_msg) -> error!("{err_msg}") (uninlined_format_args)
- 2x test format!()/assert!() positional -> inlined (uninlined_format_args)
- removed 6 now-unnecessary allow attributes
2026-05-27 14:28:08 +02:00
81624850c0
chore: add must_use to 27 non-Result public functions
...
- constructors: KnowledgeEntity, TextChunk, Scratchpad, IngestionTask,
Conversation, KnowledgeRelationship, Message, TextContent,
KnowledgeEntityEmbedding, TextChunkEmbedding
- accessors: Theme::as_str, Theme::initial_theme, TaskState::as_str,
TaskState::display_label, StorageManager::backend_kind,
StorageManager::local_base_path, EmbeddingProvider::backend_label,
EmbeddingProvider::dimension, EmbeddingProvider::model_code
- queries: TaskState::is_terminal, IngestionTask::can_retry,
KnowledgeEntityType::variants, StorageManager::resolve_local_path,
resolve_base_dir, IngestionTask::lease_duration
- helpers: Message::format_history
- builders: StorageManager::with_backend
2026-05-27 14:23:56 +02:00
99db54b2e7
fix: replace manual embedding serialization with serde_json
...
- replaced write!() loops with serde_json::to_string in 4 re-embedding methods
- standardized SQL building to use write!() with proper error propagation
- eliminates manual f32 vector string building (memory waste + loop risk)
2026-05-27 14:13:19 +02:00
cd0d95abaa
fix: revoke_api_key sets NONE, remove unused bind, lowercase error msgs
...
- fix bug where revoke_api_key set literal 'test_string_nullish' instead of NONE
- remove unused table_name bind in update_timezone
- lowercase ~16 error messages across 4 crates
2026-05-27 13:56:32 +02:00
e2284b1e69
chore: removed anyhow from apperror for improved error handling
2026-05-27 13:33:02 +02:00
76fcdcd6ce
chore: index slicing and lowercase errors
2026-05-27 12:41:26 +02:00
cd0b0368a0
chore: tightening and removing super fn
2026-05-27 11:23:39 +02:00
48985a61d3
chore: clippy and nix fmt
2026-05-27 11:23:08 +02:00
056f116885
perf: avoid small own clones and intermediate Vec allocations
...
- Derive Copy on 6 small enums (MessageRole, TaskState, StorageKind, EmbeddingBackend, PdfIngestMode, KnowledgeEntityType)
- Change create_ingestion_payload files param from Vec<FileInfo> to &[FileInfo]
- Remove 5 intermediate Vec allocations (4 embedding serialization + 1 format_history) using write! loop
- Remove 7 unnecessary .clone() calls exposed by Copy derive
2026-05-27 10:28:08 +02:00
Per Stark