Commit Graph

36 Commits

Author SHA1 Message Date
Per Stark 84eff3af3c chore: git-hooks rustfmt and clippy 2026-06-20 10:10:29 +02:00
Per Stark 4cd428185f fix: schedule nightly index rebuild on worker and skip per-ingest rebuild.
Ingest relies on SurrealDB incremental index maintenance; the worker runs native REBUILD INDEX on a configurable interval with lease state on system_settings.
2026-06-12 15:01:53 +02:00
Per Stark 81797504d4 fix: load embedding dimensions once per persist and trim vector search select. 2026-06-12 13:54:51 +02:00
Per Stark 4559ee0aa8 fix: arc-share retrieved chunks, centralize entity embeddings, and trim hot-path clones. 2026-06-06 23:05:53 +02:00
Per Stark 20de557294 feat: configure FastEmbed model in config and admin, with restart to apply
Expose fastembed_model in config and a model dropdown on Admin → Models.
Persist dimension from the chosen model, require restart to load it, and
align legacy OpenAI default settings so fresh local-embedding installs
start cleanly.
2026-06-04 21:51:57 +02:00
Per Stark c3b68e8bd3 feat: pool fastembed, batch embeddings, and reconcile embedding config on startup 2026-06-04 21:51:57 +02:00
Per Stark e9d8654324 chore: refactor retrieval pipeline to chunk-first RRF with derived entities and slimmer eval surface.
Collapse the multi-strategy entity engine into one benchmarked chunk retrieval path, derive entities from retrieved chunks, and update consumers, docs, and clippy fixes across the workspace.
2026-05-30 22:19:08 +02:00
Per Stark d90319f3b0 chore: harden common storage bootstrap and slim embedded db assets
Unify embedding config, build providers from system settings, and fail
startup when index builds error or time out. Move Surreal assets under
common/db so embeds exclude crate source, and read storage via streams.
2026-05-29 14:44:23 +02:00
Per Stark 017d6c5ba9 chore: additional clippy fixes after rebasing 2026-05-27 07:37:18 +02:00
Per Stark d76f86f56f refactor: simplify and improve testing for initialization 2026-05-26 20:21:24 +02:00
Per Stark 5ce7a76c75 clippy: adhere to pedantic clippy, uniform test error handling 2026-05-26 20:21:13 +02:00
Per Stark 7c718712c9 refactor: replace Box<dyn Error> with anyhow::Result
- ingestion_pipeline::run_worker_loop returns anyhow::Result<()>
- api_router::ApiState::new returns anyhow::Result<Self>
- html_router::HtmlState::new_with_resources is infallible, returns Self
- main/server/worker binary entry points return anyhow::Result<()>
2026-05-26 20:14:11 +02:00
Per Stark f5d72b66b0 test: add admin auth integration coverage 2026-02-14 23:11:35 +01:00
Per Stark 5b16e88991 refactor: extendable templates
refactor: simplification

refactor: simplification
2026-01-13 22:18:00 +01:00
Per Stark 8664abdf01 release: 1.0.0
fix: cargo dist
2026-01-11 20:35:01 +01:00
Per Stark b4b2ab8974 fix: ordering of index creation 2025-12-22 21:59:35 +01:00
Per Stark 86270de873 tidying stuff up, dto for search 2025-12-20 22:30:31 +01:00
Per Stark 90bac299a3 passed wide smoke check 2025-12-10 13:54:08 +01:00
Per Stark 8121e04125 retrieval simplfied 2025-12-09 20:35:42 +01:00
Per Stark 5e5053039a fix: removed stale embeddings handler 2025-11-29 20:07:48 +01:00
Per Stark 05bdaac672 evals: v3, ebeddings at the side
additional indexes
2025-11-26 15:15:10 +01:00
Per Stark 97d35a8982 retrieval-pipeline: v0 2025-11-18 22:46:35 +01:00
Per Stark 3196e65172 fix: improved storage manager, prep for s3 2025-11-03 12:39:15 +01:00
Per Stark 72578296db feat: reranking with fastembed added 2025-10-27 13:05:10 +01:00
Per Stark c4c76efe92 test: startup smoke test 2025-09-29 21:15:34 +02:00
Per Stark a0632c9768 fix: write to stderr for journalctl compatability 2025-08-22 13:41:30 +02:00
Per Stark 37584ed9fd Merge branch 'custom_llm_base'
fix: updated readme and corrected server and worker to updates

added migration

fix: openai url typo & displaying models

chore: tidying up
2025-06-08 08:28:14 +02:00
Per Stark a363c6cc05 feat: support for other providers of ai models 2025-06-06 23:16:41 +02:00
Per Stark d2772bd09c feat: port selection 2025-05-30 07:44:26 +02:00
Per Stark 850878d5c3 feat: customizable data storage path 2025-05-09 23:28:36 +02:00
Per Stark c2fbdecce0 feat: database migrations 2025-05-04 21:16:09 +02:00
Per Stark 02198dc21a feat: readability parsing, screenshot of page, file serving 2025-04-30 08:06:18 +02:00
Per Stark 776a454a88 updated dependencies application wide 2025-04-24 13:50:20 +02:00
Per Stark 233df1b79a fix: own implementation of stream parser 2025-04-10 08:23:55 +02:00
Per Stark 804461ac01 feat: improved configuration
configuration now works with both env variables and config file
2025-04-09 11:32:23 +02:00
Per Stark 5bc48fb30b refactor: better separation of dependencies to crates
node stuff to html crate only
2025-04-04 12:50:38 +02:00