Compare commits

...
127 Commits
Author SHA1 Message Date
eitchtee 32976a8d2e chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-20 03:39:13 +00:00
Herculino Trotta 26fbbcf378 Merge pull request #690 from eitchtee/weblate
Translations update from Weblate
2026-11-20 00:38:47 -03:00
Weblate d225133009 Merge remote-tracking branch 'origin/main' 2026-11-20 02:45:30 +00:00
eitchtee 91275e0256 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-20 02:45:26 +00:00
Weblate dddd757bb5 Merge remote-tracking branch 'origin/main' 2026-11-20 02:44:56 +00:00
Herculino Trotta 0f5f925683 Merge pull request #695 from eitchtee/feat/auto-tagging
feat(transactions): add lightweight auto-tagging for unclassified transactions
2026-11-19 23:44:53 -03:00
Herculino Trotta c29a6bea8f feat(transactions): add lightweight auto-tagging for unclassified transactions 2026-11-19 20:00:51 -03:00
Dimitri Decrock 35f406d3f9 locale(Dutch): update translation
Currently translated at 100.0% (991 of 991 strings)

Translation: WYGIWYH/App
Translate-URL: https://translations.herculino.com/projects/wygiwyh/app/nl/
2026-11-19 05:58:59 +00:00
Dimitri Decrock 3f3855e905 locale(Dutch): update translation
Currently translated at 93.4% (926 of 991 strings)

Translation: WYGIWYH/App
Translate-URL: https://translations.herculino.com/projects/wygiwyh/app/nl/
2026-11-18 05:58:59 +00:00
Weblate 5ad69061ba Merge remote-tracking branch 'origin/main' 2026-11-17 16:06:32 +00:00
eitchtee f36593cce8 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-17 16:06:28 +00:00
Weblate 75ab7d3e56 Merge remote-tracking branch 'origin/main' 2026-11-17 16:06:02 +00:00
Herculino Trotta 0ff29ba465 Merge pull request #693 from eitchtee/fix/export-restore
fix(export): multi-user restore, large exports, and export/restore on the page
2026-11-17 13:05:58 -03:00
Herculino Trotta 8d6b86b7ca feat(export): export and restore on the page, with progress and messages
The export and restore forms now sit on the Export and Restore page
instead of an offcanvas. Exporting shows a progress line and a toast
when the download starts; restoring splits ZIP and CSV uploads into
tabs and shows errors in the form.

- An empty export warns instead of downloading the message as a file
- Export errors show a message instead of a stuck spinner
- The download link is kept long enough for Safari to save the file
- Non-superusers get a 403 instead of a redirect loop with the login
- The asset classes CSV can be uploaded on its own
- Drop the export form's unknown investment_classes layout field
2026-11-17 13:01:25 -03:00
Herculino Trotta 5ea7f79f56 fix(export): keep each user's names on restore and speed up big exports
Restore looked categories, tags and entities up through the admin's
filtered manager, so other users' transactions moved to the admin's
categories and lost their tags. They are now matched to the row's
owner (then to ones that owner can see) and created for that owner.
Export also read tags and entities through that manager and left other
users' private ones out.

Recurring transactions and installment plans now export account_owner,
which restore already read but never received.

Transaction exports ran a few queries per row and paged prefetched rows
with OFFSET, so large histories timed out. They now load relations up
front and read in cursor chunks: 200k transactions export in ~43s with
~200 queries (10k took 34s and ~21k queries before).

Fixes #691
Fixes #692
2026-11-17 13:01:25 -03:00
Dimitri Decrock bc12332619 locale(Dutch): update translation
Currently translated at 88.0% (867 of 985 strings)

Translation: WYGIWYH/App
Translate-URL: https://translations.herculino.com/projects/wygiwyh/app/nl/
2026-11-16 04:58:59 +00:00
eitchtee 627e47341e chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-15 20:01:24 +00:00
Herculino Trotta 3752682f63 Merge pull request #686 from eitchtee/weblate
Translations update from Weblate
2026-11-15 17:00:59 -03:00
Weblate 679e96a0b3 Merge remote-tracking branch 'origin/main' 2026-11-15 19:58:42 +00:00
eitchtee 75c854d034 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-15 19:58:38 +00:00
Weblate 791d35ad20 Merge remote-tracking branch 'origin/main' 2026-11-15 19:58:17 +00:00
Herculino Trotta c58129ccfe Merge pull request #689 from eitchtee/feat/oauth-consent-screen
feat(oauth): match the consent screen to the app
2026-11-15 16:58:13 -03:00
Herculino Trotta 615bc0033e feat(oauth): match the consent screen to the app
Override django-oauth-toolkit's unstyled authorize template with the login
page's layout. It says who's signed in, what the app will be able to do, and
where the answer is sent, so a look-alike app with someone else's redirect
is easier to spot.
2026-11-15 16:54:14 -03:00
Weblate b4740e4e00 Merge remote-tracking branch 'origin/main' 2026-11-15 19:45:17 +00:00
eitchtee cb8f5128ab chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-15 19:45:11 +00:00
Weblate 82bcb635ef Merge remote-tracking branch 'origin/main' 2026-11-15 19:44:44 +00:00
Herculino Trotta faa47d2d7c Merge pull request #688 from eitchtee/feat/native-mcp
feat(api): native MCP server at /mcp
2026-11-15 16:44:18 -03:00
Weblate 9fcea382d6 Merge remote-tracking branch 'origin/main' 2026-11-15 19:44:18 +00:00
Herculino Trotta c38ad7e294 Merge pull request #687 from eitchtee/fix/recurring-end-date
fix(transactions): drop unpaid recurring occurrences past a shorter end date
2026-11-15 16:44:15 -03:00
Herculino Trotta 9ba19bd3b5 docs: point to the built-in MCP server's generic tools, drop the third-party one
WYGIWYH ships its own MCP server now, and that project looks abandoned.
2026-11-15 16:39:19 -03:00
Herculino Trotta f5414223ab fix(transactions): drop unpaid recurring occurrences past a shorter end date
Editing a recurring transaction to end earlier kept its unpaid occurrences
after the new end date, counting them in projected balances, and left
last_generated_date past it. Remove them (both sides of a generated transfer;
a partner linked from elsewhere is unlinked and kept) and move the bookmark
back to the last remaining occurrence. Paid occurrences stay.

Also pin ENABLE_SOFT_DELETE=False on the test that asserts hard deletes, so a
.env that turns soft delete on no longer fails it.
2026-11-15 16:36:29 -03:00
Herculino Trotta 4754f7e022 feat(api): serve a native MCP server at /mcp
Off unless MCP_ENABLED=true. Runs inside the web app (stateless Streamable
HTTP, JSON responses), so no extra container or supervisord program.

- Tools run the REST views in-process as the caller, so permissions,
  sharing, validation and transaction rules stay the API's.
- 11 curated tools with compact output, plus search_api/call_api to reach
  every other API endpoint from the OpenAPI schema.
- OAuth protected-resource metadata (RFC 9728) and a WWW-Authenticate hint
  on 401s, so MCP clients find the existing OAuth server and DCR.
- GET /api/transactions/summary/: totals per currency and type, optionally
  grouped by category, tag, entity, account or month.

API changes found while testing it:

- Writes no longer need Django model permissions, which nothing granted;
  access now matches the web UI (signed-in user, scoped querysets, owner-only
  edits of shared objects). API tokens and OAuth apps can now write for
  regular users.
- Account group owner is read-only; it could be set to another user.
- Reject negative installment, recurring, DCA and exchange-rate amounts, a
  zero recurrence interval, and performance on non-investment accounts.
- Accounts can be archived; group_id/exchange_currency_id are optional;
  category accepts null; currency prefix/suffix keep their spaces.
- Transactions filter by tags and entities.
- OpenAPI: separate request schemas (COMPONENT_SPLIT_REQUEST) and real
  descriptions for the viewsets and DCA chart actions.
2026-11-15 16:29:06 -03:00
Herculino Trotta 9ad872a903 locale(Portuguese (Brazil)): update translation
Currently translated at 100.0% (978 of 978 strings)

Translation: WYGIWYH/App
Translate-URL: https://translations.herculino.com/projects/wygiwyh/app/pt_BR/
2026-11-15 03:21:39 +00:00
Herculino Trotta eb51e240ac Merge pull request #672 from officialasishkumar/fix/qif-dedup-account
fix(import): make QIF deduplication account-aware and keep identical transactions
2026-11-14 12:20:49 -03:00
Herculino Trotta d763d0ffb8 fix(import): leave QIF headers out of the hash and skip non-transaction sections
The !Type header line was buffered into the first transaction's raw
lines, so a transaction hashed differently depending on whether it
opened the export. An overlapping export with a different first
transaction imported it twice. Header lines are now left out of the
internal ID. The previous-hash check still hashes the full lines,
headers included, and runs on the first appearance of each of those,
so a first transaction and an identical later one imported before
this change are both still recognized.

Account blocks (!Account and !Option:AutoSwitch account lists) and
non-transaction sections (!Type:Cat, !Type:Class, !Type:Memorized and
any type not in QIF_TRANSACTION_SECTIONS) were read with the
transaction codes, where T is an account's type and D a description,
so files containing them failed to import. They are now skipped, and
each skipped section is logged.
2026-11-14 12:16:15 -03:00
eitchtee b6a62a3e8c chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-14 14:41:29 +00:00
Herculino Trotta 304817470d Merge pull request #673 from officialasishkumar/feat/filter-preset-relative-dates
feat(transactions): keep filter preset dates relative to today
2026-11-14 11:41:04 -03:00
Herculino Trotta 72502c8873 fix(transactions): renumber filter preset migration after sub-transactions
main added 0053_sub_transaction, which also depends on 0052, so the
filter preset migration becomes 0054 to keep a single leaf.
2026-11-14 11:36:30 -03:00
Herculino Trotta 262ac3fbb6 Merge remote-tracking branch 'origin/main' into feat/filter-preset-relative-dates
# Conflicts:
#	app/apps/transactions/filters.py
2026-11-14 11:29:15 -03:00
Herculino Trotta 35c288582b fix(transactions): store filter preset dates as ISO
Presets kept dates as typed, such as "Novembro 2026" or "14/11/2026",
so changing the language or date format made them unreadable and they
were silently dropped. Dates are now stored as ISO and shown in the
user's current format when the preset is applied. Presets saved before
this keep working while the language and format stay the same.
2026-11-14 11:28:36 -03:00
Herculino Trotta c5717d41c2 feat(transactions): keep whole-month preset ranges on whole months
A preset whose dates run from the 1st to the last day of a month moved
by days, so a "this month" range drifted into the next month. Such a
range now moves by months, ending on the last day of its new month.
2026-11-14 11:27:38 -03:00
Herculino Trotta 5cc68f4bfc fix(transactions): follow visual order when tabbing through the save preset dialog
The relative dates option was shown below the preset name with CSS order
but came before it in the page, so Tab reached it first. It is now
inserted right after the name input when the dialog opens.
2026-11-14 11:26:53 -03:00
eitchtee 6c4b117d11 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-14 14:06:20 +00:00
Herculino Trotta cc13280160 Merge pull request #685 from officialasishkumar/fix/recurring-edit-unpaid-account
fix(transactions): move unpaid recurring transactions when the account or type changes
2026-11-14 11:05:53 -03:00
Herculino Trotta ed513a2171 fix(transactions): unlink outside transfers broken by a recurrence edit
An unpaid occurrence linked to a transaction outside its recurrence (e.g. through a DCA entry) could end up as two expenses, or both sides in one account, after the recurrence's type or account changed. The link is now dropped in that case; the partner is left untouched.
2026-11-14 11:02:38 -03:00
Asish Kumar 4532677e49 fix(transactions): move unpaid recurring transactions when the account or type changes
Editing a recurring transaction only refreshed the amount, description,
category, notes, tags and entities of its unpaid transactions. Changing its
account or type left the already generated transactions behind: a monthly
expense moved to another account kept its upcoming months in the old one,
and an expense turned into an income kept pulling projected balances down.
New occurrences followed the new settings, so the recurrence ended up split
across two accounts.

Unpaid transactions now take the recurrence's account and type as well,
keeping their dates, the same way installment plans already update theirs.
For recurring transfers:

- the "to" sides follow the "to" account and amount;
- turning a recurrence into a transfer adds the missing "to" sides;
- turning a transfer into an income or expense removes the unpaid "to"
  sides and leaves the paid transfers linked.

A "to" side is now recognised by its partner belonging to the same
recurrence, so an occurrence the user linked by hand to another
transaction is updated like any other occurrence instead of being treated
as a generated "to" side.
2026-11-14 17:00:17 +05:30
eitchtee 93fa7b272e chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-14 06:23:31 +00:00
Herculino Trotta 709a8a89f0 Merge pull request #684 from eitchtee/fix/sharing-help-text
fix(sharing): clarify what shared and public visibility allow
2026-11-14 03:23:07 -03:00
Herculino Trotta 66351aee49 fix(sharing): clarify what shared and public visibility allow
"Only editable by the owner" read as covering an account's transactions
too, which led to GHSA-36w2-769w-h62g being reported. Sharing an account
or making it public deliberately opens its ledger to everyone who can
see it; only the object itself stays owner-only.

State that in the generic visibility help text, and spell out the
account case on the account share panel.
2026-11-14 03:18:37 -03:00
Herculino Trotta bdda13343e Merge pull request #683 from eitchtee/GHSA-vcr4-g82g-wcq8
fix(transactions): scope installment plan and recurring transaction lookups
2026-11-14 03:06:41 -03:00
Herculino Trotta 348bc6e519 fix(transactions): scope installment plan and recurring transaction lookups
InstallmentPlan and RecurringTransaction declare the unfiltered
all_objects manager first and set no default_manager_name, so
get_object_or_404(Model, id=...) bypassed account visibility. Any
logged-in user could read, refresh, pause, finish or delete another
user's plans (and their transactions) by id.

Resolve the nine per-object views through the filtered objects manager,
as the list views, the API and the transaction views already do.

Refs GHSA-vcr4-g82g-wcq8
2026-11-14 03:01:49 -03:00
eitchtee 11c9d60954 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-14 05:38:09 +00:00
Herculino Trotta 3b47562cfb Merge pull request #681 from eitchtee/fix/api-docs-warnings
fix(api): clear the API docs schema warnings
2026-11-14 02:37:25 -03:00
Herculino Trotta 59dccd72d8 Merge pull request #680 from eitchtee/fix/rule-dry-run-connection
fix(rules): rule dry runs fail with a 500
2026-11-14 02:37:12 -03:00
Herculino Trotta 6576435168 Merge pull request #679 from eitchtee/feat/sub-transactions-2
feat: sub-transactions
2026-11-14 02:36:58 -03:00
Herculino Trotta a970c99e12 chore(demo): add sub-transaction examples to the demo data 2026-11-14 02:13:22 -03:00
Herculino Trotta 3dcc2316fb feat(export): back up sub-transactions and quick transactions 2026-11-14 02:13:21 -03:00
Herculino Trotta 188876fdf0 feat(rules): add, change and remove sub-transactions in rules
Adds a Sub-transactions field to edit actions and update-or-create
actions. Expressions get a sub_transactions variable and return a list of
{name, amount}; an empty list removes the parts. Rules skip transfers, and
setting the amount of a split transaction is skipped.
2026-11-14 02:13:21 -03:00
Herculino Trotta 5459865366 feat(api): read and write sub-transactions
Transactions, installment plans and recurring transactions take a nested
sub_transactions list; their amount becomes the parts' sum. Changing only
the amount of a split transaction, or adding parts to a transfer, is a
validation error.
2026-11-14 02:13:21 -03:00
Herculino Trotta ae245fb1d1 feat(transactions): add sub-transactions
Splits a transaction into named parts whose amounts add up to its amount,
e.g. the items of a big purchase or the fees inside an investment
contribution. Parts can be set on transactions, installment plans,
recurring and quick transactions, and are copied to the transactions they
generate. Transfers can't have parts.

Transactions show their parts under a tab on the card and in the
investment detail. The content filter matches part names, and bulk edits
leave the amount of split transactions alone.
2026-11-14 02:13:21 -03:00
Herculino Trotta a432fae632 fix(api): clear the API docs schema warnings
Describe the API token authentication, name the colliding transaction
type enums, and use nullable instead of an OpenAPI 3.1 null type. A test
keeps the schema free of warnings.
2026-11-14 02:12:16 -03:00
Herculino Trotta 00f014ccae fix(rules): keep the connection when tasks run inside a transaction
Rule dry runs call tasks directly inside an atomic block they roll back.
The connection cleanup wrapper closed that connection, so every dry run
failed with a 500.
2026-11-14 02:11:15 -03:00
eitchtee 9714df8820 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-13 21:56:52 +00:00
Herculino Trotta a0588e33d6 Merge pull request #678 from eitchtee/feat/convert-to-transfer
feat(transactions): convert a transaction into a transfer
2026-11-13 18:56:18 -03:00
Herculino Trotta 2a06bd4a06 feat(transactions): convert a transaction into a transfer
Adds Convert > Transfer to the transaction menu. The transfer form opens
prefilled on the side matching the transaction type (expense = from,
income = to); saving creates the transfer and deletes the original.
Not available for transfers or plan/recurrence transactions.
2026-11-13 18:50:46 -03:00
eitchtee 40a1364ebd chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-13 21:50:10 +00:00
Herculino Trotta 9d6379c324 Merge pull request #676 from eitchtee/feat/negative-filters-2
feat(transactions): add exclude tab to transaction filter
2026-11-13 18:49:28 -03:00
Herculino Trotta 3a44f6fa1c Merge pull request #677 from eitchtee/refactor/action-bar-improvements-3
feat(transactions): action bar improvements and new bulk actions
2026-11-13 18:49:22 -03:00
Herculino Trotta 15c29ecfb0 feat(transactions): rework the selection action bar
- Split the selection control into an unselect button and a dropdown
- Expose the new bulk actions in the edit dropdown
- Fold the actions into a single menu and hide the sum on small screens
2026-11-13 18:44:12 -03:00
Herculino Trotta 623131eb2f feat(transactions): add bulk move to today, change month and hide from summaries
Hiding skips transactions whose visibility is controlled by their account or category.
2026-11-13 18:44:12 -03:00
Herculino Trotta 6bf0dafaee fix(transactions): keep transaction rows from overflowing on small screens 2026-11-13 18:44:11 -03:00
Herculino Trotta 0d8de481d6 feat(transactions): add exclude tab to transaction filter
Split the filter into Include and Exclude tabs. Exclusions for content, accounts, currencies, categories, tags and entities are applied together with the existing filters, and are kept by presets, Clear and the active-filter indicator.
2026-11-13 18:43:29 -03:00
Herculino Trotta e552723078 Merge pull request #674 from eitchtee/dependabot/uv/soupsieve-4.0
chore(deps): bump soupsieve from 2.9 to 4.0
2026-11-13 15:40:59 -03:00
Herculino Trotta 01dd0920a9 Merge pull request #675 from eitchtee/fix/airdatepicker-hx-boosted
fix(datepicker): keep preserved pickers working after boosted navigation
2026-11-13 15:40:28 -03:00
Herculino Trotta e2981bf1f6 fix(datepicker): keep pickers working after history back and forward
A history restore swaps in htmx's snapshot of the body, which carries a
lifeless copy of AirDatepicker's global container. The previous fix looked
the container up by id, found that copy, and left the library's real
container detached, so pickers opened into nothing.

Track the container through datepicker.$container instead, remove any
snapshot copies after a swap, and reattach the real one when detached.
2026-11-13 15:18:06 -03:00
Herculino Trotta 9cf8cca1cb fix(datepicker): keep preserved pickers working after boosted navigation
The filter on the Monthly and All pages is wrapped in hx-preserve, so a
boosted navigation keeps its pickers instead of recreating them. The
AirDatepicker global container they render into lives on the body and is
swapped out with it, leaving those pickers drawing into a detached node.

After each swap, move the detached container's pickers into the new
container, or reattach it when no new picker created one.
2026-11-13 15:04:41 -03:00
dependabot[bot] 1ec0cfc02f chore(deps): bump soupsieve from 2.9 to 4.0
Bumps [soupsieve](https://github.com/facelessuser/soupsieve) from 2.9 to 4.0.
- [Release notes](https://github.com/facelessuser/soupsieve/releases)
- [Commits](https://github.com/facelessuser/soupsieve/compare/2.9...4.0)

---
updated-dependencies:
- dependency-name: soupsieve
  dependency-version: '4.0'
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-11-13 14:09:59 +00:00
Asish Kumar ced7f3826e feat(transactions): keep filter preset dates relative to today
Filter presets save the exact dates they were created with, so a preset
for "from today" or "this month" goes stale the next day or month.

The save dialog now offers "Keep dates relative to today". When ticked,
the preset remembers the day it was saved (`FilterPreset.relative_to`)
and, when applied, moves its dates forward by the time elapsed since:
"Date from"/"Until" by days and the reference months by months. A preset
saved with today as the start date always starts today, and one saved
for the current reference month always shows the current month.

- The option is only stored when the preset has a date filter.
- Presets with relative dates show a clock icon in the presets menu.
- Presets saved before this change, or without the option, keep their
  fixed dates.
2026-11-12 22:59:59 +05:30
Asish Kumar 70be7b3161 fix(import): make QIF deduplication account-aware and keep identical transactions
QIF imports deduplicate each transaction by storing a SHA-256 of its raw
lines as the transaction's internal_id and skipping rows whose hash is
already taken. The hash didn't include the account, which broke two
cases:

- Importing the same file into two accounts (possible since the account
  can be picked per run) skipped every row of the second import.
- Two real transactions with the same date, amount and payee in one file
  hashed to the same ID, so the second one was always dropped.

The internal ID now hashes the account's ID and the transaction's
occurrence (how many identical transactions came before it in the file)
together with the raw lines. Using the occurrence instead of the
position in the file keeps the IDs stable when a later export of the
same statement has more transactions before or between the repeated
ones.

Transactions imported before this change keep their old IDs. So that
re-importing an old file doesn't duplicate them, the first occurrence
of a transaction is also matched against the previous hash, limited to
the account being imported into. Repeats of identical transactions,
which the old hash always skipped, are imported now.

The duplicate check also looks at soft-deleted transactions, like the
CSV/Excel deduplication does. internal_id is unique, so a soft-deleted
transaction with the same ID used to fail the import with an integrity
error instead of being skipped.

Fixes #671
2026-11-12 15:44:27 +05:30
eitchtee 1c891a0a60 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-12 03:24:33 +00:00
Herculino Trotta 03c22b8fc2 Merge pull request #670 from officialasishkumar/feat/import-run-account
feat(import): pick the account when running an import
2026-11-12 00:24:10 -03:00
Herculino Trotta df6ccda7b3 feat(import): read any line of the file with a line transformation
Some banks name the account once in a header cell above the table, which
skip_lines and start_row throw away. A `line` transformation returns a line
of the file (or one of its columns) as text, so a mapping can read it and
clean it up with regex or replace.

CSV keeps only the requested lines during the existing counting pass; Excel
reads them straight from each sheet. An account picked on the run still
overrides the mapped one.
2026-11-12 00:17:14 -03:00
Herculino Trotta e617e809c0 fix(import): reject archived accounts when importing through the API
The upload form only lists active accounts, but the API accepted any account
the user can access, archived ones included.
2026-11-12 00:14:06 -03:00
eitchtee 1662fe70e0 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-11 18:26:08 +00:00
Herculino Trotta ca09f37b51 Merge pull request #669 from officialasishkumar/feat/net-worth-consolidated-shares
feat(net-worth): break consolidated totals down by currency share
2026-11-11 15:25:44 -03:00
Asish Kumar 1ed5dc5d31 feat(import): pick the account when running an import
An import run can now be given an account, and every transaction it
creates goes into that account instead of the one the profile maps. This
lets a single profile serve several accounts at the same bank, for
statements that only name the account in a header cell or not at all,
instead of keeping one copy of the profile per account.

- ImportRun gets an optional `account` (SET_NULL), shown on the run card.
- The upload form offers the user's active accounts when the profile
  imports transactions, and leaves the field out for profiles that import
  accounts, currencies, categories, tags or entities.
- CSV and Excel rows take the run's account before deduplication, so
  rules comparing `account` check against the picked account. The
  profile's account mapping is skipped for these rows.
- QIF files use the run's account instead of the account named after the
  file.
- The import API accepts an optional `account_id`, limited to accounts
  the user can access, rejected for profiles that don't import
  transactions, and returns `account` on import runs.

Leaving the account empty keeps the current behaviour.

Closes #538
2026-11-11 22:41:37 +05:30
Herculino Trotta dea8cfee4e test(net-worth): assert share text instead of badge markup 2026-11-11 14:02:56 -03:00
Herculino Trotta 4bf2c2552b feat(net-worth): collapse consolidated breakdown and align share badges
Consolidated rows are collapsible and remember their state per currency in localStorage. Share percentages render as badges on a shared subgrid column so they line up regardless of amount width. Currency and account menus drop their extra padding to align with the card title.
2026-11-11 13:57:21 -03:00
Asish Kumar d590dc39ab feat(net-worth): break consolidated totals down by currency share
A consolidated total on the Net Worth page was a single number: it said
what everything is worth in one currency, but not how much of it each
currency makes up, which is what you want to know when you hold the same
money in several currencies (or in currencies standing for gold, stocks
or crypto).

calculate_currency_totals now keeps, next to each consolidated total, the
components it was built from: the currency's own total and the exchanged
total of every currency that exchanges into it, all in the consolidated
currency. calculate_consolidated_shares gives each of them its share of
the total as a percentage, drops the ones that add nothing and sorts the
rest from largest to smallest. Shares are taken over the absolute total,
so a currency that adds to it always has a positive share and one that
takes from it (debt) a negative one; with a zero total there's no share.

Currencies that hide their exchanged amounts still show up, since they
still count towards the consolidated total, and so do exchange targets
that only exist through the deep search.

On the Net Worth page, each Consolidated row now lists its components
with their share and amount in the consolidated currency. Clicking one
shows that currency's line on the charts, like the currency rows above.
2026-11-11 21:27:17 +05:30
Herculino Trotta 6c3c4c03ec Merge pull request #668 from eitchtee/fix/insights-overview-exchanged
fix(insights): show exchanged amounts in the overview
2026-11-11 10:53:21 -03:00
Herculino Trotta 220ccee140 fix(insights): show exchanged amounts in the overview 2026-11-11 10:53:15 -03:00
Herculino Trotta 1c7bc3db1d Merge pull request #667 from eitchtee/fix/yearly-filter-pills-wrap
fix(yearly-overview): wrap long names in account and currency filters
2026-11-11 10:53:13 -03:00
Herculino Trotta bf83975b97 fix(yearly-overview): wrap long names in account and currency filters 2026-11-11 10:53:04 -03:00
Herculino Trotta 1ab06d544d Merge pull request #666 from eitchtee/fix/badge-overflow
fix(ui): let badges grow to fit long text
2026-11-11 10:52:58 -03:00
Herculino Trotta d2cd4d577a fix(ui): let badges grow to fit long text 2026-11-11 10:52:50 -03:00
eitchtee 4d886a4fea chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-11 13:41:24 +00:00
Herculino Trotta 3a619dc197 Merge pull request #665 from eitchtee/feat/investments-exchanged-amounts
feat(investments): show exchanged amounts in movements and honor the display switch
2026-11-11 10:40:56 -03:00
Herculino Trotta cd33b82b4e feat(investments): show exchanged amounts in movements and honor the display switch
Movements without an other-currency transfer side now show the exchanged
amount, and the converted value card follows show_exchanged_amounts.
2026-11-11 10:39:06 -03:00
eitchtee 7aed2e8574 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-11 13:27:09 +00:00
Herculino Trotta 9d36af78c4 Merge pull request #664 from officialasishkumar/feat/optional-exchanged-amounts
feat(currencies): make displaying exchanged amounts optional
2026-11-11 10:26:39 -03:00
Herculino Trotta 00d37d8c81 refactor(currencies): keep the exchanged amounts display rule in one place
Account.displayed_exchange_currency holds the account/currency precedence for transactions, and shown_exchange_currency() expresses the same rule as a query expression for the insights and account totals, replacing the per-module checks.
2026-11-11 10:22:50 -03:00
Asish Kumar d3f29f5ad1 feat(currencies): make displaying exchanged amounts optional
Currencies and accounts get a "Show exchanged amounts" switch, on by
default. Turning it off hides the amounts converted to the exchange
currency without removing the exchange currency, so consolidated totals
on the Net Worth page keep working.

The switch belongs to whichever one provides the exchange currency:

- Transactions (including both legs of combined transfers) use the
  account's switch when the account has its own exchange currency, and
  the currency's switch otherwise, matching how the exchanged amount is
  picked.
- Account cards and Net Worth account rows follow the account.
- Currency cards, the monthly summary and daily allowance, Net Worth
  currency rows and the month-by-month and year-by-year insights follow
  the currency.

Currency totals still convert hidden currencies, because the
consolidated totals are built from those values; the setting only
leaves them out of the displayed data. The API keeps returning
exchanged_amount for transactions, and the account serializer exposes
the new field (currencies already serialize all fields).

Closes #486
2026-11-11 15:26:04 +05:30
eitchtee 60a0365b23 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-11 01:05:02 +00:00
Herculino Trotta edd045607f Merge pull request #663 from eitchtee/feat/dca-improvements
feat(dca): optional amounts, currency-checked links and entities
2026-11-10 22:04:34 -03:00
Herculino Trotta 3468b10e94 feat(dca): limit create-transaction accounts to the strategy currencies 2026-11-10 21:57:39 -03:00
Herculino Trotta 750b86394e feat(dca): optional amounts, currency-checked links and entities on entries
- Amounts default to the linked transaction's amount when left empty
- Linked transactions must be in the strategy's payment/target currency; pickers only search those
- Allow setting entities on transactions created from an entry
- Fix crash when only the expense transaction was linked
2026-11-10 21:55:29 -03:00
Herculino Trotta 706f8fea2b feat(transactions): allow filtering the transaction picker search
TransactionSelect takes a filters dict sent as query params; the search endpoint applies allowlisted ones (currency, account).
2026-11-10 21:55:29 -03:00
Herculino Trotta eafaf0027c Merge pull request #662 from eitchtee/fix/investments-short-period-return
fix(investments): solve short-period return over the holding period
2026-11-10 20:38:42 -03:00
Herculino Trotta 0093cabfc9 fix(investments): solve short-period return over the holding period
Holdings under a year annualized their rate before turning it back into
the period return, so a few days with a modest gain (+1.3% in one day is
~10,600% a year) fell outside xirr's search range and showed no percentage.
Solve the rate over the period held instead.
2026-11-10 20:35:03 -03:00
eitchtee 37ecc7da1b chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-10 13:59:54 +00:00
Herculino Trotta 5d5a423db5 Merge pull request #661 from eitchtee/weblate
Translations update from Weblate
2026-11-10 10:59:29 -03:00
Weblate b6ac69716c Merge remote-tracking branch 'origin/main' 2026-11-10 13:55:36 +00:00
eitchtee c256671acc chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-10 13:55:32 +00:00
Weblate 0e62594133 Merge remote-tracking branch 'origin/main' 2026-11-10 13:55:06 +00:00
Herculino Trotta 07933ac5c1 Merge pull request #660 from jayhemnani9910/fix/recurring-month-end-keeps-day
fix(transactions): keep a recurrence on its day after a short month
2026-11-10 10:55:03 -03:00
masttera 3c30116072 locale(Russian): update translation
Currently translated at 56.6% (543 of 959 strings)

Translation: WYGIWYH/App
Translate-URL: https://translations.herculino.com/projects/wygiwyh/app/ru/
2026-11-10 07:58:57 +00:00
eitchtee 7ebcccdeda chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-10 02:41:08 +00:00
Herculino Trotta 674870641c Merge pull request #659 from eitchtee/fix/linked-transfer-exchanged-values
feat(transactions): redesign combined transfer items + show exchanged amounts for both sides
2026-11-09 23:40:42 -03:00
Herculino Trotta 0854805630 feat(transactions): show combined transfers as two legs on a timeline
- Shared header (date, description, badges); each side gets its own row:
  account with its category, tags and entities, amount + exchanged amount
- Timeline rail from the transfer icon branching into the out/in icons
- Amounts drop below on small screens and sit bottom-right on lg, like
  regular items; spacing matches regular item metadata rows
- Notes move below the legs on transfers so they don't sit in the rail
2026-11-09 23:35:56 -03:00
Herculino Trotta 4826c4932a fix(transactions): show exchanged amount for both sides of combined transfers 2026-11-09 23:16:47 -03:00
Jay Hemnani 6a289327b6 fix(transactions): keep a recurrence on its day after a short month
A monthly recurrence on the 31st clamped to Feb 28 and stayed on the
28th for every later month.
2026-11-10 03:51:15 +05:30
eitchtee bdc0603327 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-09 02:35:42 +00:00
Herculino Trotta a7ae1c7ec4 Merge pull request #657 from eitchtee/feat/distribution-totals-bracket
feat(distribution): show combined projected + current percentage
2026-11-08 23:34:56 -03:00
Herculino Trotta 34ba0c6e6a Merge pull request #658 from eitchtee/feat/investments-redesign
feat(investments): calmer list and detail pages
2026-11-08 23:34:49 -03:00
Herculino Trotta d0b6be14e7 feat(investments): calmer list and detail pages, group holdings by asset class
- One hero number with a single return line; secondary totals behind a Details toggle
- Holdings in one list card with quiet group subheaders, value and %/yr per row
- Group holdings by account group or asset class, remembered in the session
- Detail page: value and chart up top, Your position and home-currency sidebar
- Replace tooltips with visible text; keep amounts from wrapping on phones
2026-11-08 21:48:40 -03:00
Herculino Trotta 55f064ee7c feat(distribution): bracket projected + current segments with their sum
Remove the unused exchanged branch from calculate_percentage_distribution; nothing read it and it mixed up income_current with expense_projected.
2026-11-08 20:16:13 -03:00
eitchtee 10cb4465d8 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-08 22:24:02 +00:00
149 changed files with 21836 additions and 11251 deletions

No files matched your search

+3
View File
@@ -47,6 +47,9 @@ TASK_WORKERS=1 # This only work if you're using the single container option. Inc
# Personal access tokens. How often (seconds) a token's last_used_at is rewritten. # Personal access tokens. How often (seconds) a token's last_used_at is rewritten.
#API_TOKEN_LAST_USED_UPDATE_INTERVAL=600 #API_TOKEN_LAST_USED_UPDATE_INTERVAL=600
# Native MCP server at /mcp/, served by the web app itself. Off by default.
#MCP_ENABLED=false
# MCP OAuth Application. Uncomment to auto-create/update the OAuth client # MCP OAuth Application. Uncomment to auto-create/update the OAuth client
# used by remote MCP integrations after migrations complete. # used by remote MCP integrations after migrations complete.
#MCP_OAUTH_CLIENT_NAME="WYGIWYH MCP" #MCP_OAUTH_CLIENT_NAME="WYGIWYH MCP"
+8 -1
View File
@@ -295,7 +295,14 @@ Check out our [Wiki](https://github.com/eitchtee/WYGIWYH/wiki) for more informat
# MCP Server # MCP Server
[IZIme07](https://github.com/IZIme07) has kindly created an MCP Server for WYGIWYH that you can self-host. [Check it out at MCP-WYGIWYH](https://github.com/ReNewator/MCP-WYGIWYH)! WYGIWYH has a built-in MCP server, so AI assistants (Claude, ChatGPT, Claude Code, Cursor...) can read and record your transactions. It runs inside the web app; no extra container is needed. Set `MCP_ENABLED=true` to turn it on, then add `https://your.wygiwyh.domain/mcp` as a remote (Streamable HTTP) MCP server in your client.
Clients sign in in one of two ways:
- **OAuth** (Claude.ai, ChatGPT and most desktop clients): the client discovers WYGIWYH's OAuth server from `/.well-known/oauth-protected-resource`, and you log in and approve it in the browser. The client must either register itself, which needs `OAUTH2_DCR_ENABLED=true`, or use an OAuth app you created (see [MCP OAuth Application Bootstrap](#mcp-oauth-application-bootstrap)).
- **API token** (clients that let you set headers): send `Authorization: Token wygiwyh_pat_...`, created as in [API Tokens](#api-tokens-for-n8n-and-other-automations).
The client sees and changes what the signed-in user can in the app. It can list accounts, categories, tags and entities, check balances, search and summarize transactions, and create, edit or delete transactions. Everything else in the [API](#api-tokens-for-n8n-and-other-automations) (recurring transactions, installment plans, currencies, exchange rates, DCA...) is reachable through its `search_api` and `call_api` tools.
# Caveats and Warnings # Caveats and Warnings
+15 -3
View File
@@ -474,11 +474,11 @@ if DEBUG:
REST_FRAMEWORK = { REST_FRAMEWORK = {
# Use Django's standard `django.contrib.auth` permissions, # Same access as the web UI: any signed-in user, limited by the user-scoped
# or allow read-only access for unauthenticated users. # querysets and, for shared objects, SharedObjectPermission.
"DEFAULT_PERMISSION_CLASSES": [ "DEFAULT_PERMISSION_CLASSES": [
"apps.api.permissions.NotInDemoMode", "apps.api.permissions.NotInDemoMode",
"rest_framework.permissions.DjangoModelPermissions", "rest_framework.permissions.IsAuthenticated",
], ],
"DEFAULT_FILTER_BACKENDS": [ "DEFAULT_FILTER_BACKENDS": [
"django_filters.rest_framework.DjangoFilterBackend", "django_filters.rest_framework.DjangoFilterBackend",
@@ -512,11 +512,23 @@ OAUTH2_PROVIDER = {
OAUTH2_DCR_ENABLED = os.getenv("OAUTH2_DCR_ENABLED", "false").lower() == "true" OAUTH2_DCR_ENABLED = os.getenv("OAUTH2_DCR_ENABLED", "false").lower() == "true"
OAUTH2_DCR_INITIAL_ACCESS_TOKEN = os.getenv("OAUTH2_DCR_INITIAL_ACCESS_TOKEN", "") OAUTH2_DCR_INITIAL_ACCESS_TOKEN = os.getenv("OAUTH2_DCR_INITIAL_ACCESS_TOKEN", "")
# Native MCP server at /mcp/. Off by default; when off, /mcp/ and its OAuth
# protected-resource metadata return 404.
MCP_ENABLED = os.getenv("MCP_ENABLED", "false").lower() == "true"
SPECTACULAR_SETTINGS = { SPECTACULAR_SETTINGS = {
"TITLE": "WYGIWYH API", "TITLE": "WYGIWYH API",
"DESCRIPTION": "A no-frills expense tracker", "DESCRIPTION": "A no-frills expense tracker",
"VERSION": "1.0.0", "VERSION": "1.0.0",
"SERVE_INCLUDE_SCHEMA": False, "SERVE_INCLUDE_SCHEMA": False,
# Separate request schemas so read-only fields (id, owner, nested
# account...) don't show up as inputs for generated clients.
"COMPONENT_SPLIT_REQUEST": True,
# Several models have a "type" field with different choices
"ENUM_NAME_OVERRIDES": {
"TransactionTypeEnum": "apps.transactions.models.Transaction.Type",
"TemplateTypeEnum": "apps.transactions.models.QuickTransaction.Type",
},
# OTHER SETTINGS # OTHER SETTINGS
} }
+10 -1
View File
@@ -16,7 +16,7 @@ Including another URLconf
""" """
from django.contrib import admin from django.contrib import admin
from django.urls import path, include from django.urls import path, include, re_path
from drf_spectacular.views import ( from drf_spectacular.views import (
SpectacularAPIView, SpectacularAPIView,
SpectacularSwaggerView, SpectacularSwaggerView,
@@ -26,7 +26,9 @@ from apps.common.decorators.demo import disabled_on_demo
from apps.common.oauth_views import ( from apps.common.oauth_views import (
authorization_server_metadata, authorization_server_metadata,
dynamic_client_registration, dynamic_client_registration,
protected_resource_metadata,
) )
from apps.api.mcp import MCPView
from oauth2_provider import urls as _dot_urls from oauth2_provider import urls as _dot_urls
@@ -71,6 +73,13 @@ urlpatterns = [
disabled_on_demo(authorization_server_metadata), disabled_on_demo(authorization_server_metadata),
name="oauth-authorization-server-metadata", name="oauth-authorization-server-metadata",
), ),
# Clients may look it up with or without the resource path (RFC 9728)
re_path(
r"^\.well-known/oauth-protected-resource(?:/mcp)?/?$",
disabled_on_demo(protected_resource_metadata),
name="oauth-protected-resource-metadata",
),
re_path(r"^mcp/?$", MCPView.as_view(), name="mcp"),
path( path(
"oauth/register/", "oauth/register/",
disabled_on_demo(dynamic_client_registration), disabled_on_demo(dynamic_client_registration),
+2
View File
@@ -103,6 +103,7 @@ class AccountForm(forms.ModelForm):
"group", "group",
"currency", "currency",
"exchange_currency", "exchange_currency",
"show_exchanged_amounts",
"is_asset", "is_asset",
"is_archived", "is_archived",
"is_investment", "is_investment",
@@ -161,6 +162,7 @@ class AccountForm(forms.ModelForm):
Switch("is_archived"), Switch("is_archived"),
"currency", "currency",
"exchange_currency", "exchange_currency",
Switch("show_exchanged_amounts"),
Switch("is_investment"), Switch("is_investment"),
# Only relevant for investments: shown while the switch above is on # Only relevant for investments: shown while the switch above is on
Div( Div(
@@ -0,0 +1,22 @@
# Generated by Django 5.2.17 on 2026-11-11 09:49
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("accounts", "0021_alter_account_income_categories_help"),
]
operations = [
migrations.AddField(
model_name="account",
name="show_exchanged_amounts",
field=models.BooleanField(
default=True,
help_text="Show amounts converted to this account's exchange currency next to its transactions and totals.",
verbose_name="Show exchanged amounts",
),
),
]
+21
View File
@@ -104,6 +104,14 @@ class Account(SharedObject):
blank=True, blank=True,
help_text=_("Default currency for exchange calculations"), help_text=_("Default currency for exchange calculations"),
) )
show_exchanged_amounts = models.BooleanField(
default=True,
verbose_name=_("Show exchanged amounts"),
help_text=_(
"Show amounts converted to this account's exchange currency next to its "
"transactions and totals."
),
)
is_asset = models.BooleanField( is_asset = models.BooleanField(
default=False, default=False,
@@ -191,6 +199,19 @@ class Account(SharedObject):
user = get_current_user() user = get_current_user()
return self.untracked_by.filter(pk=user.pk).exists() return self.untracked_by.filter(pk=user.pk).exists()
@property
def displayed_exchange_currency(self):
"""The exchange currency to show amounts in, or None when they're hidden.
The account's exchange currency takes precedence over its currency's, so
whichever one provides it also decides whether it's shown.
"""
if self.exchange_currency_id:
return self.exchange_currency if self.show_exchanged_amounts else None
if self.currency.show_exchanged_amounts:
return self.currency.exchange_currency
return None
def clean(self): def clean(self):
super().clean() super().clean()
if self.exchange_currency == self.currency: if self.exchange_currency == self.currency:
+3
View File
@@ -4,3 +4,6 @@ from django.apps import AppConfig
class ApiConfig(AppConfig): class ApiConfig(AppConfig):
default_auto_field = "django.db.models.BigAutoField" default_auto_field = "django.db.models.BigAutoField"
name = "apps.api" name = "apps.api"
def ready(self):
import apps.api.schema # noqa: F401 registers the API token scheme for the docs
+2 -1
View File
@@ -10,7 +10,8 @@ from apps.transactions.models import (
@extend_schema_field( @extend_schema_field(
{ {
"oneOf": [{"type": "string"}, {"type": "integer"}, {"type": "null"}], "oneOf": [{"type": "string"}, {"type": "integer"}],
"nullable": True,
"description": "TransactionCategory ID or name. If the name doesn't exist, a new one will be created. Can be null if no category is assigned.", "description": "TransactionCategory ID or name. If the name doesn't exist, a new one will be created. Can be null if no category is assigned.",
} }
) )
+681
View File
@@ -0,0 +1,681 @@
"""Native MCP server (Streamable HTTP, stateless, JSON responses only).
Each tool runs the matching REST API view in-process, as the caller, so
permissions, sharing, filters, validation and signals stay the API's.
"""
import json
import logging
import re
from decimal import Decimal
from functools import cache
from threading import Lock
from urllib.parse import urlsplit
from django.conf import settings
from django.http import Http404
from django.test import RequestFactory
from django.urls import Resolver404, resolve
from drf_spectacular.generators import SchemaGenerator
from oauth2_provider.contrib.rest_framework import OAuth2Authentication
from rest_framework.exceptions import ParseError
from rest_framework.parsers import JSONParser
from rest_framework.permissions import IsAuthenticated
from rest_framework.renderers import JSONRenderer
from rest_framework.response import Response
from rest_framework.views import APIView
from apps.api import views
from apps.api.authentication import APITokenAuthentication
from apps.api.permissions import NotInDemoMode
from apps.common.oauth_views import _base_url
PROTOCOL_VERSIONS = ("2025-11-25", "2025-06-18", "2025-03-26", "2024-11-05")
INSTRUCTIONS = (
"WYGIWYH is a personal finance tracker. Amounts are never negative: `type` "
"IN is income and EX is expense. `reference_date` is the month a "
"transaction counts toward: YYYY-MM in these tools, YYYY-MM-DD for recurring "
"transactions and installment plans in call_api. Use summarize_transactions for totals instead "
"of adding up search results. Look up ids with list_accounts and "
"list_categories/list_tags/list_entities. For anything else, find the "
"endpoint with search_api and run it with call_api."
)
_factory = RequestFactory()
logger = logging.getLogger(__name__)
class ToolError(Exception):
pass
def _dispatch(request, view, method, *, path="/", kwargs=None, query=None, data=None):
"""Run an API view as the MCP caller and return its response data."""
host = {"HTTP_HOST": request._request.get_host(), "secure": request.is_secure()}
if method == "get":
inner = _factory.get(path, query or {}, **host)
else:
inner = getattr(_factory, method)(
path, json.dumps(data or {}), content_type="application/json", **host
)
inner._force_auth_user = request.user
inner._force_auth_token = request.auth
response = view(inner, **(kwargs or {}))
if response.status_code >= 400:
raise ToolError(response.data)
return response.data
def _call_api(request, viewset, action, *, method="get", pk=None, query=None, data=None):
return _dispatch(
request,
viewset.as_view({method: action}),
method,
kwargs={"pk": pk} if pk is not None else None,
query=query,
data=data,
)
def _num(value):
"""'50.000000000000000000000000000000' -> '50'."""
if value is None:
return None
return format(Decimal(str(value)).normalize(), "f")
def _clean(item):
return {k: v for k, v in item.items() if v not in (None, "", [])}
def _names(items):
return [i["name"] for i in items or []]
def _transaction(t):
account = t["account"]
exchanged = t["exchanged_amount"] # {amount, prefix, suffix, ...} or None
return _clean(
{
"id": t["id"],
"date": t["date"],
"reference_date": t["reference_date"],
"type": t["type"],
"is_paid": t["is_paid"],
"amount": _num(t["amount"]),
"currency": account["currency"]["code"],
"exchanged": (
f"{exchanged['prefix']}{_num(exchanged['amount'])}{exchanged['suffix']}"
if exchanged
else None
),
"description": t["description"],
"account": {"id": account["id"], "name": account["name"]},
"category": t["category"]["name"] if t["category"] else None,
"tags": _names(t["tags"]),
"entities": _names(t["entities"]),
"notes": t["notes"],
"sub_transactions": [
{"name": s["name"], "amount": _num(s["amount"])}
for s in t["sub_transactions"]
],
"mute": t["mute"] or None,
"transfer_with": t["linked_transaction"],
"installment_plan": t["installment_plan"],
"recurring_transaction": t["recurring_transaction"],
}
)
def _page(data, compact):
return {
"count": data["count"],
"has_more": data["next"] is not None,
"results": [compact(item) for item in data["results"]],
}
# --- tools -------------------------------------------------------------------
DATE = {"type": "string", "format": "date", "description": "YYYY-MM-DD"}
MONTH = {"type": "string", "description": "YYYY-MM"}
TYPE = {"type": "string", "enum": ["IN", "EX"], "description": "IN = income, EX = expense"}
ID = {"type": "integer"}
FILTERS = {
"date_from": DATE,
"date_to": DATE,
"month_from": MONTH,
"month_to": MONTH,
"account_id": ID,
"category_id": ID,
"tag_id": ID,
"entity_id": ID,
"type": TYPE,
"is_paid": {"type": "boolean"},
"description": {"type": "string", "description": "Part of the description, case-insensitive"},
}
FILTER_PARAMS = {
"date_from": "date__gte",
"date_to": "date__lte",
"month_from": "reference_date__gte",
"month_to": "reference_date__lte",
"account_id": "account",
"category_id": "category",
"tag_id": "tags",
"entity_id": "entities",
"type": "type",
"is_paid": "is_paid",
"description": "description__icontains",
}
TRANSACTION_FIELDS = {
"account_id": ID,
"type": TYPE,
"amount": {"type": "string", "description": "Decimal, never negative"},
"date": DATE,
"reference_date": {**MONTH, "description": "YYYY-MM; when creating, defaults to the month of date"},
"description": {"type": "string"},
"is_paid": {"type": "boolean", "description": "false for a planned transaction; when creating, defaults to true"},
"category": {
"type": ["integer", "string", "null"],
"description": "Category id or name; an unknown name creates the category; null clears it",
},
"tags": {
"type": "array",
"items": {"type": ["integer", "string"]},
"description": "Tag ids or names; unknown names are created. Replaces the current tags.",
},
"entities": {
"type": "array",
"items": {"type": ["integer", "string"]},
"description": "Entity ids or names; unknown names are created. Replaces the current entities.",
},
"notes": {"type": "string"},
"mute": {"type": "boolean", "description": "Muted transactions are left out of totals"},
}
def _filters(args):
query = {}
for key, param in FILTER_PARAMS.items():
if key in args:
value = args[key]
if isinstance(value, bool):
value = str(value).lower()
elif key.startswith("month_"): # the filter takes full dates
if not re.fullmatch(r"\d{4}-\d{2}", str(value)):
raise ToolError({key: "Use YYYY-MM."})
value = f"{value}-01"
query[param] = value
return query
def _transaction_body(args):
body = {k: v for k, v in args.items() if k in TRANSACTION_FIELDS}
if "amount" in body:
body["amount"] = str(body["amount"])
return body
def list_accounts(request, args):
query = {} if args.get("include_archived") else {"is_archived": "false"}
data = _call_api(request, views.AccountViewSet, "list", query={**query, "page_size": 1000})
return [
_clean(
{
"id": a["id"],
"name": a["name"],
"group": a["group"]["name"] if a["group"] else None,
"currency": a["currency"]["code"],
"exchange_currency": (a["exchange_currency"] or {}).get("code"),
"is_asset": a["is_asset"] or None,
"is_investment": a["is_investment"] or None,
}
)
for a in data["results"]
]
def get_account_balance(request, args):
data = _call_api(request, views.AccountViewSet, "balance", pk=args["account_id"])
return {
"current_balance": _num(data["current_balance"]),
"projected_balance": _num(data["projected_balance"]),
"currency": data["currency"]["code"],
}
def _filtered_transactions(request, action, query):
"""Like _call_api, but errors name the tool's filters, not the API's."""
try:
return _call_api(request, views.TransactionViewSet, action, query=query)
except ToolError as e:
detail = e.args[0]
if isinstance(detail, dict):
names = {param: arg for arg, param in FILTER_PARAMS.items()}
detail = {names.get(k, k): v for k, v in detail.items()}
raise ToolError(detail) from e
def search_transactions(request, args):
query = _filters(args)
query["page"] = args.get("page", 1)
query["page_size"] = min(int(args.get("page_size", 50)), 100)
query["ordering"] = "-date"
return _page(_filtered_transactions(request, "list", query), _transaction)
def summarize_transactions(request, args):
query = _filters(args)
for key in ("group_by", "include_muted"):
if key in args:
query[key] = str(args[key]).lower()
rows = _filtered_transactions(request, "summary", query)
return [{**row, "total": _num(row["total"])} for row in rows]
def get_transaction(request, args):
return _transaction(
_call_api(request, views.TransactionViewSet, "retrieve", pk=args["id"])
)
def create_transaction(request, args):
return _transaction(
_call_api(
request,
views.TransactionViewSet,
"create",
method="post",
data=_transaction_body(args),
)
)
def update_transaction(request, args):
return _transaction(
_call_api(
request,
views.TransactionViewSet,
"partial_update",
method="patch",
pk=args["id"],
data=_transaction_body(args),
)
)
def delete_transaction(request, args):
_call_api(request, views.TransactionViewSet, "destroy", method="delete", pk=args["id"])
return {"deleted": args["id"]}
def _lister(viewset):
def handler(request, args):
query = {"page_size": 1000}
if not args.get("include_inactive"):
query["active"] = "true"
if args.get("name"):
query["name__icontains"] = args["name"]
data = _call_api(request, viewset, "list", query=query)
return [
_clean(
{
"id": i["id"],
"name": i["name"],
"inactive": not i["active"] or None,
"mute": i.get("mute") or None, # categories only
}
)
for i in data["results"]
]
return handler
_operations_lock = Lock()
def _operations():
# Schema generation isn't thread-safe; concurrent first calls broke it
with _operations_lock:
return _build_operations()
@cache
def _build_operations():
"""The REST API's operations from its OpenAPI schema, with $refs inlined."""
schema = SchemaGenerator().get_schema(request=None, public=True)
components = schema["components"]["schemas"]
def inline(node, depth=0):
if isinstance(node, list):
return [inline(i, depth) for i in node]
if not isinstance(node, dict):
return node
if "$ref" in node:
if depth > 4: # recursive schemas
return {"type": "object"}
return inline(components[node["$ref"].rsplit("/", 1)[-1]], depth + 1)
return {k: inline(v, depth) for k, v in node.items()}
operations = []
for path, item in schema["paths"].items():
if not path.startswith("/api/"):
continue
for method, op in item.items():
content = op.get("requestBody", {}).get("content", {})
body = content.get("application/json") or next(iter(content.values()), None)
operations.append(
_clean(
{
"method": method.upper(),
"path": path,
"summary": op.get("summary"),
"description": op.get("description"),
# name -> enum, format or type; path params are in the path
"query": {
p["name"]: (lambda s: s.get("enum") or s.get("format") or s.get("type"))(
inline(p.get("schema") or {})
)
for p in op.get("parameters", [])
if p["in"] == "query"
},
"body": inline(body["schema"]) if body else None,
}
)
)
return operations
def search_api(request, args):
words = str(args.get("query", "")).lower().split()
if not words:
return [f"{op['method']} {op['path']}" + (f" — {op['summary']}" if "summary" in op else "") for op in _operations()]
scored = []
for op in _operations():
exact = {op["method"].lower(), *op["path"].strip("/").split("/")}
head = f"{op['method']} {op['path']} {op.get('summary', '')}".lower()
text = f"{head} {op.get('description', '')}".lower()
if all(w in text for w in words):
score = sum(2 * (w in exact) + (w in head) for w in words)
scored.append((-score, len(op["path"]), op))
scored.sort(key=lambda s: s[:2])
return [op for _, _, op in scored[: min(int(args.get("limit", 5)), 20)]]
def call_api(request, args):
method = str(args["method"]).lower()
if method not in ("get", "post", "put", "patch", "delete"):
raise ToolError({"method": "Must be GET, POST, PUT, PATCH or DELETE."})
url = urlsplit(str(args["path"]))
if url.scheme or url.netloc:
raise ToolError({"path": "Pass only the path, e.g. /api/accounts/."})
path = url.path
path = path if path.endswith("/") else f"{path}/"
try:
match = resolve(path)
except Resolver404:
match = None
view_class = getattr(match.func, "cls", None) if match else None
if not (view_class and view_class.__module__.startswith("apps.api.views")):
raise ToolError({"path": "Not a WYGIWYH API endpoint; find one with search_api."})
query = {
k: str(v).lower() if isinstance(v, bool) else v
for k, v in (args.get("query") or {}).items()
}
data = _dispatch(
request, match.func, method, path=path, kwargs=match.kwargs, query=query, data=args.get("body")
)
return {"ok": True} if data is None else data
LISTER_INPUT = {
"type": "object",
"properties": {
"name": {"type": "string", "description": "Part of the name, case-insensitive"},
"include_inactive": {"type": "boolean"},
},
}
READ = {"readOnlyHint": True, "openWorldHint": False}
WRITE = {"readOnlyHint": False, "destructiveHint": False, "openWorldHint": False}
TOOLS = [
{
"name": "list_accounts",
"description": "Accounts with their group and currency. Archived accounts are left out unless include_archived is true.",
"inputSchema": {"type": "object", "properties": {"include_archived": {"type": "boolean"}}},
"annotations": READ,
"handler": list_accounts,
},
{
"name": "get_account_balance",
"description": "Current balance (paid transactions only) and projected balance (paid and unpaid) of an account.",
"inputSchema": {"type": "object", "properties": {"account_id": ID}, "required": ["account_id"]},
"annotations": READ,
"handler": get_account_balance,
},
{
"name": "search_transactions",
"description": "Transactions matching the filters, newest date first. Paginated; page_size is at most 100.",
"inputSchema": {
"type": "object",
"properties": {
**FILTERS,
"page": {"type": "integer", "minimum": 1},
"page_size": {"type": "integer", "minimum": 1, "maximum": 100},
},
},
"annotations": READ,
"handler": search_transactions,
},
{
"name": "summarize_transactions",
"description": (
"Totals and counts of the transactions matching the filters, per currency and type, "
"optionally grouped by category, tag, entity, account or month. Muted transactions and "
"those in muted categories (see list_categories) are left out unless include_muted is true. With tag or entity grouping, a transaction counts "
"once per tag/entity. Transfers between accounts count as an expense and an income."
),
"inputSchema": {
"type": "object",
"properties": {
**FILTERS,
"group_by": {"type": "string", "enum": ["category", "tag", "entity", "account", "month"]},
"include_muted": {"type": "boolean"},
},
},
"annotations": READ,
"handler": summarize_transactions,
},
{
"name": "get_transaction",
"description": "One transaction by id.",
"inputSchema": {"type": "object", "properties": {"id": ID}, "required": ["id"]},
"annotations": READ,
"handler": get_transaction,
},
{
"name": "create_transaction",
"description": "Records an income or expense. Your transaction rules run on it, as when created in the app.",
"inputSchema": {
"type": "object",
"properties": TRANSACTION_FIELDS,
"required": ["account_id", "type", "amount", "date"],
},
"annotations": WRITE,
"handler": create_transaction,
},
{
"name": "update_transaction",
"description": "Changes only the fields sent. Your transaction rules run on it, as when edited in the app.",
"inputSchema": {
"type": "object",
"properties": {"id": ID, **TRANSACTION_FIELDS},
"required": ["id"],
},
"annotations": {**WRITE, "idempotentHint": True},
"handler": update_transaction,
},
{
"name": "delete_transaction",
"description": "Deletes a transaction. Deleting either side of a transfer deletes both.",
"inputSchema": {"type": "object", "properties": {"id": ID}, "required": ["id"]},
"annotations": {**WRITE, "destructiveHint": True, "idempotentHint": True},
"handler": delete_transaction,
},
{
"name": "list_categories",
"description": "Transaction categories. Inactive ones are left out unless include_inactive is true.",
"inputSchema": LISTER_INPUT,
"annotations": READ,
"handler": _lister(views.TransactionCategoryViewSet),
},
{
"name": "list_tags",
"description": "Transaction tags. Inactive ones are left out unless include_inactive is true.",
"inputSchema": LISTER_INPUT,
"annotations": READ,
"handler": _lister(views.TransactionTagViewSet),
},
{
"name": "list_entities",
"description": "People and businesses on the other side of transactions. Inactive ones are left out unless include_inactive is true.",
"inputSchema": LISTER_INPUT,
"annotations": READ,
"handler": _lister(views.TransactionEntityViewSet),
},
{
"name": "search_api",
"description": (
"Finds REST API endpoints for anything the other tools don't cover: recurring "
"transactions, installment plans, currencies, exchange rates, account groups, "
"investment classes, DCA strategies, import runs. Returns each match's method, path, "
"parameters and JSON body schema, to use with call_api. Without a query, lists every endpoint."
),
"inputSchema": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "Words that must all appear, e.g. 'recurring list'"},
"limit": {"type": "integer", "minimum": 1, "maximum": 20, "description": "Defaults to 5"},
},
},
"annotations": READ,
"handler": search_api,
},
{
"name": "call_api",
"description": (
"Calls a REST API endpoint found with search_api, as you. Replace path parameters "
"such as {id} with values. Responses are the API's full JSON; list endpoints are "
"paginated (page, page_size). File uploads aren't supported."
),
"inputSchema": {
"type": "object",
"properties": {
"method": {"type": "string", "enum": ["GET", "POST", "PUT", "PATCH", "DELETE"]},
"path": {"type": "string", "description": "e.g. /api/recurring-transactions/12/"},
"query": {"type": "object", "description": "Query string parameters"},
"body": {"type": "object", "description": "JSON body for POST, PUT and PATCH"},
},
"required": ["method", "path"],
},
"annotations": {**WRITE, "destructiveHint": True},
"handler": call_api,
},
]
TOOLS_BY_NAME = {tool["name"]: tool for tool in TOOLS}
# --- protocol ----------------------------------------------------------------
def _tool_result(data, is_error=False):
return {
"content": [{"type": "text", "text": json.dumps(data, ensure_ascii=False, default=str)}],
"isError": is_error,
}
class RpcError(Exception):
def __init__(self, code, message):
self.code, self.message = code, message
class MCPView(APIView):
authentication_classes = [OAuth2Authentication, APITokenAuthentication]
permission_classes = [IsAuthenticated, NotInDemoMode]
parser_classes = [JSONParser]
renderer_classes = [JSONRenderer]
schema = None # not part of the REST API docs
def initial(self, request, *args, **kwargs):
if not settings.MCP_ENABLED:
raise Http404
super().initial(request, *args, **kwargs)
def get_authenticate_header(self, request):
# Points MCP clients at the OAuth metadata (RFC 9728).
url = f"{_base_url(request)}/.well-known/oauth-protected-resource/mcp"
return f'Bearer resource_metadata="{url}"'
def handle_exception(self, exc):
# The body may be parsed during authentication, before post() runs
if isinstance(exc, ParseError):
return self._error(None, -32700, "Parse error")
return super().handle_exception(exc)
def post(self, request):
message = request.data
if (
not isinstance(message, dict)
or message.get("jsonrpc") != "2.0"
or not isinstance(message.get("method"), str)
):
return self._error(message.get("id") if isinstance(message, dict) else None, -32600, "Invalid Request")
if "id" not in message: # notification or response
return Response(status=202)
params = message.get("params") or {}
try:
result = self.handle(request, message["method"], params)
except RpcError as e:
return self._error(message["id"], e.code, e.message)
return Response({"jsonrpc": "2.0", "id": message["id"], "result": result})
def handle(self, request, method, params):
if method == "initialize":
requested = params.get("protocolVersion")
return {
"protocolVersion": requested if requested in PROTOCOL_VERSIONS else PROTOCOL_VERSIONS[0],
"capabilities": {"tools": {}},
"serverInfo": {"name": "WYGIWYH", "version": settings.APP_VERSION},
"instructions": INSTRUCTIONS,
}
if method == "ping":
return {}
if method == "tools/list":
return {"tools": [{k: v for k, v in t.items() if k != "handler"} for t in TOOLS]}
if method == "tools/call":
tool = TOOLS_BY_NAME.get(params.get("name"))
if tool is None:
raise RpcError(-32602, f"Unknown tool: {params.get('name')}")
args = params.get("arguments") or {}
if not isinstance(args, dict):
raise RpcError(-32602, "arguments must be an object")
missing = [k for k in tool["inputSchema"].get("required", []) if k not in args]
if missing:
return _tool_result({"missing": missing}, is_error=True)
try:
return _tool_result(tool["handler"](request, args))
except ToolError as e:
return _tool_result(e.args[0], is_error=True)
except Exception:
logger.exception("MCP tool %s failed", tool["name"])
return _tool_result({"detail": "Internal server error."}, is_error=True)
raise RpcError(-32601, f"Method not found: {method}")
@staticmethod
def _error(id_, code, message):
return Response({"jsonrpc": "2.0", "id": id_, "error": {"code": code, "message": message}})
+5 -7
View File
@@ -1,7 +1,7 @@
from rest_framework.permissions import ( from rest_framework.permissions import (
SAFE_METHODS, SAFE_METHODS,
BasePermission, BasePermission,
DjangoModelPermissions, IsAuthenticated,
) )
from django.conf import settings from django.conf import settings
@@ -17,11 +17,9 @@ class NotInDemoMode(BasePermission):
class SharedObjectPermission(BasePermission): class SharedObjectPermission(BasePermission):
"""Object-level ownership check for SharedObject-backed viewsets. """Object-level ownership check for SharedObject-backed viewsets.
DjangoModelPermissions is model-level: a user holding ``change_account`` The viewset's queryset includes other people's public and shared-with-them
may write any object the viewset's queryset returns, and for SharedObject objects. Sharing grants read access only, so writes are restricted to the
that queryset includes other people's public and shared-with-them objects. owner here, as the web UI does with ``get_shared_object_or_error``.
Sharing grants read access only, so writes are restricted to the owner
here as well.
Set ``shared_object_via`` on the viewset when the governing SharedObject is Set ``shared_object_via`` on the viewset when the governing SharedObject is
reached through a relation (e.g. ``"strategy"`` for a DCA entry). reached through a relation (e.g. ``"strategy"`` for a DCA entry).
@@ -43,6 +41,6 @@ class SharedObjectPermission(BasePermission):
#: ``permission_classes`` replaces the defaults, so they are repeated here. #: ``permission_classes`` replaces the defaults, so they are repeated here.
SHARED_OBJECT_PERMISSIONS = [ SHARED_OBJECT_PERMISSIONS = [
NotInDemoMode, NotInDemoMode,
DjangoModelPermissions, IsAuthenticated,
SharedObjectPermission, SharedObjectPermission,
] ]
+20
View File
@@ -0,0 +1,20 @@
from drf_spectacular.extensions import OpenApiAuthenticationExtension
from apps.users.models import APIToken
class APITokenAuthenticationScheme(OpenApiAuthenticationExtension):
"""Describes APITokenAuthentication in the API docs."""
target_class = "apps.api.authentication.APITokenAuthentication"
name = "apiToken"
def get_security_definition(self, auto_schema):
return {
"type": "apiKey",
"in": "header",
"name": "Authorization",
"description": (
f"A personal API token, sent as `Token {APIToken.TOKEN_PREFIX}…`."
),
}
+5
View File
@@ -17,6 +17,7 @@ class AccountGroupSerializer(serializers.ModelSerializer):
class Meta: class Meta:
model = AccountGroup model = AccountGroup
fields = "__all__" fields = "__all__"
read_only_fields = ["id", "owner"]
class InvestmentClassSerializer(serializers.ModelSerializer): class InvestmentClassSerializer(serializers.ModelSerializer):
@@ -44,6 +45,7 @@ class AccountSerializer(serializers.ModelSerializer):
source="group", source="group",
write_only=True, write_only=True,
allow_null=True, allow_null=True,
required=False,
) )
currency = CurrencySerializer(read_only=True) currency = CurrencySerializer(read_only=True)
@@ -56,6 +58,7 @@ class AccountSerializer(serializers.ModelSerializer):
source="exchange_currency", source="exchange_currency",
write_only=True, write_only=True,
allow_null=True, allow_null=True,
required=False,
) )
investment_class = InvestmentClassSerializer(read_only=True) investment_class = InvestmentClassSerializer(read_only=True)
@@ -80,6 +83,7 @@ class AccountSerializer(serializers.ModelSerializer):
"currency_id", "currency_id",
"exchange_currency", "exchange_currency",
"exchange_currency_id", "exchange_currency_id",
"show_exchanged_amounts",
"is_asset", "is_asset",
"is_investment", "is_investment",
"investment_class", "investment_class",
@@ -87,6 +91,7 @@ class AccountSerializer(serializers.ModelSerializer):
"income_entities", "income_entities",
"income_categories", "income_categories",
"income_tags", "income_tags",
"is_archived",
] ]
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
+11
View File
@@ -1,3 +1,4 @@
from django.utils.translation import gettext_lazy as _
from rest_framework import serializers from rest_framework import serializers
from rest_framework.permissions import IsAuthenticated from rest_framework.permissions import IsAuthenticated
@@ -10,6 +11,11 @@ class CurrencySerializer(serializers.ModelSerializer):
class Meta: class Meta:
model = Currency model = Currency
fields = "__all__" fields = "__all__"
# Prefixes and suffixes carry their spacing, e.g. "R$ "
extra_kwargs = {
"prefix": {"trim_whitespace": False},
"suffix": {"trim_whitespace": False},
}
class ExchangeRateSerializer(serializers.ModelSerializer): class ExchangeRateSerializer(serializers.ModelSerializer):
@@ -33,3 +39,8 @@ class ExchangeRateSerializer(serializers.ModelSerializer):
class Meta: class Meta:
model = ExchangeRate model = ExchangeRate
fields = "__all__" fields = "__all__"
def validate_rate(self, value):
if value <= 0:
raise serializers.ValidationError(_("The rate must be greater than zero."))
return value
+4
View File
@@ -37,6 +37,10 @@ class DCAEntrySerializer(serializers.ModelSerializer):
"entry_price", "entry_price",
] ]
read_only_fields = ["created_at", "updated_at"] read_only_fields = ["created_at", "updated_at"]
extra_kwargs = {
"amount_paid": {"min_value": 0},
"amount_received": {"min_value": 0},
}
class DCAStrategySerializer(serializers.ModelSerializer): class DCAStrategySerializer(serializers.ModelSerializer):
+18 -1
View File
@@ -1,5 +1,7 @@
from django.utils.translation import gettext_lazy as _
from rest_framework import serializers from rest_framework import serializers
from apps.accounts.models import Account
from apps.import_app.models import ImportProfile, ImportRun from apps.import_app.models import ImportProfile, ImportRun
@@ -21,6 +23,7 @@ class ImportRunSerializer(serializers.ModelSerializer):
"status", "status",
"profile", "profile",
"file_name", "file_name",
"account",
"logs", "logs",
"processed_rows", "processed_rows",
"total_rows", "total_rows",
@@ -39,8 +42,22 @@ class ImportFileSerializer(serializers.Serializer):
queryset=ImportProfile.objects.all(), source="profile" queryset=ImportProfile.objects.all(), source="profile"
) )
file = serializers.FileField() file = serializers.FileField()
account_id = serializers.PrimaryKeyRelatedField(
queryset=Account.objects.filter(is_archived=False),
source="account",
required=False,
allow_null=True,
)
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs) super().__init__(*args, **kwargs)
# Reload the queryset to get an updated version with the requesting user # Reload the querysets to get an updated version with the requesting user
self.fields["profile_id"].queryset = ImportProfile.objects.all() self.fields["profile_id"].queryset = ImportProfile.objects.all()
self.fields["account_id"].queryset = Account.objects.filter(is_archived=False)
def validate(self, data):
if data.get("account") and not data["profile"].imports_transactions:
raise serializers.ValidationError(
{"account_id": _("Only transaction imports can use an account.")}
)
return data
+116 -32
View File
@@ -1,5 +1,4 @@
from decimal import Decimal from django.db import transaction as db_transaction
from django.utils.translation import gettext_lazy as _ from django.utils.translation import gettext_lazy as _
from drf_spectacular import openapi from drf_spectacular import openapi
from drf_spectacular.types import OpenApiTypes from drf_spectacular.types import OpenApiTypes
@@ -22,10 +21,85 @@ from apps.transactions.models import (
InstallmentPlan, InstallmentPlan,
TransactionEntity, TransactionEntity,
RecurringTransaction, RecurringTransaction,
SubTransaction,
) )
from apps.common.middleware.thread_local import get_current_user from apps.common.middleware.thread_local import get_current_user
class SubTransactionSerializer(serializers.ModelSerializer):
class Meta:
model = SubTransaction
fields = ["id", "name", "amount"]
read_only_fields = ["id"]
class SubTransactionsMixin:
# `sub_transactions`, when sent, replaces the parts; a non-empty list sets
# the amount to their sum. Declare the field on the serializer itself.
sub_amount_field = "amount"
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.fields[self.sub_amount_field].required = False
def is_transfer(self, data):
return False
def validate(self, data):
field = self.sub_amount_field
subs = data.get("sub_transactions")
has_parts = self.instance is not None and self.instance.sub_transactions.exists()
if self.is_transfer(data):
if subs:
raise serializers.ValidationError(
{"sub_transactions": _("Transfers can't have sub-transactions.")}
)
if has_parts:
data["sub_transactions"] = []
elif subs:
data[field] = sum(sub["amount"] for sub in subs)
elif subs is None and field in data and has_parts:
raise serializers.ValidationError(
{
field: _(
"The amount is the sum of the sub-transactions; "
"send sub_transactions instead."
)
}
)
if not self.partial and data.get(field) is None:
raise serializers.ValidationError({field: _("This field is required.")})
return super().validate(data)
@staticmethod
def set_sub_transactions(instance, subs):
if subs is None:
return
instance.sub_transactions.all().delete()
for sub in subs:
instance.sub_transactions.create(**sub)
def create(self, validated_data):
subs = validated_data.pop("sub_transactions", None)
with db_transaction.atomic():
instance = super().create(validated_data)
self.set_sub_transactions(instance, subs)
self.after_save(instance, created=True)
return instance
def update(self, instance, validated_data):
subs = validated_data.pop("sub_transactions", None)
with db_transaction.atomic():
instance = super().update(instance, validated_data)
self.set_sub_transactions(instance, subs)
self.after_save(instance, created=False)
return instance
def after_save(self, instance, created):
"""Runs once the parts are saved, e.g. to generate transactions from them."""
class TransactionCategorySerializer(serializers.ModelSerializer): class TransactionCategorySerializer(serializers.ModelSerializer):
permission_classes = [IsAuthenticated] permission_classes = [IsAuthenticated]
@@ -62,8 +136,10 @@ class TransactionEntitySerializer(serializers.ModelSerializer):
] ]
class InstallmentPlanSerializer(serializers.ModelSerializer): class InstallmentPlanSerializer(SubTransactionsMixin, serializers.ModelSerializer):
category: str | int = TransactionCategoryField(required=False) sub_amount_field = "installment_amount"
sub_transactions = SubTransactionSerializer(many=True, required=False)
category: str | int = TransactionCategoryField(required=False, allow_null=True)
tags: str | int = TransactionTagField(required=False) tags: str | int = TransactionTagField(required=False)
entities: str | int = TransactionEntityField(required=False) entities: str | int = TransactionEntityField(required=False)
@@ -84,26 +160,25 @@ class InstallmentPlanSerializer(serializers.ModelSerializer):
"end_date", "end_date",
"recurrence", "recurrence",
"installment_amount", "installment_amount",
"sub_transactions",
"category", "category",
"tags", "tags",
"entities", "entities",
"notes", "notes",
] ]
read_only_fields = ["installment_total_number", "end_date"] read_only_fields = ["installment_total_number", "end_date"]
extra_kwargs = {"installment_amount": {"min_value": 0}}
def create(self, validated_data): def after_save(self, instance, created):
instance = super().create(validated_data) if created:
instance.create_transactions() instance.create_transactions()
return instance else:
instance.update_transactions()
def update(self, instance, validated_data):
instance = super().update(instance, validated_data)
instance.update_transactions()
return instance
class RecurringTransactionSerializer(serializers.ModelSerializer): class RecurringTransactionSerializer(SubTransactionsMixin, serializers.ModelSerializer):
category: str | int = TransactionCategoryField(required=False) sub_transactions = SubTransactionSerializer(many=True, required=False)
category: str | int = TransactionCategoryField(required=False, allow_null=True)
tags: str | int = TransactionTagField(required=False) tags: str | int = TransactionTagField(required=False)
entities: str | int = TransactionEntityField(required=False) entities: str | int = TransactionEntityField(required=False)
@@ -115,6 +190,7 @@ class RecurringTransactionSerializer(serializers.ModelSerializer):
"account", "account",
"type", "type",
"amount", "amount",
"sub_transactions",
"to_account", "to_account",
"to_amount", "to_amount",
"description", "description",
@@ -131,6 +207,11 @@ class RecurringTransactionSerializer(serializers.ModelSerializer):
"last_generated_reference_date", "last_generated_reference_date",
] ]
read_only_fields = ["last_generated_date", "last_generated_reference_date"] read_only_fields = ["last_generated_date", "last_generated_reference_date"]
extra_kwargs = {
"amount": {"min_value": 0},
"to_amount": {"min_value": 0},
"recurrence_interval": {"min_value": 1},
}
def validate(self, attrs): def validate(self, attrs):
# Same rule as TransferTemplate.clean(), which DRF doesn't run # Same rule as TransferTemplate.clean(), which DRF doesn't run
@@ -142,20 +223,21 @@ class RecurringTransactionSerializer(serializers.ModelSerializer):
) )
return super().validate(attrs) return super().validate(attrs)
def create(self, validated_data): def is_transfer(self, data):
instance = super().create(validated_data) type_ = data.get("type", getattr(self.instance, "type", None))
instance.create_upcoming_transactions() return type_ == RecurringTransaction.Type.TRANSFER
return instance
def update(self, instance, validated_data): def after_save(self, instance, created):
instance = super().update(instance, validated_data) if created:
instance.update_unpaid_transactions() instance.create_upcoming_transactions()
instance.generate_upcoming_transactions() else:
return instance instance.update_unpaid_transactions()
instance.generate_upcoming_transactions()
class TransactionSerializer(serializers.ModelSerializer): class TransactionSerializer(SubTransactionsMixin, serializers.ModelSerializer):
category: str | int = TransactionCategoryField(required=False) sub_transactions = SubTransactionSerializer(many=True, required=False)
category: str | int = TransactionCategoryField(required=False, allow_null=True)
tags: str | int = TransactionTagField(required=False) tags: str | int = TransactionTagField(required=False)
entities: str | int = TransactionEntityField(required=False) entities: str | int = TransactionEntityField(required=False)
@@ -194,6 +276,9 @@ class TransactionSerializer(serializers.ModelSerializer):
self.fields["account_id"].queryset = Account.objects.all() self.fields["account_id"].queryset = Account.objects.all()
def is_transfer(self, data):
return bool(self.instance and self.instance.linked_transaction_id)
def validate(self, data): def validate(self, data):
if not self.partial: if not self.partial:
if "date" in data and "reference_date" not in data: if "date" in data and "reference_date" not in data:
@@ -204,12 +289,12 @@ class TransactionSerializer(serializers.ModelSerializer):
raise serializers.ValidationError( raise serializers.ValidationError(
_("Either 'date' or 'reference_date' must be provided.") _("Either 'date' or 'reference_date' must be provided.")
) )
return data return super().validate(data)
def create(self, validated_data): def create(self, validated_data):
tags = validated_data.pop("tags", []) tags = validated_data.pop("tags", [])
entities = validated_data.pop("entities", []) entities = validated_data.pop("entities", [])
transaction = Transaction.objects.create(**validated_data) transaction = super().create(validated_data)
transaction.tags.set(tags) transaction.tags.set(tags)
transaction.entities.set(entities) transaction.entities.set(entities)
return transaction return transaction
@@ -217,9 +302,7 @@ class TransactionSerializer(serializers.ModelSerializer):
def update(self, instance, validated_data): def update(self, instance, validated_data):
tags = validated_data.pop("tags", None) tags = validated_data.pop("tags", None)
entities = validated_data.pop("entities", None) entities = validated_data.pop("entities", None)
for attr, value in validated_data.items(): instance = super().update(instance, validated_data)
setattr(instance, attr, value)
instance.save()
if tags is not None: if tags is not None:
instance.tags.set(tags) instance.tags.set(tags)
@@ -229,5 +312,6 @@ class TransactionSerializer(serializers.ModelSerializer):
return instance return instance
@staticmethod @staticmethod
def get_exchanged_amount(obj) -> Decimal: @extend_schema_field(OpenApiTypes.OBJECT)
def get_exchanged_amount(obj):
return obj.exchanged_amount() return obj.exchanged_amount()
+307
View File
@@ -0,0 +1,307 @@
import json
from datetime import date
from decimal import Decimal
from unittest import mock
from django.contrib.auth import get_user_model
from django.test import TestCase, override_settings
from apps.accounts.models import Account
from apps.common.middleware.thread_local import write_current_user, delete_current_user
from apps.accounts.models import AccountGroup
from apps.currencies.models import Currency, ExchangeRate
from apps.transactions.models import Transaction, TransactionCategory, TransactionTag
from apps.users.models import APIToken
User = get_user_model()
@override_settings(MCP_ENABLED=True, PUBLIC_BASE_URL="https://money.example")
class MCPTests(TestCase):
def setUp(self):
self.user = User.objects.create_user(email="me@example.com", password="x")
self.other = User.objects.create_user(email="other@example.com", password="x")
_token, self.raw_token = APIToken.objects.create_token(user=self.user, name="mcp")
self.brl = Currency.objects.create(code="BRL", name="Real", decimal_places=2)
self.account = self._as(self.user, Account.objects.create, name="Checking", currency=self.brl)
self.food = self._as(self.user, TransactionCategory.objects.create, name="Food")
self.muted = self._as(self.user, TransactionCategory.objects.create, name="Hidden", mute=True)
for amount, category in ((10, self.food), (5.5, self.food), (100, self.muted)):
self._as(
self.user,
Transaction.objects.create,
account=self.account,
type=Transaction.Type.EXPENSE,
amount=Decimal(str(amount)),
date=date(2026, 3, 5),
reference_date=date(2026, 3, 1),
category=category,
is_paid=True,
)
other_account = self._as(self.other, Account.objects.create, name="Theirs", currency=self.brl)
self.other_tx = self._as(
self.other,
Transaction.objects.create,
account=other_account,
type=Transaction.Type.EXPENSE,
amount=Decimal("999"),
date=date(2026, 3, 5),
reference_date=date(2026, 3, 1),
)
@staticmethod
def _as(user, create, **kwargs):
write_current_user(user) # owner comes from the current user
try:
return create(**kwargs)
finally:
delete_current_user()
def _rpc(self, method, params=None, id_=1, auth=True):
body = {"jsonrpc": "2.0", "method": method}
if id_ is not None:
body["id"] = id_
if params is not None:
body["params"] = params
headers = {"HTTP_AUTHORIZATION": f"Token {self.raw_token}"} if auth else {}
return self.client.post(
"/mcp", json.dumps(body), content_type="application/json", **headers
)
def _tool(self, name, **arguments):
response = self._rpc("tools/call", {"name": name, "arguments": arguments})
self.assertEqual(response.status_code, 200, response.content)
result = response.json()["result"]
return result["isError"], json.loads(result["content"][0]["text"])
@override_settings(MCP_ENABLED=False)
def test_disabled_by_setting(self):
self.assertEqual(self._rpc("ping").status_code, 404)
self.assertEqual(self.client.get("/.well-known/oauth-protected-resource/mcp").status_code, 404)
def test_unauthenticated_points_to_resource_metadata(self):
response = self._rpc("ping", auth=False)
self.assertEqual(response.status_code, 401)
self.assertEqual(
response["WWW-Authenticate"],
'Bearer resource_metadata="https://money.example/.well-known/oauth-protected-resource/mcp"',
)
metadata = self.client.get("/.well-known/oauth-protected-resource/mcp").json()
self.assertEqual(metadata["resource"], "https://money.example/mcp")
self.assertEqual(metadata["authorization_servers"], ["https://money.example"])
def test_handshake(self):
result = self._rpc("initialize", {"protocolVersion": "2025-06-18"}).json()["result"]
self.assertEqual(result["protocolVersion"], "2025-06-18")
self.assertIn("tools", result["capabilities"])
self.assertEqual(self._rpc("notifications/initialized", id_=None).status_code, 202)
tools = self._rpc("tools/list").json()["result"]["tools"]
self.assertIn("summarize_transactions", {t["name"] for t in tools})
self.assertNotIn("handler", tools[0])
def test_protocol_errors(self):
self.assertEqual(self._rpc("nope").json()["error"]["code"], -32601)
unknown_tool = self._rpc("tools/call", {"name": "nope"}).json()
self.assertEqual(unknown_tool["error"]["code"], -32602)
bad_json = self.client.post(
"/mcp", "{", content_type="application/json",
HTTP_AUTHORIZATION=f"Token {self.raw_token}",
)
self.assertEqual(bad_json.json()["error"]["code"], -32700)
def test_summary_leaves_out_muted_and_other_users(self):
is_error, rows = self._tool("summarize_transactions", group_by="category", month_from="2026-03")
self.assertFalse(is_error, rows)
self.assertEqual(
rows,
[{"currency": "BRL", "type": "EX", "group": "Food", "group_id": self.food.id, "total": "15.5", "count": 2}],
)
_, with_muted = self._tool("summarize_transactions", include_muted=True)
self.assertEqual(with_muted[0]["total"], "115.5")
def test_search_is_compact_and_scoped(self):
is_error, page = self._tool("search_transactions", category_id=self.food.id, page_size=1)
self.assertFalse(is_error, page)
self.assertEqual(page["count"], 2)
self.assertTrue(page["has_more"])
tx = page["results"][0]
self.assertEqual(tx["account"], {"id": self.account.id, "name": "Checking"})
self.assertEqual(tx["currency"], "BRL")
self.assertEqual(tx["category"], "Food")
self.assertIn(tx["amount"], ("10", "5.5"))
is_error, error = self._tool("get_transaction", id=self.other_tx.id)
self.assertTrue(is_error)
def test_create_update_delete(self):
is_error, created = self._tool(
"create_transaction",
account_id=self.account.id,
type="IN",
amount="1200.50",
date="2026-04-02",
description="Salary",
tags=["Work"],
)
self.assertFalse(is_error, created)
self.assertEqual(created["amount"], "1200.5")
self.assertEqual(created["reference_date"], "2026-04")
self.assertEqual(created["tags"], ["Work"])
is_error, updated = self._tool("update_transaction", id=created["id"], is_paid=True)
self.assertFalse(is_error, updated)
self.assertTrue(updated["is_paid"])
self.assertEqual(updated["description"], "Salary")
is_error, _ = self._tool("delete_transaction", id=created["id"])
self.assertFalse(is_error)
self.assertFalse(Transaction.objects.filter(pk=created["id"]).exists())
def test_search_api_finds_endpoints_with_their_inputs(self):
_, everything = self._tool("search_api")
self.assertIn("GET /api/currencies/", everything)
self.assertFalse(any("/api/schema/" in line for line in everything))
_, matches = self._tool("search_api", query="recurring post", limit=1)
self.assertEqual(matches[0]["path"], "/api/recurring-transactions/")
self.assertEqual(matches[0]["method"], "POST")
self.assertIn("recurrence_type", matches[0]["body"]["properties"])
def test_call_api_runs_endpoints_as_the_caller(self):
is_error, own = self._tool("call_api", method="GET", path="/api/currencies", query={"code": "BRL"})
self.assertFalse(is_error, own)
self.assertEqual(own["results"][0]["code"], "BRL")
is_error, _ = self._tool("call_api", method="GET", path=f"/api/transactions/{self.other_tx.id}/")
self.assertTrue(is_error)
is_error, deleted = self._tool(
"call_api", method="DELETE", path=f"/api/categories/{self.muted.id}/"
)
self.assertFalse(is_error, deleted)
self.assertFalse(TransactionCategory.objects.filter(pk=self.muted.id).exists())
def test_call_api_only_reaches_the_api(self):
for path in ("/api/schema/", "/api/", "/admin/", "/mcp"):
is_error, error = self._tool("call_api", method="GET", path=path)
self.assertTrue(is_error, path)
self.assertIn("path", error)
def test_exchanged_amount(self):
usd = Currency.objects.create(code="USD", name="Dollar", decimal_places=2, prefix="US$ ")
ExchangeRate.objects.create(from_currency=self.brl, to_currency=usd, rate=Decimal("0.2"), date=date(2026, 1, 1))
self.account.exchange_currency = usd
self.account.save()
is_error, page = self._tool("search_transactions", category_id=self.food.id, date_from="2026-03-01")
self.assertFalse(is_error, page)
self.assertEqual({t["exchanged"] for t in page["results"]}, {"US$ 2", "US$ 1.1"})
def test_category_can_be_cleared(self):
tx = Transaction.objects.filter(category=self.food).first()
is_error, updated = self._tool("update_transaction", id=tx.id, category=None)
self.assertFalse(is_error, updated)
self.assertNotIn("category", updated)
def test_filter_errors_use_tool_argument_names(self):
is_error, errors = self._tool("search_transactions", date_from="not-a-date")
self.assertTrue(is_error)
self.assertIn("date_from", errors)
is_error, errors = self._tool("summarize_transactions", month_from="2026-03-01")
self.assertTrue(is_error)
self.assertEqual(errors, {"month_from": "Use YYYY-MM."})
def test_exchange_rates_must_be_positive(self):
usd = Currency.objects.create(code="USD", name="Dollar", decimal_places=2)
is_error, errors = self._tool(
"call_api", method="POST", path="/api/exchange-rates/",
body={"from_currency_id": self.brl.id, "to_currency_id": usd.id, "rate": "-2", "date": "2026-01-01T00:00:00Z"},
)
self.assertTrue(is_error)
self.assertIn("rate", errors)
def test_unexpected_errors_are_tool_errors(self):
with mock.patch.dict(
"apps.api.mcp.TOOLS_BY_NAME",
{"list_tags": {"name": "list_tags", "inputSchema": {}, "handler": mock.Mock(side_effect=ValueError)}},
):
is_error, error = self._tool("list_tags")
self.assertTrue(is_error)
self.assertEqual(error, {"detail": "Internal server error."})
def test_api_keeps_currency_spacing_and_group_owner(self):
is_error, currency = self._tool(
"call_api", method="POST", path="/api/currencies/",
body={"code": "EUR", "name": "Euro", "prefix": "€ ", "suffix": " eur"},
)
self.assertFalse(is_error, currency)
self.assertEqual((currency["prefix"], currency["suffix"]), ("€ ", " eur"))
is_error, group = self._tool(
"call_api", method="POST", path="/api/account-groups/",
body={"name": "Planted", "owner": self.other.id},
)
self.assertFalse(is_error, group)
self.assertEqual(AccountGroup.all_objects.get(pk=group["id"]).owner, self.user)
def test_api_rejects_bad_amounts_and_intervals(self):
common = {"account": self.account.id, "type": "EX", "description": "x", "start_date": "2026-04-01"}
cases = [
("/api/installment-plans/", {**common, "installment_amount": "-10", "number_of_installments": 2, "recurrence": "monthly"}, "installment_amount"),
("/api/recurring-transactions/", {**common, "amount": "10", "recurrence_type": "month", "recurrence_interval": 0}, "recurrence_interval"),
]
for path, body, field in cases:
is_error, errors = self._tool("call_api", method="POST", path=path, body=body)
self.assertTrue(is_error, path)
self.assertIn(field, errors)
def test_accounts_can_be_archived_and_performance_needs_an_investment(self):
is_error, _ = self._tool(
"call_api", method="PATCH", path=f"/api/accounts/{self.account.id}/", body={"is_archived": True}
)
self.assertFalse(is_error)
self.account.refresh_from_db()
self.assertTrue(self.account.is_archived)
is_error, _ = self._tool("call_api", method="GET", path=f"/api/accounts/{self.account.id}/performance/")
self.assertTrue(is_error)
def test_tag_filter_muted_categories_and_month_order(self):
_, created = self._tool(
"create_transaction", account_id=self.account.id, type="EX", amount="7", date="2026-01-02", tags=["Trip"]
)
tag = TransactionTag.all_objects.get(name="Trip")
_, page = self._tool("search_transactions", tag_id=tag.id)
self.assertEqual([t["id"] for t in page["results"]], [created["id"]])
_, categories = self._tool("list_categories")
self.assertEqual({c["name"]: c.get("mute") for c in categories}, {"Food": None, "Hidden": True})
_, months = self._tool("summarize_transactions", group_by="month")
self.assertEqual([m["group"] for m in months], ["2026-01", "2026-03"])
def test_search_api_ranking_and_call_api_rejects_urls(self):
_, matches = self._tool("search_api", query="transactions post", limit=1)
self.assertEqual(matches[0]["path"], "/api/transactions/")
is_error, error = self._tool("call_api", method="GET", path="http://elsewhere.invalid/api/accounts/")
self.assertTrue(is_error)
self.assertIn("path", error)
def test_tool_validation_errors_are_tool_results(self):
is_error, missing = self._tool("create_transaction", account_id=self.account.id)
self.assertTrue(is_error)
self.assertEqual(set(missing["missing"]), {"type", "amount", "date"})
is_error, errors = self._tool(
"create_transaction", account_id=self.account.id, type="EX", amount="-1", date="2026-04-02"
)
self.assertTrue(is_error)
self.assertIn("amount", errors)
+5 -33
View File
@@ -1,10 +1,8 @@
"""Object-level ownership on the SharedObject API viewsets. """Object-level ownership on the SharedObject API viewsets.
DjangoModelPermissions is model-level: a user holding ``change_account`` could Any signed-in user may write through the API, and for SharedObject the
write any object the viewset's queryset returned, and for SharedObject that viewset's queryset includes other people's public and shared-with-them objects,
queryset includes other people's public and shared-with-them objects. Ordinary so the object-level check is what keeps those read-only.
users hold no model permissions, so this was not reachable for them, but the
object-level check was missing entirely.
Reads are unaffected -- they stay governed by SharedObjectManager. Reads are unaffected -- they stay governed by SharedObjectManager.
""" """
@@ -13,7 +11,6 @@ from datetime import date
from decimal import Decimal from decimal import Decimal
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
from django.contrib.auth.models import Permission
from django.test import TestCase, override_settings from django.test import TestCase, override_settings
from rest_framework import status from rest_framework import status
from rest_framework.test import APIClient from rest_framework.test import APIClient
@@ -35,12 +32,7 @@ from apps.transactions.models import TransactionCategory
DEMO=False, DEMO=False,
) )
class SharedObjectAPIPermissionTests(TestCase): class SharedObjectAPIPermissionTests(TestCase):
"""The attacker here deliberately HOLDS the Django model permissions. """The attacker is an ordinary signed-in user who can see the objects."""
Without them DjangoModelPermissions already answers 403 and the
object-level check is never consulted, so the test would pass whether or
not it exists.
"""
def setUp(self): def setUp(self):
User = get_user_model() User = get_user_model()
@@ -50,23 +42,6 @@ class SharedObjectAPIPermissionTests(TestCase):
self.attacker = User.objects.create_user( self.attacker = User.objects.create_user(
email="attacker@test.com", password="testpass123" email="attacker@test.com", password="testpass123"
) )
self.attacker.user_permissions.set(
Permission.objects.filter(
codename__in=[
"add_account",
"change_account",
"delete_account",
"add_transactioncategory",
"change_transactioncategory",
"delete_transactioncategory",
"add_dcaentry",
"change_dcaentry",
"delete_dcaentry",
]
)
)
# Permissions are cached on the user instance.
self.attacker = User.objects.get(pk=self.attacker.pk)
self.currency = Currency.objects.create( self.currency = Currency.objects.create(
code="USD", name="US Dollar", decimal_places=2 code="USD", name="US Dollar", decimal_places=2
@@ -149,10 +124,7 @@ class SharedObjectAPIPermissionTests(TestCase):
def test_owner_can_still_write_their_own_objects(self): def test_owner_can_still_write_their_own_objects(self):
owner_client = APIClient() owner_client = APIClient()
self.owner.user_permissions.set( owner = self.owner
Permission.objects.filter(codename__in=["change_account", "delete_account"])
)
owner = get_user_model().objects.get(pk=self.owner.pk)
owner_client.force_authenticate(user=owner) owner_client.force_authenticate(user=owner)
account = Account.all_objects.create( account = Account.all_objects.create(
+11
View File
@@ -0,0 +1,11 @@
import tempfile
from django.core.management import call_command
from django.test import SimpleTestCase
class SchemaTests(SimpleTestCase):
def test_schema_is_valid_without_warnings(self):
"""Keeps /api/docs/ complete: drf-spectacular warns when it can't describe something."""
with tempfile.NamedTemporaryFile(suffix=".yml") as schema:
call_command("spectacular", "--validate", "--fail-on-warn", "--file", schema.name)
+9 -3
View File
@@ -1,6 +1,7 @@
from drf_spectacular.utils import extend_schema, extend_schema_view from drf_spectacular.utils import extend_schema, extend_schema_view
from rest_framework import viewsets from rest_framework import viewsets
from rest_framework.decorators import action from rest_framework.decorators import action
from rest_framework.exceptions import ValidationError
from rest_framework.permissions import IsAuthenticated from rest_framework.permissions import IsAuthenticated
from rest_framework.response import Response from rest_framework.response import Response
@@ -18,7 +19,7 @@ from apps.investments.services import get_holding_performance
class AccountGroupViewSet(viewsets.ModelViewSet): class AccountGroupViewSet(viewsets.ModelViewSet):
"""ViewSet for managing account groups.""" """Groups that organize accounts, e.g. by bank."""
permission_classes = SHARED_OBJECT_PERMISSIONS permission_classes = SHARED_OBJECT_PERMISSIONS
queryset = AccountGroup.objects.all() queryset = AccountGroup.objects.all()
@@ -36,7 +37,8 @@ class AccountGroupViewSet(viewsets.ModelViewSet):
class InvestmentClassViewSet(viewsets.ModelViewSet): class InvestmentClassViewSet(viewsets.ModelViewSet):
"""ViewSet for managing asset classes and their target allocation.""" """Asset classes (e.g. stocks, fixed income) that group investment accounts,
each with a target share of the portfolio in percent."""
queryset = InvestmentClass.objects.all() queryset = InvestmentClass.objects.all()
serializer_class = InvestmentClassSerializer serializer_class = InvestmentClassSerializer
@@ -62,7 +64,9 @@ class InvestmentClassViewSet(viewsets.ModelViewSet):
), ),
) )
class AccountViewSet(viewsets.ModelViewSet): class AccountViewSet(viewsets.ModelViewSet):
"""ViewSet for managing accounts.""" """Accounts hold transactions in one currency; `exchange_currency` is the
currency their amounts are converted to. Asset accounts count towards net
worth but not towards the month; archived ones count towards neither."""
permission_classes = SHARED_OBJECT_PERMISSIONS permission_classes = SHARED_OBJECT_PERMISSIONS
queryset = Account.objects.all() queryset = Account.objects.all()
@@ -109,6 +113,8 @@ class AccountViewSet(viewsets.ModelViewSet):
def performance(self, request, pk=None): def performance(self, request, pk=None):
"""Get investment performance (cost basis, value, returns) for an account.""" """Get investment performance (cost basis, value, returns) for an account."""
account = self.get_object() account = self.get_object()
if not account.is_investment:
raise ValidationError({"detail": "Only investment accounts have performance."})
return Response( return Response(
InvestmentPerformanceSerializer(get_holding_performance(account)).data InvestmentPerformanceSerializer(get_holding_performance(account)).data
) )
+5
View File
@@ -7,6 +7,9 @@ from apps.currencies.models import ExchangeRate
class CurrencyViewSet(viewsets.ModelViewSet): class CurrencyViewSet(viewsets.ModelViewSet):
"""Currencies and assets (e.g. crypto). `exchange_currency` is the default
currency their amounts are converted to."""
queryset = Currency.objects.all() queryset = Currency.objects.all()
serializer_class = CurrencySerializer serializer_class = CurrencySerializer
filterset_fields = { filterset_fields = {
@@ -23,6 +26,8 @@ class CurrencyViewSet(viewsets.ModelViewSet):
class ExchangeRateViewSet(viewsets.ModelViewSet): class ExchangeRateViewSet(viewsets.ModelViewSet):
"""How much one unit of `from_currency` is worth in `to_currency` on `date`."""
queryset = ExchangeRate.objects.all() queryset = ExchangeRate.objects.all()
serializer_class = ExchangeRateSerializer serializer_class = ExchangeRateSerializer
filterset_fields = { filterset_fields = {
+27
View File
@@ -1,3 +1,5 @@
from drf_spectacular.types import OpenApiTypes
from drf_spectacular.utils import extend_schema
from rest_framework import viewsets from rest_framework import viewsets
from rest_framework.decorators import action from rest_framework.decorators import action
from rest_framework.response import Response from rest_framework.response import Response
@@ -7,6 +9,9 @@ from apps.api.serializers import DCAStrategySerializer, DCAEntrySerializer
class DCAStrategyViewSet(viewsets.ModelViewSet): class DCAStrategyViewSet(viewsets.ModelViewSet):
"""Dollar-cost averaging plans: recurring buys of `target_currency` paid in
`payment_currency`."""
permission_classes = SHARED_OBJECT_PERMISSIONS permission_classes = SHARED_OBJECT_PERMISSIONS
queryset = DCAStrategy.objects.all() queryset = DCAStrategy.objects.all()
serializer_class = DCAStrategySerializer serializer_class = DCAStrategySerializer
@@ -24,16 +29,36 @@ class DCAStrategyViewSet(viewsets.ModelViewSet):
def get_queryset(self): def get_queryset(self):
return DCAStrategy.objects.all() return DCAStrategy.objects.all()
@extend_schema(
summary="Buys over time",
description="Chart data for the strategy's buys: `labels` are entry dates in the server's display format.",
responses=OpenApiTypes.OBJECT,
)
@action(detail=True, methods=["get"]) @action(detail=True, methods=["get"])
def investment_frequency(self, request, pk=None): def investment_frequency(self, request, pk=None):
strategy = self.get_object() strategy = self.get_object()
return Response(strategy.investment_frequency_data()) return Response(strategy.investment_frequency_data())
@extend_schema(
summary="Paid vs current value per entry",
description=(
"Chart data, one item per entry. Despite their names, `entry_prices` holds "
"the amount paid and `current_prices` the entry's current value, both in the "
"payment currency; `amounts_bought` is the amount received. `labels` are entry "
"dates in the server's display format."
),
responses=OpenApiTypes.OBJECT,
)
@action(detail=True, methods=["get"]) @action(detail=True, methods=["get"])
def price_comparison(self, request, pk=None): def price_comparison(self, request, pk=None):
strategy = self.get_object() strategy = self.get_object()
return Response(strategy.price_comparison_data()) return Response(strategy.price_comparison_data())
@extend_schema(
summary="Latest price of the target currency",
description="The latest exchange rate from the target to the payment currency, and its date.",
responses=OpenApiTypes.OBJECT,
)
@action(detail=True, methods=["get"]) @action(detail=True, methods=["get"])
def current_price(self, request, pk=None): def current_price(self, request, pk=None):
strategy = self.get_object() strategy = self.get_object()
@@ -45,6 +70,8 @@ class DCAStrategyViewSet(viewsets.ModelViewSet):
class DCAEntryViewSet(viewsets.ModelViewSet): class DCAEntryViewSet(viewsets.ModelViewSet):
"""One buy in a DCA strategy: what was paid and what was received."""
permission_classes = SHARED_OBJECT_PERMISSIONS permission_classes = SHARED_OBJECT_PERMISSIONS
shared_object_via = "strategy" shared_object_via = "strategy"
queryset = DCAEntry.objects.all() queryset = DCAEntry.objects.all()
+5 -1
View File
@@ -102,6 +102,7 @@ class ImportRunViewSet(viewsets.ReadOnlyModelViewSet):
"properties": { "properties": {
"profile_id": {"type": "integer", "description": "ID of the ImportProfile to use"}, "profile_id": {"type": "integer", "description": "ID of the ImportProfile to use"},
"file": {"type": "string", "format": "binary", "description": "CSV or XLSX file to import"}, "file": {"type": "string", "format": "binary", "description": "CSV or XLSX file to import"},
"account_id": {"type": "integer", "description": "ID of the Account to import every transaction into. Leave empty to let the profile pick the account. Only for profiles that import transactions"},
}, },
"required": ["profile_id", "file"], "required": ["profile_id", "file"],
}, },
@@ -137,7 +138,10 @@ class ImportViewSet(viewsets.ViewSet):
# Create ImportRun record # Create ImportRun record
import_run = ImportRun.objects.create( import_run = ImportRun.objects.create(
profile=profile, file_name=filename, owner=request.user profile=profile,
file_name=filename,
account=serializer.validated_data.get("account"),
owner=request.user,
) )
# Queue import task # Queue import task
+103 -4
View File
@@ -1,6 +1,12 @@
from copy import deepcopy from copy import deepcopy
from rest_framework import viewsets from django.db.models import Count, Sum
from drf_spectacular.types import OpenApiTypes
from drf_spectacular.utils import OpenApiParameter, extend_schema, inline_serializer
from rest_framework import serializers, viewsets
from rest_framework.decorators import action
from rest_framework.exceptions import ValidationError
from rest_framework.response import Response
from apps.api.serializers import ( from apps.api.serializers import (
TransactionSerializer, TransactionSerializer,
@@ -21,8 +27,23 @@ from apps.transactions.models import (
from apps.rules.signals import transaction_updated, transaction_created from apps.rules.signals import transaction_updated, transaction_created
from apps.api.permissions import SHARED_OBJECT_PERMISSIONS from apps.api.permissions import SHARED_OBJECT_PERMISSIONS
# group_by -> (id field, name field)
SUMMARY_GROUPS = {
"category": ("category__id", "category__name"),
"tag": ("tags__id", "tags__name"),
"entity": ("entities__id", "entities__name"),
"account": ("account__id", "account__name"),
"month": (None, "reference_date"),
}
class TransactionViewSet(viewsets.ModelViewSet): class TransactionViewSet(viewsets.ModelViewSet):
"""Income (`IN`) and expense (`EX`) entries. `amount` is never negative;
`type` gives the direction. `amount` is required unless `sub_transactions` is
sent; then it's their sum. `reference_date` is the month the transaction
counts toward, which may differ from `date`. Transfers are an expense and
an income linked through `linked_transaction`; deleting one deletes both."""
queryset = Transaction.objects.all() queryset = Transaction.objects.all()
serializer_class = TransactionSerializer serializer_class = TransactionSerializer
filterset_fields = { filterset_fields = {
@@ -36,6 +57,8 @@ class TransactionViewSet(viewsets.ModelViewSet):
"description": ["exact", "icontains"], "description": ["exact", "icontains"],
"notes": ["exact", "icontains"], "notes": ["exact", "icontains"],
"category": ["exact", "isnull"], "category": ["exact", "isnull"],
"tags": ["exact"],
"entities": ["exact"],
"installment_plan": ["exact", "isnull"], "installment_plan": ["exact", "isnull"],
"installment_id": ["exact", "gte", "lte"], "installment_id": ["exact", "gte", "lte"],
"recurring_transaction": ["exact", "isnull"], "recurring_transaction": ["exact", "isnull"],
@@ -52,7 +75,7 @@ class TransactionViewSet(viewsets.ModelViewSet):
ordering = ["-id"] ordering = ["-id"]
def get_queryset(self): def get_queryset(self):
return Transaction.objects.all() return Transaction.objects.prefetch_related("sub_transactions")
def perform_create(self, serializer): def perform_create(self, serializer):
instance = serializer.save() instance = serializer.save()
@@ -67,8 +90,76 @@ class TransactionViewSet(viewsets.ModelViewSet):
kwargs["partial"] = True kwargs["partial"] = True
return self.update(request, *args, **kwargs) return self.update(request, *args, **kwargs)
@extend_schema(
summary="Sum transactions",
description=(
"Totals and counts of the transactions matching the same filters as "
"the list, split by currency and type, optionally grouped. Muted "
"transactions and those in muted categories are left out unless "
"`include_muted` is true. Grouping by tag or entity counts a "
"transaction once per tag/entity; untagged ones have a null group."
),
parameters=[
OpenApiParameter("group_by", enum=list(SUMMARY_GROUPS)),
OpenApiParameter("include_muted", OpenApiTypes.BOOL),
],
responses=inline_serializer(
"TransactionSummaryRow",
{
"currency": serializers.CharField(),
"type": serializers.CharField(),
"group": serializers.CharField(allow_null=True, required=False),
"group_id": serializers.IntegerField(allow_null=True, required=False),
"total": serializers.DecimalField(max_digits=42, decimal_places=30),
"count": serializers.IntegerField(),
},
many=True,
),
)
@action(detail=False, methods=["get"], pagination_class=None)
def summary(self, request):
group_by = request.query_params.get("group_by")
if group_by and group_by not in SUMMARY_GROUPS:
raise ValidationError(
{"group_by": f"Must be one of: {', '.join(SUMMARY_GROUPS)}."}
)
queryset = self.filter_queryset(self.get_queryset())
if request.query_params.get("include_muted", "").lower() != "true":
queryset = queryset.exclude(mute=True).exclude(category__mute=True)
id_field, group_field = SUMMARY_GROUPS.get(group_by, (None, None))
fields = ["account__currency__code", "type"] + [
f for f in (id_field, group_field) if f
]
rows = (
queryset.order_by() # the default ordering would split the groups
.values(*fields)
.annotate(total=Sum("amount"), count=Count("id"))
# by group first, so months come out in order
.order_by(*[f for f in (group_field,) if f], "account__currency__code", "type")
)
data = []
for row in rows:
item = {
"currency": row["account__currency__code"],
"type": row["type"],
"total": row["total"],
"count": row["count"],
}
if group_field:
group = row[group_field]
item["group"] = group.strftime("%Y-%m") if group_by == "month" and group else group
if id_field:
item["group_id"] = row[id_field]
data.append(item)
return Response(data)
class TransactionCategoryViewSet(viewsets.ModelViewSet): class TransactionCategoryViewSet(viewsets.ModelViewSet):
"""Categories, one per transaction. Transactions in a muted category are left out of totals. Sending a category name that doesn't exist on a transaction creates it."""
permission_classes = SHARED_OBJECT_PERMISSIONS permission_classes = SHARED_OBJECT_PERMISSIONS
queryset = TransactionCategory.objects.all() queryset = TransactionCategory.objects.all()
serializer_class = TransactionCategorySerializer serializer_class = TransactionCategorySerializer
@@ -87,6 +178,8 @@ class TransactionCategoryViewSet(viewsets.ModelViewSet):
class TransactionTagViewSet(viewsets.ModelViewSet): class TransactionTagViewSet(viewsets.ModelViewSet):
"""Tags; a transaction can have many. Sending a tag name that doesn't exist on a transaction creates it."""
permission_classes = SHARED_OBJECT_PERMISSIONS permission_classes = SHARED_OBJECT_PERMISSIONS
queryset = TransactionTag.objects.all() queryset = TransactionTag.objects.all()
serializer_class = TransactionTagSerializer serializer_class = TransactionTagSerializer
@@ -104,6 +197,8 @@ class TransactionTagViewSet(viewsets.ModelViewSet):
class TransactionEntityViewSet(viewsets.ModelViewSet): class TransactionEntityViewSet(viewsets.ModelViewSet):
"""Entities are the people or businesses on the other side of a transaction. Sending an entity name that doesn't exist on a transaction creates it."""
permission_classes = SHARED_OBJECT_PERMISSIONS permission_classes = SHARED_OBJECT_PERMISSIONS
queryset = TransactionEntity.objects.all() queryset = TransactionEntity.objects.all()
serializer_class = TransactionEntitySerializer serializer_class = TransactionEntitySerializer
@@ -121,6 +216,8 @@ class TransactionEntityViewSet(viewsets.ModelViewSet):
class InstallmentPlanViewSet(viewsets.ModelViewSet): class InstallmentPlanViewSet(viewsets.ModelViewSet):
"""A purchase split into a fixed number of installments. Saving a plan creates or updates one transaction per installment. `installment_amount` is required unless `sub_transactions` is sent; then it's their sum."""
queryset = InstallmentPlan.objects.all() queryset = InstallmentPlan.objects.all()
serializer_class = InstallmentPlanSerializer serializer_class = InstallmentPlanSerializer
filterset_fields = { filterset_fields = {
@@ -145,10 +242,12 @@ class InstallmentPlanViewSet(viewsets.ModelViewSet):
ordering = ["-id"] ordering = ["-id"]
def get_queryset(self): def get_queryset(self):
return InstallmentPlan.objects.all() return InstallmentPlan.objects.prefetch_related("sub_transactions")
class RecurringTransactionViewSet(viewsets.ModelViewSet): class RecurringTransactionViewSet(viewsets.ModelViewSet):
"""A transaction or transfer that repeats on a schedule. Saving one generates its upcoming transactions and updates the unpaid ones. `amount` is required unless `sub_transactions` is sent; then it's their sum."""
queryset = RecurringTransaction.objects.all() queryset = RecurringTransaction.objects.all()
serializer_class = RecurringTransactionSerializer serializer_class = RecurringTransactionSerializer
filterset_fields = { filterset_fields = {
@@ -175,4 +274,4 @@ class RecurringTransactionViewSet(viewsets.ModelViewSet):
ordering = ["-id"] ordering = ["-id"]
def get_queryset(self): def get_queryset(self):
return RecurringTransaction.objects.all() return RecurringTransaction.objects.prefetch_related("sub_transactions")
+5 -1
View File
@@ -7,6 +7,7 @@ from decimal import Decimal
from unittest.mock import MagicMock, patch from unittest.mock import MagicMock, patch
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
from django.contrib.messages import get_messages
from django.core.exceptions import ValidationError from django.core.exceptions import ValidationError
from django.core.files.uploadedfile import SimpleUploadedFile from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import TestCase from django.test import TestCase
@@ -879,7 +880,10 @@ class ExportRestoreTests(TestCase):
response = self.client.post( response = self.client.post(
reverse("restore_form"), {"zip_file": upload}, HTTP_HX_REQUEST="true" reverse("restore_form"), {"zip_file": upload}, HTTP_HX_REQUEST="true"
) )
self.assertEqual(response.status_code, 204) self.assertIn(
"Data restored successfully",
[m.message for m in get_messages(response.wsgi_request)],
)
service = BankSyncService._base_manager.get() service = BankSyncService._base_manager.get()
self.assertEqual(service.owner, self.other) self.assertEqual(service.owner, self.other)
+1 -1
View File
@@ -66,7 +66,7 @@ def calendar_list(request, month: int, year: int):
@require_http_methods(["GET"]) @require_http_methods(["GET"])
def calendar_transactions_list(request, day: int, month: int, year: int): def calendar_transactions_list(request, day: int, month: int, year: int):
date = datetime.date(year=year, month=month, day=day) date = datetime.date(year=year, month=month, day=day)
transactions = Transaction.objects.filter(date=date).collapse_transfers().order_by( transactions = Transaction.objects.filter(date=date).collapse_transfers().prefetch_related("sub_transactions").order_by(
"date", "date",
"-type", "-type",
"-is_paid", "-is_paid",
+4 -2
View File
@@ -40,9 +40,11 @@ class SharedObjectForm(forms.Form):
label=_("Visibility"), label=_("Visibility"),
widget=TomSelect(clear_button=False), widget=TomSelect(clear_button=False),
help_text=_( help_text=_(
"Private: Only shown for the owner and shared users. Only editable by the owner." "Private: Only shown for the owner and shared users."
"<br/>" "<br/>"
"Public: Shown for all users. Only editable by the owner." "Public: Shown for all users."
"<br/>"
"Only the owner can edit, share or delete it."
), ),
) )
@@ -118,6 +118,12 @@ class Command(BaseCommand):
out.save() out.save()
return out, inc return out, inc
def split(self, owner, *parts):
"""Breaks `owner` into named parts; call before it generates transactions."""
for name, amount in parts:
owner.sub_transactions.create(name=name, amount=Decimal(amount))
return owner
def rate(self, series, months): def rate(self, series, months):
return series[months + HISTORY_MONTHS] return series[months + HISTORY_MONTHS]
@@ -278,6 +284,7 @@ class Command(BaseCommand):
notes="12 interest-free installments on the credit card.", notes="12 interest-free installments on the credit card.",
) )
laptop.entities.set([self.ents["BuyMore"]]) laptop.entities.set([self.ents["BuyMore"]])
self.split(laptop, ("Laptop", "99.90"), ("Extended warranty", "8.35"))
laptop.create_transactions() laptop.create_transactions()
dental = InstallmentPlan.objects.create( dental = InstallmentPlan.objects.create(
account=checking, type=EX, description="Dental treatment", account=checking, type=EX, description="Dental treatment",
@@ -289,7 +296,8 @@ class Command(BaseCommand):
# Recurring transactions # Recurring transactions
def recurring(account, amount, description, category, start, entity=None, def recurring(account, amount, description, category, start, entity=None,
recurrence=RecurringTransaction.RecurrenceType.MONTH, paused=False): recurrence=RecurringTransaction.RecurrenceType.MONTH, paused=False,
parts=()):
r = RecurringTransaction.objects.create( r = RecurringTransaction.objects.create(
account=account, type=EX, amount=amount, description=description, account=account, type=EX, amount=amount, description=description,
category=self.cats[category], start_date=start, recurrence_type=recurrence, category=self.cats[category], start_date=start, recurrence_type=recurrence,
@@ -297,6 +305,7 @@ class Command(BaseCommand):
) )
if entity: if entity:
r.entities.set([self.ents[entity]]) r.entities.set([self.ents[entity]])
self.split(r, *parts)
if not paused: if not paused:
r.create_upcoming_transactions() r.create_upcoming_transactions()
return r return r
@@ -304,7 +313,8 @@ class Command(BaseCommand):
recurring(card, Decimal("15.99"), "StreamFlix", "Subscriptions", recurring(card, Decimal("15.99"), "StreamFlix", "Subscriptions",
self.day(-HISTORY_MONTHS, 3), "StreamFlix") self.day(-HISTORY_MONTHS, 3), "StreamFlix")
recurring(checking, Decimal("59.99"), "Internet", "Utilities", recurring(checking, Decimal("59.99"), "Internet", "Utilities",
self.day(-HISTORY_MONTHS, 15), "FiberNet") self.day(-HISTORY_MONTHS, 15), "FiberNet",
parts=[("Internet plan", "49.99"), ("Router rental", "10.00")])
recurring(checking, Decimal("39.90"), "Gym membership", "Health", recurring(checking, Decimal("39.90"), "Gym membership", "Health",
self.day(-HISTORY_MONTHS, 2), "Iron Gym") self.day(-HISTORY_MONTHS, 2), "Iron Gym")
recurring(card, Decimal("14.99"), "Domain renewal", "Subscriptions", recurring(card, Decimal("14.99"), "Domain renewal", "Subscriptions",
@@ -458,8 +468,9 @@ class Command(BaseCommand):
self.transfer(stock_acc, checking, Decimal("3"), self.day(-2, 21), "Sell AAPL", self.transfer(stock_acc, checking, Decimal("3"), self.day(-2, 21), "Sell AAPL",
to_amount=q(3 * self.rate(aapl_usd, -2)), mute=False, to_amount=q(3 * self.rate(aapl_usd, -2)), mute=False,
category="Investments") category="Investments")
self.tx(retirement, EX, Decimal("45.00"), self.day(-6, 30), "Annual management fee", fee = self.tx(retirement, EX, Decimal("45.00"), self.day(-6, 30),
"Fees") "Annual management fee", "Fees")
self.split(fee, ("Management fee", "40.00"), ("Custody fee", "5.00"))
for m, spent in ((-9, 1500), (-3, 800)): for m, spent in ((-9, 1500), (-3, 800)):
self.transfer(checking, gold_acc, Decimal(spent), self.day(m, 12), "Buy gold", self.transfer(checking, gold_acc, Decimal(spent), self.day(m, 12), "Buy gold",
to_amount=q(Decimal(spent) / self.rate(gold_usd, m)), mute=False, to_amount=q(Decimal(spent) / self.rate(gold_usd, m)), mute=False,
@@ -489,6 +500,11 @@ class Command(BaseCommand):
notes="Very old transactions are no problem.") notes="Very old transactions are no problem.")
self.transfer(old_bank, checking, Decimal("1200.00"), self.day(-HISTORY_MONTHS, 3), self.transfer(old_bank, checking, Decimal("1200.00"), self.day(-HISTORY_MONTHS, 3),
"Closing old bank account", category="Transfers") "Closing old bank account", category="Transfers")
# One receipt, split into what was bought
shop = self.tx(card, EX, Decimal("87.35"), self.day(-1, 24), "Big grocery shop",
"Groceries", ["essentials", "home"], ["Fresh Market"])
self.split(shop, ("Produce", "24.80"), ("Meat & fish", "38.15"),
("Household supplies", "16.40"), ("Snacks", "8.00"))
# Balance reconciliation: the wallet had less cash than recorded # Balance reconciliation: the wallet had less cash than recorded
self.tx(cash, EX, Decimal("12.40"), self.day(-2, 28), "Balance reconciliation") self.tx(cash, EX, Decimal("12.40"), self.day(-2, 28), "Balance reconciliation")
@@ -511,6 +527,7 @@ class Command(BaseCommand):
description="Lunch", category=self.cats["Dining"], description="Lunch", category=self.cats["Dining"],
) )
lunch.tags.set([self.tags["work"]]) lunch.tags.set([self.tags["work"]])
self.split(lunch, ("Meal", "11.90"), ("Tip", "2.00"))
# Rules run on transactions you create or edit # Rules run on transactions you create or edit
ride = TransactionRule.objects.create( ride = TransactionRule.objects.create(
+18 -1
View File
@@ -6,7 +6,7 @@ from secrets import token_urlsafe
from django.conf import settings from django.conf import settings
from django.core.exceptions import NON_FIELD_ERRORS, ValidationError from django.core.exceptions import NON_FIELD_ERRORS, ValidationError
from django.http import JsonResponse from django.http import Http404, JsonResponse
from django.views.decorators.csrf import csrf_exempt from django.views.decorators.csrf import csrf_exempt
from django.views.decorators.http import require_http_methods from django.views.decorators.http import require_http_methods
from oauth2_provider.models import get_application_model from oauth2_provider.models import get_application_model
@@ -116,6 +116,23 @@ def authorization_server_metadata(request):
return JsonResponse(metadata) return JsonResponse(metadata)
@require_http_methods(["GET"])
def protected_resource_metadata(request):
"""OAuth Protected Resource Metadata (RFC 9728) for the MCP endpoint."""
if not settings.MCP_ENABLED:
raise Http404
base_url = _base_url(request)
return JsonResponse(
{
"resource": f"{base_url}/mcp",
"resource_name": "WYGIWYH",
"authorization_servers": [base_url],
"scopes_supported": sorted(settings.OAUTH2_PROVIDER["SCOPES"].keys()),
"bearer_methods_supported": ["header"],
}
)
@csrf_exempt @csrf_exempt
@require_http_methods(["POST"]) @require_http_methods(["POST"])
def dynamic_client_registration(request): def dynamic_client_registration(request):
+9 -1
View File
@@ -2,12 +2,20 @@ import functools
import inspect import inspect
import procrastinate import procrastinate
from django.db import close_old_connections from django.db import close_old_connections as _close_old_connections
from django.db import connection
_CONNECTION_CLEANUP_WRAPPED = "_wygiwyh_connection_cleanup_wrapped" _CONNECTION_CLEANUP_WRAPPED = "_wygiwyh_connection_cleanup_wrapped"
def close_old_connections():
# Rule dry runs call tasks directly inside a transaction they roll back;
# closing its connection there would break it. Worker jobs never run in one.
if not connection.in_atomic_block:
_close_old_connections()
def _wrap_task_with_django_connection_cleanup(task): def _wrap_task_with_django_connection_cleanup(task):
if getattr(task.func, _CONNECTION_CLEANUP_WRAPPED, False): if getattr(task.func, _CONNECTION_CLEANUP_WRAPPED, False):
return return
+21 -1
View File
@@ -10,7 +10,12 @@ from apps.accounts.models import Account
from apps.common.middleware.thread_local import delete_current_user, write_current_user from apps.common.middleware.thread_local import delete_current_user, write_current_user
from apps.dca.models import DCAStrategy from apps.dca.models import DCAStrategy
from apps.investments.services import get_holding_performance from apps.investments.services import get_holding_performance
from apps.transactions.models import Transaction from apps.transactions.models import (
InstallmentPlan,
QuickTransaction,
RecurringTransaction,
Transaction,
)
@override_settings( @override_settings(
@@ -73,6 +78,21 @@ class LoadDemoDataTests(TestCase):
self.assertNotEqual(apple.realized, 0) # partial sale self.assertNotEqual(apple.realized, 0) # partial sale
self.assertTrue(perfs["Retirement Fund"].results) # gains and losses self.assertTrue(perfs["Retirement Fund"].results) # gains and losses
def test_sub_transactions_add_up(self):
self.load(date(2031, 7, 14))
owners = [
*Transaction.all_objects.filter(sub_transactions__isnull=False).distinct(),
*RecurringTransaction.all_objects.filter(sub_transactions__isnull=False).distinct(),
*QuickTransaction.objects.filter(sub_transactions__isnull=False).distinct(),
]
plans = InstallmentPlan.all_objects.filter(sub_transactions__isnull=False).distinct()
self.assertTrue(plans)
self.assertGreater(len(owners), len(plans))
for owner in [*owners, *plans]:
with self.subTest(owner=str(owner)):
amount = getattr(owner, "installment_amount", None) or owner.amount
self.assertEqual(sum(s.amount for s in owner.sub_transactions.all()), amount)
def test_running_twice_is_a_no_op(self): def test_running_twice_is_a_no_op(self):
self.load(date(2031, 7, 14)) self.load(date(2031, 7, 14))
count = Transaction.all_objects.count() count = Transaction.all_objects.count()
+16 -1
View File
@@ -1,7 +1,7 @@
from unittest.mock import patch from unittest.mock import patch
import procrastinate import procrastinate
from django.db import connection from django.db import connection, transaction
from django.test import SimpleTestCase, TransactionTestCase from django.test import SimpleTestCase, TransactionTestCase
from procrastinate.testing import InMemoryConnector from procrastinate.testing import InMemoryConnector
@@ -87,3 +87,18 @@ class ProcrastinateConnectionRecoveryTests(TransactionTestCase):
connection.connection.close() connection.connection.close()
self.assertEqual(task.func(), 1) self.assertEqual(task.func(), 1)
def test_task_called_inside_a_transaction_keeps_its_connection(self):
"""Rule dry runs call tasks directly inside a transaction they roll back."""
def sample_task():
with connection.cursor() as cursor:
cursor.execute("SELECT 1")
return cursor.fetchone()[0]
app, task = make_app_with_task(sample_task)
on_app_ready(app)
with transaction.atomic():
connection.close_at = 0 # obsolete, as with CONN_MAX_AGE=0
self.assertEqual(task.func(), 1)
+6
View File
@@ -1,4 +1,5 @@
import json import json
from urllib.parse import urlencode
from django.forms import SelectMultiple, widgets from django.forms import SelectMultiple, widgets
from django.urls import reverse from django.urls import reverse
@@ -141,6 +142,9 @@ class TransactionSelect(TomSelect):
self.load_income = income self.load_income = income
self.load_expense = expense self.load_expense = expense
self.create = False self.create = False
# Extra search filters sent as query params, e.g. {"currency": 1};
# keys must be in SEARCH_FILTERS of the transactions_search view
self.filters = {}
def build_attrs(self, base_attrs, extra_attrs=None): def build_attrs(self, base_attrs, extra_attrs=None):
attrs = super().build_attrs(base_attrs, extra_attrs) attrs = super().build_attrs(base_attrs, extra_attrs)
@@ -156,6 +160,8 @@ class TransactionSelect(TomSelect):
attrs["data-load"] = reverse( attrs["data-load"] = reverse(
"transactions_search", kwargs={"filter_type": "expenses"} "transactions_search", kwargs={"filter_type": "expenses"}
) )
if self.filters:
attrs["data-load"] += "?" + urlencode(self.filters)
return attrs return attrs
+2
View File
@@ -29,6 +29,7 @@ class CurrencyForm(forms.ModelForm):
"suffix", "suffix",
"code", "code",
"exchange_currency", "exchange_currency",
"show_exchanged_amounts",
"is_archived", "is_archived",
] ]
widgets = { widgets = {
@@ -49,6 +50,7 @@ class CurrencyForm(forms.ModelForm):
"prefix", "prefix",
"suffix", "suffix",
"exchange_currency", "exchange_currency",
Switch("show_exchanged_amounts"),
) )
if self.instance and self.instance.pk: if self.instance and self.instance.pk:
@@ -0,0 +1,22 @@
# Generated by Django 5.2.17 on 2026-11-11 09:53
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("currencies", "0027_exchangerateservice_retention"),
]
operations = [
migrations.AddField(
model_name="currency",
name="show_exchanged_amounts",
field=models.BooleanField(
default=True,
help_text="Show amounts converted to the exchange currency next to transactions and totals in this currency. Accounts with their own exchange currency use their own setting, and consolidated totals are calculated either way.",
verbose_name="Show exchanged amounts",
),
),
]
+9
View File
@@ -31,6 +31,15 @@ class Currency(models.Model):
blank=True, blank=True,
help_text=_("Default currency for exchange calculations"), help_text=_("Default currency for exchange calculations"),
) )
show_exchanged_amounts = models.BooleanField(
default=True,
verbose_name=_("Show exchanged amounts"),
help_text=_(
"Show amounts converted to the exchange currency next to transactions and "
"totals in this currency. Accounts with their own exchange currency use "
"their own setting, and consolidated totals are calculated either way."
),
)
is_archived = models.BooleanField( is_archived = models.BooleanField(
default=False, default=False,
+13
View File
@@ -0,0 +1,13 @@
from django.db.models import Case, F, When
def shown_exchange_currency(path):
"""Query expression for the exchange currency id at ``path`` (an account or a
currency lookup, such as ``"account__currency__"``), or NULL when that object
hides its exchanged amounts."""
return Case(
When(
**{f"{path}show_exchanged_amounts": True},
then=F(f"{path}exchange_currency"),
)
)
+72 -25
View File
@@ -9,7 +9,12 @@ from apps.common.widgets.datepicker import AirDatePickerInput
from apps.common.widgets.decimal import ArbitraryDecimalDisplayNumberInput from apps.common.widgets.decimal import ArbitraryDecimalDisplayNumberInput
from apps.common.widgets.tom_select import TomSelect, TransactionSelect from apps.common.widgets.tom_select import TomSelect, TransactionSelect
from apps.dca.models import DCAEntry, DCAStrategy from apps.dca.models import DCAEntry, DCAStrategy
from apps.transactions.models import Transaction, TransactionCategory, TransactionTag from apps.transactions.models import (
Transaction,
TransactionCategory,
TransactionEntity,
TransactionTag,
)
from crispy_forms.bootstrap import AccordionGroup, FormActions, Accordion from crispy_forms.bootstrap import AccordionGroup, FormActions, Accordion
from crispy_forms.helper import FormHelper from crispy_forms.helper import FormHelper
from crispy_forms.layout import HTML, Column, Div, Layout, Row from crispy_forms.layout import HTML, Column, Div, Layout, Row
@@ -104,6 +109,23 @@ class DCAEntryForm(forms.ModelForm):
queryset=TransactionTag.objects.filter(active=True), queryset=TransactionTag.objects.filter(active=True),
) )
from_entities = DynamicModelMultipleChoiceField(
model=TransactionEntity,
to_field_name="name",
create_field="name",
required=False,
label=_("Entities"),
queryset=TransactionEntity.objects.filter(active=True),
)
to_entities = DynamicModelMultipleChoiceField(
model=TransactionEntity,
to_field_name="name",
create_field="name",
required=False,
label=_("Entities"),
queryset=TransactionEntity.objects.filter(active=True),
)
expense_transaction = DynamicModelChoiceField( expense_transaction = DynamicModelChoiceField(
model=Transaction, model=Transaction,
to_field_name="id", to_field_name="id",
@@ -163,6 +185,7 @@ class DCAEntryForm(forms.ModelForm):
Column("from_category"), Column("from_category"),
Column("from_tags"), Column("from_tags"),
), ),
"from_entities",
css_class="bg-base-100 rounded-box p-4 border-base-content/60 border my-3", css_class="bg-base-100 rounded-box p-4 border-base-content/60 border my-3",
), ),
Div( Div(
@@ -171,6 +194,7 @@ class DCAEntryForm(forms.ModelForm):
Column("to_category"), Column("to_category"),
Column("to_tags"), Column("to_tags"),
), ),
"to_entities",
css_class="bg-base-100 rounded-box p-4 border-base-content/60 border", css_class="bg-base-100 rounded-box p-4 border-base-content/60 border",
), ),
active=False, active=False,
@@ -203,36 +227,30 @@ class DCAEntryForm(forms.ModelForm):
self.fields["amount_received"].widget = ArbitraryDecimalDisplayNumberInput() self.fields["amount_received"].widget = ArbitraryDecimalDisplayNumberInput()
self.fields["date"].widget = AirDatePickerInput(clear_button=False) self.fields["date"].widget = AirDatePickerInput(clear_button=False)
expense_transaction = None # Amounts can come from the linked transactions instead
income_transaction = None self.fields["amount_paid"].required = False
if self.instance and self.instance.pk: self.fields["amount_received"].required = False
# Edit mode - get from instance
expense_transaction = self.instance.expense_transaction
income_transaction = self.instance.income_transaction
elif self.data.get("expense_transaction"):
# Form validation - get from submitted data
try:
expense_transaction = Transaction.objects.get(
id=self.data["expense_transaction"]
)
income_transaction = Transaction.objects.get(
id=self.data["income_transaction"]
)
except Transaction.DoesNotExist:
pass
# If we have a current transaction, ensure it's in the queryset # The picker only searches the currency each side expects (enforced in clean)
if income_transaction: for name, currency in (
self.fields["income_transaction"].queryset = Transaction.objects.filter( ("expense_transaction", self.strategy.payment_currency),
id=income_transaction.id ("income_transaction", self.strategy.target_currency),
):
field = self.fields[name]
field.widget.filters = {"currency": currency.id}
selected = (
self.data.get(name)
if self.is_bound
else getattr(self.instance, f"{name}_id", None)
) )
if expense_transaction: # Only the selected one: the picker loads the rest by search
self.fields["expense_transaction"].queryset = Transaction.objects.filter( field.queryset = Transaction.objects.filter(
id=expense_transaction.id id=selected if str(selected or "").isdigit() else None
) )
self.fields["from_account"].queryset = Account.objects.filter( self.fields["from_account"].queryset = Account.objects.filter(
is_archived=False, is_archived=False,
currency=self.strategy.payment_currency,
) )
self.fields["from_category"].queryset = TransactionCategory.objects.filter( self.fields["from_category"].queryset = TransactionCategory.objects.filter(
@@ -242,16 +260,43 @@ class DCAEntryForm(forms.ModelForm):
self.fields["to_account"].queryset = Account.objects.filter( self.fields["to_account"].queryset = Account.objects.filter(
is_archived=False, is_archived=False,
currency=self.strategy.target_currency,
) )
self.fields["to_category"].queryset = TransactionCategory.objects.filter( self.fields["to_category"].queryset = TransactionCategory.objects.filter(
active=True active=True
) )
self.fields["to_tags"].queryset = TransactionTag.objects.filter(active=True) self.fields["to_tags"].queryset = TransactionTag.objects.filter(active=True)
self.fields["from_entities"].queryset = TransactionEntity.objects.filter(
active=True
)
self.fields["to_entities"].queryset = TransactionEntity.objects.filter(
active=True
)
def clean(self): def clean(self):
cleaned_data = super().clean() cleaned_data = super().clean()
creating = cleaned_data.get("create_transaction")
for amount_field, link_field, currency in (
("amount_paid", "expense_transaction", self.strategy.payment_currency),
("amount_received", "income_transaction", self.strategy.target_currency),
):
linked = cleaned_data.get(link_field)
if linked and linked.account.currency_id != currency.id:
self.add_error(
link_field,
_("Transaction must be in %(currency)s.")
% {"currency": currency.name},
)
linked = cleaned_data[link_field] = None
if cleaned_data.get(amount_field) is not None:
continue
if linked and not creating:
cleaned_data[amount_field] = linked.amount
elif amount_field not in self.errors:
self.add_error(amount_field, _("This field is required."))
if cleaned_data.get("create_transaction"): if cleaned_data.get("create_transaction"):
from_account = cleaned_data.get("from_account") from_account = cleaned_data.get("from_account")
to_account = cleaned_data.get("to_account") to_account = cleaned_data.get("to_account")
@@ -315,6 +360,7 @@ class DCAEntryForm(forms.ModelForm):
notes=notes, notes=notes,
) )
from_transaction.tags.set(self.cleaned_data.get("from_tags", [])) from_transaction.tags.set(self.cleaned_data.get("from_tags", []))
from_transaction.entities.set(self.cleaned_data.get("from_entities", []))
# Create "To" transaction # Create "To" transaction
to_transaction = Transaction.objects.create( to_transaction = Transaction.objects.create(
@@ -328,6 +374,7 @@ class DCAEntryForm(forms.ModelForm):
notes=notes, notes=notes,
) )
to_transaction.tags.set(self.cleaned_data.get("to_tags", [])) to_transaction.tags.set(self.cleaned_data.get("to_tags", []))
to_transaction.entities.set(self.cleaned_data.get("to_entities", []))
from_transaction.link_transfer(to_transaction) from_transaction.link_transfer(to_transaction)
+68 -1
View File
@@ -10,7 +10,7 @@ from apps.common.middleware.thread_local import delete_current_user, write_curre
from apps.currencies.models import Currency from apps.currencies.models import Currency
from apps.dca.forms import DCAEntryForm from apps.dca.forms import DCAEntryForm
from apps.dca.models import DCAStrategy from apps.dca.models import DCAStrategy
from apps.transactions.models import Transaction from apps.transactions.models import Transaction, TransactionEntity
IN, EX = Transaction.Type.INCOME, Transaction.Type.EXPENSE IN, EX = Transaction.Type.INCOME, Transaction.Type.EXPENSE
@@ -97,3 +97,70 @@ class DCATransferLinkingTests(TestCase):
expense.link_transfer(income) expense.link_transfer(income)
expense.link_transfer(income) # already linked to each other: no-op expense.link_transfer(income) # already linked to each other: no-op
self.assertEqual(Transaction.objects.get(pk=income.pk).linked_transaction, expense) self.assertEqual(Transaction.objects.get(pk=income.pk).linked_transaction, expense)
def test_amounts_come_from_linked_transactions(self):
expense, income = self.tx(self.cash, EX, "500"), self.tx(self.wallet, IN, "0.001")
form = DCAEntryForm(
data=self.entry_data(
amount_paid="",
amount_received="",
expense_transaction=str(expense.id),
income_transaction=str(income.id),
),
strategy=self.strategy,
)
self.assertTrue(form.is_valid(), form.errors)
entry = form.save()
self.assertEqual(entry.amount_paid, Decimal("500"))
self.assertEqual(entry.amount_received, Decimal("0.001"))
def test_amount_required_without_link(self):
form = DCAEntryForm(
data=self.entry_data(amount_received=""), strategy=self.strategy
)
self.assertFalse(form.is_valid())
self.assertIn("amount_received", form.errors)
def test_link_in_wrong_currency_is_rejected(self):
# Expense must be in the payment currency, income in the target currency
expense, income = self.tx(self.wallet, EX, "1"), self.tx(self.cash, IN, "500")
form = DCAEntryForm(
data=self.entry_data(
expense_transaction=str(expense.id), income_transaction=str(income.id)
),
strategy=self.strategy,
)
self.assertFalse(form.is_valid())
self.assertIn("expense_transaction", form.errors)
self.assertIn("income_transaction", form.errors)
def test_created_transactions_get_entities(self):
broker = TransactionEntity.objects.create(name="Broker")
form = DCAEntryForm(
data=self.entry_data(
create_transaction="on",
from_account=self.cash.id,
to_account=self.wallet.id,
from_entities=["Broker"],
to_entities=["Broker"],
),
strategy=self.strategy,
)
self.assertTrue(form.is_valid(), form.errors)
entry = form.save()
self.assertEqual(list(entry.expense_transaction.entities.all()), [broker])
self.assertEqual(list(entry.income_transaction.entities.all()), [broker])
def test_create_accounts_must_hold_strategy_currency(self):
# Swapped: paying from the BTC wallet into the BRL cash account
form = DCAEntryForm(
data=self.entry_data(
create_transaction="on",
from_account=self.wallet.id,
to_account=self.cash.id,
),
strategy=self.strategy,
)
self.assertFalse(form.is_valid())
self.assertIn("from_account", form.errors)
self.assertIn("to_account", form.errors)
+40 -88
View File
@@ -1,132 +1,107 @@
from apps.common.widgets.crispy.submit import NoClassSubmit
from crispy_forms.bootstrap import FormActions
from crispy_forms.helper import FormHelper
from crispy_forms.layout import HTML, Layout
from django import forms from django import forms
from django.utils.translation import gettext_lazy as _ from django.utils.translation import gettext_lazy as _
class ExportForm(forms.Form): class ExportForm(forms.Form):
# Rendered by export_app/fragments/export.html, in declaration order
users = forms.BooleanField( users = forms.BooleanField(
required=False, required=False,
widget=forms.CheckboxInput(), widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Users"), label=_("Users"),
initial=True, initial=True,
) )
accounts = forms.BooleanField( accounts = forms.BooleanField(
required=False, required=False,
widget=forms.CheckboxInput(), widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Accounts"), label=_("Accounts"),
initial=True, initial=True,
) )
currencies = forms.BooleanField( currencies = forms.BooleanField(
required=False, required=False,
widget=forms.CheckboxInput(), widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Currencies"), label=_("Currencies"),
initial=True, initial=True,
) )
transactions = forms.BooleanField( transactions = forms.BooleanField(
required=False, required=False,
widget=forms.CheckboxInput(), widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Transactions"), label=_("Transactions"),
initial=True, initial=True,
) )
categories = forms.BooleanField( categories = forms.BooleanField(
required=False, required=False,
widget=forms.CheckboxInput(), widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Categories"), label=_("Categories"),
initial=True, initial=True,
) )
tags = forms.BooleanField(
required=False,
widget=forms.CheckboxInput(),
label=_("Tags"),
initial=False,
)
entities = forms.BooleanField( entities = forms.BooleanField(
required=False, required=False,
widget=forms.CheckboxInput(), widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Entities"), label=_("Entities"),
initial=False, initial=False,
) )
recurring_transactions = forms.BooleanField( tags = forms.BooleanField(
required=False, required=False,
widget=forms.CheckboxInput(), widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Recurring Transactions"), label=_("Tags"),
initial=True, initial=False,
) )
installment_plans = forms.BooleanField( installment_plans = forms.BooleanField(
required=False, required=False,
widget=forms.CheckboxInput(), widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Installment Plans"), label=_("Installment Plans"),
initial=True, initial=True,
) )
exchange_rates = forms.BooleanField( recurring_transactions = forms.BooleanField(
required=False, required=False,
widget=forms.CheckboxInput(), widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Exchange Rates"), label=_("Recurring Transactions"),
initial=False, initial=True,
)
quick_transactions = forms.BooleanField(
required=False,
widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Quick Transactions"),
initial=True,
) )
exchange_rates_services = forms.BooleanField( exchange_rates_services = forms.BooleanField(
required=False, required=False,
widget=forms.CheckboxInput(), widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Automatic Exchange Rates"), label=_("Automatic Exchange Rates"),
initial=False, initial=False,
) )
exchange_rates = forms.BooleanField(
required=False,
widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Exchange Rates"),
initial=False,
)
bank_sync = forms.BooleanField( bank_sync = forms.BooleanField(
required=False, required=False,
widget=forms.CheckboxInput(), widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Bank Sync"), label=_("Bank Sync"),
help_text=_("Includes the providers' API keys"), help_text=_("Includes the providers' API keys"),
initial=False, initial=False,
) )
rules = forms.BooleanField( rules = forms.BooleanField(
required=False, required=False,
widget=forms.CheckboxInput(), widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Rules"), label=_("Rules"),
initial=True, initial=True,
) )
dca = forms.BooleanField( dca = forms.BooleanField(
required=False, required=False,
widget=forms.CheckboxInput(), widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("DCA"), label=_("DCA"),
initial=False, initial=False,
) )
import_profiles = forms.BooleanField( import_profiles = forms.BooleanField(
required=False, required=False,
widget=forms.CheckboxInput(), widget=forms.CheckboxInput(attrs={"class": "checkbox"}),
label=_("Import Profiles"), label=_("Import Profiles"),
initial=True, initial=True,
) )
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.helper = FormHelper()
self.helper.form_tag = False
self.helper.form_method = "post"
self.helper.layout = Layout(
"users",
"accounts",
"investment_classes",
"currencies",
"transactions",
"categories",
"entities",
"tags",
"installment_plans",
"recurring_transactions",
"exchange_rates_services",
"exchange_rates",
"bank_sync",
"rules",
"dca",
"import_profiles",
FormActions(
NoClassSubmit("submit", _("Export"), css_class="btn btn-primary"),
),
)
class RestoreForm(forms.Form): class RestoreForm(forms.Form):
zip_file = forms.FileField( zip_file = forms.FileField(
@@ -142,10 +117,12 @@ class RestoreForm(forms.Form):
transactions_tags = forms.FileField(required=False, label=_("Tags")) transactions_tags = forms.FileField(required=False, label=_("Tags"))
transactions_entities = forms.FileField(required=False, label=_("Entities")) transactions_entities = forms.FileField(required=False, label=_("Entities"))
transactions = forms.FileField(required=False, label=_("Transactions")) transactions = forms.FileField(required=False, label=_("Transactions"))
sub_transactions = forms.FileField(required=False, label=_("Sub-transactions"))
installment_plans = forms.FileField(required=False, label=_("Installment Plans")) installment_plans = forms.FileField(required=False, label=_("Installment Plans"))
recurring_transactions = forms.FileField( recurring_transactions = forms.FileField(
required=False, label=_("Recurring Transactions") required=False, label=_("Recurring Transactions")
) )
quick_transactions = forms.FileField(required=False, label=_("Quick Transactions"))
automatic_exchange_rates = forms.FileField( automatic_exchange_rates = forms.FileField(
required=False, label=_("Automatic Exchange Rates") required=False, label=_("Automatic Exchange Rates")
) )
@@ -169,36 +146,11 @@ class RestoreForm(forms.Form):
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs) super().__init__(*args, **kwargs)
# Rendered by export_app/fragments/restore.html
self.helper = FormHelper() for name, field in self.fields.items():
self.helper.form_tag = False field.widget.attrs.update(
self.helper.form_method = "post" {"class": "file-input w-full", "accept": ".zip" if name == "zip_file" else ".csv"}
self.helper.layout = Layout( )
"zip_file",
HTML('<hr class="hr my-3"/>'),
"users",
"accounts",
"currencies",
"transactions",
"transactions_categories",
"transactions_entities",
"transactions_tags",
"installment_plans",
"recurring_transactions",
"automatic_exchange_rates",
"exchange_rates",
"bank_sync_services",
"bank_sync_accounts",
"transaction_rules",
"transaction_rules_actions",
"transaction_rules_update_or_create",
"dca_strategies",
"dca_entries",
"import_profiles",
FormActions(
NoClassSubmit("submit", _("Restore"), css_class="btn btn-primary"),
),
)
def clean(self): def clean(self):
cleaned_data = super().clean() cleaned_data = super().clean()
+151 -6
View File
@@ -1,4 +1,9 @@
from functools import reduce
from operator import or_
from django.db.models import Prefetch, Q
from import_export import fields, resources from import_export import fields, resources
from import_export.widgets import ForeignKeyWidget
from apps.accounts.models import Account from apps.accounts.models import Account
from apps.export_app.widgets.foreign_key import ( from apps.export_app.widgets.foreign_key import (
@@ -14,11 +19,28 @@ from apps.transactions.models import (
TransactionEntity, TransactionEntity,
RecurringTransaction, RecurringTransaction,
InstallmentPlan, InstallmentPlan,
QuickTransaction,
SubTransaction,
) )
from apps.export_app.widgets.numbers import UniversalDecimalWidget from apps.export_app.widgets.numbers import UniversalDecimalWidget
class TransactionResource(resources.ModelResource): def with_relations(queryset, *related):
"""Loads relations up front, and every owner's tags and entities, not just the exporter's."""
return queryset.select_related(*related).prefetch_related(
Prefetch("tags", queryset=TransactionTag.all_objects.all()),
Prefetch("entities", queryset=TransactionEntity.all_objects.all()),
)
class PrefetchedExportResource(resources.ModelResource):
def iter_queryset(self, queryset):
# import-export pages prefetched querysets with OFFSET, which slows down
# quadratically; Django's iterator prefetches per chunk with a cursor instead
yield from queryset.iterator(chunk_size=2000)
class TransactionResource(PrefetchedExportResource):
account = fields.Field( account = fields.Field(
attribute="account", attribute="account",
column_name="account", column_name="account",
@@ -57,7 +79,15 @@ class TransactionResource(resources.ModelResource):
model = Transaction model = Transaction
def get_queryset(self): def get_queryset(self):
return Transaction.userless_all_objects.all() return with_relations(
Transaction.userless_all_objects.all(),
"account",
"owner",
"category",
"installment_plan",
"recurring_transaction",
"linked_transaction",
)
class TransactionTagResource(resources.ModelResource): class TransactionTagResource(resources.ModelResource):
@@ -84,7 +114,7 @@ class TransactionCategoyResource(resources.ModelResource):
return TransactionCategory.all_objects.all() return TransactionCategory.all_objects.all()
class RecurringTransactionResource(resources.ModelResource): class RecurringTransactionResource(PrefetchedExportResource):
account = fields.Field( account = fields.Field(
attribute="account", attribute="account",
column_name="account", column_name="account",
@@ -115,18 +145,23 @@ class RecurringTransactionResource(resources.ModelResource):
widget=UniversalDecimalWidget(), widget=UniversalDecimalWidget(),
) )
# Lets restore find the account and its names under the right user
account_owner = fields.Field(column_name="account_owner", readonly=True)
class Meta: class Meta:
model = RecurringTransaction model = RecurringTransaction
def get_queryset(self): def get_queryset(self):
return RecurringTransaction.all_objects.all() return with_relations(
RecurringTransaction.all_objects.all(), "account", "to_account", "category"
)
def dehydrate_account_owner(self, obj): def dehydrate_account_owner(self, obj):
"""Export the account's owner ID for proper import matching.""" """Export the account's owner ID for proper import matching."""
return obj.account.owner_id if obj.account else None return obj.account.owner_id if obj.account else None
class InstallmentPlanResource(resources.ModelResource): class InstallmentPlanResource(PrefetchedExportResource):
account = fields.Field( account = fields.Field(
attribute="account", attribute="account",
column_name="account", column_name="account",
@@ -157,12 +192,122 @@ class InstallmentPlanResource(resources.ModelResource):
widget=UniversalDecimalWidget(), widget=UniversalDecimalWidget(),
) )
# Lets restore find the account and its names under the right user
account_owner = fields.Field(column_name="account_owner", readonly=True)
class Meta: class Meta:
model = InstallmentPlan model = InstallmentPlan
def get_queryset(self): def get_queryset(self):
return InstallmentPlan.all_objects.all() return with_relations(
InstallmentPlan.all_objects.all(), "account", "category"
)
def dehydrate_account_owner(self, obj): def dehydrate_account_owner(self, obj):
"""Export the account's owner ID for proper import matching.""" """Export the account's owner ID for proper import matching."""
return obj.account.owner_id if obj.account else None return obj.account.owner_id if obj.account else None
class AnyOwnerForeignKeyWidget(ForeignKeyWidget):
"""Finds the parent among every user's rows, soft-deleted ones included."""
def get_queryset(self, value, row, *args, **kwargs):
return self.model._base_manager.all()
class QuickTransactionResource(PrefetchedExportResource):
account = fields.Field(
attribute="account",
column_name="account",
widget=AllObjectsForeignKeyWidget(Account, "name"),
)
to_account = fields.Field(
attribute="to_account",
column_name="to_account",
widget=AllObjectsForeignKeyWidget(Account, "name"),
)
category = fields.Field(
attribute="category",
column_name="category",
widget=AutoCreateForeignKeyWidget(TransactionCategory, "name"),
)
tags = fields.Field(
attribute="tags",
column_name="tags",
widget=AutoCreateManyToManyWidget(TransactionTag, field="name"),
)
entities = fields.Field(
attribute="entities",
column_name="entities",
widget=AutoCreateManyToManyWidget(TransactionEntity, field="name"),
)
internal_id = EmptyStringToNoneField(
column_name="internal_id", attribute="internal_id"
)
amount = fields.Field(
attribute="amount",
column_name="amount",
widget=UniversalDecimalWidget(),
)
to_amount = fields.Field(
attribute="to_amount",
column_name="to_amount",
widget=UniversalDecimalWidget(),
)
class Meta:
model = QuickTransaction
def get_queryset(self):
return with_relations(
QuickTransaction.all_objects.all(), "account", "to_account", "category", "owner"
)
SUB_TRANSACTION_BACKUP_PARENTS = (
"transaction",
"installment_plan",
"recurring_transaction",
"quick_transaction",
)
class SubTransactionResource(resources.ModelResource):
transaction = fields.Field(
attribute="transaction",
column_name="transaction",
widget=AnyOwnerForeignKeyWidget(Transaction),
)
installment_plan = fields.Field(
attribute="installment_plan",
column_name="installment_plan",
widget=AnyOwnerForeignKeyWidget(InstallmentPlan),
)
recurring_transaction = fields.Field(
attribute="recurring_transaction",
column_name="recurring_transaction",
widget=AnyOwnerForeignKeyWidget(RecurringTransaction),
)
quick_transaction = fields.Field(
attribute="quick_transaction",
column_name="quick_transaction",
widget=AnyOwnerForeignKeyWidget(QuickTransaction),
)
amount = fields.Field(
attribute="amount",
column_name="amount",
widget=UniversalDecimalWidget(),
)
def __init__(self, parents=SUB_TRANSACTION_BACKUP_PARENTS, **kwargs):
"""`parents`: the kinds of parent being exported, whose parts go along."""
super().__init__(**kwargs)
self.parents = parents
class Meta:
model = SubTransaction
def get_queryset(self):
return SubTransaction.objects.filter(
reduce(or_, (Q(**{f"{p}__isnull": False}) for p in self.parents), Q(pk__in=[]))
)
+196 -2
View File
@@ -1,17 +1,39 @@
import io import io
from datetime import date
import zipfile import zipfile
from decimal import Decimal from decimal import Decimal
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
from django.db import connection
from django.contrib.messages import get_messages
from django.test import TestCase from django.test import TestCase
from django.urls import reverse
from django.test.utils import CaptureQueriesContext
from apps.accounts.models import Account, InvestmentClass from apps.accounts.models import Account, InvestmentClass
from apps.common.middleware.thread_local import delete_current_user, write_current_user from apps.common.middleware.thread_local import delete_current_user, write_current_user
from apps.currencies.models import Currency from apps.currencies.models import Currency
from apps.export_app.resources.accounts import AccountResource, InvestmentClassResource from apps.export_app.resources.accounts import AccountResource, InvestmentClassResource
from apps.export_app.resources.transactions import TransactionEntityResource from apps.export_app.resources.transactions import (
InstallmentPlanResource,
RecurringTransactionResource,
TransactionCategoyResource,
TransactionTagResource,
QuickTransactionResource,
SubTransactionResource,
TransactionEntityResource,
TransactionResource,
)
from apps.export_app.views import process_imports from apps.export_app.views import process_imports
from apps.transactions.models import TransactionEntity from apps.transactions.models import (
InstallmentPlan,
QuickTransaction,
SubTransaction,
Transaction,
TransactionCategory,
TransactionEntity,
TransactionTag,
)
class InvestmentRestoreTests(TestCase): class InvestmentRestoreTests(TestCase):
@@ -51,3 +73,175 @@ class InvestmentRestoreTests(TestCase):
self.assertEqual(account.investment_class.name, "Stocks") self.assertEqual(account.investment_class.name, "Stocks")
self.assertEqual(account.investment_class.target, Decimal("60")) self.assertEqual(account.investment_class.target, Decimal("60"))
self.assertEqual([e.name for e in account.income_entities.all()], ["Broker"]) self.assertEqual([e.name for e in account.income_entities.all()], ["Broker"])
class SubTransactionBackupTests(TestCase):
def setUp(self):
self.user = get_user_model().objects.create_user(email="s@test.com", password="pw")
write_current_user(self.user)
self.addCleanup(delete_current_user)
account = Account.objects.create(
name="A", currency=Currency.objects.create(code="USD", name="USD", decimal_places=2)
)
self.transaction = Transaction.objects.create(
account=account, type="EX", date=date(2030, 1, 10), amount=5
)
self.plan = InstallmentPlan.objects.create(
account=account, type="EX", description="Laptop", number_of_installments=2,
start_date=date(2030, 1, 10), installment_amount=100,
)
SubTransaction.objects.create(transaction=self.transaction, name="Fee", amount="1.25")
SubTransaction.objects.create(transaction=self.transaction, name="Item", amount="3.75")
SubTransaction.objects.create(installment_plan=self.plan, name="Principal", amount=100)
lunch = QuickTransaction.objects.create(
name="Lunch", account=account, type="EX", amount=Decimal("13.90")
)
SubTransaction.objects.create(quick_transaction=lunch, name="Meal", amount="11.90")
SubTransaction.objects.create(quick_transaction=lunch, name="Tip", amount=2)
def _parts(self):
return sorted(
SubTransaction.objects.values_list(
"name", "transaction", "installment_plan", "quick_transaction__name", "amount"
)
)
def test_round_trip_puts_parts_back_on_their_parents(self):
before = self._parts()
backup = io.BytesIO()
with zipfile.ZipFile(backup, "w") as z:
for resource, name in (
(TransactionResource(), "transactions"),
(InstallmentPlanResource(), "installment_plans"),
(QuickTransactionResource(), "quick_transactions"),
(SubTransactionResource(), "sub_transactions"),
):
z.writestr(f"{name}.csv", resource.export().csv)
SubTransaction.objects.all().delete()
QuickTransaction.all_objects.all().delete()
backup.seek(0)
process_imports(None, {"zip_file": backup})
self.assertEqual(self._parts(), before)
def test_exports_only_the_parts_of_exported_parents(self):
dataset = SubTransactionResource(parents=["installment_plan"]).export()
self.assertEqual([row["name"] for row in dataset.dict], ["Principal"])
class MultiUserRestoreTests(TestCase):
"""Export and restore run as the admin; other users' private names must survive (#691)."""
def setUp(self):
User = get_user_model()
self.admin = User.objects.create_superuser(email="admin@x.com", password="pw")
self.bob = User.objects.create_user(email="bob@x.com", password="pw")
usd = Currency.objects.create(code="USD", name="USD", decimal_places=2)
for user in (self.admin, self.bob):
write_current_user(user)
account = Account.objects.create(name="Checking", currency=usd)
food = TransactionCategory.objects.create(name="Food")
transaction = Transaction.objects.create(
account=account, type="EX", date=date(2030, 1, 1), amount=1,
category=food, description="lunch",
)
transaction.tags.add(TransactionTag.objects.create(name="groceries"))
InstallmentPlan.objects.create(
account=account, type="EX", description="TV", number_of_installments=1,
start_date=date(2030, 1, 1), installment_amount=10, category=food,
)
write_current_user(self.admin)
self.addCleanup(delete_current_user)
def _state(self):
return sorted(
(t.account.owner.email, t.category.owner.email, [g.owner.email for g in t.tags(manager="all_objects").all()])
for t in Transaction.userless_all_objects.all()
) + sorted(
(p.account.owner.email, p.category.owner.email)
for p in InstallmentPlan.all_objects.all()
)
def test_round_trip_keeps_each_users_categories_and_tags(self):
before = self._state()
backup = io.BytesIO()
with zipfile.ZipFile(backup, "w") as z:
for resource, name in (
(TransactionCategoyResource(), "transactions_categories"),
(TransactionTagResource(), "transactions_tags"),
(TransactionResource(), "transactions"),
(InstallmentPlanResource(), "installment_plans"),
):
z.writestr(f"{name}.csv", resource.export().csv)
Transaction.userless_all_objects.all().delete()
InstallmentPlan.all_objects.all().delete()
TransactionCategory.all_objects.all().delete()
TransactionTag.all_objects.all().delete()
backup.seek(0)
process_imports(None, {"zip_file": backup})
self.assertEqual(self._state(), before)
def test_missing_names_are_created_for_the_rows_owner(self):
csv = TransactionResource().export().csv
Transaction.userless_all_objects.all().delete()
InstallmentPlan.all_objects.all().delete()
TransactionCategory.all_objects.all().delete()
TransactionTag.all_objects.all().delete()
process_imports(None, {"transactions": io.BytesIO(csv.encode())})
self.assertEqual(
sorted(TransactionCategory.all_objects.values_list("owner__email", flat=True)),
["admin@x.com", "bob@x.com"],
)
self.assertEqual(
sorted(TransactionTag.all_objects.values_list("owner__email", flat=True)),
["admin@x.com", "bob@x.com"],
)
def test_export_queries_do_not_grow_with_rows(self):
# A few queries per row timed big exports out (#692)
def export_queries():
with CaptureQueriesContext(connection) as queries:
TransactionResource().export()
return len(queries)
few = export_queries()
template = Transaction.userless_all_objects.first()
for _ in range(5):
copy = Transaction.all_objects.create(
account=template.account, type="EX", date=date(2030, 1, 2), amount=1,
category=template.category, owner=template.owner,
)
copy.tags.set(template.tags(manager="all_objects").all())
self.assertEqual(export_queries(), few)
class ExportViewTests(TestCase):
def setUp(self):
self.client.force_login(
get_user_model().objects.create_superuser(email="a@test.com", password="pw")
)
def test_nothing_selected_warns_instead_of_downloading(self):
response = self.client.post(reverse("export_form"), {}, HTTP_HX_REQUEST="true")
self.assertEqual(response.status_code, 204)
self.assertEqual(
[m.level_tag for m in get_messages(response.wsgi_request)], ["warning"]
)
def test_non_superuser_is_refused_not_sent_back_to_login(self):
# The login page sends signed-in users back to ?next=, so redirecting there looped
self.client.force_login(
get_user_model().objects.create_user(email="b@test.com", password="pw")
)
self.assertEqual(self.client.get(reverse("export_index")).status_code, 403)
def test_restore_without_files_shows_the_error_in_the_form(self):
response = self.client.post(reverse("restore_form"), {}, HTTP_HX_REQUEST="true")
self.assertContains(response, "Please upload either a ZIP file or at least one CSV file")
self.assertContains(response, 'id="restore-form"')
+115 -76
View File
@@ -3,7 +3,7 @@ import zipfile
from io import BytesIO from io import BytesIO
from django.contrib import messages from django.contrib import messages
from django.contrib.auth.decorators import login_required, user_passes_test from django.contrib.auth.decorators import login_required
from django.db import transaction from django.db import transaction
from django.http import HttpResponse from django.http import HttpResponse
from django.shortcuts import render from django.shortcuts import render
@@ -46,24 +46,31 @@ from apps.export_app.resources.transactions import (
TransactionCategoyResource, TransactionCategoyResource,
InstallmentPlanResource, InstallmentPlanResource,
RecurringTransactionResource, RecurringTransactionResource,
QuickTransactionResource,
SubTransactionResource,
) )
from apps.export_app.resources.users import UserResource from apps.export_app.resources.users import UserResource
from apps.common.decorators.demo import disabled_on_demo from apps.common.decorators.demo import disabled_on_demo
from apps.common.decorators.user import is_superuser
logger = logging.getLogger() logger = logging.getLogger()
@login_required @login_required
@disabled_on_demo @disabled_on_demo
@user_passes_test(lambda u: u.is_superuser) @is_superuser
@require_http_methods(["GET"]) @require_http_methods(["GET"])
def export_index(request): def export_index(request):
return render(request, "export_app/pages/index.html") return render(
request,
"export_app/pages/index.html",
{"form": ExportForm(), "restore_form": RestoreForm()},
)
@login_required @login_required
@disabled_on_demo @disabled_on_demo
@user_passes_test(lambda u: u.is_superuser) @is_superuser
@require_http_methods(["GET", "POST"]) @require_http_methods(["GET", "POST"])
def export_form(request): def export_form(request):
timestamp = timezone.localtime(timezone.now()).strftime("%Y-%m-%dT%H-%M-%S") timestamp = timezone.localtime(timezone.now()).strftime("%Y-%m-%dT%H-%M-%S")
@@ -84,6 +91,9 @@ def export_form(request):
export_recurring_transactions = form.cleaned_data.get( export_recurring_transactions = form.cleaned_data.get(
"recurring_transactions", False "recurring_transactions", False
) )
export_quick_transactions = form.cleaned_data.get(
"quick_transactions", False
)
export_exchange_rates_services = form.cleaned_data.get( export_exchange_rates_services = form.cleaned_data.get(
"exchange_rates_services", False "exchange_rates_services", False
@@ -93,72 +103,104 @@ def export_form(request):
export_dca = form.cleaned_data.get("dca", False) export_dca = form.cleaned_data.get("dca", False)
export_import_profiles = form.cleaned_data.get("import_profiles", False) export_import_profiles = form.cleaned_data.get("import_profiles", False)
exports = [] try:
if export_users: exports = []
exports.append((UserResource().export(), "users")) if export_users:
if export_accounts: exports.append((UserResource().export(), "users"))
exports.append((AccountResource().export(), "accounts")) if export_accounts:
exports.append( exports.append((AccountResource().export(), "accounts"))
(InvestmentClassResource().export(), "investment_classes") exports.append(
) (InvestmentClassResource().export(), "investment_classes")
if export_currencies:
exports.append((CurrencyResource().export(), "currencies"))
if export_transactions:
exports.append((TransactionResource().export(), "transactions"))
if export_categories:
exports.append(
(TransactionCategoyResource().export(), "transactions_categories")
)
if export_tags:
exports.append((TransactionTagResource().export(), "transactions_tags"))
if export_entities:
exports.append(
(TransactionEntityResource().export(), "transactions_entities")
)
if export_installment_plans:
exports.append(
(InstallmentPlanResource().export(), "installment_plans")
)
if export_recurring_transactions:
exports.append(
(RecurringTransactionResource().export(), "recurring_transactions")
)
if export_exchange_rates_services:
exports.append(
(ExchangeRateServiceResource().export(), "automatic_exchange_rates")
)
if export_exchange_rates:
exports.append((ExchangeRateResource().export(), "exchange_rates"))
if form.cleaned_data.get("bank_sync"):
exports.append((BankSyncServiceResource().export(), "bank_sync_services"))
exports.append((BankSyncAccountResource().export(), "bank_sync_accounts"))
if export_rules:
exports.append(
(TransactionRuleResource().export(), "transaction_rules")
)
exports.append(
(
TransactionRuleActionResource().export(),
"transaction_rules_actions",
) )
) if export_currencies:
exports.append( exports.append((CurrencyResource().export(), "currencies"))
( if export_transactions:
UpdateOrCreateTransactionRuleResource().export(), exports.append((TransactionResource().export(), "transactions"))
"transaction_rules_update_or_create", if export_categories:
exports.append(
(TransactionCategoyResource().export(), "transactions_categories")
) )
) if export_tags:
if export_dca: exports.append((TransactionTagResource().export(), "transactions_tags"))
exports.append((DCAStrategyResource().export(), "dca_strategies")) if export_entities:
exports.append( exports.append(
( (TransactionEntityResource().export(), "transactions_entities")
DCAEntryResource().export(),
"dca_entries",
) )
) if export_installment_plans:
if export_import_profiles: exports.append(
exports.append((ImportProfileResource().export(), "import_profiles")) (InstallmentPlanResource().export(), "installment_plans")
)
if export_recurring_transactions:
exports.append(
(RecurringTransactionResource().export(), "recurring_transactions")
)
if export_quick_transactions:
exports.append(
(QuickTransactionResource().export(), "quick_transactions")
)
sub_transaction_parents = [
parent
for parent, exported in (
("transaction", export_transactions),
("installment_plan", export_installment_plans),
("recurring_transaction", export_recurring_transactions),
("quick_transaction", export_quick_transactions),
)
if exported
]
if sub_transaction_parents:
exports.append(
(
SubTransactionResource(parents=sub_transaction_parents).export(),
"sub_transactions",
)
)
if export_exchange_rates_services:
exports.append(
(ExchangeRateServiceResource().export(), "automatic_exchange_rates")
)
if export_exchange_rates:
exports.append((ExchangeRateResource().export(), "exchange_rates"))
if form.cleaned_data.get("bank_sync"):
exports.append((BankSyncServiceResource().export(), "bank_sync_services"))
exports.append((BankSyncAccountResource().export(), "bank_sync_accounts"))
if export_rules:
exports.append(
(TransactionRuleResource().export(), "transaction_rules")
)
exports.append(
(
TransactionRuleActionResource().export(),
"transaction_rules_actions",
)
)
exports.append(
(
UpdateOrCreateTransactionRuleResource().export(),
"transaction_rules_update_or_create",
)
)
if export_dca:
exports.append((DCAStrategyResource().export(), "dca_strategies"))
exports.append(
(
DCAEntryResource().export(),
"dca_entries",
)
)
if export_import_profiles:
exports.append((ImportProfileResource().export(), "import_profiles"))
except Exception as e:
logger.error("Error exporting", exc_info=e)
messages.error(
request,
_("There was an error exporting your data. Check the logs for more details."),
)
return HttpResponse(status=204, headers={"HX-Trigger": "updated"})
if exports:
messages.success(request, _("Export ready, your download has started"))
if len(exports) >= 2: if len(exports) >= 2:
with zipfile.ZipFile(zip_buffer, "w", zipfile.ZIP_DEFLATED) as zip_file: with zipfile.ZipFile(zip_buffer, "w", zipfile.ZIP_DEFLATED) as zip_file:
for dataset, name in exports: for dataset, name in exports:
@@ -186,9 +228,9 @@ def export_form(request):
) )
return response return response
else: else:
return HttpResponse( # 204, or the download extension would save this message as a file
_("You have to select at least one export"), messages.warning(request, _("You have to select at least one export"))
) return HttpResponse(status=204, headers={"HX-Trigger": "updated"})
else: else:
form = ExportForm() form = ExportForm()
@@ -199,7 +241,7 @@ def export_form(request):
@only_htmx @only_htmx
@login_required @login_required
@disabled_on_demo @disabled_on_demo
@user_passes_test(lambda u: u.is_superuser) @is_superuser
@require_http_methods(["GET", "POST"]) @require_http_methods(["GET", "POST"])
def import_form(request): def import_form(request):
if request.method == "POST": if request.method == "POST":
@@ -208,12 +250,7 @@ def import_form(request):
try: try:
process_imports(request, form.cleaned_data) process_imports(request, form.cleaned_data)
messages.success(request, _("Data restored successfully")) messages.success(request, _("Data restored successfully"))
return HttpResponse( form = RestoreForm() # a fresh form clears the chosen files and old errors
status=204,
headers={
"HX-Trigger": "hide_offcanvas, updated",
},
)
except Exception as e: except Exception as e:
logger.error("Error importing", exc_info=e) logger.error("Error importing", exc_info=e)
messages.error( messages.error(
@@ -250,7 +287,9 @@ def process_imports(request, cleaned_data):
("exchange_rates", ExchangeRateResource), ("exchange_rates", ExchangeRateResource),
("installment_plans", InstallmentPlanResource), ("installment_plans", InstallmentPlanResource),
("recurring_transactions", RecurringTransactionResource), ("recurring_transactions", RecurringTransactionResource),
("quick_transactions", QuickTransactionResource),
("transactions", TransactionResource), ("transactions", TransactionResource),
("sub_transactions", SubTransactionResource),
("bank_sync_services", BankSyncServiceResource), ("bank_sync_services", BankSyncServiceResource),
("bank_sync_accounts", BankSyncAccountResource), ("bank_sync_accounts", BankSyncAccountResource),
("dca_strategies", DCAStrategyResource), ("dca_strategies", DCAStrategyResource),
+60 -44
View File
@@ -1,6 +1,63 @@
from django.db.models import Q
from import_export.widgets import ForeignKeyWidget from import_export.widgets import ForeignKeyWidget
def row_owner_id(row):
"""The id of the user who owns the record being restored from ``row``."""
if not row:
return None
# Check for direct owner field first
owner_id = row.get("owner") if "owner" in row else None
# Fall back to account_owner for models like InstallmentPlan
if not owner_id and "account_owner" in row:
owner_id = row.get("account_owner")
# If still no owner, try to get it from the existing record's account
# This handles backward compatibility with older exports
if not owner_id and "id" in row and row.get("id"):
try:
from apps.transactions.models import (
InstallmentPlan,
RecurringTransaction,
)
record_id = row.get("id")
for model_class in [InstallmentPlan, RecurringTransaction]:
try:
existing = model_class.all_objects.get(id=record_id)
if existing.account:
owner_id = existing.account.owner_id
break
except model_class.DoesNotExist:
continue
except Exception:
pass
# Final fallback: use the current logged-in user
# This handles restoring to a fresh database with older exports
if not owner_id:
from apps.common.middleware.thread_local import get_current_user
user = get_current_user()
if user and user.is_authenticated:
owner_id = user.id
return owner_id or None
def get_or_create_for_owner(model, name, owner_id):
"""
The owner's own object named ``name``, else one they can see (public or
shared with them), else a new one they own. Restore runs as the admin, so
the user-filtered ``objects`` manager would pick the admin's objects.
"""
named = model.all_objects.filter(name=name)
return (
named.filter(owner_id=owner_id).first()
or named.filter(
Q(visibility="public") | Q(shared_with=owner_id) | Q(owner=None)
).first()
or model.all_objects.create(name=name, owner_id=owner_id)
)
class AllObjectsForeignKeyWidget(ForeignKeyWidget): class AllObjectsForeignKeyWidget(ForeignKeyWidget):
""" """
ForeignKeyWidget that uses 'all_objects' manager for lookups, ForeignKeyWidget that uses 'all_objects' manager for lookups,
@@ -9,48 +66,10 @@ class AllObjectsForeignKeyWidget(ForeignKeyWidget):
""" """
def get_queryset(self, value, row, *args, **kwargs): def get_queryset(self, value, row, *args, **kwargs):
# Use all_objects manager if available, otherwise fall back to default
if hasattr(self.model, "all_objects"): if hasattr(self.model, "all_objects"):
qs = self.model.all_objects.all() qs = self.model.all_objects.all()
# Filter by owner if the row has an owner field and the model has owner if owner_id := row_owner_id(row):
if row: qs = qs.filter(owner_id=owner_id)
# Check for direct owner field first
owner_id = row.get("owner") if "owner" in row else None
# Fall back to account_owner for models like InstallmentPlan
if not owner_id and "account_owner" in row:
owner_id = row.get("account_owner")
# If still no owner, try to get it from the existing record's account
# This handles backward compatibility with older exports
if not owner_id and "id" in row and row.get("id"):
try:
# Try to find the existing record and get owner from its account
from apps.transactions.models import (
InstallmentPlan,
RecurringTransaction,
)
record_id = row.get("id")
# Try to find the existing InstallmentPlan or RecurringTransaction
for model_class in [InstallmentPlan, RecurringTransaction]:
try:
existing = model_class.all_objects.get(id=record_id)
if existing.account:
owner_id = existing.account.owner_id
break
except model_class.DoesNotExist:
continue
except Exception:
pass
# Final fallback: use the current logged-in user
# This handles restoring to a fresh database with older exports
if not owner_id:
from apps.common.middleware.thread_local import get_current_user
user = get_current_user()
if user and user.is_authenticated:
owner_id = user.id
if owner_id:
qs = qs.filter(owner_id=owner_id)
return qs return qs
return super().get_queryset(value, row, *args, **kwargs) return super().get_queryset(value, row, *args, **kwargs)
@@ -58,10 +77,7 @@ class AllObjectsForeignKeyWidget(ForeignKeyWidget):
class AutoCreateForeignKeyWidget(ForeignKeyWidget): class AutoCreateForeignKeyWidget(ForeignKeyWidget):
def clean(self, value, row=None, *args, **kwargs): def clean(self, value, row=None, *args, **kwargs):
if value: if value:
try: return get_or_create_for_owner(self.model, value, row_owner_id(row))
return super().clean(value, row, **kwargs)
except self.model.DoesNotExist:
return self.model.objects.create(name=value)
return None return None
+8 -13
View File
@@ -1,21 +1,16 @@
from import_export.widgets import ManyToManyWidget from import_export.widgets import ManyToManyWidget
from apps.export_app.widgets.foreign_key import get_or_create_for_owner, row_owner_id
class AutoCreateManyToManyWidget(ManyToManyWidget): class AutoCreateManyToManyWidget(ManyToManyWidget):
def clean(self, value, row=None, *args, **kwargs): def clean(self, value, row=None, *args, **kwargs):
if not value: if not value:
return [] return []
values = value.split(self.separator) owner_id = row_owner_id(row)
cleaned_values = [] return [
get_or_create_for_owner(self.model, val.strip(), owner_id)
for val in values: for val in value.split(self.separator)
val = val.strip() if val.strip()
if val: ]
try:
obj = self.model.objects.get(**{self.field: val})
except self.model.DoesNotExist:
obj = self.model.objects.create(name=val)
cleaned_values.append(obj)
return cleaned_values
+23 -3
View File
@@ -1,4 +1,6 @@
from apps.accounts.models import Account
from apps.common.widgets.crispy.submit import NoClassSubmit from apps.common.widgets.crispy.submit import NoClassSubmit
from apps.common.widgets.tom_select import TomSelect
from apps.import_app.models import ImportProfile from apps.import_app.models import ImportProfile
from crispy_forms.bootstrap import FormActions from crispy_forms.bootstrap import FormActions
from crispy_forms.helper import FormHelper from crispy_forms.helper import FormHelper
@@ -42,15 +44,33 @@ class ImportProfileForm(forms.ModelForm):
class ImportRunFileUploadForm(forms.Form): class ImportRunFileUploadForm(forms.Form):
file = forms.FileField(label=_("Select a file")) file = forms.FileField(label=_("Select a file"))
account = forms.ModelChoiceField(
queryset=Account.objects.filter(is_archived=False),
required=False,
label=_("Account"),
help_text=_(
"Import every transaction into this account. Leave empty to let the "
"profile pick the account."
),
widget=TomSelect(group_by="group"),
)
def __init__(self, *args, **kwargs): def __init__(self, *args, profile, **kwargs):
super().__init__(*args, **kwargs) super().__init__(*args, **kwargs)
self.helper = FormHelper() self.helper = FormHelper()
self.helper.form_tag = False self.helper.form_tag = False
self.helper.form_method = "post" self.helper.form_method = "post"
self.helper.layout = Layout( self.helper.layout = Layout("file")
"file",
if profile.imports_transactions:
self.fields["account"].queryset = Account.objects.filter(is_archived=False)
self.helper.layout.append("account")
else:
# Only transactions belong to an account
del self.fields["account"]
self.helper.layout.append(
FormActions( FormActions(
NoClassSubmit("submit", _("Import"), css_class="btn btn-primary"), NoClassSubmit("submit", _("Import"), css_class="btn btn-primary"),
), ),
@@ -0,0 +1,20 @@
# Generated by Django 5.2.17 on 2026-11-11 17:04
import django.db.models.deletion
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('accounts', '0022_account_show_exchanged_amounts'),
('import_app', '0003_owner_and_sharing'),
]
operations = [
migrations.AddField(
model_name='importrun',
name='account',
field=models.ForeignKey(blank=True, help_text='Import every transaction into this account. Leave empty to let the profile pick the account.', null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='import_runs', to='accounts.account', verbose_name='Account'),
),
]
+21
View File
@@ -38,6 +38,15 @@ class ImportProfile(SharedObject):
version_number = self.Versions(self.version).name.split("_")[1] version_number = self.Versions(self.version).name.split("_")[1]
return _("Version {number}").format(number=version_number) return _("Version {number}").format(number=version_number)
@property
def imports_transactions(self):
"""Whether this profile creates transactions, as opposed to accounts, tags, etc."""
try:
config = version_1.ImportProfileSchema(**yaml.safe_load(self.yaml_config))
except Exception:
return False
return config.settings.importing == "transactions"
def clean(self): def clean(self):
if self.version and self.version == self.Versions.VERSION_1: if self.version and self.version == self.Versions.VERSION_1:
try: try:
@@ -70,6 +79,18 @@ class ImportRun(OwnedObject):
max_length=10000, max_length=10000,
help_text=_("File name"), help_text=_("File name"),
) )
account = models.ForeignKey(
"accounts.Account",
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name="import_runs",
verbose_name=_("Account"),
help_text=_(
"Import every transaction into this account. Leave empty to let the "
"profile pick the account."
),
)
transactions = models.ManyToManyField( transactions = models.ManyToManyField(
"transactions.Transaction", related_name="import_runs" "transactions.Transaction", related_name="import_runs"
) )
+15
View File
@@ -75,6 +75,20 @@ class SubtractTransformationRule(BaseModel):
) )
class LineTransformationRule(BaseModel):
type: Literal["line"]
line: int = Field(
...,
ge=1,
description="Line of the file to read, counted from 1 at the top of the file, including skipped lines",
)
column: int | None = Field(
default=None,
ge=1,
description="Column of that line to read, counted from 1. Empty to read the whole line (Excel cells are joined with commas).",
)
class CSVImportSettings(BaseModel): class CSVImportSettings(BaseModel):
skip_errors: bool = Field( skip_errors: bool = Field(
default=False, default=False,
@@ -133,6 +147,7 @@ class ColumnMapping(BaseModel):
| SplitTransformationRule | SplitTransformationRule
| AddTransformationRule | AddTransformationRule
| SubtractTransformationRule | SubtractTransformationRule
| LineTransformationRule
] ]
] = Field(default_factory=list) ] = Field(default_factory=list)
+150 -12
View File
@@ -4,7 +4,9 @@ import logging
import os import os
import re import re
import zipfile import zipfile
from collections import Counter
from django.db import transaction from django.db import transaction
from django.db.models import Q
from datetime import datetime, date from datetime import datetime, date
from decimal import Decimal, InvalidOperation from decimal import Decimal, InvalidOperation
from pathlib import Path from pathlib import Path
@@ -39,6 +41,17 @@ from apps.import_app.schemas.v1 import (
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
# QIF sections that hold transactions. Others, like category lists (!Type:Cat)
# or memorized transactions (!Type:Memorized), are skipped
QIF_TRANSACTION_SECTIONS = {
"!type:bank",
"!type:cash",
"!type:ccard",
"!type:invst",
"!type:oth a",
"!type:oth l",
}
class ImportService: class ImportService:
def __init__(self, import_run: ImportRun): def __init__(self, import_run: ImportRun):
@@ -53,6 +66,15 @@ class ImportService:
) )
self.mapping: Dict[str, version_1.ColumnMapping] = self.config.mapping self.mapping: Dict[str, version_1.ColumnMapping] = self.config.mapping
# Lines read by `line` transformations, captured as (text, cells) while the file is read
self._wanted_lines = {
transform.line
for mapping in self.mapping.values()
for transform in mapping.transformations or []
if transform.type == "line"
}
self._file_lines: Dict[int, tuple[str, list[str]]] = {}
# Ensure temp directory exists # Ensure temp directory exists
os.makedirs(settings.IMPORT_TEMP_ROOT, exist_ok=True) os.makedirs(settings.IMPORT_TEMP_ROOT, exist_ok=True)
@@ -189,6 +211,15 @@ class ImportService:
transform.pattern, transform.replacement, transformed transform.pattern, transform.replacement, transformed
) )
elif transform.type == "line":
text, cells = self._file_lines.get(transform.line, ("", []))
if transform.column is None:
transformed = text
elif transform.column <= len(cells):
transformed = cells[transform.column - 1]
else:
transformed = ""
elif transform.type == "date_format": elif transform.type == "date_format":
transformed = datetime.strptime( transformed = datetime.strptime(
transformed, transform.original_format transformed, transform.original_format
@@ -289,7 +320,8 @@ class ImportService:
# Ignore if category doesn't exist and create is False or not set # Ignore if category doesn't exist and create is False or not set
data["category"] = None data["category"] = None
if "account" in data: # Look the account up, unless the run already picked one
if "account" in data and not isinstance(data["account"], Account):
account_id = data.pop("account") account_id = data.pop("account")
account_mapping = next( account_mapping = next(
( (
@@ -710,6 +742,10 @@ class ImportService:
if mapped_data: if mapped_data:
# Handle different import types # Handle different import types
if self.settings.importing == "transactions": if self.settings.importing == "transactions":
if self.import_run.account:
# The run's account wins over the profile's account mapping
mapped_data["account"] = self.import_run.account
if self.deduplication and self._check_duplicate_transaction( if self.deduplication and self._check_duplicate_transaction(
mapped_data mapped_data
): ):
@@ -744,14 +780,31 @@ class ImportService:
logger.error(f"Fatal error processing row {row_number}", exc_info=e) logger.error(f"Fatal error processing row {row_number}", exc_info=e)
def _capture_csv_lines(self, csv_file):
"""Pass the file's lines through, keeping the ones `line` transformations read."""
for number, line in enumerate(csv_file, start=1):
if number in self._wanted_lines:
text = line.rstrip("\r\n")
cells = next(csv.reader([text], delimiter=self.settings.delimiter), [])
self._file_lines[number] = (text, cells)
yield line
def _capture_excel_lines(self, read_row):
"""Keep the sheet rows `line` transformations read. read_row(n) returns row n's values."""
self._file_lines = {}
for number in self._wanted_lines:
cells = ["" if value is None else str(value) for value in read_row(number)]
self._file_lines[number] = (",".join(cells), cells)
def _process_csv(self, file_path): def _process_csv(self, file_path):
# First pass: count rows # First pass: count rows, and capture the lines `line` transformations read
with open(file_path, "r", encoding=self.settings.encoding) as csv_file: with open(file_path, "r", encoding=self.settings.encoding) as csv_file:
lines = self._capture_csv_lines(csv_file)
# Skip specified number of rows # Skip specified number of rows
for _ in range(self.settings.skip_lines): for _ in range(self.settings.skip_lines):
next(csv_file) next(lines)
reader = csv.DictReader(csv_file, delimiter=self.settings.delimiter) reader = csv.DictReader(lines, delimiter=self.settings.delimiter)
self._update_totals("total", value=sum(1 for _ in reader)) self._update_totals("total", value=sum(1 for _ in reader))
with open(file_path, "r", encoding=self.settings.encoding) as csv_file: with open(file_path, "r", encoding=self.settings.encoding) as csv_file:
@@ -803,6 +856,14 @@ class ImportService:
sheet = workbook[sheet_name] sheet = workbook[sheet_name]
self._log("info", f"Processing sheet: {sheet_name}") self._log("info", f"Processing sheet: {sheet_name}")
self._capture_excel_lines(
lambda number: next(
sheet.iter_rows(
min_row=number, max_row=number, values_only=True
),
(),
)
)
headers = [ headers = [
str(cell.value or "") for cell in sheet[self.settings.start_row] str(cell.value or "") for cell in sheet[self.settings.start_row]
] ]
@@ -863,6 +924,13 @@ class ImportService:
continue continue
sheet = workbook.sheet_by_name(sheet_name) sheet = workbook.sheet_by_name(sheet_name)
self._log("info", f"Processing sheet: {sheet_name}") self._log("info", f"Processing sheet: {sheet_name}")
self._capture_excel_lines(
lambda number: (
sheet.row_values(number - 1)
if number <= sheet.nrows
else []
)
)
headers = [ headers = [
str(sheet.cell_value(self.settings.start_row - 1, col) or "") str(sheet.cell_value(self.settings.start_row - 1, col) or "")
for col in range(sheet.ncols) for col in range(sheet.ncols)
@@ -913,14 +981,36 @@ class ImportService:
f"Invalid {self.settings.file_type.upper()} file format: {str(e)}" f"Invalid {self.settings.file_type.upper()} file format: {str(e)}"
) )
@staticmethod
def _qif_internal_id(raw_content: str, account: Account, occurrence: int) -> str:
"""
Builds the internal ID of a QIF transaction from its raw lines.
The account is part of the hash so the same file can be imported into
different accounts, and the occurrence (how many identical transactions
came before it in the file) tells identical transactions apart, e.g. two
purchases with the same date, amount and payee.
"""
return hashlib.sha256(
f"{account.pk}:{occurrence}:{raw_content}".encode("utf-8")
).hexdigest()
def _parse_and_import_qif(self, content_lines: list[str], filename: str) -> None: def _parse_and_import_qif(self, content_lines: list[str], filename: str) -> None:
# Infer account from filename (remove extension) # Infer account from filename (remove extension)
account_name = os.path.splitext(os.path.basename(filename))[0] account_name = os.path.splitext(os.path.basename(filename))[0]
current_transaction = {} current_transaction = {}
raw_lines_buffer = [] raw_lines_buffer = []
# How many times each transaction's raw lines have been seen in this file
occurrences = Counter()
legacy_contents_seen = set()
in_account_list = False
in_account_block = False
in_other_section = False
account = Account.objects.filter(name=account_name).first() account = (
self.import_run.account or Account.objects.filter(name=account_name).first()
)
if not account: if not account:
raise ValueError(f"Account '{account_name}' not found.") raise ValueError(f"Account '{account_name}' not found.")
@@ -931,23 +1021,66 @@ class ImportService:
if not line: if not line:
continue continue
# Account blocks (!Account, or an !Option:AutoSwitch account list)
# and non-transaction sections use codes that mean something else
# in a transaction, e.g. T is an account's type and D its description
header = line.lower()
if header == "!option:autoswitch":
in_account_list = True
elif header == "!clear:autoswitch":
in_account_list = False
elif header == "!account":
in_account_block = True
elif header.startswith("!type:"):
in_other_section = header not in QIF_TRANSACTION_SECTIONS
if in_other_section:
self._log("info", f"Skipped {line} section in {filename}")
if in_account_list or in_account_block or in_other_section:
if line == "^":
in_account_block = False
raw_lines_buffer = []
continue
raw_lines_buffer.append(line) raw_lines_buffer.append(line)
if line == "^": if line == "^":
if current_transaction: if current_transaction:
# Deduplication using hash of raw lines # Deduplication using hash of raw lines. Header lines (!Type,
raw_content = "".join(raw_lines_buffer) # !Account) are left out so a transaction hashes the same
internal_id = hashlib.sha256( # whether or not it comes right after one
raw_content.encode("utf-8") legacy_content = "".join(raw_lines_buffer)
).hexdigest() raw_content = "".join(
raw_line
for raw_line in raw_lines_buffer
if not raw_line.startswith("!")
)
occurrence = occurrences[raw_content]
occurrences[raw_content] += 1
internal_id = self._qif_internal_id(
raw_content, account, occurrence
)
# Reset buffer for next transaction # Reset buffer for next transaction
raw_lines_buffer = [] raw_lines_buffer = []
duplicate_filter = Q(internal_id=internal_id)
if legacy_content not in legacy_contents_seen:
# Transactions imported before the account and the
# occurrence were part of the hash used the raw lines
# alone, headers included, and only ever kept the first
# of identical ones
legacy_contents_seen.add(legacy_content)
duplicate_filter |= Q(
internal_id=hashlib.sha256(
legacy_content.encode("utf-8")
).hexdigest(),
account=account,
)
try: try:
with transaction.atomic(): with transaction.atomic():
if Transaction.objects.filter( if Transaction.all_objects.filter(
internal_id=internal_id duplicate_filter
).exists(): ).exists():
self._increment_totals("skipped", 1) self._increment_totals("skipped", 1)
self._log( self._log(
@@ -1149,6 +1282,11 @@ class ImportService:
self.import_run.save(update_fields=["started_at"]) self.import_run.save(update_fields=["started_at"])
self._log("info", "Starting import process") self._log("info", "Starting import process")
if self.import_run.account:
self._log(
"info",
f"Importing transactions into account '{self.import_run.account.name}'",
)
try: try:
if isinstance(self.settings, version_1.CSVImportSettings): if isinstance(self.settings, version_1.CSVImportSettings):
@@ -0,0 +1,167 @@
"""Tests for the `line` transformation, which reads a line outside the table, like a header cell."""
import shutil
import tempfile
from pathlib import Path
import openpyxl
from django.contrib.auth import get_user_model
from django.test import TestCase, override_settings
from apps.accounts.models import Account
from apps.common.middleware.thread_local import delete_current_user, write_current_user
from apps.currencies.models import Currency
from apps.import_app.models import ImportProfile, ImportRun
from apps.import_app.services.v1 import ImportService
from apps.transactions.models import Transaction
MAPPING = """
mapping:
date:
target: date
source: date
format: "%Y-%m-%d"
amount:
target: amount
source: amount
type:
target: type
source: amount
detection_method: sign
description:
target: description
source: description
account:
target: account
transformations:
{account_transformations}
"""
# The account name sits in the header, above the table
CSV_CONTENT = """Bank statement
Account:;Savings
Account number:;Savings 1234
date;amount;description
2026-01-05;-50.00;Groceries
2026-01-06;1000.00;Salary
"""
class LineTransformationTests(TestCase):
def setUp(self):
self.temp_root = Path(tempfile.mkdtemp()).resolve()
self.addCleanup(shutil.rmtree, self.temp_root, ignore_errors=True)
temp_root_override = override_settings(IMPORT_TEMP_ROOT=self.temp_root)
temp_root_override.enable()
self.addCleanup(temp_root_override.disable)
self.user = get_user_model().objects.create_user(
email="test@example.com", password="password"
)
write_current_user(self.user)
self.addCleanup(delete_current_user)
currency = Currency.objects.create(
code="EUR", name="Euro", decimal_places=2, prefix="€ "
)
Account.objects.create(name="Checking", currency=currency, owner=self.user)
self.savings = Account.objects.create(
name="Savings", currency=currency, owner=self.user
)
def _import(self, settings, account_transformations, file_name, write):
yaml_config = settings + MAPPING.format(
account_transformations=account_transformations
)
profile = ImportProfile.objects.create(
name="Profile", yaml_config=yaml_config, owner=self.user
)
run = ImportRun.objects.create(
profile=profile, file_name=file_name, owner=self.user
)
file_path = self.temp_root / file_name
write(file_path)
ImportService(run).process_file(str(file_path))
run.refresh_from_db()
return run
def _import_csv(self, account_transformations):
settings = """
settings:
file_type: csv
importing: transactions
delimiter: ";"
skip_lines: 4
trigger_transaction_rules: false
"""
return self._import(
settings,
account_transformations,
"statement.csv",
lambda path: path.write_text(CSV_CONTENT, encoding="utf-8"),
)
def assert_imported_into_savings(self, run):
self.assertEqual(run.status, ImportRun.Status.FINISHED, run.logs)
self.assertEqual(run.successful_rows, 2)
self.assertEqual(Transaction.objects.filter(account=self.savings).count(), 2)
def test_csv_reads_a_cell_from_a_skipped_line(self):
run = self._import_csv(
"""
- type: line
line: 2
column: 2
"""
)
self.assert_imported_into_savings(run)
def test_csv_whole_line_cleaned_with_regex(self):
run = self._import_csv(
"""
- type: line
line: 3
- type: regex
pattern: '^Account number:;(\\w+) \\d+$'
replacement: '\\1'
"""
)
self.assert_imported_into_savings(run)
def test_xlsx_reads_a_cell_above_the_header(self):
def write(path):
workbook = openpyxl.Workbook()
sheet = workbook.active
sheet["A1"] = "Bank statement"
sheet["A4"] = "Account:"
sheet["B4"] = "Savings"
sheet.append([])
sheet.append(["date", "amount", "description"])
sheet.append(["2026-01-05", "-50.00", "Groceries"])
sheet.append(["2026-01-06", "1000.00", "Salary"])
workbook.save(path)
settings = """
settings:
file_type: xlsx
importing: transactions
start_row: 6
trigger_transaction_rules: false
"""
run = self._import(
settings,
"""
- type: line
line: 4
column: 2
""",
"statement.xlsx",
write,
)
self.assert_imported_into_savings(run)
@@ -1,4 +1,6 @@
from datetime import date
from decimal import Decimal from decimal import Decimal
import hashlib
import os import os
import shutil import shutil
from django.test import TestCase, override_settings from django.test import TestCase, override_settings
@@ -12,6 +14,30 @@ from apps.transactions.models import (
Transaction, Transaction,
) )
QIF_YAML_CONFIG = """
settings:
file_type: qif
importing: transactions
date_format: "%d/%m/%Y"
mapping: {}
"""
# A salary, then two purchases with the same date, amount and payee
IDENTICAL_TRANSACTIONS_CONTENT = """!Type:Bank
D04/01/2015
T100.00
PSalary
^
D05/01/2015
T-4.50
PCoffee Shop
^
D05/01/2015
T-4.50
PCoffee Shop
^
"""
class QIFImportTests(TestCase): class QIFImportTests(TestCase):
def setUp(self): def setUp(self):
@@ -45,6 +71,23 @@ class QIFImportTests(TestCase):
if os.path.exists(self.test_dir): if os.path.exists(self.test_dir):
shutil.rmtree(self.test_dir) shutil.rmtree(self.test_dir)
def _import_qif(self, content, account=None, filename="bradesco-checking.qif"):
file_path = os.path.join(self.test_dir, filename)
with open(file_path, "w", encoding="utf-8") as f:
f.write(content)
profile = ImportProfile.objects.create(
name="QIF Profile",
yaml_config=QIF_YAML_CONFIG,
version=ImportProfile.Versions.VERSION_1,
)
run = ImportRun.objects.create(
profile=profile, file_name=filename, account=account
)
ImportService(run).process_file(file_path)
run.refresh_from_db()
return run
def test_import_single_qif_valid_mapping(self): def test_import_single_qif_valid_mapping(self):
content = """!Type:Bank content = """!Type:Bank
D04/01/2015 D04/01/2015
@@ -257,3 +300,210 @@ mapping: {}
self.assertEqual( self.assertEqual(
Transaction.objects.first().description, "" Transaction.objects.first().description, ""
) # empty desc if no memo ) # empty desc if no memo
def test_import_same_file_into_two_accounts(self):
savings = Account.objects.create(
name="bradesco-savings",
group=self.group,
currency=self.currency,
owner=self.user,
)
into_checking = self._import_qif(
IDENTICAL_TRANSACTIONS_CONTENT, account=self.account
)
into_savings = self._import_qif(IDENTICAL_TRANSACTIONS_CONTENT, account=savings)
into_savings_again = self._import_qif(
IDENTICAL_TRANSACTIONS_CONTENT, account=savings
)
self.assertEqual(into_checking.successful_rows, 3)
self.assertEqual(into_savings.successful_rows, 3)
self.assertEqual(into_savings.skipped_rows, 0)
self.assertEqual(into_savings_again.successful_rows, 0)
self.assertEqual(into_savings_again.skipped_rows, 3)
self.assertEqual(Transaction.objects.filter(account=self.account).count(), 3)
self.assertEqual(Transaction.objects.filter(account=savings).count(), 3)
def test_import_identical_transactions_in_one_file(self):
first_run = self._import_qif(IDENTICAL_TRANSACTIONS_CONTENT)
self.assertEqual(first_run.successful_rows, 3)
self.assertEqual(first_run.skipped_rows, 0)
coffees = Transaction.objects.filter(entities__name="Coffee Shop")
self.assertEqual(coffees.count(), 2)
self.assertEqual(coffees.values("internal_id").distinct().count(), 2)
# Re-importing the file still recognizes both of them
second_run = self._import_qif(IDENTICAL_TRANSACTIONS_CONTENT)
self.assertEqual(second_run.successful_rows, 0)
self.assertEqual(second_run.skipped_rows, 3)
self.assertEqual(Transaction.objects.count(), 3)
def test_import_skips_transactions_imported_with_the_previous_hash(self):
# Imports made before the account was part of the hash used the raw
# lines alone, and skipped every repeat of identical transactions
legacy_internal_id = hashlib.sha256(
"D05/01/2015T-4.50PCoffee Shop^".encode("utf-8")
).hexdigest()
Transaction.objects.create(
account=self.account,
type=Transaction.Type.EXPENSE,
date=date(2015, 1, 5),
amount=Decimal("4.50"),
internal_id=legacy_internal_id,
)
savings = Account.objects.create(
name="bradesco-savings",
group=self.group,
currency=self.currency,
owner=self.user,
)
into_checking = self._import_qif(IDENTICAL_TRANSACTIONS_CONTENT)
into_savings = self._import_qif(IDENTICAL_TRANSACTIONS_CONTENT, account=savings)
# The first coffee is the one imported before, the second one was
# skipped back then and is imported now
self.assertEqual(into_checking.successful_rows, 2)
self.assertEqual(into_checking.skipped_rows, 1)
self.assertEqual(Transaction.objects.filter(account=self.account).count(), 3)
# The previous hash doesn't say which account it was for, so it only
# counts as a duplicate in the account the transaction is in
self.assertEqual(into_savings.successful_rows, 3)
self.assertEqual(into_savings.skipped_rows, 0)
self.assertEqual(Transaction.objects.filter(account=savings).count(), 3)
def test_import_overlapping_exports_with_a_different_first_transaction(self):
# The first coffee opens the second export, right after the header
later_export = """!Type:Bank
D05/01/2015
T-4.50
PCoffee Shop
^
D06/01/2015
T-20.00
PMarket
^
"""
self._import_qif(IDENTICAL_TRANSACTIONS_CONTENT)
run = self._import_qif(later_export)
self.assertEqual(run.successful_rows, 1)
self.assertEqual(run.skipped_rows, 1)
self.assertEqual(Transaction.objects.count(), 4)
def test_import_skips_previous_hash_of_a_first_transaction_repeated_later(self):
# The previous hash included the header in the first transaction, so a
# first transaction and an identical later one were both imported
content = """!Type:Bank
D05/01/2015
T-4.50
PCoffee Shop
^
D05/01/2015
T-4.50
PCoffee Shop
^
"""
for raw_content in (
"!Type:BankD05/01/2015T-4.50PCoffee Shop^",
"D05/01/2015T-4.50PCoffee Shop^",
):
Transaction.objects.create(
account=self.account,
type=Transaction.Type.EXPENSE,
date=date(2015, 1, 5),
amount=Decimal("4.50"),
internal_id=hashlib.sha256(raw_content.encode("utf-8")).hexdigest(),
)
run = self._import_qif(content)
self.assertEqual(run.successful_rows, 0)
self.assertEqual(run.skipped_rows, 2)
self.assertEqual(Transaction.objects.count(), 2)
def test_import_skips_account_blocks(self):
# Quicken-style export: an account list, then an account block before
# the transactions. T and D mean the account type and description there
content = """!Option:AutoSwitch
!Account
NChecking
TBank
DMain checking account
^
NSavings
TBank
^
!Clear:AutoSwitch
!Account
NChecking
TBank
DMain checking account
^
!Type:Bank
D04/01/2015
T100.00
PSalary
^
"""
run = self._import_qif(content)
self.assertEqual(run.status, ImportRun.Status.FINISHED)
self.assertEqual(run.successful_rows, 1)
self.assertEqual(run.failed_rows, 0)
transaction = Transaction.objects.get()
self.assertEqual(transaction.amount, Decimal("100.00"))
self.assertEqual(transaction.date, date(2015, 1, 4))
def test_import_skips_non_transaction_sections(self):
# Category and class lists, and memorized transactions (templates, not
# real transactions), around a section of real ones
content = """!Type:Cat
NFood
DGroceries and restaurants
E
^
!Type:Class
NPersonal
DPersonal expenses
^
!Type:Bank
D04/01/2015
T100.00
PSalary
^
!Type:Memorized
KC
T-50.00
PRent
^
"""
run = self._import_qif(content)
self.assertEqual(run.status, ImportRun.Status.FINISHED)
self.assertEqual(run.successful_rows, 1)
self.assertEqual(run.failed_rows, 0)
self.assertEqual(Transaction.objects.get().amount, Decimal("100.00"))
@override_settings(ENABLE_SOFT_DELETE=True)
def test_import_skips_soft_deleted_transactions(self):
content = """!Type:Bank
D04/01/2015
T100.00
POK
^
"""
self._import_qif(content)
Transaction.objects.get().delete()
self.assertTrue(Transaction.deleted_objects.exists())
run = self._import_qif(content)
self.assertEqual(run.status, ImportRun.Status.FINISHED)
self.assertEqual(run.successful_rows, 0)
self.assertEqual(run.skipped_rows, 1)
self.assertFalse(Transaction.objects.exists())
@@ -0,0 +1,380 @@
"""Tests for importing a file into an account picked for the import run."""
import shutil
import tempfile
from decimal import Decimal
from pathlib import Path
from unittest.mock import patch
from django.contrib.auth import get_user_model
from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import TestCase, override_settings
from django.urls import reverse
from rest_framework import status
from rest_framework.test import APIClient
from apps.accounts.models import Account
from apps.common.middleware.thread_local import delete_current_user, write_current_user
from apps.currencies.models import Currency
from apps.import_app.models import ImportProfile, ImportRun
from apps.import_app.services.v1 import ImportService
from apps.import_app.tests.test_view_permissions import (
HTMX,
ImportObjectPermissionTestCase,
)
from apps.transactions.models import Transaction
CSV_SETTINGS = """
settings:
file_type: csv
importing: transactions
trigger_transaction_rules: false
"""
CSV_MAPPING = """
mapping:
date:
target: date
source: date
format: "%Y-%m-%d"
amount:
target: amount
source: amount
type:
target: type
source: amount
detection_method: sign
description:
target: description
source: description
"""
CSV_ACCOUNT_MAPPING = """
account:
target: account
default: Checking
"""
CSV_ACCOUNT_DEDUPLICATION = """
deduplication:
- type: compare
fields:
- date
- amount
- account
match_type: strict
"""
CSV_CONTENT = """date,amount,description
2026-01-05,-50.00,Groceries
2026-01-06,1000.00,Salary
"""
TAGS_YAML_CONFIG = """
settings:
file_type: csv
importing: tags
mapping:
name:
target: tag_name
source: name
"""
class ImportRunAccountServiceTests(TestCase):
def setUp(self):
self.temp_root = Path(tempfile.mkdtemp()).resolve()
self.addCleanup(shutil.rmtree, self.temp_root, ignore_errors=True)
temp_root_override = override_settings(IMPORT_TEMP_ROOT=self.temp_root)
temp_root_override.enable()
self.addCleanup(temp_root_override.disable)
self.user = get_user_model().objects.create_user(
email="test@example.com", password="password"
)
write_current_user(self.user)
self.addCleanup(delete_current_user)
currency = Currency.objects.create(
code="EUR", name="Euro", decimal_places=2, prefix="€ "
)
self.checking = Account.objects.create(
name="Checking", currency=currency, owner=self.user
)
self.savings = Account.objects.create(
name="Savings", currency=currency, owner=self.user
)
def _import(self, yaml_config, file_name, content, account=None):
profile = ImportProfile.objects.create(
name="Profile", yaml_config=yaml_config, owner=self.user
)
run = ImportRun.objects.create(
profile=profile, file_name=file_name, account=account, owner=self.user
)
file_path = self.temp_root / file_name
file_path.write_text(content, encoding="utf-8")
ImportService(run).process_file(str(file_path))
run.refresh_from_db()
return run
def test_run_account_overrides_the_profile_account(self):
run = self._import(
CSV_SETTINGS + CSV_MAPPING + CSV_ACCOUNT_MAPPING,
"statement.csv",
CSV_CONTENT,
account=self.savings,
)
self.assertEqual(run.status, ImportRun.Status.FINISHED)
self.assertEqual(run.successful_rows, 2)
self.assertEqual(Transaction.objects.filter(account=self.savings).count(), 2)
self.assertFalse(Transaction.objects.filter(account=self.checking).exists())
self.assertIn("Importing transactions into account 'Savings'", run.logs)
def test_profile_account_is_used_without_a_run_account(self):
run = self._import(
CSV_SETTINGS + CSV_MAPPING + CSV_ACCOUNT_MAPPING,
"statement.csv",
CSV_CONTENT,
)
self.assertEqual(run.successful_rows, 2)
self.assertEqual(Transaction.objects.filter(account=self.checking).count(), 2)
def test_run_account_imports_with_a_profile_without_account_mapping(self):
run = self._import(
CSV_SETTINGS + CSV_MAPPING,
"statement.csv",
CSV_CONTENT,
account=self.checking,
)
self.assertEqual(run.successful_rows, 2)
expense = Transaction.objects.get(description="Groceries")
self.assertEqual(expense.account, self.checking)
self.assertEqual(expense.type, Transaction.Type.EXPENSE)
self.assertEqual(expense.amount, Decimal("50.00"))
def test_deduplication_by_account_uses_the_run_account(self):
yaml_config = CSV_SETTINGS + CSV_MAPPING + CSV_ACCOUNT_DEDUPLICATION
self._import(yaml_config, "first.csv", CSV_CONTENT, account=self.checking)
# Same rows, other account: not duplicates
into_savings = self._import(
yaml_config, "second.csv", CSV_CONTENT, account=self.savings
)
# Same rows, same account: duplicates
into_savings_again = self._import(
yaml_config, "third.csv", CSV_CONTENT, account=self.savings
)
self.assertEqual(into_savings.successful_rows, 2)
self.assertEqual(into_savings.skipped_rows, 0)
self.assertEqual(into_savings_again.successful_rows, 0)
self.assertEqual(into_savings_again.skipped_rows, 2)
self.assertEqual(Transaction.objects.filter(account=self.savings).count(), 2)
def test_qif_run_account_is_used_instead_of_the_file_name(self):
yaml_config = """
settings:
file_type: qif
importing: transactions
date_format: "%d/%m/%Y"
mapping: {}
"""
content = """!Type:Bank
D05/01/2026
T-100.00
PSupermarket
^
"""
run = self._import(
yaml_config, "export-2026.qif", content, account=self.savings
)
self.assertEqual(run.status, ImportRun.Status.FINISHED)
transaction = Transaction.objects.get()
self.assertEqual(transaction.account, self.savings)
self.assertEqual(transaction.amount, Decimal("100.00"))
class ImportRunAccountTestCase(ImportObjectPermissionTestCase):
def setUp(self):
super().setUp()
self.temp_root = Path(tempfile.mkdtemp()).resolve()
self.addCleanup(shutil.rmtree, self.temp_root, ignore_errors=True)
temp_root_override = override_settings(IMPORT_TEMP_ROOT=self.temp_root)
temp_root_override.enable()
self.addCleanup(temp_root_override.disable)
currency = Currency.objects.create(
code="EUR", name="Euro", decimal_places=2, prefix="€ "
)
self.account = Account.all_objects.create(
name="Checking", currency=currency, owner=self.owner
)
self.archived_account = Account.all_objects.create(
name="Old checking", currency=currency, owner=self.owner, is_archived=True
)
self.strangers_account = Account.all_objects.create(
name="Stranger's checking", currency=currency, owner=self.stranger
)
self.tags_profile = ImportProfile.all_objects.create(
name="Tags profile", yaml_config=TAGS_YAML_CONFIG, owner=self.owner
)
class ImportRunAccountViewTests(ImportRunAccountTestCase):
def _upload_form(self, profile):
self.login(self.owner)
response = self.client.get(
reverse("import_run_add", kwargs={"profile_id": profile.id}), **HTMX
)
self.assertEqual(response.status_code, 200)
return response.context["form"]
def test_upload_form_offers_active_accounts_for_transaction_profiles(self):
form = self._upload_form(self.private_profile)
self.assertIn("account", form.fields)
self.assertFalse(form.fields["account"].required)
self.assertEqual(list(form.fields["account"].queryset), [self.account])
def test_upload_form_has_no_account_for_other_profiles(self):
form = self._upload_form(self.tags_profile)
self.assertNotIn("account", form.fields)
@patch("apps.import_app.views.process_import.defer")
def test_web_upload_stores_the_account_on_the_run(self, mock_defer):
self.login(self.owner)
response = self.client.post(
reverse("import_run_add", kwargs={"profile_id": self.private_profile.id}),
{
"file": SimpleUploadedFile("web.csv", b"date,amount\n"),
"account": self.account.id,
},
**HTMX,
)
self.assertEqual(response.status_code, 204)
run = ImportRun.all_objects.get(file_name="web.csv")
self.assertEqual(run.account, self.account)
mock_defer.assert_called_once()
@patch("apps.import_app.views.process_import.defer")
def test_web_upload_without_account_keeps_the_profile_account(self, mock_defer):
self.login(self.owner)
response = self.client.post(
reverse("import_run_add", kwargs={"profile_id": self.private_profile.id}),
{"file": SimpleUploadedFile("web.csv", b"date,amount\n")},
**HTMX,
)
self.assertEqual(response.status_code, 204)
self.assertIsNone(ImportRun.all_objects.get(file_name="web.csv").account)
@patch("apps.import_app.views.process_import.defer")
def test_web_upload_rejects_another_users_account(self, mock_defer):
self.login(self.owner)
response = self.client.post(
reverse("import_run_add", kwargs={"profile_id": self.private_profile.id}),
{
"file": SimpleUploadedFile("web.csv", b"date,amount\n"),
"account": self.strangers_account.id,
},
**HTMX,
)
self.assertEqual(response.status_code, 200)
self.assertIn("account", response.context["form"].errors)
self.assertFalse(ImportRun.all_objects.filter(file_name="web.csv").exists())
mock_defer.assert_not_called()
def test_runs_list_shows_the_run_account(self):
ImportRun.all_objects.create(
profile=self.private_profile,
file_name="statement.csv",
account=self.account,
owner=self.owner,
)
self.login(self.owner)
response = self.client.get(
reverse(
"import_profile_runs_list",
kwargs={"profile_id": self.private_profile.id},
),
**HTMX,
)
self.assertContains(response, "Checking")
class ImportRunAccountAPITests(ImportRunAccountTestCase):
def setUp(self):
super().setUp()
self.api = APIClient()
self.api.force_authenticate(user=self.owner)
def _post(self, profile, **data):
return self.api.post(
"/api/import/import/",
{
"profile_id": profile.id,
"file": SimpleUploadedFile("api.csv", b"date,amount\n"),
**data,
},
format="multipart",
)
@patch("apps.api.views.imports.process_import.defer")
def test_api_upload_stores_the_account_on_the_run(self, mock_defer):
response = self._post(self.private_profile, account_id=self.account.id)
self.assertEqual(response.status_code, status.HTTP_202_ACCEPTED)
run = ImportRun.all_objects.get(id=response.data["import_run_id"])
self.assertEqual(run.account, self.account)
response = self.api.get(f"/api/import/runs/{run.id}/")
self.assertEqual(response.data["account"], self.account.id)
@patch("apps.api.views.imports.process_import.defer")
def test_api_upload_without_account_keeps_the_profile_account(self, mock_defer):
response = self._post(self.private_profile)
self.assertEqual(response.status_code, status.HTTP_202_ACCEPTED)
run = ImportRun.all_objects.get(id=response.data["import_run_id"])
self.assertIsNone(run.account)
@patch("apps.api.views.imports.process_import.defer")
def test_api_rejects_another_users_account(self, mock_defer):
response = self._post(
self.private_profile, account_id=self.strangers_account.id
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertIn("account_id", response.data)
mock_defer.assert_not_called()
@patch("apps.api.views.imports.process_import.defer")
def test_api_rejects_an_archived_account(self, mock_defer):
response = self._post(
self.private_profile, account_id=self.archived_account.id
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertIn("account_id", response.data)
mock_defer.assert_not_called()
@patch("apps.api.views.imports.process_import.defer")
def test_api_rejects_an_account_for_profiles_not_importing_transactions(
self, mock_defer
):
response = self._post(self.tags_profile, account_id=self.account.id)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertIn("account_id", response.data)
mock_defer.assert_not_called()
+11 -4
View File
@@ -168,7 +168,11 @@ def import_runs_list(request, profile_id):
# ImportRun.objects only returns the requesting user's runs, so a shared # ImportRun.objects only returns the requesting user's runs, so a shared
# profile does not expose the runs other users made with it. # profile does not expose the runs other users made with it.
runs = ImportRun.objects.filter(profile=profile).order_by("-id") runs = (
ImportRun.objects.filter(profile=profile)
.select_related("account")
.order_by("-id")
)
return render( return render(
request, request,
@@ -201,7 +205,7 @@ def import_run_add(request, profile_id):
) )
if request.method == "POST": if request.method == "POST":
form = ImportRunFileUploadForm(request.POST, request.FILES) form = ImportRunFileUploadForm(request.POST, request.FILES, profile=profile)
if form.is_valid(): if form.is_valid():
uploaded_file = request.FILES["file"] uploaded_file = request.FILES["file"]
@@ -210,7 +214,10 @@ def import_run_add(request, profile_id):
file_path = fs.path(filename) file_path = fs.path(filename)
import_run = ImportRun.objects.create( import_run = ImportRun.objects.create(
profile=profile, file_name=filename, owner=request.user profile=profile,
file_name=filename,
account=form.cleaned_data.get("account"),
owner=request.user,
) )
# Defer the procrastinate task # Defer the procrastinate task
@@ -229,7 +236,7 @@ def import_run_add(request, profile_id):
}, },
) )
else: else:
form = ImportRunFileUploadForm() form = ImportRunFileUploadForm(profile=profile)
return render( return render(
request, request,
+6 -6
View File
@@ -8,6 +8,7 @@ from django.utils import timezone
from apps.currencies.models import Currency from apps.currencies.models import Currency
from apps.currencies.utils.convert import convert from apps.currencies.utils.convert import convert
from apps.currencies.utils.display import shown_exchange_currency
from apps.transactions.models import Transaction from apps.transactions.models import Transaction
@@ -94,7 +95,7 @@ def get_month_by_month_data(year=None, group_by="categories"):
"account__currency__decimal_places", "account__currency__decimal_places",
"account__currency__prefix", "account__currency__prefix",
"account__currency__suffix", "account__currency__suffix",
"account__currency__exchange_currency", exchange_currency_id=shown_exchange_currency("account__currency__"),
) )
.annotate( .annotate(
expense_total=Coalesce( expense_total=Coalesce(
@@ -139,6 +140,7 @@ def get_month_by_month_data(year=None, group_by="categories"):
item_name = metric[name_field] item_name = metric[name_field]
month = metric["reference_date__month"] month = metric["reference_date__month"]
currency_id = metric["account__currency"] currency_id = metric["account__currency"]
exchange_currency_id = metric["exchange_currency_id"]
# Use a consistent key for None (uncategorized/untagged/no entity) # Use a consistent key for None (uncategorized/untagged/no entity)
item_key = item_id if item_id is not None else "__none__" item_key = item_id if item_id is not None else "__none__"
@@ -164,7 +166,7 @@ def get_month_by_month_data(year=None, group_by="categories"):
"decimal_places": metric["account__currency__decimal_places"], "decimal_places": metric["account__currency__decimal_places"],
"prefix": metric["account__currency__prefix"], "prefix": metric["account__currency__prefix"],
"suffix": metric["account__currency__suffix"], "suffix": metric["account__currency__suffix"],
"exchange_currency_id": metric["account__currency__exchange_currency"], "exchange_currency_id": exchange_currency_id,
} }
currency_data = { currency_data = {
@@ -181,11 +183,9 @@ def get_month_by_month_data(year=None, group_by="categories"):
} }
# Handle currency conversion if exchange currency is set # Handle currency conversion if exchange currency is set
if metric["account__currency__exchange_currency"]: if exchange_currency_id:
from_currency = Currency.objects.get(id=currency_id) from_currency = Currency.objects.get(id=currency_id)
exchange_currency = Currency.objects.get( exchange_currency = Currency.objects.get(id=exchange_currency_id)
id=metric["account__currency__exchange_currency"]
)
converted_amount, prefix, suffix, decimal_places = convert( converted_amount, prefix, suffix, decimal_places = convert(
amount=final_total, amount=final_total,
+16 -7
View File
@@ -6,6 +6,7 @@ from django.db.models.functions import Coalesce
from apps.currencies.models import Currency from apps.currencies.models import Currency
from apps.currencies.utils.convert import convert from apps.currencies.utils.convert import convert
from apps.currencies.utils.display import shown_exchange_currency
from apps.transactions.models import Transaction from apps.transactions.models import Transaction
# Grouping levels the overview can be built from. Any ordering of these keys is # Grouping levels the overview can be built from. Any ordering of these keys is
@@ -25,11 +26,11 @@ CURRENCY_FIELDS = (
"account__currency__decimal_places", "account__currency__decimal_places",
"account__currency__prefix", "account__currency__prefix",
"account__currency__suffix", "account__currency__suffix",
"account__currency__exchange_currency",
) )
# Aggregated amounts, plus the totals derived from them. Every one of these is # Aggregated amounts, plus the totals derived from them. Every one of these is
# converted when the account's currency has an exchange currency. # converted when the account's currency has an exchange currency and shows
# exchanged amounts.
AGGREGATED_FIELDS = ( AGGREGATED_FIELDS = (
"expense_current", "expense_current",
"expense_projected", "expense_projected",
@@ -78,7 +79,12 @@ def _sum_of(transaction_type, is_paid):
def _aggregate(transactions_queryset, group_fields, name_field): def _aggregate(transactions_queryset, group_fields, name_field):
"""Sum income/expense per currency, grouped by the given fields.""" """Sum income/expense per currency, grouped by the given fields."""
return ( return (
transactions_queryset.values(*group_fields, name_field, *CURRENCY_FIELDS) transactions_queryset.values(
*group_fields,
name_field,
*CURRENCY_FIELDS,
exchange_currency_id=shown_exchange_currency("account__currency__"),
)
.annotate( .annotate(
expense_current=_sum_of(Transaction.Type.EXPENSE, is_paid=True), expense_current=_sum_of(Transaction.Type.EXPENSE, is_paid=True),
expense_projected=_sum_of(Transaction.Type.EXPENSE, is_paid=False), expense_projected=_sum_of(Transaction.Type.EXPENSE, is_paid=False),
@@ -120,11 +126,9 @@ def _build_currency_data(metric, showing):
"total": currency_data[total], "total": currency_data[total],
} }
if metric["account__currency__exchange_currency"]: if metric["exchange_currency_id"]:
from_currency = Currency.objects.get(id=metric["account__currency"]) from_currency = Currency.objects.get(id=metric["account__currency"])
exchange_currency = Currency.objects.get( exchange_currency = Currency.objects.get(id=metric["exchange_currency_id"])
id=metric["account__currency__exchange_currency"]
)
exchanged = {} exchanged = {}
for field in AGGREGATED_FIELDS + TOTAL_FIELDS: for field in AGGREGATED_FIELDS + TOTAL_FIELDS:
@@ -144,6 +148,11 @@ def _build_currency_data(metric, showing):
"name": exchange_currency.name, "name": exchange_currency.name,
} }
if exchanged: if exchanged:
exchanged["shown"] = {
"income": exchanged.get(income),
"expense": exchanged.get(expense),
"total": exchanged.get(total),
}
currency_data["exchanged"] = exchanged currency_data["exchanged"] = exchanged
return currency_data return currency_data
+6 -6
View File
@@ -7,6 +7,7 @@ from django.db.models.functions import Coalesce
from apps.currencies.models import Currency from apps.currencies.models import Currency
from apps.currencies.utils.convert import convert from apps.currencies.utils.convert import convert
from apps.currencies.utils.display import shown_exchange_currency
from apps.transactions.models import Transaction from apps.transactions.models import Transaction
@@ -83,7 +84,7 @@ def get_year_by_year_data(group_by="categories"):
"account__currency__decimal_places", "account__currency__decimal_places",
"account__currency__prefix", "account__currency__prefix",
"account__currency__suffix", "account__currency__suffix",
"account__currency__exchange_currency", exchange_currency_id=shown_exchange_currency("account__currency__"),
) )
.annotate( .annotate(
expense_total=Coalesce( expense_total=Coalesce(
@@ -126,6 +127,7 @@ def get_year_by_year_data(group_by="categories"):
item_name = metric[name_field] item_name = metric[name_field]
year = metric["reference_date__year"] year = metric["reference_date__year"]
currency_id = metric["account__currency"] currency_id = metric["account__currency"]
exchange_currency_id = metric["exchange_currency_id"]
# Use a consistent key for None (uncategorized/untagged/no entity) # Use a consistent key for None (uncategorized/untagged/no entity)
item_key = item_id if item_id is not None else "__none__" item_key = item_id if item_id is not None else "__none__"
@@ -151,7 +153,7 @@ def get_year_by_year_data(group_by="categories"):
"decimal_places": metric["account__currency__decimal_places"], "decimal_places": metric["account__currency__decimal_places"],
"prefix": metric["account__currency__prefix"], "prefix": metric["account__currency__prefix"],
"suffix": metric["account__currency__suffix"], "suffix": metric["account__currency__suffix"],
"exchange_currency_id": metric["account__currency__exchange_currency"], "exchange_currency_id": exchange_currency_id,
} }
currency_data = { currency_data = {
@@ -168,11 +170,9 @@ def get_year_by_year_data(group_by="categories"):
} }
# Handle currency conversion if exchange currency is set # Handle currency conversion if exchange currency is set
if metric["account__currency__exchange_currency"]: if exchange_currency_id:
from_currency = Currency.objects.get(id=currency_id) from_currency = Currency.objects.get(id=currency_id)
exchange_currency = Currency.objects.get( exchange_currency = Currency.objects.get(id=exchange_currency_id)
id=metric["account__currency__exchange_currency"]
)
converted_amount, prefix, suffix, decimal_places = convert( converted_amount, prefix, suffix, decimal_places = convert(
amount=final_total, amount=final_total,
+2 -1
View File
@@ -308,6 +308,7 @@ def latest_transactions(request):
transactions = ( transactions = (
Transaction.objects.filter(created_at__gte=limit) Transaction.objects.filter(created_at__gte=limit)
.collapse_transfers() .collapse_transfers()
.prefetch_related("sub_transactions")
.order_by("-id")[:30] .order_by("-id")[:30]
) )
@@ -325,7 +326,7 @@ def late_transactions(request):
now = timezone.localdate(timezone.now()) now = timezone.localdate(timezone.now())
transactions = Transaction.objects.filter( transactions = Transaction.objects.filter(
is_paid=False, date__lt=now is_paid=False, date__lt=now
).collapse_transfers() ).collapse_transfers().prefetch_related("sub_transactions")
return render( return render(
request, request,
+11 -11
View File
@@ -239,13 +239,14 @@ def get_holding_performance(account, as_of=None, with_history=False):
return perf return perf
def xirr(flows): def xirr(flows, period_days=365):
"""Annual rate r where the dated flows' present value is zero, as a float """Rate r per ``period_days`` (a year by default) where the dated flows'
(0.138 = 13.8% a year). None when it can't be solved.""" present value is zero, as a float (0.138 = 13.8% a period). None when it
can't be solved."""
if not flows: if not flows:
return None return None
start = min(flow[0] for flow in flows) start = min(flow[0] for flow in flows)
points = [((flow[0] - start).days / 365, float(flow[1])) for flow in flows] points = [((flow[0] - start).days / period_days, float(flow[1])) for flow in flows]
if not (any(a < 0 for _, a in points) and any(a > 0 for _, a in points)): if not (any(a < 0 for _, a in points) and any(a > 0 for _, a in points)):
return None return None
@@ -272,17 +273,16 @@ def money_weighted_return(flows, market_value, as_of):
``{"percentage": Decimal, "annualized": bool}`` or None. ``{"percentage": Decimal, "annualized": bool}`` or None.
Over a year or more it is annualized (XIRR). Below a year an annual rate Over a year or more it is annualized (XIRR). Below a year an annual rate
exaggerates, so it is turned back into the return over the period held.""" exaggerates, so the rate is solved over the period held instead: annualizing
a few days blows past what xirr can solve (+1.3% in one day is ~10,600% a year)."""
if not flows or market_value is None: if not flows or market_value is None:
return None return None
rate = xirr(flows + [(as_of, market_value)]) days = (as_of - min(flow[0] for flow in flows)).days
annualized = days >= 365
rate = xirr(flows + [(as_of, market_value)], 365 if annualized else max(days, 1))
if rate is None: if rate is None:
return None return None
days = (as_of - min(flow[0] for flow in flows)).days return {"percentage": Decimal(str(round(rate * 100, 4))), "annualized": annualized}
if days >= 365:
return {"percentage": Decimal(str(round(rate * 100, 4))), "annualized": True}
period = (1 + rate) ** (days / 365) - 1
return {"percentage": Decimal(str(round(period * 100, 4))), "annualized": False}
def _month_ends(transactions, as_of): def _month_ends(transactions, as_of):
@@ -306,6 +306,12 @@ class HoldingPerformanceTests(TestCase):
self.assertFalse(mw["annualized"]) self.assertFalse(mw["annualized"])
self.assertAlmostEqual(float(mw["percentage"]), 5, places=3) self.assertAlmostEqual(float(mw["percentage"]), 5, places=3)
# A day old with a big move: annualized it'd be far past what xirr can solve
day = datetime.date(2025, 1, 2)
mw = money_weighted_return([(start, D(-1000), "in")], D(1050), day)
self.assertFalse(mw["annualized"])
self.assertAlmostEqual(float(mw["percentage"]), 5, places=3)
# Money added late weighs less: +100 on 1000 held all year beats the # Money added late weighs less: +100 on 1000 held all year beats the
# simple +100 / 2000 = 5% when the second 1000 only came in at the end # simple +100 / 2000 = 5% when the second 1000 only came in at the end
flows = [(start, D(-1000), "in"), (datetime.date(2025, 12, 31), D(-1000), "in")] flows = [(start, D(-1000), "in"), (datetime.date(2025, 12, 31), D(-1000), "in")]
@@ -441,6 +447,34 @@ class InvestmentViewTests(TestCase):
response = self.client.get(reverse("investments_charts"), **self.HTMX) response = self.client.get(reverse("investments_charts"), **self.HTMX)
self.assertContains(response, "investment-charts-0") self.assertContains(response, "investment-charts-0")
def test_detail_follows_show_exchanged_amounts(self):
brl = self.account.currency
usd = Currency.objects.create(
code="USD", name="Dollar", decimal_places=2, prefix="US$ ",
exchange_currency=brl,
)
ExchangeRate.objects.create(
from_currency=usd, to_currency=brl, rate=D("5"),
date=timezone.make_aware(datetime.datetime(2026, 1, 1)),
)
# Tracked by value in USD: movements and the "In Real" card come from USD's chain
account = Account.all_objects.create(
name="Broker", currency=usd, is_investment=True, owner=self.owner
)
tx(account, IN, "10", datetime.date(2026, 1, 2))
self.client.force_login(self.owner)
url = reverse("investment_detail", args=[account.id])
response = self.client.get(url, **self.HTMX)
self.assertContains(response, 'data-amount="50"', count=3) # 10 USD at 5: ≈ line, card, movement
self.assertContains(response, "In Real")
usd.show_exchanged_amounts = False
usd.save()
response = self.client.get(url, **self.HTMX)
self.assertNotContains(response, 'data-amount="50"')
self.assertNotContains(response, "In Real")
def test_stranger_cannot_see_detail(self): def test_stranger_cannot_see_detail(self):
self.client.force_login(self.stranger) self.client.force_login(self.stranger)
response = self.client.get( response = self.client.get(
+5
View File
@@ -5,6 +5,11 @@ from . import views
urlpatterns = [ urlpatterns = [
path("investments/", views.investments_index, name="investments_index"), path("investments/", views.investments_index, name="investments_index"),
path("investments/list/", views.investments_list, name="investments_list"), path("investments/list/", views.investments_list, name="investments_list"),
path(
"investments/group-by/",
views.investments_group_by,
name="investments_group_by",
),
path("investments/charts/", views.investments_charts, name="investments_charts"), path("investments/charts/", views.investments_charts, name="investments_charts"),
path( path(
"investments/rebalance/", "investments/rebalance/",
+37 -14
View File
@@ -25,7 +25,7 @@ from apps.transactions.models import Transaction
def _holdings(): def _holdings():
return Account.objects.filter(is_investment=True, is_archived=False).select_related( return Account.objects.filter(is_investment=True, is_archived=False).select_related(
"group", "currency", "exchange_currency" "group", "investment_class", "currency", "exchange_currency"
) )
@@ -34,31 +34,47 @@ def investments_index(request):
return render(request, "investments/pages/index.html") return render(request, "investments/pages/index.html")
GROUP_BY = ("group", "investment_class")
def _grouped(performances, attr):
groups = {}
for perf in performances:
group = getattr(perf.account, attr)
groups.setdefault(group.id if group else None, (group, []))[1].append(perf)
return [
{"group": group, "holdings": items, "totals": summarize(items)}
for group, items in sorted(
groups.values(), key=lambda g: (g[0] is None, g[0] and g[0].name)
)
]
@only_htmx @only_htmx
@login_required @login_required
def investments_list(request): def investments_list(request):
performances = [get_holding_performance(account) for account in _holdings()] performances = [get_holding_performance(account) for account in _holdings()]
groups = {} # Both groupings are rendered so the toggle flips client-side without recomputing
for perf in performances:
group = perf.account.group
groups.setdefault(group.id if group else None, (group, []))[1].append(perf)
return render( return render(
request, request,
"investments/fragments/list.html", "investments/fragments/list.html",
{ {
"groups": [ "groupings": [(attr, _grouped(performances, attr)) for attr in GROUP_BY],
{"group": group, "holdings": items, "totals": summarize(items)}
for group, items in sorted(
groups.values(), key=lambda g: (g[0] is None, g[0] and g[0].name)
)
],
"totals": summarize(performances, convert=True), "totals": summarize(performances, convert=True),
"group_by": request.session.get("investments_group_by", "group"),
}, },
) )
@only_htmx
@login_required
def investments_group_by(request):
if request.GET.get("group_by") in GROUP_BY:
request.session["investments_group_by"] = request.GET["group_by"]
return HttpResponse(status=204)
def _money(currency): def _money(currency):
"""What chartMoney (charts.js) needs to format amounts like the rest of the app.""" """What chartMoney (charts.js) needs to format amounts like the rest of the app."""
return { return {
@@ -193,7 +209,13 @@ def investment_detail(request, account_id):
perf = get_holding_performance(account, with_history=True) perf = get_holding_performance(account, with_history=True)
transactions = ( transactions = (
Transaction.objects.filter(account=account) Transaction.objects.filter(account=account)
.select_related("linked_transaction__account__currency", "category") .select_related(
"linked_transaction__account__currency",
"category",
"account__currency__exchange_currency",
"account__exchange_currency",
)
.prefetch_related("sub_transactions")
.order_by("-date", "-id")[:50] .order_by("-date", "-id")[:50]
) )
income = [] income = []
@@ -230,7 +252,8 @@ def investment_detail(request, account_id):
{ {
"account": account, "account": account,
"perf": perf, "perf": perf,
"home": in_home_currency(perf), # The valuation currency provides the next hop, so its switch decides
"home": in_home_currency(perf) if perf.currency.show_exchanged_amounts else None,
"transactions": transactions, "transactions": transactions,
"income": income, "income": income,
"income_entities": list(account.income_entities.all()), "income_entities": list(account.income_entities.all()),
+1
View File
@@ -93,6 +93,7 @@ def transactions_list(request, month: int, year: int):
"installment_plan", "installment_plan",
"entities", "entities",
"dca_expense_entries", "dca_expense_entries",
"sub_transactions",
"dca_income_entries", "dca_income_entries",
"linked_transaction__account__currency", "linked_transaction__account__currency",
"linked_transaction__account__group", "linked_transaction__account__group",
+41
View File
@@ -80,3 +80,44 @@ class NetWorthCurrencyChartTests(TestCase):
'<span class="text-start shrink">Consolidated</span>', '<span class="text-start shrink">Consolidated</span>',
html=False, html=False,
) )
def test_consolidated_total_lists_each_currency_share(self):
user = get_user_model().objects.create_user(
email="shares@example.com", password="password"
)
usd = Currency.objects.create(code="USD", name="US Dollar", prefix="$ ")
eur = Currency.objects.create(
code="EUR", name="Euro", prefix="€ ", exchange_currency=usd
)
ExchangeRate.objects.create(
from_currency=eur,
to_currency=usd,
rate=Decimal("2"),
date=timezone.now(),
)
for currency, amount in ((usd, "300"), (eur, "50")):
Transaction.userless_all_objects.create(
account=Account.all_objects.create(
name=f"{currency.code} account", currency=currency, owner=user
),
owner=user,
type=Transaction.Type.INCOME,
amount=Decimal(amount),
date=date(2026, 1, 15),
reference_date=date(2026, 1, 1),
is_paid=True,
)
self.client.force_login(user)
response = self.client.get(reverse("net_worth"))
self.assertEqual(response.status_code, 200)
components = response.context["currency_net_worth"][usd.id]["consolidated"][
"components"
]
self.assertEqual(
[(part["currency"]["name"], part["share"]) for part in components],
[("US Dollar", Decimal("75")), ("Euro", Decimal("25"))],
)
self.assertContains(response, "75.0%")
self.assertContains(response, "25.0%")
+2
View File
@@ -190,6 +190,7 @@ class UpdateOrCreateTransactionRuleActionForm(forms.ModelForm):
"set_date": _("Date"), "set_date": _("Date"),
"set_reference_date": _("Reference Date"), "set_reference_date": _("Reference Date"),
"set_amount": _("Amount"), "set_amount": _("Amount"),
"set_sub_transactions": _("Sub-transactions"),
"set_description": _("Description"), "set_description": _("Description"),
"set_tags": _("Tags"), "set_tags": _("Tags"),
"set_entities": _("Entities"), "set_entities": _("Entities"),
@@ -367,6 +368,7 @@ class UpdateOrCreateTransactionRuleActionForm(forms.ModelForm):
Field("set_reference_date", rows=1), Field("set_reference_date", rows=1),
Field("set_description", rows=1), Field("set_description", rows=1),
Field("set_amount", rows=1), Field("set_amount", rows=1),
Field("set_sub_transactions", rows=1),
Field("set_category", rows=1), Field("set_category", rows=1),
Field("set_tags", rows=1), Field("set_tags", rows=1),
Field("set_notes", rows=1), Field("set_notes", rows=1),
@@ -0,0 +1,23 @@
# Generated by Django 5.2.17 on 2026-11-14 03:08
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('rules', '0018_transactionrule_order'),
]
operations = [
migrations.AddField(
model_name='updateorcreatetransactionruleaction',
name='set_sub_transactions',
field=models.TextField(blank=True, verbose_name='Sub-transactions'),
),
migrations.AlterField(
model_name='transactionruleaction',
name='field',
field=models.CharField(choices=[('account', 'Account'), ('type', 'Type'), ('is_paid', 'Paid'), ('date', 'Date'), ('reference_date', 'Reference Date'), ('mute', 'Mute'), ('amount', 'Amount'), ('sub_transactions', 'Sub-transactions'), ('description', 'Description'), ('notes', 'Notes'), ('category', 'Category'), ('tags', 'Tags'), ('entities', 'Entities'), ('internal_nome', 'Internal Note'), ('internal_id', 'Internal ID')], max_length=50, verbose_name='Field'),
),
]
+5
View File
@@ -39,6 +39,7 @@ class TransactionRuleAction(models.Model):
reference_date = "reference_date", _("Reference Date") reference_date = "reference_date", _("Reference Date")
mute = "mute", _("Mute") mute = "mute", _("Mute")
amount = "amount", _("Amount") amount = "amount", _("Amount")
sub_transactions = "sub_transactions", _("Sub-transactions")
description = "description", _("Description") description = "description", _("Description")
notes = "notes", _("Notes") notes = "notes", _("Notes")
category = "category", _("Category") category = "category", _("Category")
@@ -315,6 +316,10 @@ class UpdateOrCreateTransactionRuleAction(models.Model):
verbose_name=_("Tags"), verbose_name=_("Tags"),
blank=True, blank=True,
) )
set_sub_transactions = models.TextField(
verbose_name=_("Sub-transactions"),
blank=True,
)
set_mute = models.TextField( set_mute = models.TextField(
verbose_name=_("Mute"), verbose_name=_("Mute"),
blank=True, blank=True,
+105 -6
View File
@@ -17,12 +17,15 @@ from procrastinate.contrib.django import app
from simpleeval import EvalWithCompoundTypes from simpleeval import EvalWithCompoundTypes
from apps.accounts.models import Account from apps.accounts.models import Account
from apps.common.functions.decimals import truncate_decimal
from apps.common.middleware.thread_local import write_current_user, delete_current_user from apps.common.middleware.thread_local import write_current_user, delete_current_user
from apps.rules.models import ( from apps.rules.models import (
TransactionRule, TransactionRule,
TransactionRuleAction, TransactionRuleAction,
) )
from apps.transactions.auto_tagging.service import apply_auto_tagging
from apps.transactions.models import ( from apps.transactions.models import (
SubTransaction,
Transaction, Transaction,
TransactionCategory, TransactionCategory,
TransactionTag, TransactionTag,
@@ -32,6 +35,39 @@ from apps.rules.utils import transactions
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
SUB_TRANSACTIONS_SHAPE = (
"Sub-transactions must be a list like [{'name': 'Fee', 'amount': 1.5}, ...]"
)
def build_sub_transactions(transaction: Transaction, value) -> list[SubTransaction]:
"""Validated sub-transactions for `transaction` from a rule expression's result."""
if not isinstance(value, (list, tuple)):
raise ValueError(SUB_TRANSACTIONS_SHAPE)
places = transaction.account.currency.decimal_places
subs = []
for item in value:
if not isinstance(item, dict) or item.get("amount") is None:
raise ValueError(SUB_TRANSACTIONS_SHAPE)
sub = SubTransaction(
transaction=transaction,
name=str(item.get("name") or "").strip(),
amount=truncate_decimal(
value=Decimal(str(item["amount"])), decimal_places=places
),
)
# the transaction may not be saved yet, so its link isn't checked here
sub.full_clean(exclude=["transaction"], validate_constraints=False)
subs.append(sub)
return subs
def _snapshot(transaction: Transaction) -> Transaction:
"""A copy to show in dry-run results, as the transaction is right now."""
copy = transaction.deepcopy()
copy.copy_sub_transactions_from(transaction)
return copy
class DryRunResults: class DryRunResults:
def __init__(self, dry_run: bool): def __init__(self, dry_run: bool):
@@ -49,7 +85,7 @@ class DryRunResults:
if not self.dry_run: if not self.dry_run:
return return
if isinstance(instance, Transaction): if isinstance(instance, Transaction):
instance = instance.deepcopy() instance = _snapshot(instance)
elif isinstance(instance, dict): elif isinstance(instance, dict):
instance = deepcopy(instance) instance = deepcopy(instance)
@@ -65,7 +101,7 @@ class DryRunResults:
if not self.dry_run: if not self.dry_run:
return return
if isinstance(instance, Transaction): if isinstance(instance, Transaction):
instance = instance.deepcopy() instance = _snapshot(instance)
elif isinstance(instance, dict): elif isinstance(instance, dict):
instance = deepcopy(instance) instance = deepcopy(instance)
@@ -95,7 +131,7 @@ class DryRunResults:
return return
if isinstance(end_instance, Transaction): if isinstance(end_instance, Transaction):
end_instance = end_instance.deepcopy() end_instance = _snapshot(end_instance)
elif isinstance(end_instance, dict): elif isinstance(end_instance, dict):
end_instance = deepcopy(end_instance) end_instance = deepcopy(end_instance)
@@ -201,6 +237,14 @@ def check_for_transaction_rules(
f"{prefix}entities_ids": ( f"{prefix}entities_ids": (
[x.id for x in transaction.entities.all()] if transaction.id else [] [x.id for x in transaction.entities.all()] if transaction.id else []
), ),
f"{prefix}sub_transactions": (
[
{"name": s.name, "amount": s.amount}
for s in transaction.sub_transactions.all()
]
if transaction.id
else []
),
f"{prefix}is_expense": transaction.type == Transaction.Type.EXPENSE, f"{prefix}is_expense": transaction.type == Transaction.Type.EXPENSE,
f"{prefix}is_income": transaction.type == Transaction.Type.INCOME, f"{prefix}is_income": transaction.type == Transaction.Type.INCOME,
f"{prefix}is_paid": transaction.is_paid, f"{prefix}is_paid": transaction.is_paid,
@@ -250,6 +294,10 @@ def check_for_transaction_rules(
f"{prefix}entities_ids": [ f"{prefix}entities_ids": [
x[0] for x in transaction.get("entities", []) x[0] for x in transaction.get("entities", [])
], ],
f"{prefix}sub_transactions": [
{"name": s["name"], "amount": Decimal(s["amount"])}
for s in transaction.get("sub_transactions", [])
],
f"{prefix}is_expense": transaction.get("type") f"{prefix}is_expense": transaction.get("type")
== Transaction.Type.EXPENSE, == Transaction.Type.EXPENSE,
f"{prefix}is_income": transaction.get("type") f"{prefix}is_income": transaction.get("type")
@@ -274,6 +322,22 @@ def check_for_transaction_rules(
f"{prefix}installment_total": transaction.get("installment_total"), f"{prefix}installment_total": transaction.get("installment_total"),
} }
def _evaluate_sub_transactions(transaction, value):
"""The parts an expression's `value` gives `transaction`, with its amount set
to their sum, or None for a transfer, which can't have any."""
if transaction.linked_transaction_id:
_log("Skipped: transfers can't have sub-transactions")
return None
subs = build_sub_transactions(transaction, value)
if subs:
transaction.amount = sum(s.amount for s in subs)
_log(f"Sub-transactions will be set as: {[(s.name, s.amount) for s in subs]}")
return subs
def _save_sub_transactions(transaction, subs):
transaction.sub_transactions.all().delete()
SubTransaction.objects.bulk_create(subs)
def _process_update_or_create_transaction_action(processed_action): def _process_update_or_create_transaction_action(processed_action):
"""Helper to process a single linked transaction action""" """Helper to process a single linked transaction action"""
@@ -295,7 +359,7 @@ def check_for_transaction_rules(
existing = True existing = True
if dry_run: if dry_run:
starting_instance = transaction.deepcopy() starting_instance = _snapshot(transaction)
_log("Found at least one matching transaction, using latest:") _log("Found at least one matching transaction, using latest:")
_log("{}".format(pformat(model_to_dict(transaction)))) _log("{}".format(pformat(model_to_dict(transaction))))
@@ -348,7 +412,15 @@ def check_for_transaction_rules(
processed_action.set_reference_date processed_action.set_reference_date
) )
if processed_action.set_amount: subs = None
if processed_action.set_sub_transactions:
subs = _evaluate_sub_transactions(
transaction, simple.eval(processed_action.set_sub_transactions)
)
# A transaction with parts gets its amount from them
has_parts = bool(subs) if subs is not None else transaction.has_sub_transactions
if processed_action.set_amount and not has_parts:
transaction.amount = simple.eval(processed_action.set_amount) transaction.amount = simple.eval(processed_action.set_amount)
if processed_action.set_description: if processed_action.set_description:
@@ -383,6 +455,8 @@ def check_for_transaction_rules(
) )
) )
transaction.save() transaction.save()
if subs is not None:
_save_sub_transactions(transaction, subs)
# Handle M2M fields after save # Handle M2M fields after save
tags = [] tags = []
@@ -441,7 +515,10 @@ def check_for_transaction_rules(
dry_run_results.header("edit_transaction", action=processed_action) dry_run_results.header("edit_transaction", action=processed_action)
field = processed_action.field field = processed_action.field
original_value = getattr(transaction, field) if field == TransactionRuleAction.Field.sub_transactions:
original_value = list(transaction.sub_transactions.values("name", "amount"))
else:
original_value = getattr(transaction, field)
new_value = simple.eval(processed_action.value) new_value = simple.eval(processed_action.value)
tags = [] tags = []
@@ -518,6 +595,18 @@ def check_for_transaction_rules(
transaction.entities.add(entity) transaction.entities.add(entity)
entities.append(entity) entities.append(entity)
elif field == TransactionRuleAction.Field.sub_transactions:
subs = _evaluate_sub_transactions(transaction, new_value)
if subs is not None:
_save_sub_transactions(transaction, subs)
new_value = ", ".join(f"{s.name}: {s.amount}" for s in subs) or "[]"
elif (
field == TransactionRuleAction.Field.amount
and transaction.has_sub_transactions
):
_log("Skipped: the amount is the sum of the transaction's sub-transactions")
else: else:
setattr( setattr(
transaction, transaction,
@@ -761,6 +850,16 @@ def check_for_transaction_rules(
error="Initial trigger didn't match, this rule will be skipped", error="Initial trigger didn't match, this rule will be skipped",
) )
_log("Initial trigger didn't match, this rule will be skipped") _log("Initial trigger didn't match, this rule will be skipped")
if not dry_run and signal == "transaction_created":
try:
apply_auto_tagging(instance, user)
except Exception:
logger.error(
"Error while applying auto-tagging for transaction %s",
instance.id,
exc_info=True,
)
except Exception as e: except Exception as e:
_log( _log(
"** Error while executing 'check_for_transaction_rules' task", "** Error while executing 'check_for_transaction_rules' task",
+108 -2
View File
@@ -6,10 +6,15 @@ from django.contrib.auth import get_user_model
from django.test import TransactionTestCase from django.test import TransactionTestCase
from apps.accounts.models import Account from apps.accounts.models import Account
from apps.common.middleware.thread_local import write_current_user
from apps.currencies.models import Currency from apps.currencies.models import Currency
from apps.rules.models import TransactionRule, UpdateOrCreateTransactionRuleAction from apps.rules.models import (
TransactionRule,
TransactionRuleAction,
UpdateOrCreateTransactionRuleAction,
)
from apps.rules.tasks import check_for_transaction_rules from apps.rules.tasks import check_for_transaction_rules
from apps.transactions.models import Transaction from apps.transactions.models import SubTransaction, Transaction
def run_check_for_transaction_rules_without_worker_wrapper(**kwargs): def run_check_for_transaction_rules_without_worker_wrapper(**kwargs):
@@ -122,3 +127,104 @@ class CheckForTransactionRulesTests(TransactionTestCase):
generated_transaction = Transaction.objects.get(description="Muted copy") generated_transaction = Transaction.objects.get(description="Muted copy")
self.assertTrue(generated_transaction.mute) self.assertTrue(generated_transaction.mute)
self.assertFalse(generated_transaction.is_paid) self.assertFalse(generated_transaction.is_paid)
@patch("apps.rules.signals.check_for_transaction_rules.defer")
def test_edit_amount_skips_transactions_with_sub_transactions(self, mock_defer):
rule = TransactionRule.objects.create(
active=True, on_update=True, name="Set amount", trigger="True", owner=self.user
)
TransactionRuleAction.objects.create(rule=rule, field="amount", value="99")
TransactionRuleAction.objects.create(rule=rule, field="description", value="'Ruled'")
amounts = {}
for parts in ([], [Decimal("4"), Decimal("6")]):
transaction = Transaction.objects.create(
account=self.account, type=Transaction.Type.EXPENSE, amount=sum(parts) or 1,
date=date(2026, 5, 4), owner=self.user,
)
for amount in parts:
SubTransaction.objects.create(transaction=transaction, name="Part", amount=amount)
run_check_for_transaction_rules_without_worker_wrapper(
instance_id=transaction.id, user_id=self.user.id, signal="transaction_updated"
)
transaction.refresh_from_db()
self.assertEqual(transaction.description, "Ruled")
amounts[len(parts)] = transaction.amount
self.assertEqual(amounts, {0: Decimal("99"), 2: Decimal("10")})
class SubTransactionRuleTests(TransactionTestCase):
def setUp(self):
self.user = get_user_model().objects.create_user(email="subs@example.com", password="x")
currency = Currency.objects.create(code="USD", name="US Dollar", decimal_places=2)
self.account = Account.objects.create(name="Main", currency=currency, owner=self.user)
self.rule = TransactionRule.objects.create(
active=True, on_update=True, name="Parts", trigger="True", owner=self.user
)
def _transaction(self, *parts, amount=None):
transaction = Transaction.objects.create(
account=self.account, type=Transaction.Type.EXPENSE, date=date(2026, 5, 4),
amount=amount if amount is not None else sum(a for _, a in parts), owner=self.user,
)
for name, part in parts:
SubTransaction.objects.create(transaction=transaction, name=name, amount=part)
return transaction
def _run(self, transaction):
with patch("apps.rules.signals.check_for_transaction_rules.defer"):
run_check_for_transaction_rules_without_worker_wrapper(
instance_id=transaction.id, user_id=self.user.id, signal="transaction_updated"
)
transaction.refresh_from_db()
return transaction.amount, list(transaction.sub_transactions.values_list("name", "amount"))
def _edit(self, expression):
TransactionRuleAction.objects.create(rule=self.rule, field="sub_transactions", value=expression)
def test_edit_action_adds_changes_and_deletes_parts(self):
self._edit(
'[{"name": s["name"], "amount": s["amount"] * 2} for s in sub_transactions'
' if s["name"] != "Tip"] + [{"name": "Fee", "amount": decimal("0.50")}]'
)
result = self._run(self._transaction(("Meal", Decimal("10")), ("Tip", Decimal("2"))))
self.assertEqual(result, (Decimal("20.50"), [("Meal", Decimal("20")), ("Fee", Decimal("0.5"))]))
def test_edit_action_splits_a_plain_transaction_and_clearing_keeps_amount(self):
self._edit('[{"name": "Principal", "amount": amount - 5}, {"name": "Fee", "amount": 5}]')
self.assertEqual(
self._run(self._transaction(amount=Decimal("100"))),
(Decimal("100"), [("Principal", Decimal("95")), ("Fee", Decimal("5"))]),
)
self.rule.transaction_actions.update(value="[]")
self.assertEqual(self._run(self._transaction(("A", Decimal("3")))), (Decimal("3"), []))
def test_invalid_parts_leave_the_transaction_alone(self):
transaction = self._transaction(("A", Decimal("3")))
for expression in ('"Fee"', '[{"name": "Fee"}]', '[{"name": "", "amount": 1}]',
'[{"name": "Refund", "amount": -1}]'):
with self.subTest(expression=expression):
self.rule.transaction_actions.all().delete()
self._edit(expression)
self.assertEqual(self._run(transaction), (Decimal("3"), [("A", Decimal("3"))]))
def test_transfers_are_skipped(self):
out, inc = self._transaction(amount=Decimal("5")), self._transaction(amount=Decimal("5"))
write_current_user(self.user)
with patch("apps.rules.signals.check_for_transaction_rules.defer"):
Transaction.objects.filter(pk=out.pk).update(linked_transaction=inc)
self._edit('[{"name": "Fee", "amount": 1}]')
self.assertEqual(self._run(out), (Decimal("5"), []))
def test_update_or_create_creates_a_transaction_from_its_parts(self):
UpdateOrCreateTransactionRuleAction.objects.create(
rule=self.rule, set_account="account_id", set_type="'EX'", set_date="date",
set_description="'Copy'", set_amount="999",
set_sub_transactions="sub_transactions + [{'name': 'Fee', 'amount': 1}]",
)
self._run(self._transaction(("A", Decimal("3"))))
copy = Transaction.objects.get(description="Copy")
self.assertEqual(copy.amount, Decimal("4"))
self.assertEqual(
list(copy.sub_transactions.values_list("name", "amount")),
[("A", Decimal("3")), ("Fee", Decimal("1"))],
)
+4
View File
@@ -86,6 +86,10 @@ def serialize_transaction(sender: Transaction, deleted: bool):
"notes": sender.notes, "notes": sender.notes,
"tags": list(sender.tags.values_list("id", "name")), "tags": list(sender.tags.values_list("id", "name")),
"entities": list(sender.entities.values_list("id", "name")), "entities": list(sender.entities.values_list("id", "name")),
"sub_transactions": [
{"name": name, "amount": str(amount)}
for name, amount in sender.sub_transactions.values_list("name", "amount")
],
"deleted": deleted, "deleted": deleted,
"internal_note": sender.internal_note, "internal_note": sender.internal_note,
"internal_id": sender.internal_id, "internal_id": sender.internal_id,
+8
View File
@@ -1,6 +1,7 @@
from django.contrib import admin from django.contrib import admin
from apps.transactions.models import ( from apps.transactions.models import (
AutoTaggerModel,
Transaction, Transaction,
TransactionCategory, TransactionCategory,
TransactionTag, TransactionTag,
@@ -79,3 +80,10 @@ class TransactionTagModelAdmin(SharedObjectModelAdmin):
@admin.register(TransactionEntity) @admin.register(TransactionEntity)
class TransactionEntityModelAdmin(SharedObjectModelAdmin): class TransactionEntityModelAdmin(SharedObjectModelAdmin):
pass pass
@admin.register(AutoTaggerModel)
class AutoTaggerModelAdmin(admin.ModelAdmin):
list_display = ["user", "transaction_count", "version", "trained_at"]
readonly_fields = ["trained_at", "transaction_count", "version"]
exclude = ["data"]
@@ -0,0 +1,21 @@
from apps.transactions.auto_tagging.classifier import (
MODEL_VERSION,
Prediction,
TrainingRecord,
TransactionAutoTagger,
)
from apps.transactions.auto_tagging.service import (
apply_auto_tagging,
ensure_model_fresh_async,
train_for_user,
)
__all__ = [
"MODEL_VERSION",
"Prediction",
"TrainingRecord",
"TransactionAutoTagger",
"apply_auto_tagging",
"ensure_model_fresh_async",
"train_for_user",
]
@@ -0,0 +1,427 @@
from __future__ import annotations
import heapq
import json
import math
import re
import unicodedata
import zlib
from collections import defaultdict
from dataclasses import dataclass, field
from decimal import Decimal
MODEL_VERSION = 1
MAX_TRAINING_DOCS = 5000
TOP_K = 10
MIN_SIMILARITY = 0.35
NEAR_EXACT_SIMILARITY = 0.85
MIN_SUPPORT_NEIGHBORS = 2
CATEGORY_CONFIDENCE_THRESHOLD = 0.65
MULTI_LABEL_CONFIDENCE_THRESHOLD = 0.60
# Strip dates (2025-01-20, 20/01/2025, 20/01) and installment counters (01/12)
_DATE_OR_INSTALLMENT_RE = re.compile(
r"\b\d{1,4}[-/.]\d{1,2}(?:[-/.]\d{1,4})?\b"
)
_WORD_RE = re.compile(r"[a-z0-9]+")
@dataclass(frozen=True)
class TrainingRecord:
description: str
notes: str
account_id: int
tx_type: str
amount: Decimal | float | int
category_id: int | None
tag_ids: tuple[int, ...] = ()
entity_ids: tuple[int, ...] = ()
@dataclass(frozen=True)
class Prediction:
category_id: int | None = None
tag_ids: tuple[int, ...] = ()
entity_ids: tuple[int, ...] = ()
@property
def is_empty(self) -> bool:
return (
self.category_id is None
and not self.tag_ids
and not self.entity_ids
)
def _normalize_text(text: str) -> str:
if not text:
return ""
decomposed = unicodedata.normalize("NFKD", text)
stripped = "".join(
ch for ch in decomposed if unicodedata.category(ch) != "Mn"
)
lowered = stripped.lower()
return _DATE_OR_INSTALLMENT_RE.sub(" ", lowered)
def _tokenize_text(text: str) -> list[str]:
normalized = _normalize_text(text)
if not normalized:
return []
return [
tok
for tok in _WORD_RE.findall(normalized)
if len(tok) >= 2 and not tok.isdigit()
]
def _add_text_features(
features: dict[str, float],
tokens: list[str],
*,
unigram_weight: float,
bigram_weight: float,
char_ngram_weight: float,
) -> None:
for tok in tokens:
features[tok] = features.get(tok, 0.0) + unigram_weight
if char_ngram_weight > 0.0 and len(tok) >= 6 and tok.isalpha():
for i in range(len(tok) - 3):
ngram = f"c:{tok[i : i + 4]}"
features[ngram] = features.get(ngram, 0.0) + char_ngram_weight
for i in range(len(tokens) - 1):
bigram = f"b:{tokens[i]}_{tokens[i + 1]}"
features[bigram] = features.get(bigram, 0.0) + bigram_weight
def _amount_bucket(amount: Decimal | float | int | None) -> int:
if amount is None:
return 0
val = abs(float(amount))
if val < 1.0:
return 0
return int(math.floor(math.log10(val))) + 1
def _extract_feature_parts(
description: str,
notes: str,
account_id: int,
tx_type: str,
amount: Decimal | float | int | None,
) -> tuple[dict[str, float], set[str], bool]:
"""Extract combined features plus the set of tokens from description.
Returns ``(features, desc_feature_keys, has_description)``.
"""
desc_tokens = _tokenize_text(description)
notes_tokens = _tokenize_text(notes)
if not desc_tokens and not notes_tokens:
return {}, set(), False
desc_features: dict[str, float] = {}
if desc_tokens:
_add_text_features(
desc_features,
desc_tokens,
unigram_weight=2.0,
bigram_weight=1.5,
char_ngram_weight=0.6,
)
features = dict(desc_features)
if notes_tokens:
_add_text_features(
features,
notes_tokens,
unigram_weight=1.0,
bigram_weight=0.75,
char_ngram_weight=0.0,
)
features[f"__type_{tx_type}"] = 0.5
features[f"__acct_{account_id}"] = 0.6
features[f"__amt_{_amount_bucket(amount)}"] = 0.4
return features, set(desc_features.keys()), bool(desc_tokens)
def extract_features(
description: str,
notes: str,
account_id: int,
tx_type: str,
amount: Decimal | float | int | None,
) -> dict[str, float]:
"""Extract weighted text + structured features for a transaction.
Returns an empty dict when neither description nor notes yields any
meaningful text tokens, preventing blind matches on account/type alone.
"""
features, _, _ = _extract_feature_parts(
description=description,
notes=notes,
account_id=account_id,
tx_type=tx_type,
amount=amount,
)
return features
@dataclass
class TransactionAutoTagger:
"""Zero-dependency TF-IDF + Cosine Similarity k-NN classifier."""
idf: dict[str, float] = field(default_factory=dict)
# Flat posting lists: [doc_idx_0, weight_0, doc_idx_1, weight_1, ...]
inverted_index: dict[str, list[float | int]] = field(default_factory=dict)
doc_categories: list[int | None] = field(default_factory=list)
doc_tags: list[list[int]] = field(default_factory=list)
doc_entities: list[list[int]] = field(default_factory=list)
doc_counts: list[int] = field(default_factory=list)
total_records: int = 0
version: int = MODEL_VERSION
@property
def num_docs(self) -> int:
return self.total_records or len(self.doc_categories)
@classmethod
def fit(cls, records: list[TrainingRecord]) -> TransactionAutoTagger:
# Collapse identical (features, labels) into weighted prototypes
prototypes: dict[tuple, int] = {}
doc_features: list[dict[str, float]] = []
doc_categories: list[int | None] = []
doc_tags: list[list[int]] = []
doc_entities: list[list[int]] = []
doc_counts: list[int] = []
df: dict[str, int] = defaultdict(int)
total_records = 0
for rec in records[:MAX_TRAINING_DOCS]:
feats = extract_features(
description=rec.description,
notes=rec.notes,
account_id=rec.account_id,
tx_type=rec.tx_type,
amount=rec.amount,
)
if not feats:
continue
total_records += 1
for tok in feats:
df[tok] += 1
sorted_tags = tuple(sorted(rec.tag_ids))
sorted_entities = tuple(sorted(rec.entity_ids))
sig = (
tuple(sorted(feats.items())),
rec.category_id,
sorted_tags,
sorted_entities,
)
existing_idx = prototypes.get(sig)
if existing_idx is not None:
doc_counts[existing_idx] += 1
else:
prototypes[sig] = len(doc_features)
doc_features.append(feats)
doc_categories.append(rec.category_id)
doc_tags.append(list(sorted_tags))
doc_entities.append(list(sorted_entities))
doc_counts.append(1)
if total_records == 0:
return cls()
idf = {
tok: round(math.log((1.0 + total_records) / (1.0 + count)) + 1.0, 4)
for tok, count in df.items()
}
inverted_index: dict[str, list[float | int]] = defaultdict(list)
for doc_idx, feats in enumerate(doc_features):
weights = {
tok: (1.0 + math.log(tf)) * idf[tok]
for tok, tf in feats.items()
}
norm = math.sqrt(sum(w * w for w in weights.values()))
if norm <= 0.0:
continue
for tok, w in weights.items():
postings = inverted_index[tok]
postings.append(doc_idx)
postings.append(round(w / norm, 4))
return cls(
idf=idf,
inverted_index=dict(inverted_index),
doc_categories=doc_categories,
doc_tags=doc_tags,
doc_entities=doc_entities,
doc_counts=doc_counts,
total_records=total_records,
version=MODEL_VERSION,
)
def to_bytes(self) -> bytes:
payload = {
"v": self.version,
"n": self.total_records,
"idf": self.idf,
"idx": self.inverted_index,
"cat": self.doc_categories,
"tag": self.doc_tags,
"ent": self.doc_entities,
"cnt": self.doc_counts,
}
raw = json.dumps(payload, separators=(",", ":")).encode("utf-8")
return zlib.compress(raw, level=6)
@classmethod
def from_bytes(cls, blob: bytes | memoryview) -> TransactionAutoTagger | None:
if not blob:
return None
try:
payload = json.loads(zlib.decompress(bytes(blob)).decode("utf-8"))
except (zlib.error, json.JSONDecodeError, UnicodeDecodeError):
return None
if payload.get("v") != MODEL_VERSION or "cnt" not in payload:
return None
return cls(
idf=payload["idf"],
inverted_index=payload["idx"],
doc_categories=payload["cat"],
doc_tags=payload["tag"],
doc_entities=payload["ent"],
doc_counts=payload["cnt"],
total_records=payload.get("n", len(payload["cat"])),
version=payload["v"],
)
def predict(
self,
*,
description: str,
notes: str,
account_id: int,
tx_type: str,
amount: Decimal | float | int | None,
allowed_category_ids: set[int] | None = None,
allowed_tag_ids: set[int] | None = None,
allowed_entity_ids: set[int] | None = None,
) -> Prediction:
if self.num_docs == 0:
return Prediction()
feats, desc_feature_keys, has_description = _extract_feature_parts(
description=description,
notes=notes,
account_id=account_id,
tx_type=tx_type,
amount=amount,
)
if not feats:
return Prediction()
query_weights: dict[str, float] = {}
matched_text_tokens = 0
unknown_idf = math.log(1.0 + self.num_docs) + 1.0
norm_sq = 0.0
for tok, tf in feats.items():
tok_idf = self.idf.get(tok)
if tok_idf is not None:
w = (1.0 + math.log(tf)) * tok_idf
query_weights[tok] = w
norm_sq += w * w
if not tok.startswith("__"):
matched_text_tokens += 1
elif not tok.startswith("__"):
# Penalize unknown description tokens so unrelated descriptions
# abstain, but don't let unique notes (e.g. itemized details)
# discard a matching description.
if not has_description or tok in desc_feature_keys:
w = (1.0 + math.log(tf)) * unknown_idf
norm_sq += w * w
if matched_text_tokens == 0 or norm_sq <= 0.0:
return Prediction()
norm = math.sqrt(norm_sq)
scores: dict[int, float] = defaultdict(float)
for tok, w in query_weights.items():
q_w = w / norm
postings = self.inverted_index.get(tok)
if postings:
for i in range(0, len(postings), 2):
scores[postings[i]] += q_w * postings[i + 1]
candidates = [
(sim, doc_idx)
for doc_idx, sim in scores.items()
if sim >= MIN_SIMILARITY
]
if not candidates:
return Prediction()
top_neighbors = heapq.nlargest(TOP_K, candidates, key=lambda x: x[0])
total_support = sum(self.doc_counts[doc_idx] for _, doc_idx in top_neighbors)
if total_support < MIN_SUPPORT_NEIGHBORS:
if top_neighbors[0][0] < NEAR_EXACT_SIMILARITY:
return Prediction()
total_weight = 0.0
cat_weights: dict[int, float] = defaultdict(float)
cat_present_weight = 0.0
tag_weights: dict[int, float] = defaultdict(float)
entity_weights: dict[int, float] = defaultdict(float)
for sim, doc_idx in top_neighbors:
count = self.doc_counts[doc_idx]
w = (sim * sim) * (1.0 + math.log(count))
total_weight += w
cat_id = self.doc_categories[doc_idx]
if cat_id is not None:
cat_present_weight += w
cat_weights[cat_id] += w
for tid in self.doc_tags[doc_idx]:
tag_weights[tid] += w
for eid in self.doc_entities[doc_idx]:
entity_weights[eid] += w
predicted_category: int | None = None
if cat_weights and (cat_present_weight / total_weight) >= 0.5:
best_cat, best_w = max(cat_weights.items(), key=lambda item: item[1])
if (best_w / cat_present_weight) >= CATEGORY_CONFIDENCE_THRESHOLD:
if allowed_category_ids is None or best_cat in allowed_category_ids:
predicted_category = best_cat
predicted_tags = tuple(
sorted(
tid
for tid, w in tag_weights.items()
if (w / total_weight) >= MULTI_LABEL_CONFIDENCE_THRESHOLD
and (allowed_tag_ids is None or tid in allowed_tag_ids)
)
)
predicted_entities = tuple(
sorted(
eid
for eid, w in entity_weights.items()
if (w / total_weight) >= MULTI_LABEL_CONFIDENCE_THRESHOLD
and (allowed_entity_ids is None or eid in allowed_entity_ids)
)
)
return Prediction(
category_id=predicted_category,
tag_ids=predicted_tags,
entity_ids=predicted_entities,
)
@@ -0,0 +1,238 @@
from __future__ import annotations
import logging
from collections import OrderedDict, defaultdict
from datetime import timedelta
from django.db.models import Q
from django.utils import timezone
from apps.common.middleware.thread_local import (
delete_current_user,
get_current_user,
write_current_user,
)
from apps.transactions.auto_tagging.classifier import (
MAX_TRAINING_DOCS,
MODEL_VERSION,
Prediction,
TrainingRecord,
TransactionAutoTagger,
)
from apps.transactions.models import (
AutoTaggerModel,
Transaction,
TransactionCategory,
TransactionEntity,
TransactionTag,
)
logger = logging.getLogger(__name__)
STALE_MODEL_AGE = timedelta(hours=1)
_TAGGER_CACHE_MAXSIZE = 4
_tagger_cache: OrderedDict[tuple[int, str, int], TransactionAutoTagger] = OrderedDict()
def _get_cached_tagger(model_obj: AutoTaggerModel) -> TransactionAutoTagger | None:
cache_key = (model_obj.user_id, model_obj.trained_at.isoformat(), model_obj.version)
cached = _tagger_cache.get(cache_key)
if cached is not None:
_tagger_cache.move_to_end(cache_key)
return cached
tagger = TransactionAutoTagger.from_bytes(model_obj.data)
if tagger is not None:
_tagger_cache[cache_key] = tagger
while len(_tagger_cache) > _TAGGER_CACHE_MAXSIZE:
_tagger_cache.popitem(last=False)
return tagger
def _with_user_context(user):
"""Context manager that temporarily sets the thread-local current user."""
class _UserContext:
def __init__(self, target_user):
self.target_user = target_user
self.previous_user = None
def __enter__(self):
self.previous_user = get_current_user()
write_current_user(self.target_user)
return self.target_user
def __exit__(self, exc_type, exc_val, exc_tb):
if self.previous_user is not None:
write_current_user(self.previous_user)
elif get_current_user() is not None:
delete_current_user()
return _UserContext(user)
def train_for_user(user) -> AutoTaggerModel:
"""Train and persist the auto-tagging model for a single user."""
with _with_user_context(user):
qs = (
Transaction.objects.filter(linked_transaction__isnull=True)
.exclude(Q(description="") & Q(notes=""))
.filter(
Q(category__isnull=False)
| Q(tags__isnull=False)
| Q(entities__isnull=False)
)
.distinct()
.order_by("-date", "-id")
.values_list(
"id",
"description",
"notes",
"account_id",
"type",
"amount",
"category_id",
)[:MAX_TRAINING_DOCS]
)
rows = list(qs)
tx_ids = [row[0] for row in rows]
tags_by_tx: dict[int, list[int]] = defaultdict(list)
entities_by_tx: dict[int, list[int]] = defaultdict(list)
if tx_ids:
for tx_id, tag_id in Transaction.tags.through.objects.filter(
transaction_id__in=tx_ids
).values_list("transaction_id", "transactiontag_id"):
tags_by_tx[tx_id].append(tag_id)
for tx_id, entity_id in Transaction.entities.through.objects.filter(
transaction_id__in=tx_ids
).values_list("transaction_id", "transactionentity_id"):
entities_by_tx[tx_id].append(entity_id)
records = [
TrainingRecord(
description=description or "",
notes=notes or "",
account_id=account_id,
tx_type=tx_type,
amount=amount,
category_id=category_id,
tag_ids=tuple(tags_by_tx.get(tx_id, ())),
entity_ids=tuple(entities_by_tx.get(tx_id, ())),
)
for tx_id, description, notes, account_id, tx_type, amount, category_id in rows
]
tagger = TransactionAutoTagger.fit(records)
model_obj, _ = AutoTaggerModel.objects.update_or_create(
user=user,
defaults={
"data": tagger.to_bytes(),
"transaction_count": tagger.num_docs,
"version": MODEL_VERSION,
},
)
return model_obj
def ensure_model_fresh_async(user_id: int) -> None:
"""Defer a background retrain if Procrastinate is available."""
from apps.transactions.tasks import train_user_auto_tagger
try:
train_user_auto_tagger.configure(
queueing_lock=f"train_auto_tagger_{user_id}"
).defer(user_id=user_id)
except Exception:
logger.debug(
"Could not defer auto-tagger training for user %s",
user_id,
exc_info=True,
)
def _load_or_train_tagger(user) -> TransactionAutoTagger | None:
model_obj = AutoTaggerModel.objects.filter(user=user).first()
if model_obj is None or model_obj.version != MODEL_VERSION:
model_obj = train_for_user(user)
elif timezone.now() - model_obj.trained_at > STALE_MODEL_AGE:
ensure_model_fresh_async(user.id)
return _get_cached_tagger(model_obj)
def apply_auto_tagging(instance: Transaction, user) -> bool:
"""Auto-assign category, tags, and entities to an unclassified transaction.
Only runs when:
- The user has auto_tagging enabled in UserSettings
- The transaction is not deleted and not part of a transfer
- The transaction currently has NO category, NO tags, and NO entities
"""
user_settings = getattr(user, "settings", None)
if user_settings is None or not user_settings.auto_tagging:
return False
if instance.deleted or instance.linked_transaction_id is not None:
return False
if (
instance.category_id is not None
or Transaction.tags.through.objects.filter(transaction_id=instance.pk).exists()
or Transaction.entities.through.objects.filter(
transaction_id=instance.pk
).exists()
):
return False
with _with_user_context(user):
tagger = _load_or_train_tagger(user)
if tagger is None or tagger.num_docs == 0:
return False
allowed_category_ids = set(
TransactionCategory.objects.filter(
active=True, auto_match=True
).values_list("id", flat=True)
)
allowed_tag_ids = set(
TransactionTag.objects.filter(
active=True, auto_match=True
).values_list("id", flat=True)
)
allowed_entity_ids = set(
TransactionEntity.objects.filter(
active=True, auto_match=True
).values_list("id", flat=True)
)
prediction: Prediction = tagger.predict(
description=instance.description,
notes=instance.notes,
account_id=instance.account_id,
tx_type=instance.type,
amount=instance.amount,
allowed_category_ids=allowed_category_ids,
allowed_tag_ids=allowed_tag_ids,
allowed_entity_ids=allowed_entity_ids,
)
if prediction.is_empty:
return False
if prediction.category_id is not None:
Transaction.all_objects.filter(pk=instance.pk).update(
emit_signal=False,
category_id=prediction.category_id,
)
instance.category_id = prediction.category_id
if prediction.tag_ids:
instance.tags.add(*prediction.tag_ids)
if prediction.entity_ids:
instance.entities.add(*prediction.entity_ids)
return True
+171 -16
View File
@@ -1,3 +1,5 @@
import datetime
import django_filters import django_filters
from apps.accounts.models import Account from apps.accounts.models import Account
from apps.common.fields.month_year import MonthYearFormField from apps.common.fields.month_year import MonthYearFormField
@@ -6,13 +8,15 @@ from apps.common.widgets.decimal import ArbitraryDecimalDisplayNumberInput
from apps.common.widgets.tom_select import TomSelectMultiple from apps.common.widgets.tom_select import TomSelectMultiple
from apps.currencies.models import Currency from apps.currencies.models import Currency
from apps.transactions.models import ( from apps.transactions.models import (
SubTransaction,
Transaction, Transaction,
TransactionCategory, TransactionCategory,
TransactionEntity, TransactionEntity,
TransactionTag, TransactionTag,
) )
from crispy_forms.helper import FormHelper from crispy_forms.helper import FormHelper
from crispy_forms.layout import Column, Field, Layout, Row from crispy_forms.layout import HTML, Column, Div, Field, Layout, Row
from dateutil.relativedelta import relativedelta
from django import forms from django import forms
from django.db.models import Q from django.db.models import Q
from django.utils.translation import gettext_lazy as _ from django.utils.translation import gettext_lazy as _
@@ -31,11 +35,26 @@ MUTE_STATUS_CHOICES = (
def content_filter(queryset, name, value): def content_filter(queryset, name, value):
queryset = queryset.filter( queryset = queryset.filter(
Q(description__icontains=value) | Q(notes__icontains=value) Q(description__icontains=value)
| Q(notes__icontains=value)
# a subquery, so a transaction with several matching parts isn't listed twice
| Q(pk__in=SubTransaction.objects.filter(name__icontains=value).values("transaction"))
) )
return queryset return queryset
def exclude_matching(positive_filter):
"""Turn a positive filter method into one that drops the rows it would keep."""
def method(queryset, name, value):
if not value:
return queryset
matching = positive_filter(queryset, name, value)
return queryset.exclude(pk__in=matching.values("pk"))
return method
class MonthYearFilter(Filter): class MonthYearFilter(Filter):
field_class = MonthYearFormField field_class = MonthYearFormField
@@ -46,6 +65,13 @@ class TransactionsFilter(django_filters.FilterSet):
"is_paid": {"1", "0"}, "is_paid": {"1", "0"},
"mute_status": {"active", "muted"}, "mute_status": {"active", "muted"},
} }
# How each date filter moves when a preset keeps its dates relative to today
relative_date_units = {
"date_start": "days",
"date_end": "days",
"reference_date_start": "months",
"reference_date_end": "months",
}
description = django_filters.CharFilter( description = django_filters.CharFilter(
label=_("Content"), label=_("Content"),
@@ -124,6 +150,42 @@ class TransactionsFilter(django_filters.FilterSet):
lookup_expr="lte", lookup_expr="lte",
label=_("Amount max"), label=_("Amount max"),
) )
exclude_description = django_filters.CharFilter(
label=_("Content"),
method=exclude_matching(content_filter),
widget=forms.TextInput(attrs={"type": "search"}),
)
exclude_account = django_filters.ModelMultipleChoiceFilter(
field_name="account__name",
queryset=Account.objects.all(),
to_field_name="name",
exclude=True,
label=_("Accounts"),
widget=TomSelectMultiple(checkboxes=True, remove_button=True, group_by="group"),
)
exclude_currency = django_filters.ModelMultipleChoiceFilter(
field_name="account__currency",
queryset=Currency.objects.all(),
to_field_name="id",
exclude=True,
label=_("Currencies"),
widget=TomSelectMultiple(checkboxes=True, remove_button=True),
)
exclude_category = django_filters.MultipleChoiceFilter(
label=_("Categories"),
widget=TomSelectMultiple(checkboxes=True, remove_button=True),
method="filter_exclude_category",
)
exclude_tags = django_filters.MultipleChoiceFilter(
label=_("Tags"),
widget=TomSelectMultiple(checkboxes=True, remove_button=True),
method="filter_exclude_tags",
)
exclude_entities = django_filters.MultipleChoiceFilter(
label=_("Entities"),
widget=TomSelectMultiple(checkboxes=True, remove_button=True),
method="filter_exclude_entities",
)
class Meta: class Meta:
model = Transaction model = Transaction
@@ -178,21 +240,53 @@ class TransactionsFilter(django_filters.FilterSet):
"mute_status", "mute_status",
template="transactions/widgets/transaction_type_filter_buttons.html", template="transactions/widgets/transaction_type_filter_buttons.html",
), ),
Field("description"), Div(
Row(Column("date_start"), Column("date_end")), HTML(
Row( '<div role="tablist" class="tabs tabs-border mb-2">'
Column("reference_date_start"), '<button type="button" role="tab" class="tab" '
Column("reference_date_end"), ":class=\"{'tab-active': tab === 'include'}\" "
":aria-selected=\"tab === 'include'\" "
"@click=\"tab = 'include'\">"
f"{_('Include')}</button>"
'<button type="button" role="tab" class="tab" '
":class=\"{'tab-active': tab === 'exclude'}\" "
":aria-selected=\"tab === 'exclude'\" "
"@click=\"tab = 'exclude'\">"
f"{_('Exclude')}</button>"
"</div>"
),
Div(
Field("description"),
Row(Column("date_start"), Column("date_end")),
Row(
Column("reference_date_start"),
Column("reference_date_end"),
),
Row(
Column("from_amount"),
Column("to_amount"),
),
Field("account", size=1),
Field("currency", size=1),
Field("category", size=1),
Field("tags", size=1),
Field("entities", size=1),
role="tabpanel",
x_show="tab === 'include'",
),
Div(
Field("exclude_description"),
Field("exclude_account", size=1),
Field("exclude_currency", size=1),
Field("exclude_category", size=1),
Field("exclude_tags", size=1),
Field("exclude_entities", size=1),
role="tabpanel",
x_show="tab === 'exclude'",
x_cloak="",
),
x_data="{ tab: 'include' }",
), ),
Row(
Column("from_amount"),
Column("to_amount"),
),
Field("account", size=1),
Field("currency", size=1),
Field("category", size=1),
Field("tags", size=1),
Field("entities", size=1),
) )
self.form.fields["to_amount"].widget = ArbitraryDecimalDisplayNumberInput() self.form.fields["to_amount"].widget = ArbitraryDecimalDisplayNumberInput()
@@ -201,6 +295,7 @@ class TransactionsFilter(django_filters.FilterSet):
self.form.fields["date_end"].widget = AirDatePickerInput() self.form.fields["date_end"].widget = AirDatePickerInput()
self.form.fields["account"].queryset = Account.objects.all() self.form.fields["account"].queryset = Account.objects.all()
self.form.fields["exclude_account"].queryset = Account.objects.all()
category_choices = list( category_choices = list(
TransactionCategory.objects.values_list("name", "name").order_by("name") TransactionCategory.objects.values_list("name", "name").order_by("name")
) )
@@ -209,11 +304,15 @@ class TransactionsFilter(django_filters.FilterSet):
("uncategorized", _("Uncategorized")), ("uncategorized", _("Uncategorized")),
] ]
self.form.fields["category"].choices = custom_choices + category_choices self.form.fields["category"].choices = custom_choices + category_choices
self.form.fields["exclude_category"].choices = (
custom_choices + category_choices
)
tag_choices = list( tag_choices = list(
TransactionTag.objects.values_list("name", "name").order_by("name") TransactionTag.objects.values_list("name", "name").order_by("name")
) )
custom_tag_choices = [("any", _("Tagged")), ("untagged", _("Untagged"))] custom_tag_choices = [("any", _("Tagged")), ("untagged", _("Untagged"))]
self.form.fields["tags"].choices = custom_tag_choices + tag_choices self.form.fields["tags"].choices = custom_tag_choices + tag_choices
self.form.fields["exclude_tags"].choices = custom_tag_choices + tag_choices
entity_choices = list( entity_choices = list(
TransactionEntity.objects.values_list("name", "name").order_by("name") TransactionEntity.objects.values_list("name", "name").order_by("name")
) )
@@ -222,6 +321,9 @@ class TransactionsFilter(django_filters.FilterSet):
("no_entity", _("No entity")), ("no_entity", _("No entity")),
] ]
self.form.fields["entities"].choices = custom_entity_choices + entity_choices self.form.fields["entities"].choices = custom_entity_choices + entity_choices
self.form.fields["exclude_entities"].choices = (
custom_entity_choices + entity_choices
)
@property @property
def has_active_filters(self): def has_active_filters(self):
@@ -240,6 +342,50 @@ class TransactionsFilter(django_filters.FilterSet):
return True return True
return False return False
def shift_dates(self, since, until):
"""
Move the date filters forward by the time elapsed between two dates.
Dates move by days and reference months by months, so a preset saved
from today, or for the current month, keeps meaning that later on.
A date range of whole months, like the 1st to the 31st, moves by
months instead, so it stays whole months.
Dates are read as ISO, or as typed in the user's format by presets saved
before they were stored as ISO, and written back in the user's format.
"""
months = (until.year - since.year) * 12 + until.month - since.month
shifts = {
"days": relativedelta(days=(until - since).days),
"months": relativedelta(months=months),
}
dates = {}
for name in self.relative_date_units:
for value in (self.data.get(name), self.form[name].data):
try:
dates[name] = datetime.date.fromisoformat(value)
break
except (TypeError, ValueError):
pass
start, end = dates.get("date_start"), dates.get("date_end")
whole_months = (
start
and end
and start.day == 1
and (end + datetime.timedelta(days=1)).day == 1
)
for name, value in dates.items():
shift = shifts[self.relative_date_units[name]]
if whole_months and name == "date_start":
shift = shifts["months"]
elif whole_months and name == "date_end":
# day=31 lands on the last day of the shifted month, whatever its length
shift = shifts["months"] + relativedelta(day=31)
self.data[name] = self.form.fields[name].widget.format_value(value + shift)
@staticmethod @staticmethod
def filter_category(queryset, name, value): def filter_category(queryset, name, value):
if not value: if not value:
@@ -309,6 +455,15 @@ class TransactionsFilter(django_filters.FilterSet):
return queryset return queryset
def filter_exclude_category(self, queryset, name, value):
return exclude_matching(self.filter_category)(queryset, name, value)
def filter_exclude_tags(self, queryset, name, value):
return exclude_matching(self.filter_tags)(queryset, name, value)
def filter_exclude_entities(self, queryset, name, value):
return exclude_matching(self.filter_entities)(queryset, name, value)
@staticmethod @staticmethod
def filter_mute_status(queryset, name, value): def filter_mute_status(queryset, name, value):
from apps.common.middleware.thread_local import get_current_user from apps.common.middleware.thread_local import get_current_user
+114 -10
View File
@@ -17,6 +17,7 @@ from apps.transactions.models import (
TransactionAttachment, TransactionAttachment,
QuickTransaction, QuickTransaction,
RecurringTransaction, RecurringTransaction,
SubTransaction,
Transaction, Transaction,
TransactionCategory, TransactionCategory,
TransactionEntity, TransactionEntity,
@@ -54,7 +55,89 @@ class MultipleFileField(forms.FileField):
return [] return []
class TransactionForm(forms.ModelForm): class SubTransactionForm(forms.ModelForm):
class Meta:
model = SubTransaction
fields = ["name", "amount"]
widgets = {
"name": forms.TextInput(
attrs={"class": "input join-item flex-1", "placeholder": _("Name")}
),
}
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
# Built per request so the mask uses the user's number format
self.fields["amount"].widget = ArbitraryDecimalDisplayNumberInput(
attrs={
"class": "input join-item w-2/5",
"inputmode": "decimal",
"placeholder": _("Amount"),
}
)
class SubTransactionsFormMixin:
"""Adds sub-transactions to a transaction or template form; while there are any,
the amount field is their sum. Transfers don't take them.
Rendered by transactions/widgets/sub_transactions.html, placed after the amount.
"""
sub_amount_field = "amount"
def __init__(self, *args, sub_transactions_initial=None, **kwargs):
super().__init__(*args, **kwargs)
data = self.data if self.is_bound else None
if data is not None and "sub-TOTAL_FORMS" not in data:
# Posts without the formset (e.g. shortcuts to /add/) leave sub-transactions alone
data = {"sub-TOTAL_FORMS": "0", "sub-INITIAL_FORMS": "0"}
initial = sub_transactions_initial or []
self.sub_transactions = forms.inlineformset_factory(
type(self.instance),
SubTransaction,
form=SubTransactionForm,
extra=len(initial),
can_delete=True,
)(data, instance=self.instance, prefix="sub", initial=initial)
# With sub-transactions the amount is their sum, see clean()
self.fields[self.sub_amount_field].required = False
def _takes_sub_transactions(self):
return self.cleaned_data.get("type") != "TR"
def clean(self):
cleaned_data = super().clean()
field = self.sub_amount_field
if self._takes_sub_transactions() and self.sub_transactions.is_valid():
amounts = [
f.cleaned_data["amount"]
for f in self.sub_transactions.forms
if f.cleaned_data and not f.cleaned_data.get("DELETE")
]
if amounts:
cleaned_data[field] = sum(amounts)
if cleaned_data.get(field) is None and field not in self.errors:
self.add_error(field, self.fields[field].error_messages["required"])
return cleaned_data
def is_valid(self):
return super().is_valid() and (
not self._takes_sub_transactions() or self.sub_transactions.is_valid()
)
def save(self, *args, **kwargs):
with db_transaction.atomic():
instance = super().save(*args, **kwargs)
if self._takes_sub_transactions():
self.sub_transactions.instance = instance
self.sub_transactions.save()
else:
instance.sub_transactions.all().delete()
return instance
class TransactionForm(SubTransactionsFormMixin, forms.ModelForm):
category = DynamicModelChoiceField( category = DynamicModelChoiceField(
create_field="name", create_field="name",
model=TransactionCategory, model=TransactionCategory,
@@ -164,6 +247,7 @@ class TransactionForm(forms.ModelForm):
), ),
"description", "description",
Field("amount", inputmode="decimal"), Field("amount", inputmode="decimal"),
HTML('{% include "transactions/widgets/sub_transactions.html" %}'),
Row( Row(
Column("category"), Column("category"),
Column("tags"), Column("tags"),
@@ -187,6 +271,7 @@ class TransactionForm(forms.ModelForm):
), ),
"description", "description",
Field("amount", inputmode="decimal"), Field("amount", inputmode="decimal"),
HTML('{% include "transactions/widgets/sub_transactions.html" %}'),
Accordion( Accordion(
AccordionGroup( AccordionGroup(
_("More"), _("More"),
@@ -322,7 +407,7 @@ def _type_aware(form, *layout):
) )
class QuickTransactionForm(forms.ModelForm): class QuickTransactionForm(SubTransactionsFormMixin, forms.ModelForm):
category = DynamicModelChoiceField( category = DynamicModelChoiceField(
create_field="name", create_field="name",
model=TransactionCategory, model=TransactionCategory,
@@ -432,6 +517,7 @@ class QuickTransactionForm(forms.ModelForm):
), ),
"description", "description",
Field("amount", inputmode="decimal"), Field("amount", inputmode="decimal"),
HTML('{% include "transactions/widgets/sub_transactions.html" %}'),
_transfer_fields(self), _transfer_fields(self),
Row( Row(
Column("category"), Column("category"),
@@ -833,7 +919,9 @@ class TransferForm(forms.Form):
return from_transaction, to_transaction return from_transaction, to_transaction
class InstallmentPlanForm(forms.ModelForm): class InstallmentPlanForm(SubTransactionsFormMixin, forms.ModelForm):
sub_amount_field = "installment_amount"
account = forms.ModelChoiceField( account = forms.ModelChoiceField(
queryset=Account.objects.filter(is_archived=False), queryset=Account.objects.filter(is_archived=False),
label=_("Account"), label=_("Account"),
@@ -956,6 +1044,7 @@ class InstallmentPlanForm(forms.ModelForm):
Column("recurrence", css_class="col-span-12 md:col-span-4"), Column("recurrence", css_class="col-span-12 md:col-span-4"),
), ),
"installment_amount", "installment_amount",
HTML('{% include "transactions/widgets/sub_transactions.html" %}'),
Row( Row(
Column("category"), Column("category"),
Column("tags"), Column("tags"),
@@ -989,6 +1078,7 @@ class InstallmentPlanForm(forms.ModelForm):
# Take the amount set on the form, even if already paid # Take the amount set on the form, even if already paid
source_transaction.amount = instance.installment_amount source_transaction.amount = instance.installment_amount
source_transaction.save() source_transaction.save()
source_transaction.copy_sub_transactions_from(instance)
instance.update_transactions() instance.update_transactions()
elif is_new: elif is_new:
instance.create_transactions() instance.create_transactions()
@@ -1001,7 +1091,7 @@ class InstallmentPlanForm(forms.ModelForm):
class TransactionTagForm(forms.ModelForm): class TransactionTagForm(forms.ModelForm):
class Meta: class Meta:
model = TransactionTag model = TransactionTag
fields = ["name", "active"] fields = ["name", "active", "auto_match"]
labels = {"name": _("Tag name")} labels = {"name": _("Tag name")}
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
@@ -1010,7 +1100,11 @@ class TransactionTagForm(forms.ModelForm):
self.helper = FormHelper() self.helper = FormHelper()
self.helper.form_tag = False self.helper.form_tag = False
self.helper.form_method = "post" self.helper.form_method = "post"
self.helper.layout = Layout(Field("name", css_class="mb-3"), Switch("active")) self.helper.layout = Layout(
Field("name", css_class="mb-3"),
Switch("active"),
Switch("auto_match"),
)
if self.instance and self.instance.pk: if self.instance and self.instance.pk:
self.helper.layout.append( self.helper.layout.append(
@@ -1029,7 +1123,7 @@ class TransactionTagForm(forms.ModelForm):
class TransactionEntityForm(forms.ModelForm): class TransactionEntityForm(forms.ModelForm):
class Meta: class Meta:
model = TransactionEntity model = TransactionEntity
fields = ["name", "active"] fields = ["name", "active", "auto_match"]
labels = {"name": _("Entity name")} labels = {"name": _("Entity name")}
def __init__(self, *args, **kwargs): def __init__(self, *args, **kwargs):
@@ -1038,7 +1132,11 @@ class TransactionEntityForm(forms.ModelForm):
self.helper = FormHelper() self.helper = FormHelper()
self.helper.form_tag = False self.helper.form_tag = False
self.helper.form_method = "post" self.helper.form_method = "post"
self.helper.layout = Layout(Field("name"), Switch("active")) self.helper.layout = Layout(
Field("name"),
Switch("active"),
Switch("auto_match"),
)
if self.instance and self.instance.pk: if self.instance and self.instance.pk:
self.helper.layout.append( self.helper.layout.append(
@@ -1057,7 +1155,7 @@ class TransactionEntityForm(forms.ModelForm):
class TransactionCategoryForm(forms.ModelForm): class TransactionCategoryForm(forms.ModelForm):
class Meta: class Meta:
model = TransactionCategory model = TransactionCategory
fields = ["name", "mute", "active"] fields = ["name", "mute", "active", "auto_match"]
labels = {"name": _("Category name")} labels = {"name": _("Category name")}
help_texts = { help_texts = {
"mute": _("Muted categories won't be displayed on monthly summaries") "mute": _("Muted categories won't be displayed on monthly summaries")
@@ -1069,7 +1167,12 @@ class TransactionCategoryForm(forms.ModelForm):
self.helper = FormHelper() self.helper = FormHelper()
self.helper.form_tag = False self.helper.form_tag = False
self.helper.form_method = "post" self.helper.form_method = "post"
self.helper.layout = Layout(Field("name"), Switch("mute"), Switch("active")) self.helper.layout = Layout(
Field("name"),
Switch("mute"),
Switch("active"),
Switch("auto_match"),
)
if self.instance and self.instance.pk: if self.instance and self.instance.pk:
self.helper.layout.append( self.helper.layout.append(
@@ -1085,7 +1188,7 @@ class TransactionCategoryForm(forms.ModelForm):
) )
class RecurringTransactionForm(forms.ModelForm): class RecurringTransactionForm(SubTransactionsFormMixin, forms.ModelForm):
account = forms.ModelChoiceField( account = forms.ModelChoiceField(
queryset=Account.objects.filter(is_archived=False), queryset=Account.objects.filter(is_archived=False),
label=_("Account"), label=_("Account"),
@@ -1209,6 +1312,7 @@ class RecurringTransactionForm(forms.ModelForm):
"description", "description",
Switch("add_description_to_transaction"), Switch("add_description_to_transaction"),
"amount", "amount",
HTML('{% include "transactions/widgets/sub_transactions.html" %}'),
_transfer_fields(self), _transfer_fields(self),
Row( Row(
Column("category"), Column("category"),
@@ -0,0 +1,34 @@
# Generated by Django 5.2.17 on 2026-11-13 22:22
import apps.transactions.validators
import django.db.models.deletion
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('transactions', '0052_transfer_templates'),
]
operations = [
migrations.CreateModel(
name='SubTransaction',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('name', models.CharField(max_length=255, verbose_name='Name')),
('amount', models.DecimalField(decimal_places=30, max_digits=42, validators=[apps.transactions.validators.validate_non_negative, apps.transactions.validators.validate_decimal_places], verbose_name='Amount')),
('installment_plan', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='sub_transactions', to='transactions.installmentplan')),
('quick_transaction', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='sub_transactions', to='transactions.quicktransaction')),
('recurring_transaction', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='sub_transactions', to='transactions.recurringtransaction')),
('transaction', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='sub_transactions', to='transactions.transaction', verbose_name='Transaction')),
],
options={
'verbose_name': 'Sub-transaction',
'verbose_name_plural': 'Sub-transactions',
'db_table': 'sub_transactions',
'ordering': ['id'],
'constraints': [models.CheckConstraint(condition=models.Q(models.Q(('installment_plan__isnull', True), ('quick_transaction__isnull', True), ('recurring_transaction__isnull', True), ('transaction__isnull', False)), models.Q(('installment_plan__isnull', True), ('quick_transaction__isnull', True), ('recurring_transaction__isnull', False), ('transaction__isnull', True)), models.Q(('installment_plan__isnull', False), ('quick_transaction__isnull', True), ('recurring_transaction__isnull', True), ('transaction__isnull', True)), models.Q(('installment_plan__isnull', True), ('quick_transaction__isnull', False), ('recurring_transaction__isnull', True), ('transaction__isnull', True)), _connector='OR'), name='sub_transaction_has_one_parent')],
},
),
]
@@ -0,0 +1,15 @@
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("transactions", "0053_sub_transaction"),
]
operations = [
migrations.AddField(
model_name="filterpreset",
name="relative_to",
field=models.DateField(blank=True, null=True),
),
]
@@ -0,0 +1,47 @@
# Generated by Django 5.2.17 on 2026-11-19 21:22
import django.db.models.deletion
from django.conf import settings
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('transactions', '0054_filterpreset_relative_to'),
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.AddField(
model_name='transactioncategory',
name='auto_match',
field=models.BooleanField(default=True, help_text='Allow auto-tagging to automatically assign this category based on past transactions', verbose_name='Auto-match'),
),
migrations.AddField(
model_name='transactionentity',
name='auto_match',
field=models.BooleanField(default=True, help_text='Allow auto-tagging to automatically assign this entity based on past transactions', verbose_name='Auto-match'),
),
migrations.AddField(
model_name='transactiontag',
name='auto_match',
field=models.BooleanField(default=True, help_text='Allow auto-tagging to automatically assign this tag based on past transactions', verbose_name='Auto-match'),
),
migrations.CreateModel(
name='AutoTaggerModel',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('data', models.BinaryField(default=b'')),
('transaction_count', models.PositiveIntegerField(default=0)),
('version', models.PositiveSmallIntegerField(default=1)),
('trained_at', models.DateTimeField(auto_now=True)),
('user', models.OneToOneField(on_delete=django.db.models.deletion.CASCADE, related_name='auto_tagger_model', to=settings.AUTH_USER_MODEL)),
],
options={
'verbose_name': 'Auto-Tagger Model',
'verbose_name_plural': 'Auto-Tagger Models',
'db_table': 'auto_tagger_models',
},
),
]
+293 -56
View File
@@ -1,5 +1,8 @@
import calendar
import decimal import decimal
import functools
import logging import logging
import operator
import uuid import uuid
from copy import deepcopy from copy import deepcopy
from pathlib import Path from pathlib import Path
@@ -44,6 +47,9 @@ class FilterPreset(models.Model):
) )
name = models.CharField(max_length=100) name = models.CharField(max_length=100)
parameters = models.JSONField(default=dict) parameters = models.JSONField(default=dict)
# When set, the saved dates are moved along with today when the preset is
# applied, keeping the distance they had to this date
relative_to = models.DateField(null=True, blank=True)
class Meta: class Meta:
ordering = ["name", "id"] ordering = ["name", "id"]
@@ -255,6 +261,13 @@ class TransactionCategory(SharedObject):
"Deactivated categories won't be able to be selected when creating new transactions" "Deactivated categories won't be able to be selected when creating new transactions"
), ),
) )
auto_match = models.BooleanField(
default=True,
verbose_name=_("Auto-match"),
help_text=_(
"Allow auto-tagging to automatically assign this category based on past transactions"
),
)
objects = SharedObjectManager() objects = SharedObjectManager()
all_objects = models.Manager() # Unfiltered manager all_objects = models.Manager() # Unfiltered manager
@@ -279,6 +292,13 @@ class TransactionTag(SharedObject):
"Deactivated tags won't be able to be selected when creating new transactions" "Deactivated tags won't be able to be selected when creating new transactions"
), ),
) )
auto_match = models.BooleanField(
default=True,
verbose_name=_("Auto-match"),
help_text=_(
"Allow auto-tagging to automatically assign this tag based on past transactions"
),
)
objects = SharedObjectManager() objects = SharedObjectManager()
all_objects = models.Manager() # Unfiltered manager all_objects = models.Manager() # Unfiltered manager
@@ -303,6 +323,13 @@ class TransactionEntity(SharedObject):
"Deactivated entities won't be able to be selected when creating new transactions" "Deactivated entities won't be able to be selected when creating new transactions"
), ),
) )
auto_match = models.BooleanField(
default=True,
verbose_name=_("Auto-match"),
help_text=_(
"Allow auto-tagging to automatically assign this entity based on past transactions"
),
)
objects = SharedObjectManager() objects = SharedObjectManager()
all_objects = models.Manager() # Unfiltered manager all_objects = models.Manager() # Unfiltered manager
@@ -443,9 +470,10 @@ class Transaction(OwnedObject):
if isinstance(self.amount, (str, int, float)): if isinstance(self.amount, (str, int, float)):
self.amount = decimal.Decimal(str(self.amount)) self.amount = decimal.Decimal(str(self.amount))
self.amount = truncate_decimal( if self.amount is not None: # missing amount is reported by field validation
value=self.amount, decimal_places=account.currency.decimal_places self.amount = truncate_decimal(
) value=self.amount, decimal_places=account.currency.decimal_places
)
# Normalize reference_date # Normalize reference_date
if self.reference_date: if self.reference_date:
@@ -571,35 +599,50 @@ class Transaction(OwnedObject):
def hard_delete(self, *args, **kwargs): def hard_delete(self, *args, **kwargs):
super().delete(*args, **kwargs) super().delete(*args, **kwargs)
@property
def has_sub_transactions(self):
return bool(self.pk) and self.sub_transactions.exists()
@property
def saved_sub_transactions(self):
"""Sub-transactions, or none for an unsaved preview (which can't query them)."""
return self.sub_transactions.all() if self.pk else []
def copy_sub_transactions_from(self, source):
"""Makes this transaction's sub-transactions copies of ``source``'s (a
transaction or a template that generates them)."""
self.sub_transactions.all().delete()
SubTransaction.objects.bulk_create(
SubTransaction(transaction=self, name=sub.name, amount=sub.amount)
for sub in source.sub_transactions.all()
)
def exchanged_amount(self): def exchanged_amount(self):
if self.account.exchange_currency: return self._exchanged_to(
converted_amount, prefix, suffix, decimal_places = convert( self.account.exchange_currency or self.account.currency.exchange_currency
self.amount, )
to_currency=self.account.exchange_currency,
from_currency=self.account.currency, def displayed_exchanged_amount(self):
date=self.date, """The exchanged amount to display, or None when it's hidden."""
) return self._exchanged_to(self.account.displayed_exchange_currency)
if converted_amount:
return { def _exchanged_to(self, exchange_currency):
"amount": converted_amount, if not exchange_currency:
"prefix": prefix, return None
"suffix": suffix,
"decimal_places": decimal_places, converted_amount, prefix, suffix, decimal_places = convert(
} self.amount,
elif self.account.currency.exchange_currency: to_currency=exchange_currency,
converted_amount, prefix, suffix, decimal_places = convert( from_currency=self.account.currency,
self.amount, date=self.date,
to_currency=self.account.currency.exchange_currency, )
from_currency=self.account.currency, if converted_amount:
date=self.date, return {
) "amount": converted_amount,
if converted_amount: "prefix": prefix,
return { "suffix": suffix,
"amount": converted_amount, "decimal_places": decimal_places,
"prefix": prefix, }
"suffix": suffix,
"decimal_places": decimal_places,
}
return None return None
@@ -660,6 +703,81 @@ class Transaction(OwnedObject):
return new_obj return new_obj
SUB_TRANSACTION_PARENTS = (
"transaction",
"recurring_transaction",
"installment_plan",
"quick_transaction",
)
class SubTransaction(models.Model):
"""A named part of a transaction, or of a template that generates them; when
present, the parts add up to its amount. Exactly one parent is set.
Soft deletion follows the parent: sub-transactions are only reached through it.
"""
transaction = models.ForeignKey(
Transaction,
on_delete=models.CASCADE,
null=True,
blank=True,
related_name="sub_transactions",
verbose_name=_("Transaction"),
)
recurring_transaction = models.ForeignKey(
"RecurringTransaction",
on_delete=models.CASCADE,
null=True,
blank=True,
related_name="sub_transactions",
)
installment_plan = models.ForeignKey(
"InstallmentPlan",
on_delete=models.CASCADE,
null=True,
blank=True,
related_name="sub_transactions",
)
quick_transaction = models.ForeignKey(
"QuickTransaction",
on_delete=models.CASCADE,
null=True,
blank=True,
related_name="sub_transactions",
)
name = models.CharField(max_length=255, verbose_name=_("Name"))
amount = models.DecimalField(
max_digits=42,
decimal_places=30,
verbose_name=_("Amount"),
validators=[validate_non_negative, validate_decimal_places],
)
class Meta:
verbose_name = _("Sub-transaction")
verbose_name_plural = _("Sub-transactions")
db_table = "sub_transactions"
ordering = ["id"]
constraints = [
models.CheckConstraint(
# One of: only "transaction" set, only "recurring_transaction" set, ...
condition=functools.reduce(
operator.or_,
(
Q(**{f"{p}__isnull": p != parent for p in SUB_TRANSACTION_PARENTS})
for parent in SUB_TRANSACTION_PARENTS
),
),
name="sub_transaction_has_one_parent",
)
]
def __str__(self):
return self.name
class TransactionAttachment(models.Model): class TransactionAttachment(models.Model):
id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False) id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
transaction = models.ForeignKey( transaction = models.ForeignKey(
@@ -857,6 +975,7 @@ class InstallmentPlan(models.Model):
) )
new_transaction.tags.set(self.tags.all()) new_transaction.tags.set(self.tags.all())
new_transaction.entities.set(self.entities.all()) new_transaction.entities.set(self.entities.all())
new_transaction.copy_sub_transactions_from(self)
@transaction.atomic @transaction.atomic
def update_transactions(self): def update_transactions(self):
@@ -898,6 +1017,7 @@ class InstallmentPlan(models.Model):
not existing_transaction.is_paid not existing_transaction.is_paid
): # Don't update value for paid transactions ): # Don't update value for paid transactions
existing_transaction.amount = self.installment_amount existing_transaction.amount = self.installment_amount
existing_transaction.copy_sub_transactions_from(self)
existing_transaction.save() existing_transaction.save()
@@ -924,6 +1044,7 @@ class InstallmentPlan(models.Model):
) )
new_transaction.tags.set(self.tags.all()) new_transaction.tags.set(self.tags.all())
new_transaction.entities.set(self.entities.all()) new_transaction.entities.set(self.entities.all())
new_transaction.copy_sub_transactions_from(self)
# Remove any extra transactions that are no longer part of the plan # Remove any extra transactions that are no longer part of the plan
self.transactions.filter( self.transactions.filter(
@@ -1129,6 +1250,7 @@ class RecurringTransaction(TransferTemplate):
# different one, and the scoped default manager would hide private rows. # different one, and the scoped default manager would hide private rows.
created_transaction.tags.set(self.tags(manager="all_objects").all()) created_transaction.tags.set(self.tags(manager="all_objects").all())
created_transaction.entities.set(self.entities(manager="all_objects").all()) created_transaction.entities.set(self.entities(manager="all_objects").all())
created_transaction.copy_sub_transactions_from(self)
self.complete_transfer(created_transaction) self.complete_transfer(created_transaction)
def adopt_transaction(self, existing_transaction, date, reference_date): def adopt_transaction(self, existing_transaction, date, reference_date):
@@ -1171,6 +1293,7 @@ class RecurringTransaction(TransferTemplate):
existing_transaction.save() existing_transaction.save()
existing_transaction.tags.set(self.tags(manager="all_objects").all()) existing_transaction.tags.set(self.tags(manager="all_objects").all())
existing_transaction.entities.set(self.entities(manager="all_objects").all()) existing_transaction.entities.set(self.entities(manager="all_objects").all())
existing_transaction.copy_sub_transactions_from(self)
def get_recurrence_delta(self): def get_recurrence_delta(self):
if self.recurrence_type == self.RecurrenceType.DAY: if self.recurrence_type == self.RecurrenceType.DAY:
@@ -1183,7 +1306,21 @@ class RecurringTransaction(TransferTemplate):
return relativedelta(years=self.recurrence_interval) return relativedelta(years=self.recurrence_interval)
def get_next_date(self, current_date): def get_next_date(self, current_date):
return current_date + self.get_recurrence_delta() next_date = current_date + self.get_recurrence_delta()
# A start on the 29th-31st is clamped in shorter months; return to that day
# afterwards instead of keeping the clamped one.
if (
self.recurrence_type
in (self.RecurrenceType.MONTH, self.RecurrenceType.YEAR)
and self.start_date.day > current_date.day
and current_date.day
== calendar.monthrange(current_date.year, current_date.month)[1]
):
last_day = calendar.monthrange(next_date.year, next_date.month)[1]
next_date = next_date.replace(day=min(self.start_date.day, last_day))
return next_date
@classmethod @classmethod
def generate_upcoming_transactions(cls): def generate_upcoming_transactions(cls):
@@ -1246,36 +1383,116 @@ class RecurringTransaction(TransferTemplate):
""" """
Updates all unpaid transactions associated with this RecurringTransaction. Updates all unpaid transactions associated with this RecurringTransaction.
Only unpaid transactions (`is_paid=False`) are modified. Updates fields like Only unpaid transactions (`is_paid=False`) are modified. They keep their dates
amount, description, category, notes, and many-to-many relationships (tags, entities). and take the account, type, amount, description, category, notes, tags and
entities of the recurrence; a transfer's "to" side takes the "to" account and
amount. Turning the recurrence into a transfer adds the missing "to" sides,
and turning it into an income or expense removes them. Unpaid ones after a
new end date are removed.
""" """
unpaid_transactions = self.transactions.filter(is_paid=False) self._remove_occurrences_after_end()
unpaid_transactions = list(
self.transactions.filter(is_paid=False).select_related(
"linked_transaction"
)
)
to_side_ids = {
existing_transaction.pk
for existing_transaction in unpaid_transactions
if self._is_to_side(existing_transaction)
}
for existing_transaction in unpaid_transactions: for existing_transaction in unpaid_transactions:
# Update fields based on RecurringTransaction if existing_transaction.pk in to_side_ids:
is_to_side = ( if self.is_transfer:
existing_transaction.linked_transaction_id self._apply_to(
and existing_transaction.type == Transaction.Type.INCOME existing_transaction,
self.to_account,
Transaction.Type.INCOME,
self.to_amount or self.amount,
existing_transaction.date,
existing_transaction.reference_date,
)
# Otherwise its unpaid "from" side removes it below
continue
if (
not self.is_transfer
and existing_transaction.linked_transaction_id in to_side_ids
):
to_side = existing_transaction.linked_transaction
existing_transaction.linked_transaction = None
to_side.linked_transaction = None
to_side.delete(_with_partner=False)
partner = existing_transaction.linked_transaction
if (
partner
and partner.recurring_transaction_id != self.pk
and (
{self.transaction_type, partner.type}
!= {Transaction.Type.INCOME, Transaction.Type.EXPENSE}
or self.account_id == partner.account_id
)
):
# A link made elsewhere (e.g. a DCA entry) that the new type or account
# would break: unlink it and keep the partner
Transaction.all_objects.filter(pk=partner.pk).update(
linked_transaction=None
)
existing_transaction.linked_transaction = None
self._apply_to(
existing_transaction,
self.account,
self.transaction_type,
self.amount,
existing_transaction.date,
existing_transaction.reference_date,
) )
existing_transaction.amount = ( if not existing_transaction.linked_transaction_id:
self.to_amount or self.amount if is_to_side else self.amount self.complete_transfer(existing_transaction)
)
existing_transaction.description = ( def _remove_occurrences_after_end(self):
self.description if self.add_description_to_transaction else "" """Removes unpaid occurrences an earlier end date no longer reaches, and
) moves the generation bookmark back so a later extension fills the gap."""
existing_transaction.category = self.category if not self.end_date:
existing_transaction.notes = ( return
self.notes if self.add_notes_to_transaction else ""
removed = set()
stale = self.transactions.filter(
is_paid=False, date__gt=self.end_date
).select_related("linked_transaction")
for existing_transaction in stale:
if existing_transaction.pk in removed:
continue
partner = existing_transaction.linked_transaction
own_partner = bool(partner and partner.recurring_transaction_id == self.pk)
if partner and not own_partner:
# A link made elsewhere (e.g. a DCA entry): keep the partner
Transaction.all_objects.filter(pk=partner.pk).update(
linked_transaction=None
)
existing_transaction.delete(_with_partner=own_partner)
removed.update({existing_transaction.pk, partner.pk if own_partner else None})
if self.last_generated_date and self.last_generated_date > self.end_date:
last = self.transactions.order_by("-date").first()
self.last_generated_date = last.date if last else None
self.last_generated_reference_date = last.reference_date if last else None
self.save(
update_fields=["last_generated_date", "last_generated_reference_date"]
) )
# Update many-to-many relationships (see create_transaction) def _is_to_side(self, existing_transaction):
existing_transaction.tags.set(self.tags(manager="all_objects").all()) """Whether a transaction is the "to" side of a transfer this generated."""
existing_transaction.entities.set( return bool(
self.entities(manager="all_objects").all() existing_transaction.linked_transaction_id
) and existing_transaction.type == Transaction.Type.INCOME
and existing_transaction.linked_transaction.recurring_transaction_id
# Save updated transaction == self.pk
existing_transaction.save() )
def delete_unpaid_transactions(self): def delete_unpaid_transactions(self):
""" """
@@ -1371,3 +1588,23 @@ class QuickTransaction(TransferTemplate, OwnedObject):
def __str__(self): def __str__(self):
return self.name return self.name
class AutoTaggerModel(models.Model):
user = models.OneToOneField(
settings.AUTH_USER_MODEL,
on_delete=models.CASCADE,
related_name="auto_tagger_model",
)
data = models.BinaryField(default=b"")
transaction_count = models.PositiveIntegerField(default=0)
version = models.PositiveSmallIntegerField(default=1)
trained_at = models.DateTimeField(auto_now=True)
class Meta:
verbose_name = _("Auto-Tagger Model")
verbose_name_plural = _("Auto-Tagger Models")
db_table = "auto_tagger_models"
def __str__(self):
return f"AutoTaggerModel({self.user_id}, n={self.transaction_count})"
+49 -1
View File
@@ -2,12 +2,15 @@ import logging
from datetime import timedelta from datetime import timedelta
from cachalot.api import cachalot_disabled, invalidate from cachalot.api import cachalot_disabled, invalidate
from django.contrib.auth import get_user_model
from django.db.models import Max
from django.utils import timezone from django.utils import timezone
from django.conf import settings from django.conf import settings
from procrastinate.contrib.django import app from procrastinate.contrib.django import app
from apps.transactions.models import RecurringTransaction, Transaction from apps.transactions.auto_tagging.service import _with_user_context, train_for_user
from apps.transactions.models import AutoTaggerModel, RecurringTransaction, Transaction
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -50,3 +53,48 @@ def cleanup_deleted_transactions(timestamp=None):
deleted_count, _ = old_transactions.hard_delete() deleted_count, _ = old_transactions.hard_delete()
return f"Hard deleted {deleted_count} objects older than {settings.KEEP_DELETED_TRANSACTIONS_FOR} days." return f"Hard deleted {deleted_count} objects older than {settings.KEEP_DELETED_TRANSACTIONS_FOR} days."
@app.task(name="train_user_auto_tagger")
def train_user_auto_tagger(user_id: int):
user = get_user_model().objects.filter(id=user_id).first()
if user is None:
return "User not found."
user_settings = getattr(user, "settings", None)
if user_settings is None or not user_settings.auto_tagging:
return "Auto-tagging disabled for user."
with cachalot_disabled():
model_obj = train_for_user(user)
return f"Trained auto-tagger for user {user_id} on {model_obj.transaction_count} transactions."
@app.periodic(cron="30 2 * * *")
@app.task(lock="retrain_auto_taggers", name="retrain_auto_taggers")
def retrain_auto_taggers(timestamp=None):
User = get_user_model()
users = User.objects.filter(settings__auto_tagging=True, is_active=True).select_related(
"settings"
)
existing_models = {
m.user_id: m.trained_at
for m in AutoTaggerModel.objects.filter(user__in=users)
}
retrained = 0
with cachalot_disabled():
for user in users:
trained_at = existing_models.get(user.id)
with _with_user_context(user):
latest_tx = Transaction.objects.aggregate(latest=Max("updated_at"))[
"latest"
]
if trained_at is not None and (
latest_tx is None or latest_tx <= trained_at
):
continue
train_for_user(user)
retrained += 1
return f"Retrained auto-tagger for {retrained} user(s)."
@@ -0,0 +1,474 @@
import datetime
from decimal import Decimal
from unittest.mock import patch
from django.contrib.auth import get_user_model
from django.test import TestCase
from apps.accounts.models import Account, AccountGroup
from apps.common.middleware.thread_local import (
delete_current_user,
get_current_user,
write_current_user,
)
from apps.currencies.models import Currency
from apps.rules.models import TransactionRule, TransactionRuleAction
from apps.rules.tasks import check_for_transaction_rules
from apps.transactions.auto_tagging.classifier import (
TrainingRecord,
TransactionAutoTagger,
)
from apps.transactions.auto_tagging.service import apply_auto_tagging, train_for_user
from apps.transactions.forms import (
TransactionCategoryForm,
TransactionEntityForm,
TransactionTagForm,
)
from apps.transactions.models import (
AutoTaggerModel,
Transaction,
TransactionCategory,
TransactionEntity,
TransactionTag,
)
from apps.transactions.tasks import retrain_auto_taggers, train_user_auto_tagger
from apps.users.forms import UserSettingsForm
User = get_user_model()
class TransactionAutoTaggerClassifierTests(TestCase):
def _sample_records(self) -> list[TrainingRecord]:
return [
TrainingRecord(
description="UBER *TRIP HELP.UBER.COM 12/01",
notes="",
account_id=1,
tx_type="EX",
amount=Decimal("24.50"),
category_id=10,
tag_ids=(100,),
entity_ids=(200,),
),
TrainingRecord(
description="UBER *TRIP 15/01/2025",
notes="",
account_id=1,
tx_type="EX",
amount=Decimal("18.90"),
category_id=10,
tag_ids=(100,),
entity_ids=(200,),
),
TrainingRecord(
description="UBER *TRIP 2025-01-20",
notes="",
account_id=1,
tx_type="EX",
amount=Decimal("31.00"),
category_id=10,
tag_ids=(100,),
entity_ids=(200,),
),
TrainingRecord(
description="PIX PADARIA SÃO JOÃO",
notes="breakfast",
account_id=1,
tx_type="EX",
amount=Decimal("15.00"),
category_id=20,
tag_ids=(101, 102),
entity_ids=(201,),
),
TrainingRecord(
description="PIX PADARIA SAO JOAO",
notes="coffee",
account_id=1,
tx_type="EX",
amount=Decimal("12.00"),
category_id=20,
tag_ids=(101, 102),
entity_ids=(201,),
),
]
def test_predicts_category_tags_and_entities_with_noise_and_accents(self):
tagger = TransactionAutoTagger.fit(self._sample_records())
uber_pred = tagger.predict(
description="UBER *TRIP 03/02/2025",
notes="",
account_id=1,
tx_type="EX",
amount=Decimal("22.00"),
)
self.assertEqual(uber_pred.category_id, 10)
self.assertEqual(uber_pred.tag_ids, (100,))
self.assertEqual(uber_pred.entity_ids, (200,))
bakery_pred = tagger.predict(
description="PIX PADARIA SAO JOAO 04/02",
notes="",
account_id=1,
tx_type="EX",
amount=Decimal("14.00"),
)
self.assertEqual(bakery_pred.category_id, 20)
self.assertEqual(bakery_pred.tag_ids, (101, 102))
self.assertEqual(bakery_pred.entity_ids, (201,))
def test_serialization_roundtrip_and_invalid_blob(self):
tagger = TransactionAutoTagger.fit(self._sample_records())
blob = tagger.to_bytes()
restored = TransactionAutoTagger.from_bytes(blob)
self.assertIsNotNone(restored)
self.assertEqual(restored.num_docs, tagger.num_docs)
pred = restored.predict(
description="UBER *TRIP",
notes="",
account_id=1,
tx_type="EX",
amount=Decimal("20.00"),
)
self.assertEqual(pred.category_id, 10)
self.assertIsNone(TransactionAutoTagger.from_bytes(b""))
self.assertIsNone(TransactionAutoTagger.from_bytes(b"not-valid-zlib"))
def test_abstains_on_empty_text_or_unrelated_description(self):
tagger = TransactionAutoTagger.fit(self._sample_records())
empty_pred = tagger.predict(
description="",
notes="",
account_id=1,
tx_type="EX",
amount=Decimal("20.00"),
)
self.assertTrue(empty_pred.is_empty)
unrelated_pred = tagger.predict(
description="COMPLETELY UNKNOWN MERCHANT XYZ",
notes="",
account_id=1,
tx_type="EX",
amount=Decimal("20.00"),
)
self.assertTrue(unrelated_pred.is_empty)
# Sharing only 1 weak token among mostly unknown words must still abstain
weak_overlap_pred = tagger.predict(
description="COMPLETELY RANDOM TRIP THING",
notes="",
account_id=1,
tx_type="EX",
amount=Decimal("20.00"),
)
self.assertTrue(weak_overlap_pred.is_empty)
def test_unique_notes_do_not_discard_matching_description(self):
tagger = TransactionAutoTagger.fit(self._sample_records())
pred = tagger.predict(
description="UBER *TRIP 05/02",
notes="airport pickup for visiting cousins with heavy suitcases",
account_id=1,
tx_type="EX",
amount=Decimal("25.00"),
)
self.assertEqual(pred.category_id, 10)
self.assertEqual(pred.tag_ids, (100,))
self.assertEqual(pred.entity_ids, (200,))
def test_respects_allowed_label_sets(self):
tagger = TransactionAutoTagger.fit(self._sample_records())
pred = tagger.predict(
description="UBER *TRIP",
notes="",
account_id=1,
tx_type="EX",
amount=Decimal("20.00"),
allowed_category_ids=set(),
allowed_tag_ids={100},
allowed_entity_ids=set(),
)
self.assertIsNone(pred.category_id)
self.assertEqual(pred.tag_ids, (100,))
self.assertEqual(pred.entity_ids, ())
class AutoTaggingServiceAndRulesTests(TestCase):
def setUp(self):
self.user = User.objects.create_user(
email="alice@example.com", password="password123"
)
self.other_user = User.objects.create_user(
email="bob@example.com", password="password123"
)
self.currency = Currency.objects.create(
code="USD", name="US Dollar", decimal_places=2, prefix="$ "
)
self.group = AccountGroup.objects.create(name="Checking", owner=self.user)
self.account = Account.objects.create(
name="Main Card",
group=self.group,
currency=self.currency,
owner=self.user,
visibility="private",
)
self.second_account = Account.objects.create(
name="Savings",
group=self.group,
currency=self.currency,
owner=self.user,
visibility="private",
)
self.cat_transport = TransactionCategory.objects.create(
name="Transport", owner=self.user, visibility="private"
)
self.cat_other = TransactionCategory.objects.create(
name="Other", owner=self.user, visibility="private"
)
self.tag_commute = TransactionTag.objects.create(
name="Commute", owner=self.user, visibility="private"
)
self.tag_rule = TransactionTag.objects.create(
name="RuleApplied", owner=self.user, visibility="private"
)
self.entity_uber = TransactionEntity.objects.create(
name="Uber", owner=self.user, visibility="private"
)
# Seed historical labeled transactions for Alice
for day in (1, 5, 10):
tx = Transaction.objects.create(
owner=self.user,
account=self.account,
type=Transaction.Type.EXPENSE,
date=datetime.date(2025, 1, day),
amount=Decimal("25.00"),
description=f"UBER *TRIP 0{day}/01",
category=self.cat_transport,
)
tx.tags.add(self.tag_commute)
tx.entities.add(self.entity_uber)
def tearDown(self):
if get_current_user() is not None:
delete_current_user()
def _tag_ids(self, tx: Transaction) -> list[int]:
return list(
Transaction.tags.through.objects.filter(transaction_id=tx.pk).values_list(
"transactiontag_id", flat=True
)
)
def _entity_ids(self, tx: Transaction) -> list[int]:
return list(
Transaction.entities.through.objects.filter(
transaction_id=tx.pk
).values_list("transactionentity_id", flat=True)
)
def _create_unclassified_uber_tx(self, **overrides) -> Transaction:
defaults = {
"owner": self.user,
"account": self.account,
"type": Transaction.Type.EXPENSE,
"date": datetime.date(2025, 2, 1),
"amount": Decimal("27.00"),
"description": "UBER *TRIP 01/02",
}
defaults.update(overrides)
return Transaction.objects.create(**defaults)
def test_auto_tags_unclassified_transaction_after_rules(self):
tx = self._create_unclassified_uber_tx()
check_for_transaction_rules(
instance_id=tx.id,
user_id=self.user.id,
signal="transaction_created",
)
tx.refresh_from_db()
self.assertEqual(tx.category_id, self.cat_transport.id)
self.assertEqual(self._tag_ids(tx), [self.tag_commute.id])
self.assertEqual(self._entity_ids(tx), [self.entity_uber.id])
def test_skips_when_user_already_set_category_tag_or_entity(self):
train_for_user(self.user)
tx_with_cat = self._create_unclassified_uber_tx(category=self.cat_other)
self.assertFalse(apply_auto_tagging(tx_with_cat, self.user))
tx_with_cat.refresh_from_db()
self.assertEqual(tx_with_cat.category_id, self.cat_other.id)
self.assertEqual(self._tag_ids(tx_with_cat), [])
self.assertEqual(self._entity_ids(tx_with_cat), [])
tx_with_tag = self._create_unclassified_uber_tx()
tx_with_tag.tags.add(self.tag_rule)
self.assertFalse(apply_auto_tagging(tx_with_tag, self.user))
tx_with_tag.refresh_from_db()
self.assertIsNone(tx_with_tag.category_id)
self.assertEqual(self._entity_ids(tx_with_tag), [])
tx_with_entity = self._create_unclassified_uber_tx()
tx_with_entity.entities.add(self.entity_uber)
self.assertFalse(apply_auto_tagging(tx_with_entity, self.user))
tx_with_entity.refresh_from_db()
self.assertIsNone(tx_with_entity.category_id)
self.assertEqual(self._tag_ids(tx_with_entity), [])
def test_rules_run_before_auto_tagging_and_prevent_auto_tagging_if_rule_classifies(self):
rule = TransactionRule.objects.create(
owner=self.user,
visibility="private",
name="Tag Uber via Rule",
trigger="'UBER' in description",
on_create=True,
active=True,
)
TransactionRuleAction.objects.create(
rule=rule,
field=TransactionRuleAction.Field.tags,
value=f"['{self.tag_rule.name}']",
)
tx = self._create_unclassified_uber_tx()
check_for_transaction_rules(
instance_id=tx.id,
user_id=self.user.id,
signal="transaction_created",
)
tx.refresh_from_db()
# Rule set the tag; auto-tagging should see the transaction is already classified and skip it
self.assertEqual(self._tag_ids(tx), [self.tag_rule.id])
self.assertIsNone(tx.category_id)
self.assertEqual(self._entity_ids(tx), [])
def test_user_settings_disable_auto_tagging(self):
self.user.settings.auto_tagging = False
self.user.settings.save()
tx = self._create_unclassified_uber_tx()
check_for_transaction_rules(
instance_id=tx.id,
user_id=self.user.id,
signal="transaction_created",
)
tx.refresh_from_db()
self.assertIsNone(tx.category_id)
self.assertEqual(self._tag_ids(tx), [])
self.assertEqual(self._entity_ids(tx), [])
def test_per_item_auto_match_opt_out_and_inactive_items(self):
train_for_user(self.user)
self.cat_transport.auto_match = False
self.cat_transport.save()
self.tag_commute.active = False
self.tag_commute.save()
tx = self._create_unclassified_uber_tx()
applied = apply_auto_tagging(tx, self.user)
self.assertTrue(applied)
tx.refresh_from_db()
self.assertIsNone(tx.category_id)
self.assertEqual(self._tag_ids(tx), [])
self.assertEqual(self._entity_ids(tx), [self.entity_uber.id])
def test_skips_transfers(self):
train_for_user(self.user)
tx_from = self._create_unclassified_uber_tx()
tx_from.add_transfer_side(self.second_account, Decimal("27.00"))
tx_from.refresh_from_db()
self.assertFalse(apply_auto_tagging(tx_from, self.user))
self.assertIsNone(tx_from.category_id)
def test_multi_user_isolation(self):
train_for_user(self.user)
bob_group = AccountGroup.objects.create(name="Bob Group", owner=self.other_user)
bob_account = Account.objects.create(
name="Bob Card",
group=bob_group,
currency=self.currency,
owner=self.other_user,
visibility="private",
)
bob_tx = Transaction.objects.create(
owner=self.other_user,
account=bob_account,
type=Transaction.Type.EXPENSE,
date=datetime.date(2025, 2, 1),
amount=Decimal("25.00"),
description="UBER *TRIP 01/02",
)
self.assertFalse(apply_auto_tagging(bob_tx, self.other_user))
bob_tx.refresh_from_db()
self.assertIsNone(bob_tx.category_id)
self.assertFalse(bob_tx.tags.exists())
self.assertFalse(bob_tx.entities.exists())
def test_procrastinate_training_tasks(self):
msg = train_user_auto_tagger(self.user.id)
self.assertIn("Trained auto-tagger", msg)
model_obj = AutoTaggerModel.objects.get(user=self.user)
self.assertEqual(model_obj.transaction_count, 3)
# Periodic task skips already-up-to-date user
with patch(
"apps.transactions.tasks.train_for_user", wraps=train_for_user
) as mock_train:
retrain_auto_taggers()
trained_user_ids = [call.args[0].id for call in mock_train.call_args_list]
self.assertNotIn(self.user.id, trained_user_ids)
def test_retrain_auto_taggers_detects_shared_account_updates(self):
# Train Bob initially (0 transactions visible)
train_for_user(self.other_user)
self.assertEqual(
AutoTaggerModel.objects.get(user=self.other_user).transaction_count, 0
)
# Share Alice's account with Bob and add a new transaction owned by Alice
self.account.shared_with.add(self.other_user)
Transaction.objects.create(
owner=self.user,
account=self.account,
type=Transaction.Type.EXPENSE,
date=datetime.date(2025, 2, 2),
amount=Decimal("30.00"),
description="UBER *TRIP SHARED",
category=self.cat_transport,
)
retrain_auto_taggers()
bob_model = AutoTaggerModel.objects.get(user=self.other_user)
self.assertGreater(bob_model.transaction_count, 0)
def test_forms_expose_auto_tagging_and_auto_match_fields(self):
write_current_user(self.user)
settings_form = UserSettingsForm(instance=self.user.settings)
self.assertIn("auto_tagging", settings_form.fields)
cat_form = TransactionCategoryForm(instance=self.cat_transport)
self.assertIn("auto_match", cat_form.fields)
tag_form = TransactionTagForm(instance=self.tag_commute)
self.assertIn("auto_match", tag_form.fields)
entity_form = TransactionEntityForm(instance=self.entity_uber)
self.assertIn("auto_match", entity_form.fields)
@@ -0,0 +1,169 @@
from datetime import date, datetime, timezone as dt_timezone
from decimal import Decimal
from django.contrib.auth import get_user_model
from django.test import TestCase
from apps.accounts.models import Account
from apps.common.middleware.thread_local import delete_current_user, write_current_user
from apps.currencies.models import Currency, ExchangeRate
from apps.transactions.models import Transaction
from apps.transactions.utils.calculations import (
calculate_consolidated_shares,
calculate_currency_totals,
)
def component(name, total_final):
return {"currency": {"name": name}, "total_final": Decimal(total_final)}
class CalculateConsolidatedSharesTests(TestCase):
def test_shares_add_up_to_the_total(self):
shares = calculate_consolidated_shares(
[component("USD", "150"), component("EUR", "50")], Decimal("200")
)
self.assertEqual(
[(share["currency"]["name"], share["share"]) for share in shares],
[("USD", Decimal("75")), ("EUR", Decimal("25"))],
)
def test_sorted_from_largest_to_smallest(self):
shares = calculate_consolidated_shares(
[component("USD", "10"), component("BTC", "-5"), component("EUR", "20")],
Decimal("25"),
)
self.assertEqual(
[share["currency"]["name"] for share in shares], ["EUR", "USD", "BTC"]
)
def test_component_that_takes_from_the_total_has_a_negative_share(self):
shares = calculate_consolidated_shares(
[component("USD", "300"), component("EUR", "-100")], Decimal("200")
)
self.assertEqual(
[share["share"] for share in shares], [Decimal("150"), Decimal("-50")]
)
def test_sign_follows_the_component_when_the_total_is_negative(self):
shares = calculate_consolidated_shares(
[component("USD", "100"), component("EUR", "-300")], Decimal("-200")
)
self.assertEqual(
[share["share"] for share in shares], [Decimal("50"), Decimal("-150")]
)
def test_components_that_add_nothing_are_left_out(self):
shares = calculate_consolidated_shares(
[component("USD", "0"), component("EUR", "50")], Decimal("50")
)
self.assertEqual([share["currency"]["name"] for share in shares], ["EUR"])
self.assertEqual(shares[0]["share"], Decimal("100"))
def test_no_share_without_a_total(self):
shares = calculate_consolidated_shares(
[component("USD", "100"), component("EUR", "-100")], Decimal("0")
)
self.assertEqual([share["share"] for share in shares], [None, None])
class CurrencyTotalsConsolidatedComponentsTests(TestCase):
def setUp(self):
self.user = get_user_model().objects.create_user(
email="shares@test.com", password="testpass123"
)
write_current_user(self.user)
self.addCleanup(delete_current_user)
self.usd = Currency.objects.create(
code="USD", name="US Dollar", decimal_places=2, prefix="$ "
)
self.eur = Currency.objects.create(
code="EUR", name="Euro", decimal_places=2, exchange_currency=self.usd
)
self.gbp = Currency.objects.create(
code="GBP", name="Pound", decimal_places=2, exchange_currency=self.usd
)
for currency in (self.eur, self.gbp):
ExchangeRate.objects.create(
from_currency=currency,
to_currency=self.usd,
rate=Decimal("2"),
date=datetime(2026, 1, 1, tzinfo=dt_timezone.utc),
)
self._add(self.usd, Transaction.Type.INCOME, "150")
self._add(self.eur, Transaction.Type.INCOME, "50")
self._add(self.gbp, Transaction.Type.EXPENSE, "25")
def _add(self, currency, transaction_type, amount):
Transaction.objects.create(
account=Account.objects.create(
name=f"{currency.code} account", currency=currency
),
type=transaction_type,
is_paid=True,
date=date(2026, 3, 10),
reference_date=date(2026, 3, 1),
amount=Decimal(amount),
)
def _components(self, totals, currency):
return [
(part["currency"]["name"], part["total_final"], part["share"])
for part in totals[currency.id]["consolidated"]["components"]
]
def test_components_are_in_the_consolidated_currency(self):
totals = calculate_currency_totals(Transaction.objects.all(), deep_search=True)
self.assertEqual(
totals[self.usd.id]["consolidated"]["total_final"], Decimal("200")
)
self.assertEqual(
self._components(totals, self.usd),
[
("US Dollar", Decimal("150"), Decimal("75")),
("Euro", Decimal("100"), Decimal("50")),
("Pound", Decimal("-50"), Decimal("-25")),
],
)
def test_currency_without_sources_is_its_only_component(self):
totals = calculate_currency_totals(Transaction.objects.all(), deep_search=True)
self.assertEqual(
self._components(totals, self.eur),
[("Euro", Decimal("50"), Decimal("100"))],
)
def test_hidden_exchanged_amounts_still_count(self):
self.eur.show_exchanged_amounts = False
self.eur.save()
totals = calculate_currency_totals(Transaction.objects.all(), deep_search=True)
self.assertNotIn("exchanged", totals[self.eur.id])
self.assertIn(
("Euro", Decimal("100"), Decimal("50")),
self._components(totals, self.usd),
)
def test_target_found_by_deep_search(self):
Transaction.objects.filter(account__currency=self.usd).delete()
totals = calculate_currency_totals(Transaction.objects.all(), deep_search=True)
self.assertEqual(
self._components(totals, self.usd),
[
("Euro", Decimal("100"), Decimal("200")),
("Pound", Decimal("-50"), Decimal("-100")),
],
)
@@ -0,0 +1,189 @@
from datetime import date, datetime, timezone as dt_timezone
from decimal import Decimal
from django.contrib.auth import get_user_model
from django.test import TestCase, override_settings
from django.urls import reverse
from apps.accounts.models import Account, AccountGroup
from apps.common.middleware.thread_local import delete_current_user, write_current_user
from apps.currencies.models import Currency, ExchangeRate
from apps.insights.utils.month_by_month import get_month_by_month_data
from apps.insights.utils.overview import get_grouped_totals
from apps.insights.utils.year_by_year import get_year_by_year_data
from apps.transactions.models import Transaction, TransactionCategory
from apps.transactions.utils.calculations import (
calculate_account_totals,
calculate_currency_totals,
)
@override_settings(
STORAGES={
"default": {"BACKEND": "django.core.files.storage.FileSystemStorage"},
"staticfiles": {
"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"
},
},
WHITENOISE_AUTOREFRESH=True,
)
class ExchangedAmountsTestBase(TestCase):
def setUp(self):
self.user = get_user_model().objects.create_user(
email="exchanged@test.com", password="testpass123"
)
self.client.login(username="exchanged@test.com", password="testpass123")
write_current_user(self.user)
self.addCleanup(delete_current_user)
self.usd = Currency.objects.create(
code="USD", name="US Dollar", decimal_places=2, prefix="$ "
)
self.eur = Currency.objects.create(
code="EUR",
name="Euro",
decimal_places=2,
prefix="€ ",
exchange_currency=self.usd,
)
ExchangeRate.objects.create(
from_currency=self.eur,
to_currency=self.usd,
rate=Decimal("2"),
date=datetime(2026, 1, 1, tzinfo=dt_timezone.utc),
)
self.account = Account.objects.create(
name="Euro Account",
group=AccountGroup.objects.create(name="Group"),
currency=self.eur,
)
self.transaction = Transaction.objects.create(
account=self.account,
type=Transaction.Type.INCOME,
is_paid=True,
date=date(2026, 3, 10),
reference_date=date(2026, 3, 1),
amount=Decimal("100"),
description="Salary",
category=TransactionCategory.objects.create(name="Work"),
)
def _hide_for(self, obj):
obj.show_exchanged_amounts = False
obj.save()
class TransactionExchangedAmountTests(ExchangedAmountsTestBase):
def test_shown_by_default(self):
exchanged = self.transaction.displayed_exchanged_amount()
self.assertEqual(exchanged["amount"], Decimal("200"))
self.assertEqual(exchanged["prefix"], "$ ")
def test_currency_can_hide_it(self):
self._hide_for(self.eur)
self.transaction.refresh_from_db()
self.assertIsNone(self.transaction.displayed_exchanged_amount())
# Only hidden from display, the API keeps returning it
self.assertEqual(self.transaction.exchanged_amount()["amount"], Decimal("200"))
def test_account_exchange_currency_follows_the_account_setting(self):
self.account.exchange_currency = self.usd
self.account.save()
self._hide_for(self.eur)
self.transaction.refresh_from_db()
self.assertEqual(
self.transaction.displayed_exchanged_amount()["amount"], Decimal("200")
)
self._hide_for(self.account)
self.transaction.refresh_from_db()
self.assertIsNone(self.transaction.displayed_exchanged_amount())
def test_monthly_list_hides_it(self):
url = reverse("monthly_transactions_list", kwargs={"month": 3, "year": 2026})
response = self.client.get(url, HTTP_HX_REQUEST="true")
self.assertContains(response, "exchanged-amount")
self._hide_for(self.eur)
response = self.client.get(url, HTTP_HX_REQUEST="true")
self.assertContains(response, f'data-transaction-id="{self.transaction.id}"')
self.assertNotContains(response, "exchanged-amount")
class TotalsExchangedAmountTests(ExchangedAmountsTestBase):
def test_currency_totals_hide_it_but_keep_consolidating(self):
self._hide_for(self.eur)
totals = calculate_currency_totals(Transaction.objects.all(), deep_search=True)
self.assertNotIn("exchanged", totals[self.eur.id])
self.assertEqual(
totals[self.usd.id]["consolidated"]["total_final"], Decimal("200")
)
def test_currency_totals_hide_it_for_currencies_found_by_deep_search(self):
brl = Currency.objects.create(code="BRL", name="Real", decimal_places=2)
self.usd.exchange_currency = brl
self.usd.save()
totals = calculate_currency_totals(Transaction.objects.all(), deep_search=True)
self.assertIn("exchanged", totals[self.usd.id])
self._hide_for(self.usd)
totals = calculate_currency_totals(Transaction.objects.all(), deep_search=True)
self.assertNotIn("exchanged", totals[self.usd.id])
def test_account_totals_follow_the_account_setting(self):
self.account.exchange_currency = self.usd
self.account.save()
totals = calculate_account_totals(Transaction.objects.all())
self.assertEqual(
totals[self.account.id]["exchanged"]["total_final"], Decimal("200")
)
self._hide_for(self.account)
totals = calculate_account_totals(Transaction.objects.all())
self.assertNotIn("exchanged", totals[self.account.id])
def test_insights_hide_it(self):
def exchanged_totals():
month_data = get_month_by_month_data(year=2026)
year_data = get_year_by_year_data()
return [
month_data["grand_total"]["currencies"][self.eur.id],
month_data["month_totals"][3]["currencies"][self.eur.id],
year_data["grand_total"]["currencies"][self.eur.id],
year_data["year_totals"][2026]["currencies"][self.eur.id],
]
def overview_total():
overview = get_grouped_totals(Transaction.objects.all(), ["categories"])
return next(iter(overview.values()))["currencies"][self.eur.id]
for total in exchanged_totals():
self.assertEqual(total["exchanged"]["final_total"], Decimal("200"))
self.assertEqual(overview_total()["exchanged"]["shown"]["total"], Decimal("200"))
self._hide_for(self.eur)
for total in exchanged_totals():
self.assertNotIn("exchanged", total)
self.assertNotIn("exchanged", overview_total())
def test_net_worth_keeps_the_consolidated_total(self):
self._hide_for(self.eur)
response = self.client.get(reverse("net_worth"), {"view_type": "current"})
currency_net_worth = response.context["currency_net_worth"]
self.assertNotIn("exchanged", currency_net_worth[self.eur.id])
self.assertEqual(
currency_net_worth[self.usd.id]["consolidated"]["total_final"],
Decimal("200"),
)
self.assertContains(response, "Consolidated")
@@ -0,0 +1,71 @@
from decimal import Decimal
from django.contrib.auth import get_user_model
from django.test import TestCase, override_settings
from django.urls import reverse
from django.utils import timezone
from apps.accounts.models import Account
from apps.currencies.models import Currency
from apps.transactions.models import Transaction, TransactionCategory, TransactionTag
@override_settings(
STORAGES={
"default": {"BACKEND": "django.core.files.storage.FileSystemStorage"},
"staticfiles": {
"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"
},
},
WHITENOISE_AUTOREFRESH=True,
)
class ExcludeFilterTests(TestCase):
def setUp(self):
self.user = get_user_model().objects.create_user(
email="exclude@test.com", password="pass"
)
self.client.force_login(self.user)
currency = Currency.objects.create(code="BRL", name="Real", decimal_places=2)
self.account = Account.objects.create(name="Wallet", currency=currency)
food = TransactionCategory.objects.create(name="Food")
home = TransactionTag.objects.create(name="home")
self.make("Groceries", category=food).tags.add(home)
self.make("Restaurant", category=food, notes="with friends")
self.make("Rent").tags.add(home)
self.make("Gift")
def make(self, description, **kwargs):
today = timezone.localdate(timezone.now())
return Transaction.objects.create(
account=self.account,
type=Transaction.Type.EXPENSE,
date=today,
reference_date=today.replace(day=1),
amount=Decimal("10"),
description=description,
**kwargs,
)
def listed(self, params):
response = self.client.get(
reverse("transactions_all_list"), params, HTTP_HX_REQUEST="true"
)
self.assertEqual(response.status_code, 200)
page = response.context["page_obj"]
late = response.context["late_transactions"] or []
return sorted(t.description for t in [*page, *late])
def test_exclusions_join_with_inclusions(self):
cases = [
({"exclude_category": ["Food"]}, ["Gift", "Rent"]),
({"exclude_category": ["uncategorized"]}, ["Groceries", "Restaurant"]),
({"exclude_tags": ["home"]}, ["Gift", "Restaurant"]),
({"exclude_tags": ["untagged"]}, ["Groceries", "Rent"]),
({"exclude_description": "friends"}, ["Gift", "Groceries", "Rent"]),
({"exclude_account": ["Wallet"]}, []),
# positive and negative filters are applied together
({"category": ["Food"], "exclude_tags": ["home"]}, ["Restaurant"]),
]
for params, expected in cases:
self.assertEqual(self.listed(params), expected, params)
@@ -1,3 +1,6 @@
import datetime
from unittest import mock
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
from django.test import TestCase, override_settings from django.test import TestCase, override_settings
from django.urls import reverse from django.urls import reverse
@@ -51,6 +54,55 @@ class FilterPresetViewTests(TestCase):
) )
self.assertContains(response, "Account X Unpaid") self.assertContains(response, "Account X Unpaid")
def test_create_keeps_dates_relative_to_today_when_asked(self):
self.user.settings.date_format = "d/m/Y"
self.user.settings.save()
today = datetime.date(2026, 8, 20)
with mock.patch("django.utils.timezone.localdate", return_value=today):
self.client.post(
reverse("filter_preset_create"),
{
"name": "Today",
"date_start": "20/08/2026",
"relative_dates": "true",
},
HTTP_HX_REQUEST="true",
)
preset = FilterPreset.objects.get(owner=self.user, name="Today")
self.assertEqual(preset.relative_to, today)
self.assertEqual(preset.parameters, {"date_start": ["2026-08-20"]})
def test_create_keeps_dates_fixed_by_default(self):
self.client.post(
reverse("filter_preset_create"),
{
"name": "August",
"reference_date_start": "August 2026",
"relative_dates": "false",
},
HTTP_HX_REQUEST="true",
)
preset = FilterPreset.objects.get(owner=self.user, name="August")
self.assertIsNone(preset.relative_to)
def test_create_ignores_relative_dates_without_a_date_filter(self):
self.client.post(
reverse("filter_preset_create"),
{
"name": "Unpaid without dates",
"is_paid": ["0"],
"date_start": "",
"relative_dates": "true",
},
HTTP_HX_REQUEST="true",
)
preset = FilterPreset.objects.get(owner=self.user, name="Unpaid without dates")
self.assertIsNone(preset.relative_to)
def test_create_rejects_a_blank_name(self): def test_create_rejects_a_blank_name(self):
response = self.client.post( response = self.client.post(
reverse("filter_preset_create"), reverse("filter_preset_create"),
@@ -82,6 +134,121 @@ class FilterPresetViewTests(TestCase):
"updated", "updated",
) )
def test_apply_moves_relative_dates_along_with_today(self):
self.user.settings.date_format = "d/m/Y"
self.user.settings.save()
preset = FilterPreset.objects.create(
owner=self.user,
name="Next days",
parameters={
"date_start": ["20/08/2026"],
"date_end": ["31/08/2026"],
"reference_date_start": ["August 2026"],
"reference_date_end": ["September 2026"],
},
relative_to=datetime.date(2026, 8, 20),
)
with mock.patch(
"django.utils.timezone.localdate",
return_value=datetime.date(2027, 1, 5),
):
response = self.client.get(
reverse("filter_preset_apply", args=[preset.pk]),
HTTP_HX_REQUEST="true",
)
data = response.context["filter"].data
self.assertEqual(data["date_start"], "05/01/2027")
self.assertEqual(data["date_end"], "16/01/2027")
self.assertEqual(data["reference_date_start"], "January 2027")
self.assertEqual(data["reference_date_end"], "February 2027")
self.assertContains(response, 'data-value="2027-01-05"')
self.assertContains(response, 'data-value="2027-01-16"')
self.assertContains(response, 'data-value="2027-01-01"')
self.assertContains(response, 'data-value="2027-02-01"')
def test_apply_keeps_a_whole_month_range_on_whole_months(self):
self.user.settings.date_format = "d/m/Y"
self.user.settings.save()
preset = FilterPreset.objects.create(
owner=self.user,
name="This month",
parameters={"date_start": ["01/01/2027"], "date_end": ["31/01/2027"]},
relative_to=datetime.date(2027, 1, 20),
)
with mock.patch(
"django.utils.timezone.localdate",
return_value=datetime.date(2027, 2, 3),
):
response = self.client.get(
reverse("filter_preset_apply", args=[preset.pk]),
HTTP_HX_REQUEST="true",
)
self.assertContains(response, 'data-value="2027-02-01"')
self.assertContains(response, 'data-value="2027-02-28"')
def test_apply_shows_stored_dates_in_the_users_current_format(self):
self.user.settings.date_format = "m/d/Y"
self.user.settings.save()
preset = FilterPreset.objects.create(
owner=self.user,
name="August",
parameters={
"date_start": ["2026-08-20"],
"reference_date_start": ["2026-08-01"],
},
)
response = self.client.get(
reverse("filter_preset_apply", args=[preset.pk]),
HTTP_HX_REQUEST="true",
)
self.assertContains(response, 'data-value="2026-08-20"')
self.assertContains(response, 'value="August 2026"')
def test_apply_keeps_fixed_dates_unchanged(self):
self.user.settings.date_format = "d/m/Y"
self.user.settings.save()
preset = FilterPreset.objects.create(
owner=self.user,
name="August",
parameters={
"date_start": ["20/08/2026"],
"reference_date_start": ["August 2026"],
},
)
with mock.patch(
"django.utils.timezone.localdate",
return_value=datetime.date(2027, 1, 5),
):
response = self.client.get(
reverse("filter_preset_apply", args=[preset.pk]),
HTTP_HX_REQUEST="true",
)
data = response.context["filter"].data
self.assertEqual(data["date_start"], "20/08/2026")
self.assertEqual(data["reference_date_start"], "August 2026")
self.assertContains(response, 'data-value="2026-08-20"')
self.assertContains(response, 'data-value="2026-08-01"')
def test_presets_with_relative_dates_are_marked_in_the_list(self):
FilterPreset.objects.create(
owner=self.user,
name="Today",
parameters={"date_start": ["20/08/2026"]},
relative_to=datetime.date(2026, 8, 20),
)
response = self.client.get(reverse("transactions_all_index"))
self.assertContains(response, "Dates relative to today", count=1)
def test_clear_returns_the_default_filter_form_without_changing_the_url(self): def test_clear_returns_the_default_filter_form_without_changing_the_url(self):
response = self.client.get( response = self.client.get(
"/transactions/filter/clear/", "/transactions/filter/clear/",
@@ -242,6 +242,30 @@ class RecurringTransactionTests(TestCase):
self.assertEqual(recurring.recurrence_interval, 1) self.assertEqual(recurring.recurrence_interval, 1)
self.assertEqual(recurring.account.currency.code, "USD") self.assertEqual(recurring.account.currency.code, "USD")
def test_month_end_recurrence_returns_to_its_day(self):
"""A recurrence on the 31st goes back to it after a short month"""
recurring = RecurringTransaction.objects.create(
account=self.account,
type=Transaction.Type.EXPENSE,
amount=Decimal("100.00"),
description="Rent",
start_date=datetime.date(2026, 1, 31),
recurrence_type=RecurringTransaction.RecurrenceType.MONTH,
recurrence_interval=1,
)
feb = recurring.get_next_date(datetime.date(2026, 1, 31))
march = recurring.get_next_date(feb)
april = recurring.get_next_date(march)
self.assertEqual(feb, datetime.date(2026, 2, 28))
self.assertEqual(march, datetime.date(2026, 3, 31))
self.assertEqual(april, datetime.date(2026, 4, 30))
self.assertEqual(
recurring.get_next_date(datetime.date(2026, 2, 1)),
datetime.date(2026, 3, 1),
)
def test_generate_upcoming_transactions_keeps_tags_and_entities(self): def test_generate_upcoming_transactions_keeps_tags_and_entities(self):
"""Generation must copy tags/entities even with no current user""" """Generation must copy tags/entities even with no current user"""
tag = TransactionTag.objects.create(name="Essential") tag = TransactionTag.objects.create(name="Essential")
@@ -0,0 +1,321 @@
from decimal import Decimal
from django.contrib.auth import get_user_model
from django.test import TestCase, override_settings
from django.urls import reverse
from django.utils import timezone
from rest_framework.test import APIClient
from apps.accounts.models import Account, AccountGroup
from apps.currencies.models import Currency
from apps.transactions.models import (
InstallmentPlan,
QuickTransaction,
RecurringTransaction,
SubTransaction,
Transaction,
)
from apps.transactions.filters import content_filter, exclude_matching
from apps.transactions.views.actions import clone_transaction
def _subs(*rows, initial=0):
data = {"sub-TOTAL_FORMS": str(len(rows)), "sub-INITIAL_FORMS": str(initial)}
for i, row in enumerate(rows):
data.update({f"sub-{i}-{k}": v for k, v in row.items()})
return data
@override_settings(
STORAGES={
"default": {"BACKEND": "django.core.files.storage.FileSystemStorage"},
"staticfiles": {"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"},
},
WHITENOISE_AUTOREFRESH=True,
)
class SubTransactionTests(TestCase):
def setUp(self):
user = get_user_model().objects.create_user(email="u@test.com", password="pw")
self.client.login(username="u@test.com", password="pw")
currency = Currency.objects.create(code="USD", name="USD", decimal_places=2)
self.account = Account.objects.create(
name="A", group=AccountGroup.objects.create(name="G"), currency=currency
)
self.base = {
"account": self.account.id,
"type": "EX",
"is_paid": True,
"date": timezone.localdate().isoformat(),
"description": "Groceries",
}
def _add(self, data):
return self.client.post(
reverse("transaction_add"), {**data, "submit": ""}, headers={"HX-Request": "true"}
)
def test_amount_is_sum_of_sub_transactions(self):
response = self._add(
{**self.base, "amount": "999", **_subs(
{"name": "Milk", "amount": "2.50"}, {"name": "Bread", "amount": "4"}
)}
)
self.assertEqual(response.status_code, 204)
transaction = Transaction.objects.get()
self.assertEqual(transaction.amount, Decimal("6.50"))
self.assertEqual(
list(transaction.sub_transactions.values_list("name", flat=True)),
["Milk", "Bread"],
)
def test_amount_required_without_sub_transactions(self):
response = self._add({**self.base, **_subs()})
self.assertEqual(response.status_code, 200)
self.assertIn("amount", response.context["form"].errors)
self.assertFalse(Transaction.objects.exists())
def test_sub_transaction_needs_name(self):
response = self._add({**self.base, **_subs({"name": "", "amount": "3"})})
self.assertEqual(response.status_code, 200)
self.assertFalse(Transaction.objects.exists())
def test_edit_removing_sub_transactions_restores_typed_amount(self):
transaction = Transaction.objects.create(
account=self.account, type="EX", date=timezone.localdate(), amount=5
)
sub = SubTransaction.objects.create(transaction=transaction, name="Fee", amount=5)
response = self.client.post(
reverse("transaction_edit", kwargs={"transaction_id": transaction.id}),
{**self.base, "amount": "12", **_subs(
{"id": sub.id, "name": "Fee", "amount": "5", "DELETE": "on"}, initial=1
)},
headers={"HX-Request": "true"},
)
self.assertEqual(response.status_code, 204)
transaction.refresh_from_db()
self.assertEqual(transaction.amount, Decimal("12"))
self.assertFalse(transaction.sub_transactions.exists())
@override_settings(ENABLE_SOFT_DELETE=True)
def test_soft_delete_hides_and_hard_delete_removes_sub_transactions(self):
transaction = Transaction.objects.create(
account=self.account, type="EX", date=timezone.localdate(), amount=5
)
SubTransaction.objects.create(transaction=transaction, name="Fee", amount=5)
url = reverse("transaction_delete", kwargs={"transaction_id": transaction.id})
self.client.delete(url, headers={"HX-Request": "true"})
# soft: sub-transactions stay with (and are hidden by) the deleted parent
self.assertTrue(Transaction.deleted_objects.filter(pk=transaction.pk).exists())
self.assertEqual(SubTransaction.objects.count(), 1)
self.client.delete(url, headers={"HX-Request": "true"})
self.assertFalse(SubTransaction.objects.exists())
def test_clone_copies_sub_transactions(self):
transaction = Transaction.objects.create(
account=self.account, type="EX", date=timezone.localdate(), amount=5
)
SubTransaction.objects.create(transaction=transaction, name="Fee", amount=5)
clone = clone_transaction(transaction)
self.assertEqual(list(clone.sub_transactions.values_list("name", "amount")), [("Fee", Decimal("5"))])
def test_content_filter_matches_sub_transaction_names(self):
shop = Transaction.objects.create(
account=self.account, type="EX", date=timezone.localdate(), amount=5, description="Shop"
)
SubTransaction.objects.create(transaction=shop, name="Milk", amount=2)
SubTransaction.objects.create(transaction=shop, name="Oat milk", amount=3)
other = Transaction.objects.create(
account=self.account, type="EX", date=timezone.localdate(), amount=1, description="Bus"
)
everything = Transaction.all_objects.all()
self.assertEqual(list(content_filter(everything, "description", "milk")), [shop])
self.assertEqual(list(exclude_matching(content_filter)(everything, "x", "milk")), [other])
def test_bulk_edit_leaves_amount_of_split_transactions(self):
split = Transaction.objects.create(
account=self.account, type="EX", date=timezone.localdate(), amount=5
)
SubTransaction.objects.create(transaction=split, name="Fee", amount=5)
plain = Transaction.objects.create(
account=self.account, type="EX", date=timezone.localdate(), amount=1
)
response = self.client.post(
reverse("transactions_bulk_edit"),
{"transactions": [split.id, plain.id], "amount": "50", "description": "Edited"},
headers={"HX-Request": "true"},
)
self.assertEqual(response.status_code, 204)
split.refresh_from_db()
plain.refresh_from_db()
self.assertEqual((split.amount, split.description), (Decimal("5"), "Edited"))
self.assertEqual(plain.amount, Decimal("50"))
def _post(self, url_name, data):
return self.client.post(reverse(url_name), data, headers={"HX-Request": "true"})
def _parts(self, transaction):
return list(transaction.sub_transactions.values_list("name", "amount"))
def test_quick_transaction_parts_are_copied_to_created_transaction(self):
self._post(
"quick_transaction_add",
{**self.base, "name": "Lunch", **_subs({"name": "Meal", "amount": "20"}, {"name": "Tip", "amount": "2"})},
)
quick = QuickTransaction.objects.get()
self.assertEqual(quick.amount, Decimal("22"))
self.client.get(
reverse("quick_transaction_add_as_transaction", kwargs={"quick_transaction_id": quick.id}),
headers={"HX-Request": "true"},
)
transaction = Transaction.objects.get()
self.assertEqual(transaction.amount, Decimal("22"))
self.assertEqual(self._parts(transaction), [("Meal", Decimal("20")), ("Tip", Decimal("2"))])
def test_quick_transfer_drops_parts(self):
other = Account.objects.create(name="B", group=self.account.group, currency=self.account.currency)
response = self._post(
"quick_transaction_add",
{**self.base, "type": "TR", "to_account": other.id, "amount": "5", "name": "Move",
**_subs({"name": "Hidden", "amount": "9"})},
)
self.assertEqual(response.status_code, 204)
quick = QuickTransaction.objects.get()
self.assertEqual(quick.amount, Decimal("5"))
self.assertFalse(quick.sub_transactions.exists())
def test_recurring_parts_follow_generated_and_unpaid_transactions(self):
self._post(
"recurring_transaction_add",
{**self.base, "start_date": timezone.localdate().isoformat(), "recurrence_type": "month",
"recurrence_interval": 1, "keep_at_most": 2, **_subs({"name": "Plan", "amount": "10"})},
)
recurring = RecurringTransaction.all_objects.get()
self.assertEqual(recurring.amount, Decimal("10"))
generated = recurring.transactions.all()
self.assertTrue(generated)
for transaction in generated:
self.assertEqual(self._parts(transaction), [("Plan", Decimal("10"))])
sub = recurring.sub_transactions.get()
self.client.post(
reverse("recurring_transaction_edit", kwargs={"recurring_transaction_id": recurring.id}),
{**self.base, "start_date": timezone.localdate().isoformat(), "recurrence_type": "month",
"recurrence_interval": 1, "keep_at_most": 2,
**_subs({"id": sub.id, "name": "Plan", "amount": "10"}, {"name": "Tax", "amount": "1"}, initial=1)},
headers={"HX-Request": "true"},
)
for transaction in recurring.transactions.all():
self.assertEqual(transaction.amount, Decimal("11"))
self.assertEqual(len(self._parts(transaction)), 2)
def test_installment_parts_are_copied_to_each_installment(self):
self._post(
"installment_plan_add",
{**self.base, "number_of_installments": 2, "installment_start": 1,
"start_date": timezone.localdate().isoformat(), "recurrence": "monthly",
**_subs({"name": "Principal", "amount": "90"}, {"name": "Interest", "amount": "10"})},
)
plan = InstallmentPlan.all_objects.get()
self.assertEqual(plan.installment_amount, Decimal("100"))
self.assertEqual(plan.transactions.count(), 2)
for transaction in plan.transactions.all():
self.assertEqual(transaction.amount, Decimal("100"))
self.assertEqual(len(self._parts(transaction)), 2)
class SubTransactionApiTests(TestCase):
def setUp(self):
user = get_user_model().objects.create_user(
email="api@test.com", password="pw", is_superuser=True
)
self.api = APIClient()
self.api.force_authenticate(user=user)
currency = Currency.objects.create(code="EUR", name="EUR", decimal_places=2)
self.account = Account.objects.create(name="A", currency=currency, owner=user)
def test_create_read_and_update_sub_transactions(self):
response = self.api.post(
"/api/transactions/",
{"account_id": self.account.id, "type": "EX", "date": "2030-01-10", "description": "Shop",
"sub_transactions": [{"name": "A", "amount": "1.50"}, {"name": "B", "amount": "2"}]},
format="json",
)
self.assertEqual(response.status_code, 201, response.data)
self.assertEqual(Decimal(response.data["amount"]), Decimal("3.50"))
self.assertEqual([s["name"] for s in response.data["sub_transactions"]], ["A", "B"])
url = f"/api/transactions/{response.data['id']}/"
# The amount can't drift from the parts
response = self.api.patch(url, {"amount": "9"}, format="json")
self.assertEqual(response.status_code, 400)
response = self.api.patch(url, {"sub_transactions": [{"name": "C", "amount": "4"}]}, format="json")
self.assertEqual(response.status_code, 200, response.data)
self.assertEqual(Decimal(response.data["amount"]), Decimal("4"))
self.assertEqual(self.api.get(url).data["sub_transactions"][0]["name"], "C")
def test_amount_required_without_sub_transactions(self):
response = self.api.post(
"/api/transactions/",
{"account_id": self.account.id, "type": "EX", "date": "2030-01-10"},
format="json",
)
self.assertEqual(response.status_code, 400)
self.assertIn("amount", response.data)
def _parts(self, owner):
return [(s.name, s.amount) for s in owner.sub_transactions.all()]
def test_installment_plan_parts_reach_every_installment(self):
response = self.api.post(
"/api/installment-plans/",
{"account": self.account.id, "type": "EX", "description": "Laptop",
"number_of_installments": 3, "start_date": "2030-01-10", "recurrence": "monthly",
"sub_transactions": [{"name": "Principal", "amount": "90"}, {"name": "Interest", "amount": "10"}]},
format="json",
)
self.assertEqual(response.status_code, 201, response.data)
self.assertEqual(Decimal(response.data["installment_amount"]), Decimal("100"))
plan = InstallmentPlan.all_objects.get()
installments = plan.transactions.all()
self.assertEqual(len(installments), 3)
for transaction in installments:
self.assertEqual(transaction.amount, Decimal("100"))
self.assertEqual(len(self._parts(transaction)), 2)
url = f"/api/installment-plans/{plan.id}/"
self.assertEqual(self.api.patch(url, {"installment_amount": "5"}, format="json").status_code, 400)
response = self.api.patch(url, {"sub_transactions": [{"name": "Principal", "amount": "80"}]}, format="json")
self.assertEqual(response.status_code, 200, response.data)
for transaction in plan.transactions.all():
self.assertEqual(self._parts(transaction), [("Principal", Decimal("80"))])
self.assertEqual(transaction.amount, Decimal("80"))
def test_recurring_parts_follow_its_transactions_and_transfers_drop_them(self):
response = self.api.post(
"/api/recurring-transactions/",
{"account": self.account.id, "type": "EX", "description": "Internet",
"start_date": timezone.localdate().isoformat(), "recurrence_type": "month", "recurrence_interval": 1,
"sub_transactions": [{"name": "Plan", "amount": "50"}, {"name": "Router", "amount": "10"}]},
format="json",
)
self.assertEqual(response.status_code, 201, response.data)
self.assertEqual(Decimal(response.data["amount"]), Decimal("60"))
self.assertEqual([s["name"] for s in response.data["sub_transactions"]], ["Plan", "Router"])
recurring = RecurringTransaction.all_objects.get()
generated = Transaction.all_objects.filter(recurring_transaction=recurring)
self.assertTrue(generated)
for transaction in generated:
self.assertEqual(transaction.amount, Decimal("60"))
self.assertEqual(len(self._parts(transaction)), 2)
other = Account.objects.create(name="B", currency=self.account.currency, owner=self.account.owner)
url = f"/api/recurring-transactions/{recurring.id}/"
self.assertEqual(
self.api.patch(url, {"type": "TR", "to_account": other.id,
"sub_transactions": [{"name": "X", "amount": "1"}]}, format="json").status_code,
400,
)
response = self.api.patch(url, {"type": "TR", "to_account": other.id}, format="json")
self.assertEqual(response.status_code, 200, response.data)
self.assertEqual(response.data["sub_transactions"], [])
@@ -32,7 +32,7 @@ class TransactionSearchTests(TestCase):
def make(self, days, description="", amount="10", **kwargs): def make(self, days, description="", amount="10", **kwargs):
return Transaction.objects.create( return Transaction.objects.create(
account=self.account, account=kwargs.pop("account", self.account),
type=Transaction.Type.EXPENSE, type=Transaction.Type.EXPENSE,
date=self.today + timedelta(days=days), date=self.today + timedelta(days=days),
reference_date=(self.today + timedelta(days=days)).replace(day=1), reference_date=(self.today + timedelta(days=days)).replace(day=1),
@@ -72,3 +72,13 @@ class TransactionSearchTests(TestCase):
self.assertEqual(self.search("wallet"), ["Groceries", "Other"]) self.assertEqual(self.search("wallet"), ["Groceries", "Other"])
self.assertEqual(self.search("groceries nomatch"), []) self.assertEqual(self.search("groceries nomatch"), [])
def test_filters_by_currency(self):
usd = Currency.objects.create(code="USD", name="Dollar", decimal_places=2)
other = Account.objects.create(name="Bank", currency=usd)
self.make(0, "brl")
self.make(0, "usd", account=other)
response = self.client.get(
reverse("transactions_search"), {"q": "", "currency": usd.id}
)
self.assertEqual([r["description"] for r in response.json()], ["usd"])
+294 -12
View File
@@ -1,6 +1,7 @@
from datetime import date from datetime import date
from decimal import Decimal from decimal import Decimal
from dateutil.relativedelta import relativedelta
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
from django.test import TestCase, override_settings from django.test import TestCase, override_settings
from django.urls import reverse from django.urls import reverse
@@ -251,19 +252,20 @@ class TransferTemplateTests(TransferTestBase):
"""Recurring and quick transactions of type Transfer.""" """Recurring and quick transactions of type Transfer."""
def _recurring(self, **overrides): def _recurring(self, **overrides):
return RecurringTransaction.objects.create( data = {
account=self.checking, "account": self.checking,
type=RecurringTransaction.Type.TRANSFER, "type": RecurringTransaction.Type.TRANSFER,
to_account=self.savings, "to_account": self.savings,
amount=Decimal("100"), "amount": Decimal("100"),
to_amount=Decimal("95"), "to_amount": Decimal("95"),
description="Savings", "description": "Savings",
start_date=timezone.localdate(), "start_date": timezone.localdate(),
recurrence_type=RecurringTransaction.RecurrenceType.MONTH, "recurrence_type": RecurringTransaction.RecurrenceType.MONTH,
recurrence_interval=1, "recurrence_interval": 1,
keep_at_most=2, "keep_at_most": 2,
**overrides, **overrides,
) }
return RecurringTransaction.objects.create(**data)
def test_recurring_transfer_generates_linked_pairs(self): def test_recurring_transfer_generates_linked_pairs(self):
recurring = self._recurring() recurring = self._recurring()
@@ -294,6 +296,227 @@ class TransferTemplateTests(TransferTestBase):
{(Transaction.Type.EXPENSE, Decimal("10")), (Transaction.Type.INCOME, Decimal("9"))}, {(Transaction.Type.EXPENSE, Decimal("10")), (Transaction.Type.INCOME, Decimal("9"))},
) )
def test_recurring_update_moves_unpaid_to_new_account_and_type(self):
recurring = self._recurring(type=RecurringTransaction.Type.EXPENSE)
recurring.create_upcoming_transactions()
paid = recurring.transactions.earliest("date")
paid.is_paid = True
paid.save()
recurring.account = self.savings
recurring.type = RecurringTransaction.Type.INCOME
recurring.save()
recurring.update_unpaid_transactions()
paid.refresh_from_db()
self.assertEqual(
(paid.account, paid.type), (self.checking, Transaction.Type.EXPENSE)
)
unpaid = recurring.transactions.filter(is_paid=False)
self.assertTrue(unpaid.exists())
self.assertEqual(
set(unpaid.values_list("account", "type")),
{(self.savings.id, Transaction.Type.INCOME)},
)
def test_recurring_transfer_update_moves_to_sides_to_new_account(self):
recurring = self._recurring()
recurring.create_upcoming_transactions()
brokerage = Account.objects.create(
name="Brokerage", group=self.checking.group, currency=self.checking.currency
)
recurring.to_account = brokerage
recurring.save()
recurring.update_unpaid_transactions()
self.assertEqual(
set(recurring.transactions.values_list("type", "account")),
{
(Transaction.Type.EXPENSE, self.checking.id),
(Transaction.Type.INCOME, brokerage.id),
},
)
def test_recurring_update_into_transfer_adds_to_sides(self):
recurring = self._recurring(type=RecurringTransaction.Type.INCOME)
recurring.create_upcoming_transactions()
occurrences = recurring.transactions.count()
recurring.type = RecurringTransaction.Type.TRANSFER
recurring.save()
recurring.update_unpaid_transactions()
expenses = recurring.transactions.filter(type=Transaction.Type.EXPENSE)
self.assertEqual(expenses.count(), occurrences)
self.assertEqual(recurring.transactions.count(), occurrences * 2)
for from_t in expenses:
to_t = from_t.linked_transaction
self.assertEqual(from_t.account, self.checking)
self.assertEqual(to_t.linked_transaction, from_t)
self.assertEqual(to_t.account, self.savings)
self.assertEqual(to_t.amount, Decimal("95"))
self.assertEqual(to_t.recurring_transaction, recurring)
def _assert_update_out_of_transfer_removes_to_sides(self):
recurring = self._recurring()
recurring.create_upcoming_transactions()
paid_from = recurring.transactions.filter(
type=Transaction.Type.EXPENSE
).earliest("date")
paid_from.is_paid = True
paid_from.save()
paid_to = paid_from.linked_transaction
occurrences = recurring.transactions.filter(
type=Transaction.Type.EXPENSE
).count()
recurring.type = RecurringTransaction.Type.EXPENSE
recurring.save()
recurring.update_unpaid_transactions()
unpaid = recurring.transactions.filter(is_paid=False)
self.assertEqual(unpaid.count(), occurrences - 1)
self.assertEqual(
set(unpaid.values_list("account", "type", "linked_transaction")),
{(self.checking.id, Transaction.Type.EXPENSE, None)},
)
paid_from.refresh_from_db()
self.assertEqual(paid_from.linked_transaction, paid_to)
@override_settings(ENABLE_SOFT_DELETE=False) # .env may turn it on
def test_recurring_update_out_of_transfer_removes_to_sides(self):
self._assert_update_out_of_transfer_removes_to_sides()
self.assertFalse(Transaction.deleted_objects.exists())
def _shorten(self, recurring, months):
"""Ends ``recurring`` after its first ``months`` + 1 occurrences."""
recurring.end_date = recurring.start_date + relativedelta(months=months)
recurring.save()
recurring.update_unpaid_transactions()
recurring.refresh_from_db()
def test_shorter_end_date_removes_later_unpaid_occurrences(self):
recurring = self._recurring(type=RecurringTransaction.Type.EXPENSE, keep_at_most=4)
recurring.create_upcoming_transactions()
last = recurring.transactions.latest("date")
last.is_paid = True # already happened: stays
last.save()
self.assertEqual(recurring.transactions.count(), 5)
self._shorten(recurring, 1)
end = recurring.end_date
self.assertEqual(
list(recurring.transactions.order_by("date").values_list("date", "is_paid")),
[(recurring.start_date, False), (end, False), (last.date, True)],
)
self.assertEqual(recurring.last_generated_date, last.date)
last.delete()
self._shorten(recurring, 1)
self.assertEqual(recurring.last_generated_date, end)
self.assertEqual(recurring.last_generated_reference_date, end.replace(day=1))
def test_shorter_end_date_removes_both_sides_of_later_transfers(self):
recurring = self._recurring(keep_at_most=3)
recurring.create_upcoming_transactions()
self.assertEqual(recurring.transactions.count(), 8)
self._shorten(recurring, 1)
self.assertEqual(recurring.transactions.count(), 4)
self.assertFalse(recurring.transactions.filter(date__gt=recurring.end_date).exists())
@override_settings(ENABLE_SOFT_DELETE=True)
def test_recurring_update_out_of_transfer_soft_deletes_to_sides(self):
self._assert_update_out_of_transfer_removes_to_sides()
self.assertTrue(Transaction.deleted_objects.exists())
self.assertFalse(
Transaction.deleted_objects.exclude(linked_transaction=None).exists()
)
def test_recurring_update_keeps_transfer_linked_by_hand(self):
recurring = self._recurring(type=RecurringTransaction.Type.INCOME)
recurring.create_upcoming_transactions()
income = recurring.transactions.earliest("date")
expense = Transaction.objects.create(
account=self.savings,
type=Transaction.Type.EXPENSE,
amount=Decimal("100"),
date=income.date,
description="Outside",
is_paid=False,
)
expense.link_transfer(income)
recurring.amount = Decimal("120")
recurring.save()
recurring.update_unpaid_transactions()
income.refresh_from_db()
self.assertEqual(income.linked_transaction, expense)
self.assertEqual(income.amount, Decimal("120"))
self.assertTrue(Transaction.objects.filter(pk=expense.pk).exists())
def test_recurring_update_unlinks_transfer_linked_by_hand_when_broken(self):
for change in ({"type": RecurringTransaction.Type.EXPENSE}, {"account": self.savings}):
with self.subTest(change=change):
recurring = self._recurring(type=RecurringTransaction.Type.INCOME)
recurring.create_upcoming_transactions()
income = recurring.transactions.earliest("date")
outside = Transaction.objects.create(
account=self.savings,
type=Transaction.Type.EXPENSE,
amount=Decimal("100"),
date=income.date,
description="Outside",
is_paid=False,
)
outside.link_transfer(income)
for field, value in change.items():
setattr(recurring, field, value)
recurring.save()
recurring.update_unpaid_transactions()
income.refresh_from_db()
outside.refresh_from_db()
self.assertIsNone(income.linked_transaction)
self.assertIsNone(outside.linked_transaction)
self.assertEqual(
(outside.account, outside.type, outside.amount),
(self.savings, Transaction.Type.EXPENSE, Decimal("100")),
)
def test_edit_view_moves_unpaid_transactions(self):
recurring = self._recurring(type=RecurringTransaction.Type.EXPENSE)
recurring.create_upcoming_transactions()
response = self.client.post(
reverse(
"recurring_transaction_edit",
kwargs={"recurring_transaction_id": recurring.id},
),
{
"type": "EX",
"account": self.savings.id,
"amount": "100",
"description": "Savings",
"start_date": recurring.start_date.isoformat(),
"recurrence_type": "month",
"recurrence_interval": "1",
"keep_at_most": "2",
},
HTTP_HX_REQUEST="true",
)
self.assertEqual(response.status_code, 204)
self.assertEqual(
set(recurring.transactions.values_list("account", flat=True)),
{self.savings.id},
)
def test_transfer_type_requires_other_account(self): def test_transfer_type_requires_other_account(self):
form = RecurringTransactionForm( form = RecurringTransactionForm(
{ {
@@ -439,3 +662,62 @@ class TransferReviewFixTests(TransferTestBase):
partner.refresh_from_db() partner.refresh_from_db()
self.assertIsNone(partner.linked_transaction) self.assertIsNone(partner.linked_transaction)
class BulkActionTests(TransferTestBase):
def _get(self, name, *ids, **kwargs):
query = "&".join(f"transactions={pk}" for pk in ids)
return self.client.get(
reverse(name, kwargs=kwargs) + f"?{query}", HTTP_HX_REQUEST="true"
)
def _expense(self, **fields):
return Transaction.objects.create(
account=self.checking,
type=Transaction.Type.EXPENSE,
amount=Decimal("10"),
date=date(2026, 3, 10),
description="Coffee",
**fields,
)
def test_change_month_updates_a_fully_selected_transfer_once_per_side(self):
from_t, to_t = self._transfer()
updated = []
receiver = lambda sender, **kwargs: updated.append(sender.pk) # noqa: E731
transaction_updated.connect(receiver, weak=False)
self.addCleanup(transaction_updated.disconnect, receiver)
response = self._get(
"transactions_bulk_change_month", from_t.id, to_t.id, change_type="next"
)
self.assertEqual(response.status_code, 204)
from_t.refresh_from_db()
to_t.refresh_from_db()
self.assertEqual(from_t.reference_date, date(2026, 4, 1))
self.assertEqual(to_t.reference_date, date(2026, 4, 1))
self.assertEqual(sorted(updated), sorted([from_t.pk, to_t.pk])) # once per side
def test_change_month_rejects_unknown_direction(self):
t = self._expense()
response = self._get("transactions_bulk_change_month", t.id, change_type="sideways")
self.assertEqual(response.status_code, 400)
def test_move_to_today(self):
t = self._expense()
self._get("transactions_bulk_move_to_today", t.id)
t.refresh_from_db()
self.assertEqual(t.date, timezone.localdate(timezone.now()))
def test_mute_skips_transactions_controlled_elsewhere(self):
muted_category = TransactionCategory.objects.create(name="Hidden", mute=True)
plain = self._expense()
by_category = self._expense(category=muted_category)
self._get("transactions_bulk_mute", plain.id, by_category.id)
plain.refresh_from_db()
by_category.refresh_from_db()
self.assertTrue(plain.mute)
self.assertFalse(by_category.mute)
+108
View File
@@ -312,6 +312,54 @@ class TransactionConvertViewTests(TestCase):
set(plan.transactions.values_list("amount", flat=True)), {70} set(plan.transactions.values_list("amount", flat=True)), {70}
) )
def test_transfer_form_prefills_side_from_type(self):
initial = self._get("transaction_convert_to_transfer").context["form"].initial
self.assertEqual(initial["from_account"], self.account.id)
self.assertEqual(initial["from_tags"], ["TestTag"])
self.assertNotIn("to_account", initial)
self.transaction.type = Transaction.Type.INCOME
self.transaction.save()
initial = self._get("transaction_convert_to_transfer").context["form"].initial
self.assertEqual(initial["to_account"], self.account.id)
self.assertNotIn("from_account", initial)
self.assertEqual(initial["from_amount"], 50)
self.assertNotIn("to_amount", initial)
def test_transfer_replaces_original_transaction(self):
other = Account.objects.create(name="Savings", currency=self.currency)
response = self._post(
"transaction_convert_to_transfer",
{
"from_account": self.account.id,
"to_account": other.id,
"from_amount": "50",
"date": "2030-01-15",
"is_paid": "on",
},
)
self.assertEqual(response.status_code, 204)
self.assertFalse(Transaction.objects.filter(id=self.transaction.id).exists())
self.assertEqual(
Transaction.objects.filter(linked_transaction__isnull=False).count(), 2
)
# A transfer can't be converted into one again
self.transaction = Transaction.objects.get(account=other)
self.assertEqual(self._get("transaction_convert_to_transfer").status_code, 404)
def test_invalid_transfer_keeps_original_transaction(self):
response = self._post(
"transaction_convert_to_transfer",
{
"from_account": self.account.id,
"to_account": self.account.id,
"from_amount": "50",
"date": "2030-01-15",
},
)
self.assertEqual(response.status_code, 200)
self.assertTrue(Transaction.objects.filter(id=self.transaction.id).exists())
def test_linked_transaction_cannot_be_converted_into_plan(self): def test_linked_transaction_cannot_be_converted_into_plan(self):
self._post( self._post(
"transaction_convert_to_installment_plan", "transaction_convert_to_installment_plan",
@@ -332,3 +380,63 @@ class TransactionConvertViewTests(TestCase):
self.assertEqual( self.assertEqual(
self._get("transaction_convert_to_installment_plan").status_code, 404 self._get("transaction_convert_to_installment_plan").status_code, 404
) )
@override_settings(
STORAGES={
"default": {"BACKEND": "django.core.files.storage.FileSystemStorage"},
"staticfiles": {"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"},
},
WHITENOISE_AUTOREFRESH=True,
)
class InstallmentRecurringIsolationTests(TestCase):
"""Per-object plan views must 404 for objects on another user's private account."""
def setUp(self):
User = get_user_model()
owner = User.objects.create_user(email="owner@test.com", password="pw")
User.objects.create_user(email="other@test.com", password="pw")
self.client.login(username="other@test.com", password="pw")
currency = Currency.objects.create(code="USD", name="US Dollar", decimal_places=2)
account = Account.all_objects.create(name="Private", currency=currency, owner=owner)
self.plan = InstallmentPlan.all_objects.create(
account=account,
type=Transaction.Type.EXPENSE,
description="Owner plan",
number_of_installments=3,
start_date=date(2025, 1, 1),
installment_amount=100,
)
self.recurring = RecurringTransaction.all_objects.create(
account=account,
type=Transaction.Type.EXPENSE,
amount=50,
description="Owner recurring",
start_date=date(2025, 1, 1),
recurrence_type=RecurringTransaction.RecurrenceType.MONTH,
recurrence_interval=1,
)
def test_other_user_gets_404(self):
cases = [
("get", "installment_plan_transactions", "installment_plan_id", self.plan),
("get", "installment_plan_edit", "installment_plan_id", self.plan),
("get", "installment_plan_refresh", "installment_plan_id", self.plan),
("delete", "installment_plan_delete", "installment_plan_id", self.plan),
("get", "recurring_transaction_transactions", "recurring_transaction_id", self.recurring),
("get", "recurring_transaction_edit", "recurring_transaction_id", self.recurring),
("get", "recurring_transaction_toggle_pause", "recurring_transaction_id", self.recurring),
("get", "recurring_transaction_finish", "recurring_transaction_id", self.recurring),
("delete", "recurring_transaction_delete", "recurring_transaction_id", self.recurring),
]
for method, name, kwarg, obj in cases:
with self.subTest(name=name):
url = reverse(name, kwargs={kwarg: obj.id})
response = getattr(self.client, method)(url, HTTP_HX_REQUEST="true")
self.assertEqual(response.status_code, 404)
self.assertTrue(InstallmentPlan.all_objects.filter(id=self.plan.id).exists())
self.recurring.refresh_from_db()
self.assertFalse(self.recurring.is_paused)
self.assertIsNone(self.recurring.end_date)
+20
View File
@@ -81,6 +81,21 @@ urlpatterns = [
views.bulk_clone_transactions, views.bulk_clone_transactions,
name="transactions_bulk_clone", name="transactions_bulk_clone",
), ),
path(
"transactions/actions/change-month/<str:change_type>/",
views.bulk_change_month,
name="transactions_bulk_change_month",
),
path(
"transactions/actions/move-to-today/",
views.bulk_move_to_today,
name="transactions_bulk_move_to_today",
),
path(
"transactions/actions/mute/",
views.bulk_mute,
name="transactions_bulk_mute",
),
path( path(
"transaction/<int:transaction_id>/pay/", "transaction/<int:transaction_id>/pay/",
views.transaction_pay, views.transaction_pay,
@@ -417,4 +432,9 @@ urlpatterns = [
views.transaction_convert_to_installment_plan, views.transaction_convert_to_installment_plan,
name="transaction_convert_to_installment_plan", name="transaction_convert_to_installment_plan",
), ),
path(
"transaction/<int:transaction_id>/convert/transfer/",
views.transaction_convert_to_transfer,
name="transaction_convert_to_transfer",
),
] ]
+65 -71
View File
@@ -6,6 +6,7 @@ from django.db.models.functions import Coalesce
from apps.transactions.models import Transaction from apps.transactions.models import Transaction
from apps.currencies.utils.convert import convert from apps.currencies.utils.convert import convert
from apps.currencies.utils.display import shown_exchange_currency
from apps.currencies.models import Currency from apps.currencies.models import Currency
@@ -29,6 +30,7 @@ def calculate_currency_totals(
"account__currency__exchange_currency__decimal_places", "account__currency__exchange_currency__decimal_places",
"account__currency__exchange_currency__prefix", "account__currency__exchange_currency__prefix",
"account__currency__exchange_currency__suffix", "account__currency__exchange_currency__suffix",
"account__currency__show_exchanged_amounts",
) )
.annotate( .annotate(
expense_current=Coalesce( expense_current=Coalesce(
@@ -177,7 +179,10 @@ def calculate_currency_totals(
converted_val if converted_val is not None else Decimal("0") converted_val if converted_val is not None else Decimal("0")
) )
currency_data["exchanged"] = exchanged_details # Consolidated totals below need the exchanged values either way;
# the currency's setting only hides them from display
if total["account__currency__show_exchanged_amounts"]:
currency_data["exchanged"] = exchanged_details
if exchange_currency_obj_for_this_total.id not in currencies_using_exchange: if exchange_currency_obj_for_this_total.id not in currencies_using_exchange:
currencies_using_exchange[exchange_currency_obj_for_this_total.id] = [] currencies_using_exchange[exchange_currency_obj_for_this_total.id] = []
@@ -255,9 +260,10 @@ def calculate_currency_totals(
converted_val if converted_val is not None else Decimal("0") converted_val if converted_val is not None else Decimal("0")
) )
currency_data_for_db_currency["exchanged"] = ( if db_currency.show_exchanged_amounts:
exchanged_details_for_db_currency currency_data_for_db_currency["exchanged"] = (
) exchanged_details_for_db_currency
)
# Ensure its own exchange_currency is registered in currencies_using_exchange # Ensure its own exchange_currency is registered in currencies_using_exchange
# and add it to the queue if it hasn't been processed yet for deep add. # and add it to the queue if it hasn't been processed yet for deep add.
@@ -297,12 +303,28 @@ def calculate_currency_totals(
"total_projected": data_consolidated_currency["total_projected"], "total_projected": data_consolidated_currency["total_projected"],
"total_final": data_consolidated_currency["total_final"], "total_final": data_consolidated_currency["total_final"],
} }
# What each currency adds to the consolidated total, in the
# consolidated currency, starting with the currency's own total
components = [
{
"currency": data_consolidated_currency["currency"],
"total_final": data_consolidated_currency["total_final"],
}
]
if currency_id_consolidated in currencies_using_exchange: if currency_id_consolidated in currencies_using_exchange:
for original_currency_info in currencies_using_exchange[ for original_currency_info in currencies_using_exchange[
currency_id_consolidated currency_id_consolidated
]: ]:
exchanged_values_from_original = original_currency_info["exchanged"] exchanged_values_from_original = original_currency_info["exchanged"]
components.append(
{
"currency": result[original_currency_info["currency_id"]][
"currency"
],
"total_final": exchanged_values_from_original["total_final"],
}
)
for field in [ for field in [
"expense_current", "expense_current",
"expense_projected", "expense_projected",
@@ -317,6 +339,9 @@ def calculate_currency_totals(
field field
] ]
consolidated_data["components"] = calculate_consolidated_shares(
components, consolidated_data["total_final"]
)
result[currency_id_consolidated]["consolidated"] = consolidated_data result[currency_id_consolidated]["consolidated"] = consolidated_data
# Sort currencies by their final_total or consolidated final_total, descending # Sort currencies by their final_total or consolidated final_total, descending
@@ -335,6 +360,36 @@ def calculate_currency_totals(
return result return result
def calculate_consolidated_shares(components, consolidated_total):
"""
Give each component of a consolidated total its share of that total, as a
percentage. Components that add nothing are left out and the rest are sorted
from the largest to the smallest.
Shares are taken over the absolute total, so a component that adds to it
always has a positive share and one that takes from it (a debt, for
example) a negative one. Without a total to divide by, the share is None.
"""
shares = []
for component in components:
if not component["total_final"]:
continue
shares.append(
{
**component,
"share": (
component["total_final"] / abs(consolidated_total) * 100
if consolidated_total
else None
),
}
)
return sorted(shares, key=lambda share: share["total_final"], reverse=True)
def calculate_percentage_distribution(currency_totals): def calculate_percentage_distribution(currency_totals):
""" """
Calculate percentage distribution of financial metrics for each currency. Calculate percentage distribution of financial metrics for each currency.
@@ -377,70 +432,9 @@ def calculate_percentage_distribution(currency_totals):
"expense_projected": 0, "expense_projected": 0,
} }
# If there's exchanged data, calculate percentages for that too p = percentages[currency_id]["percentages"]
if "exchanged" in data: p["income_total"] = p["income_current"] + p["income_projected"]
exchanged_total = sum( p["expense_total"] = p["expense_current"] + p["expense_projected"]
[
abs(data.get("exchanged", {}).get("income_current", Decimal("0"))),
abs(
data.get("exchanged", {}).get("income_projected", Decimal("0"))
),
abs(data.get("exchanged", {}).get("expense_current", Decimal("0"))),
abs(data.get("exchanged", {}).get("income_current", Decimal("0"))),
]
)
percentages[currency_id]["exchanged"] = {
"currency": data["exchanged"]["currency"],
"percentages": {},
}
if exchanged_total > 0:
percentages[currency_id]["exchanged"]["percentages"] = {
"income_current": (
abs(
data.get("exchanged", {}).get(
"income_current", Decimal("0")
)
)
/ exchanged_total
)
* 100,
"income_projected": (
abs(
data.get("exchanged", {}).get(
"income_projected", Decimal("0")
)
)
/ exchanged_total
)
* 100,
"expense_current": (
abs(
data.get("exchanged", {}).get(
"expense_current", Decimal("0")
)
)
/ exchanged_total
)
* 100,
"expense_projected": (
abs(
data.get("exchanged", {}).get(
"income_current", Decimal("0")
)
)
/ exchanged_total
)
* 100,
}
else:
percentages[currency_id]["exchanged"]["percentages"] = {
"income_current": 0,
"income_projected": 0,
"expense_current": 0,
"expense_projected": 0,
}
return percentages return percentages
@@ -460,7 +454,7 @@ def calculate_account_totals(transactions_queryset, ignore_empty=False):
"account__currency__decimal_places", "account__currency__decimal_places",
"account__currency__prefix", "account__currency__prefix",
"account__currency__suffix", "account__currency__suffix",
"account__exchange_currency", exchange_currency_id=shown_exchange_currency("account__"),
).annotate( ).annotate(
expense_current=Coalesce( expense_current=Coalesce(
Sum( Sum(
@@ -527,8 +521,8 @@ def calculate_account_totals(transactions_queryset, ignore_empty=False):
account_id = total["account"] account_id = total["account"]
currency = Currency.objects.get(id=total["account__currency__id"]) currency = Currency.objects.get(id=total["account__currency__id"])
exchange_currency = ( exchange_currency = (
Currency.objects.get(id=total["account__exchange_currency"]) Currency.objects.get(id=total["exchange_currency_id"])
if total["account__exchange_currency"] if total["exchange_currency_id"]
else None else None
) )
+77
View File
@@ -1,10 +1,12 @@
import json import json
from copy import deepcopy from copy import deepcopy
from dateutil.relativedelta import relativedelta
from django.contrib import messages from django.contrib import messages
from django.contrib.auth.decorators import login_required from django.contrib.auth.decorators import login_required
from django.db.models import Q from django.db.models import Q
from django.http import HttpResponse from django.http import HttpResponse
from django.utils import timezone
from django.utils.translation import gettext_lazy as _, ngettext_lazy from django.utils.translation import gettext_lazy as _, ngettext_lazy
from apps.common.decorators.htmx import only_htmx from apps.common.decorators.htmx import only_htmx
@@ -30,6 +32,7 @@ def _copy(transaction):
new_transaction.tags.add(*transaction.tags.all()) new_transaction.tags.add(*transaction.tags.all())
new_transaction.entities.add(*transaction.entities.all()) new_transaction.entities.add(*transaction.entities.all())
new_transaction.copy_sub_transactions_from(transaction)
return new_transaction return new_transaction
@@ -149,6 +152,80 @@ def bulk_undelete_transactions(request):
) )
def _bulk_update(request, transactions, change):
"""Applies change to each selected transaction like its single action does.
A transfer moves as a pair through save(), so a selected partner is skipped.
"""
done = set()
for transaction in transactions:
if transaction.linked_transaction_id in done:
continue
old_data = deepcopy(transaction)
change(transaction)
transaction.save()
transaction_updated.send(sender=transaction, old_data=old_data)
done.add(transaction.id)
count = len(done)
messages.success(
request,
ngettext_lazy(
"%(count)s transaction updated successfully",
"%(count)s transactions updated successfully",
count,
)
% {"count": count},
)
return HttpResponse(status=204, headers={"HX-Trigger": "updated"})
@only_htmx
@login_required
def bulk_change_month(request, change_type):
months = {"previous": -1, "next": 1}.get(change_type)
if months is None:
return HttpResponse(status=400)
def change(transaction):
transaction.reference_date += relativedelta(months=months)
return _bulk_update(
request,
Transaction.objects.filter(id__in=request.GET.getlist("transactions")),
change,
)
@only_htmx
@login_required
def bulk_move_to_today(request):
today = timezone.localdate(timezone.now())
def change(transaction):
transaction.date = today
return _bulk_update(
request,
Transaction.objects.filter(id__in=request.GET.getlist("transactions")),
change,
)
@only_htmx
@login_required
def bulk_mute(request):
# Visibility controlled by the account or category can't be changed here.
transactions = Transaction.objects.filter(
id__in=request.GET.getlist("transactions"), mute=False
).exclude(Q(account__untracked_by=request.user) | Q(category__mute=True))
def change(transaction):
transaction.mute = True
return _bulk_update(request, transactions, change)
@only_htmx @only_htmx
@login_required @login_required
def bulk_clone_transactions(request): def bulk_clone_transactions(request):
@@ -12,6 +12,7 @@ from apps.transactions.forms import (
InstallmentPlanForm, InstallmentPlanForm,
QuickTransactionForm, QuickTransactionForm,
RecurringTransactionForm, RecurringTransactionForm,
TransferForm,
) )
from apps.transactions.models import Transaction from apps.transactions.models import Transaction
@@ -36,6 +37,10 @@ def _initial_from_transaction(transaction: Transaction):
return initial return initial
def _sub_transactions_initial(transaction: Transaction):
return list(transaction.sub_transactions.values("name", "amount"))
def _get_unlinked_transaction(transaction_id): def _get_unlinked_transaction(transaction_id):
# Transactions already generated by a plan or recurrence can't be converted again # Transactions already generated by a plan or recurrence can't be converted again
return get_object_or_404( return get_object_or_404(
@@ -77,6 +82,7 @@ def transaction_convert_to_quick_transaction(request, transaction_id):
) )
else: else:
form = QuickTransactionForm( form = QuickTransactionForm(
sub_transactions_initial=_sub_transactions_initial(transaction),
initial={ initial={
**_initial_from_transaction(transaction), **_initial_from_transaction(transaction),
"is_paid": transaction.is_paid, "is_paid": transaction.is_paid,
@@ -113,6 +119,7 @@ def transaction_convert_to_recurring_transaction(request, transaction_id):
) )
else: else:
form = RecurringTransactionForm( form = RecurringTransactionForm(
sub_transactions_initial=_sub_transactions_initial(transaction),
initial={ initial={
**_initial_from_transaction(transaction), **_initial_from_transaction(transaction),
"amount": transaction.amount, "amount": transaction.amount,
@@ -153,6 +160,7 @@ def transaction_convert_to_installment_plan(request, transaction_id):
) )
else: else:
form = InstallmentPlanForm( form = InstallmentPlanForm(
sub_transactions_initial=_sub_transactions_initial(transaction),
initial={ initial={
**_initial_from_transaction(transaction), **_initial_from_transaction(transaction),
"installment_amount": transaction.amount, "installment_amount": transaction.amount,
@@ -168,3 +176,54 @@ def transaction_convert_to_installment_plan(request, transaction_id):
transaction_id, transaction_id,
"transaction_convert_to_installment_plan", "transaction_convert_to_installment_plan",
) )
@only_htmx
@login_required
@require_http_methods(["GET", "POST"])
def transaction_convert_to_transfer(request, transaction_id):
transaction = _get_unlinked_transaction(transaction_id)
if transaction.linked_transaction_id:
raise Http404() # Already a transfer
if request.method == "POST":
form = TransferForm(request.POST)
if form.is_valid():
with db_transaction.atomic():
form.save()
transaction.delete()
messages.success(request, _("Transaction converted into a transfer"))
return HttpResponse(
status=204,
headers={"HX-Trigger": "updated, hide_offcanvas"},
)
else:
# An expense leaves its account (the "from" side), an income enters it ("to")
side = "from" if transaction.type == Transaction.Type.EXPENSE else "to"
form = TransferForm(
initial={
f"{side}_account": transaction.account_id,
f"{side}_category": transaction.category_id,
f"{side}_tags": list(transaction.tags.values_list("name", flat=True)),
f"{side}_entities": list(
transaction.entities.values_list("name", flat=True)
),
# To amount stays empty, so it follows from amount
"from_amount": transaction.amount,
"date": transaction.date,
"reference_date": transaction.reference_date,
"description": transaction.description,
"notes": transaction.notes,
"mute": transaction.mute,
"is_paid": transaction.is_paid,
}
)
return _render_convert_form(
request,
form,
_("Convert into transfer"),
transaction_id,
"transaction_convert_to_transfer",
)
@@ -66,8 +66,8 @@ def finished_installment_plans_list(request):
@login_required @login_required
@require_http_methods(["GET"]) @require_http_methods(["GET"])
def installment_plan_transactions(request, installment_plan_id): def installment_plan_transactions(request, installment_plan_id):
installment_plan = get_object_or_404(InstallmentPlan, id=installment_plan_id) installment_plan = get_object_or_404(InstallmentPlan.objects, id=installment_plan_id)
transactions = installment_plan.transactions.all().order_by("reference_date", "id") transactions = installment_plan.transactions.prefetch_related("sub_transactions").order_by("reference_date", "id")
return render( return render(
request, request,
@@ -106,7 +106,7 @@ def installment_plan_add(request):
@login_required @login_required
@require_http_methods(["GET", "POST"]) @require_http_methods(["GET", "POST"])
def installment_plan_edit(request, installment_plan_id): def installment_plan_edit(request, installment_plan_id):
installment_plan = get_object_or_404(InstallmentPlan, id=installment_plan_id) installment_plan = get_object_or_404(InstallmentPlan.objects, id=installment_plan_id)
if request.method == "POST": if request.method == "POST":
form = InstallmentPlanForm(request.POST, instance=installment_plan) form = InstallmentPlanForm(request.POST, instance=installment_plan)
@@ -134,7 +134,7 @@ def installment_plan_edit(request, installment_plan_id):
@login_required @login_required
@require_http_methods(["GET"]) @require_http_methods(["GET"])
def installment_plan_refresh(request, installment_plan_id): def installment_plan_refresh(request, installment_plan_id):
installment_plan = get_object_or_404(InstallmentPlan, id=installment_plan_id) installment_plan = get_object_or_404(InstallmentPlan.objects, id=installment_plan_id)
installment_plan.update_transactions() installment_plan.update_transactions()
messages.success(request, _("Installment Plan refreshed successfully")) messages.success(request, _("Installment Plan refreshed successfully"))
@@ -151,7 +151,7 @@ def installment_plan_refresh(request, installment_plan_id):
@login_required @login_required
@require_http_methods(["DELETE"]) @require_http_methods(["DELETE"])
def installment_plan_delete(request, installment_plan_id): def installment_plan_delete(request, installment_plan_id):
installment_plan = get_object_or_404(InstallmentPlan, id=installment_plan_id) installment_plan = get_object_or_404(InstallmentPlan.objects, id=installment_plan_id)
installment_plan.delete() installment_plan.delete()
@@ -155,6 +155,7 @@ def quick_transaction_add_as_transaction(request, quick_transaction_id):
new_transaction.save() new_transaction.save()
new_transaction.tags.set(quick_transaction.tags.all()) new_transaction.tags.set(quick_transaction.tags.all())
new_transaction.entities.set(quick_transaction.entities.all()) new_transaction.entities.set(quick_transaction.entities.all())
new_transaction.copy_sub_transactions_from(quick_transaction)
to_transaction = quick_transaction.complete_transfer(new_transaction) to_transaction = quick_transaction.complete_transfer(new_transaction)
transaction_created.send(sender=new_transaction) transaction_created.send(sender=new_transaction)
@@ -88,9 +88,9 @@ def finished_recurring_transactions_list(request):
@require_http_methods(["GET"]) @require_http_methods(["GET"])
def recurring_transaction_transactions(request, recurring_transaction_id): def recurring_transaction_transactions(request, recurring_transaction_id):
recurring_transaction = get_object_or_404( recurring_transaction = get_object_or_404(
RecurringTransaction, id=recurring_transaction_id RecurringTransaction.objects, id=recurring_transaction_id
) )
transactions = recurring_transaction.transactions.collapse_transfers().order_by( transactions = recurring_transaction.transactions.collapse_transfers().prefetch_related("sub_transactions").order_by(
"reference_date", "id" "reference_date", "id"
) )
@@ -132,7 +132,7 @@ def recurring_transaction_add(request):
@require_http_methods(["GET", "POST"]) @require_http_methods(["GET", "POST"])
def recurring_transaction_edit(request, recurring_transaction_id): def recurring_transaction_edit(request, recurring_transaction_id):
recurring_transaction = get_object_or_404( recurring_transaction = get_object_or_404(
RecurringTransaction, id=recurring_transaction_id RecurringTransaction.objects, id=recurring_transaction_id
) )
if request.method == "POST": if request.method == "POST":
@@ -162,7 +162,7 @@ def recurring_transaction_edit(request, recurring_transaction_id):
@require_http_methods(["GET"]) @require_http_methods(["GET"])
def recurring_transaction_toggle_pause(request, recurring_transaction_id): def recurring_transaction_toggle_pause(request, recurring_transaction_id):
recurring_transaction = get_object_or_404( recurring_transaction = get_object_or_404(
RecurringTransaction, id=recurring_transaction_id RecurringTransaction.objects, id=recurring_transaction_id
) )
current_paused = recurring_transaction.is_paused current_paused = recurring_transaction.is_paused
recurring_transaction.is_paused = not current_paused recurring_transaction.is_paused = not current_paused
@@ -201,7 +201,7 @@ def recurring_transaction_toggle_pause(request, recurring_transaction_id):
@require_http_methods(["GET"]) @require_http_methods(["GET"])
def recurring_transaction_finish(request, recurring_transaction_id): def recurring_transaction_finish(request, recurring_transaction_id):
recurring_transaction: RecurringTransaction = get_object_or_404( recurring_transaction: RecurringTransaction = get_object_or_404(
RecurringTransaction, id=recurring_transaction_id RecurringTransaction.objects, id=recurring_transaction_id
) )
today = timezone.localdate(timezone.now()) - relativedelta(days=1) today = timezone.localdate(timezone.now()) - relativedelta(days=1)
@@ -227,7 +227,7 @@ def recurring_transaction_finish(request, recurring_transaction_id):
@require_http_methods(["DELETE"]) @require_http_methods(["DELETE"])
def recurring_transaction_delete(request, recurring_transaction_id): def recurring_transaction_delete(request, recurring_transaction_id):
recurring_transaction = get_object_or_404( recurring_transaction = get_object_or_404(
RecurringTransaction, id=recurring_transaction_id RecurringTransaction.objects, id=recurring_transaction_id
) )
recurring_transaction.delete() recurring_transaction.delete()
@@ -409,6 +409,8 @@ def transactions_bulk_edit(request):
transaction.tags.set(value) transaction.tags.set(value)
elif field_name == "entities": elif field_name == "entities":
transaction.entities.set(value) transaction.entities.set(value)
elif field_name == "amount" and transaction.has_sub_transactions:
continue # its amount is the sum of its parts
else: else:
setattr(transaction, field_name, value) setattr(transaction, field_name, value)
@@ -713,10 +715,25 @@ def filter_preset_create(request):
for key in TransactionsFilter.base_filters for key in TransactionsFilter.base_filters
if key in request.POST if key in request.POST
} }
# Dates are stored as ISO, so a later change of date format or language
# doesn't stop the preset from reading them
date_form = TransactionsFilter(request.POST).form
for key in TransactionsFilter.relative_date_units:
if date_form[key].data:
parameters[key] = [date_form[key].data]
has_dates = any(
any(parameters.get(key, [])) for key in TransactionsFilter.relative_date_units
)
relative_to = None
if has_dates and request.POST.get("relative_dates") == "true":
relative_to = timezone.localdate(timezone.now())
FilterPreset.objects.create( FilterPreset.objects.create(
owner=request.user, owner=request.user,
name=name, name=name,
parameters=parameters, parameters=parameters,
relative_to=relative_to,
) )
return render( return render(
request, request,
@@ -736,6 +753,10 @@ def filter_preset_apply(request, preset_id):
data.setlist(key, values) data.setlist(key, values)
transaction_filter = TransactionsFilter(data) transaction_filter = TransactionsFilter(data)
# Fixed presets shift by nothing, which still puts ISO dates in the user's format
today = timezone.localdate(timezone.now())
transaction_filter.shift_dates(since=preset.relative_to or today, until=today)
response = render( response = render(
request, request,
"transactions/fragments/filter_form.html", "transactions/fragments/filter_form.html",
@@ -802,6 +823,7 @@ def transaction_all_list(request):
"installment_plan", "installment_plan",
"entities", "entities",
"dca_expense_entries", "dca_expense_entries",
"sub_transactions",
"dca_income_entries", "dca_income_entries",
"linked_transaction__account__currency", "linked_transaction__account__currency",
"linked_transaction__account__group", "linked_transaction__account__group",
@@ -971,6 +993,7 @@ def transactions_trash_can_list(request):
"installment_plan", "installment_plan",
"entities", "entities",
"dca_expense_entries", "dca_expense_entries",
"sub_transactions",
"dca_income_entries", "dca_income_entries",
"linked_transaction__account__currency", "linked_transaction__account__currency",
"linked_transaction__account__group", "linked_transaction__account__group",
@@ -985,6 +1008,13 @@ def transactions_trash_can_list(request):
) )
# Query params the transaction picker may filter by (TransactionSelect.filters)
SEARCH_FILTERS = {
"currency": "account__currency_id",
"account": "account_id",
}
@login_required @login_required
@require_http_methods(["GET"]) @require_http_methods(["GET"])
def get_recent_transactions(request, filter_type=None): def get_recent_transactions(request, filter_type=None):
@@ -1002,6 +1032,11 @@ def get_recent_transactions(request, filter_type=None):
elif filter_type == "income": elif filter_type == "income":
queryset = queryset.filter(type=Transaction.Type.INCOME) queryset = queryset.filter(type=Transaction.Type.INCOME)
for param, lookup in SEARCH_FILTERS.items():
value = request.GET.get(param, "")
if value.isdigit():
queryset = queryset.filter(**{lookup: value})
if search_term: if search_term:
queryset = search_transactions(queryset, search_term) queryset = search_transactions(queryset, search_term)
Loaded 100 of 149 files, more files were not shown because too many files have changed in this diff. Show more