Compare commits

...
182 Commits
Author SHA1 Message Date
Herculino Trotta e552723078 Merge pull request #674 from eitchtee/dependabot/uv/soupsieve-4.0
chore(deps): bump soupsieve from 2.9 to 4.0
2026-11-13 15:40:59 -03:00
Herculino Trotta 01dd0920a9 Merge pull request #675 from eitchtee/fix/airdatepicker-hx-boosted
fix(datepicker): keep preserved pickers working after boosted navigation
2026-11-13 15:40:28 -03:00
Herculino Trotta e2981bf1f6 fix(datepicker): keep pickers working after history back and forward
A history restore swaps in htmx's snapshot of the body, which carries a
lifeless copy of AirDatepicker's global container. The previous fix looked
the container up by id, found that copy, and left the library's real
container detached, so pickers opened into nothing.

Track the container through datepicker.$container instead, remove any
snapshot copies after a swap, and reattach the real one when detached.
2026-11-13 15:18:06 -03:00
Herculino Trotta 9cf8cca1cb fix(datepicker): keep preserved pickers working after boosted navigation
The filter on the Monthly and All pages is wrapped in hx-preserve, so a
boosted navigation keeps its pickers instead of recreating them. The
AirDatepicker global container they render into lives on the body and is
swapped out with it, leaving those pickers drawing into a detached node.

After each swap, move the detached container's pickers into the new
container, or reattach it when no new picker created one.
2026-11-13 15:04:41 -03:00
dependabot[bot] 1ec0cfc02f chore(deps): bump soupsieve from 2.9 to 4.0
Bumps [soupsieve](https://github.com/facelessuser/soupsieve) from 2.9 to 4.0.
- [Release notes](https://github.com/facelessuser/soupsieve/releases)
- [Commits](https://github.com/facelessuser/soupsieve/compare/2.9...4.0)

---
updated-dependencies:
- dependency-name: soupsieve
  dependency-version: '4.0'
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-11-13 14:09:59 +00:00
eitchtee 1c891a0a60 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-12 03:24:33 +00:00
Herculino Trotta 03c22b8fc2 Merge pull request #670 from officialasishkumar/feat/import-run-account
feat(import): pick the account when running an import
2026-11-12 00:24:10 -03:00
Herculino Trotta df6ccda7b3 feat(import): read any line of the file with a line transformation
Some banks name the account once in a header cell above the table, which
skip_lines and start_row throw away. A `line` transformation returns a line
of the file (or one of its columns) as text, so a mapping can read it and
clean it up with regex or replace.

CSV keeps only the requested lines during the existing counting pass; Excel
reads them straight from each sheet. An account picked on the run still
overrides the mapped one.
2026-11-12 00:17:14 -03:00
Herculino Trotta e617e809c0 fix(import): reject archived accounts when importing through the API
The upload form only lists active accounts, but the API accepted any account
the user can access, archived ones included.
2026-11-12 00:14:06 -03:00
eitchtee 1662fe70e0 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-11 18:26:08 +00:00
Herculino Trotta ca09f37b51 Merge pull request #669 from officialasishkumar/feat/net-worth-consolidated-shares
feat(net-worth): break consolidated totals down by currency share
2026-11-11 15:25:44 -03:00
Asish Kumar 1ed5dc5d31 feat(import): pick the account when running an import
An import run can now be given an account, and every transaction it
creates goes into that account instead of the one the profile maps. This
lets a single profile serve several accounts at the same bank, for
statements that only name the account in a header cell or not at all,
instead of keeping one copy of the profile per account.

- ImportRun gets an optional `account` (SET_NULL), shown on the run card.
- The upload form offers the user's active accounts when the profile
  imports transactions, and leaves the field out for profiles that import
  accounts, currencies, categories, tags or entities.
- CSV and Excel rows take the run's account before deduplication, so
  rules comparing `account` check against the picked account. The
  profile's account mapping is skipped for these rows.
- QIF files use the run's account instead of the account named after the
  file.
- The import API accepts an optional `account_id`, limited to accounts
  the user can access, rejected for profiles that don't import
  transactions, and returns `account` on import runs.

Leaving the account empty keeps the current behaviour.

Closes #538
2026-11-11 22:41:37 +05:30
Herculino Trotta dea8cfee4e test(net-worth): assert share text instead of badge markup 2026-11-11 14:02:56 -03:00
Herculino Trotta 4bf2c2552b feat(net-worth): collapse consolidated breakdown and align share badges
Consolidated rows are collapsible and remember their state per currency in localStorage. Share percentages render as badges on a shared subgrid column so they line up regardless of amount width. Currency and account menus drop their extra padding to align with the card title.
2026-11-11 13:57:21 -03:00
Asish Kumar d590dc39ab feat(net-worth): break consolidated totals down by currency share
A consolidated total on the Net Worth page was a single number: it said
what everything is worth in one currency, but not how much of it each
currency makes up, which is what you want to know when you hold the same
money in several currencies (or in currencies standing for gold, stocks
or crypto).

calculate_currency_totals now keeps, next to each consolidated total, the
components it was built from: the currency's own total and the exchanged
total of every currency that exchanges into it, all in the consolidated
currency. calculate_consolidated_shares gives each of them its share of
the total as a percentage, drops the ones that add nothing and sorts the
rest from largest to smallest. Shares are taken over the absolute total,
so a currency that adds to it always has a positive share and one that
takes from it (debt) a negative one; with a zero total there's no share.

Currencies that hide their exchanged amounts still show up, since they
still count towards the consolidated total, and so do exchange targets
that only exist through the deep search.

On the Net Worth page, each Consolidated row now lists its components
with their share and amount in the consolidated currency. Clicking one
shows that currency's line on the charts, like the currency rows above.
2026-11-11 21:27:17 +05:30
Herculino Trotta 6c3c4c03ec Merge pull request #668 from eitchtee/fix/insights-overview-exchanged
fix(insights): show exchanged amounts in the overview
2026-11-11 10:53:21 -03:00
Herculino Trotta 220ccee140 fix(insights): show exchanged amounts in the overview 2026-11-11 10:53:15 -03:00
Herculino Trotta 1c7bc3db1d Merge pull request #667 from eitchtee/fix/yearly-filter-pills-wrap
fix(yearly-overview): wrap long names in account and currency filters
2026-11-11 10:53:13 -03:00
Herculino Trotta bf83975b97 fix(yearly-overview): wrap long names in account and currency filters 2026-11-11 10:53:04 -03:00
Herculino Trotta 1ab06d544d Merge pull request #666 from eitchtee/fix/badge-overflow
fix(ui): let badges grow to fit long text
2026-11-11 10:52:58 -03:00
Herculino Trotta d2cd4d577a fix(ui): let badges grow to fit long text 2026-11-11 10:52:50 -03:00
eitchtee 4d886a4fea chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-11 13:41:24 +00:00
Herculino Trotta 3a619dc197 Merge pull request #665 from eitchtee/feat/investments-exchanged-amounts
feat(investments): show exchanged amounts in movements and honor the display switch
2026-11-11 10:40:56 -03:00
Herculino Trotta cd33b82b4e feat(investments): show exchanged amounts in movements and honor the display switch
Movements without an other-currency transfer side now show the exchanged
amount, and the converted value card follows show_exchanged_amounts.
2026-11-11 10:39:06 -03:00
eitchtee 7aed2e8574 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-11 13:27:09 +00:00
Herculino Trotta 9d36af78c4 Merge pull request #664 from officialasishkumar/feat/optional-exchanged-amounts
feat(currencies): make displaying exchanged amounts optional
2026-11-11 10:26:39 -03:00
Herculino Trotta 00d37d8c81 refactor(currencies): keep the exchanged amounts display rule in one place
Account.displayed_exchange_currency holds the account/currency precedence for transactions, and shown_exchange_currency() expresses the same rule as a query expression for the insights and account totals, replacing the per-module checks.
2026-11-11 10:22:50 -03:00
Asish Kumar d3f29f5ad1 feat(currencies): make displaying exchanged amounts optional
Currencies and accounts get a "Show exchanged amounts" switch, on by
default. Turning it off hides the amounts converted to the exchange
currency without removing the exchange currency, so consolidated totals
on the Net Worth page keep working.

The switch belongs to whichever one provides the exchange currency:

- Transactions (including both legs of combined transfers) use the
  account's switch when the account has its own exchange currency, and
  the currency's switch otherwise, matching how the exchanged amount is
  picked.
- Account cards and Net Worth account rows follow the account.
- Currency cards, the monthly summary and daily allowance, Net Worth
  currency rows and the month-by-month and year-by-year insights follow
  the currency.

Currency totals still convert hidden currencies, because the
consolidated totals are built from those values; the setting only
leaves them out of the displayed data. The API keeps returning
exchanged_amount for transactions, and the account serializer exposes
the new field (currencies already serialize all fields).

Closes #486
2026-11-11 15:26:04 +05:30
eitchtee 60a0365b23 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-11 01:05:02 +00:00
Herculino Trotta edd045607f Merge pull request #663 from eitchtee/feat/dca-improvements
feat(dca): optional amounts, currency-checked links and entities
2026-11-10 22:04:34 -03:00
Herculino Trotta 3468b10e94 feat(dca): limit create-transaction accounts to the strategy currencies 2026-11-10 21:57:39 -03:00
Herculino Trotta 750b86394e feat(dca): optional amounts, currency-checked links and entities on entries
- Amounts default to the linked transaction's amount when left empty
- Linked transactions must be in the strategy's payment/target currency; pickers only search those
- Allow setting entities on transactions created from an entry
- Fix crash when only the expense transaction was linked
2026-11-10 21:55:29 -03:00
Herculino Trotta 706f8fea2b feat(transactions): allow filtering the transaction picker search
TransactionSelect takes a filters dict sent as query params; the search endpoint applies allowlisted ones (currency, account).
2026-11-10 21:55:29 -03:00
Herculino Trotta eafaf0027c Merge pull request #662 from eitchtee/fix/investments-short-period-return
fix(investments): solve short-period return over the holding period
2026-11-10 20:38:42 -03:00
Herculino Trotta 0093cabfc9 fix(investments): solve short-period return over the holding period
Holdings under a year annualized their rate before turning it back into
the period return, so a few days with a modest gain (+1.3% in one day is
~10,600% a year) fell outside xirr's search range and showed no percentage.
Solve the rate over the period held instead.
2026-11-10 20:35:03 -03:00
eitchtee 37ecc7da1b chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-10 13:59:54 +00:00
Herculino Trotta 5d5a423db5 Merge pull request #661 from eitchtee/weblate
Translations update from Weblate
2026-11-10 10:59:29 -03:00
Weblate b6ac69716c Merge remote-tracking branch 'origin/main' 2026-11-10 13:55:36 +00:00
eitchtee c256671acc chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-10 13:55:32 +00:00
Weblate 0e62594133 Merge remote-tracking branch 'origin/main' 2026-11-10 13:55:06 +00:00
Herculino Trotta 07933ac5c1 Merge pull request #660 from jayhemnani9910/fix/recurring-month-end-keeps-day
fix(transactions): keep a recurrence on its day after a short month
2026-11-10 10:55:03 -03:00
masttera 3c30116072 locale(Russian): update translation
Currently translated at 56.6% (543 of 959 strings)

Translation: WYGIWYH/App
Translate-URL: https://translations.herculino.com/projects/wygiwyh/app/ru/
2026-11-10 07:58:57 +00:00
eitchtee 7ebcccdeda chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-10 02:41:08 +00:00
Herculino Trotta 674870641c Merge pull request #659 from eitchtee/fix/linked-transfer-exchanged-values
feat(transactions): redesign combined transfer items + show exchanged amounts for both sides
2026-11-09 23:40:42 -03:00
Herculino Trotta 0854805630 feat(transactions): show combined transfers as two legs on a timeline
- Shared header (date, description, badges); each side gets its own row:
  account with its category, tags and entities, amount + exchanged amount
- Timeline rail from the transfer icon branching into the out/in icons
- Amounts drop below on small screens and sit bottom-right on lg, like
  regular items; spacing matches regular item metadata rows
- Notes move below the legs on transfers so they don't sit in the rail
2026-11-09 23:35:56 -03:00
Herculino Trotta 4826c4932a fix(transactions): show exchanged amount for both sides of combined transfers 2026-11-09 23:16:47 -03:00
Jay Hemnani 6a289327b6 fix(transactions): keep a recurrence on its day after a short month
A monthly recurrence on the 31st clamped to Feb 28 and stayed on the
28th for every later month.
2026-11-10 03:51:15 +05:30
eitchtee bdc0603327 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-09 02:35:42 +00:00
Herculino Trotta a7ae1c7ec4 Merge pull request #657 from eitchtee/feat/distribution-totals-bracket
feat(distribution): show combined projected + current percentage
2026-11-08 23:34:56 -03:00
Herculino Trotta 34ba0c6e6a Merge pull request #658 from eitchtee/feat/investments-redesign
feat(investments): calmer list and detail pages
2026-11-08 23:34:49 -03:00
Herculino Trotta d0b6be14e7 feat(investments): calmer list and detail pages, group holdings by asset class
- One hero number with a single return line; secondary totals behind a Details toggle
- Holdings in one list card with quiet group subheaders, value and %/yr per row
- Group holdings by account group or asset class, remembered in the session
- Detail page: value and chart up top, Your position and home-currency sidebar
- Replace tooltips with visible text; keep amounts from wrapping on phones
2026-11-08 21:48:40 -03:00
Herculino Trotta 55f064ee7c feat(distribution): bracket projected + current segments with their sum
Remove the unused exchanged branch from calculate_percentage_distribution; nothing read it and it mixed up income_current with expense_projected.
2026-11-08 20:16:13 -03:00
eitchtee 10cb4465d8 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-08 22:24:02 +00:00
Herculino Trotta 93c7de2888 Merge pull request #656 from eitchtee/fix/investment-returns
fix(investments): correct returns on unit-priced and value-tracked accounts
2026-11-08 19:23:37 -03:00
Herculino Trotta fda4bade62 fix(investments): count income categories on unit-priced accounts and transfers at their own amount
- Unit-priced accounts now honour their income categories, tags and entities:
  a matching transaction is a gain or loss, and any other unlinked one is money
  in or out at that day's market price. Before, every unlinked outflow was a
  loss and yields were booked as new money in.
- A transfer out of an account tracked by value counts its own amount. It used
  the other side converted at the day's rate, so a coin bought from a savings
  account moved the coin's price change onto the savings account.
2026-11-08 18:05:17 -03:00
Herculino Trotta 448e9c8f5c fix(api): save income categories, tags and entities on account create and update
AccountSerializer assigned the many-to-many fields with setattr, which Django rejects, so any write that included them returned a 500. The default ModelSerializer create/update already handles them.
2026-11-08 17:57:56 -03:00
eitchtee e7665103c0 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-08 20:20:28 +00:00
Herculino Trotta bdbcf3ea70 Merge pull request #655 from eitchtee/feat/demo-investments
feat(demo): add investment examples to the demo data
2026-11-08 17:20:04 -03:00
Herculino Trotta b6cb1e8b52 Merge pull request #654 from eitchtee/fix/transfer-account-name-width
fix(transactions): let transfer account names use the free row space
2026-11-08 17:19:55 -03:00
Herculino Trotta d9c296c26b Merge pull request #653 from eitchtee/fix/transfer-form-fields
fix(transfers): entities, paid state and inactive items in the transfer form
2026-11-08 17:19:41 -03:00
Herculino Trotta 78cb61b0b6 feat(demo): add investment examples to the demo data
Investments landed without demo data. The ETF, Bitcoin and savings accounts become investments, and new accounts cover the rest of the tracker: Apple shares (dividends paid into checking, a partial sale, gifted shares with an estimated cost), a retirement fund tracked by value (quarterly gains and a loss, a yearly fee) and gold. Asset classes have targets that add up to 100%, plus one without a target to show how rebalancing leaves it out.

The investment examples use their own random seed, so the rest of the demo is unchanged. The demo test now renders the investment pages and checks each kind of holding has figures.
2026-11-08 17:17:16 -03:00
Herculino Trotta 52eae5180f fix(transactions): let transfer account names use the free row space
Account names on a combined transfer were cut at a fixed 14rem even when the row had room. On large screens the row now stays on one line and the account side can grow up to half of it, so names only truncate past that.
2026-11-08 17:17:05 -03:00
Herculino Trotta 122f0b7a28 fix(transfers): entities, paid state and inactive items in the transfer form
The transfer form had no entities field and always created paid transfers. Each side now has its own entities, shown as from -> to in the list like category and tags, and a Projected/Paid toggle applies to both sides.

Editing a transfer also offered only active categories, tags and entities and non-archived accounts, so saving silently dropped a deactivated tag or failed on an archived account. The transfer's own items stay selectable, as in the transaction form.
2026-11-08 17:16:49 -03:00
Herculino Trotta b561ec7d77 Merge pull request #652 from eitchtee/weblate
Translations update from Weblate
2026-11-08 16:44:06 -03:00
Herculino Trotta cdc112fa00 locale(Portuguese (Brazil)): update translation
Currently translated at 100.0% (958 of 958 strings)

Translation: WYGIWYH/App
Translate-URL: https://translations.herculino.com/projects/wygiwyh/app/pt_BR/
2026-11-08 19:40:42 +00:00
eitchtee 90df6227cb chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-08 19:20:48 +00:00
Herculino Trotta 1c7be36ebb Merge pull request #650 from eitchtee/feat/bank-sync-2
feat: bank sync with external providers (Lunch Flow, SimpleFIN, Pluggy)
2026-11-08 16:20:11 -03:00
Herculino Trotta 1369c3edbf Merge remote-tracking branch 'origin/main' into feat/bank-sync-2
# Conflicts:
#	app/apps/export_app/views.py
#	app/templates/includes/sidebar.html
2026-11-08 16:17:06 -03:00
Herculino Trotta cc4c58b676 feat(bank-sync): add Pluggy provider
Brazilian banks through Open Finance. Users give their Pluggy client
credentials and the ids of their connections (items), from Meu Pluggy.

- Uses the cursor-paginated /v2/transactions; /transactions is removed
  after 2026-12-31. Our own filters are sent again with each cursor.
- The direction comes from `type`, not the amount's sign: credit cards
  keep the card's sign, purchases are positive. Checked against a real
  Meu Pluggy card and checking account; the sandbox data doesn't match
  production here.
- Pending transactions keep their id once they post (a card's open bill
  stays pending until it closes), so they're imported unpaid and settled
  later. Future-dated ones (installments, scheduled payments) wait for
  their date. Installments already come dated in their own month.
- Foreign purchases use the account's amount, with the original amount
  in the notes.
- Connections that need the user to log in again show as an error.
2026-11-08 16:10:38 -03:00
Herculino Trotta 2a4202118b feat(bank-sync): import pending transactions and settle them when they post
Providers can now return pending transactions, flagged as such, when
their ids survive posting. They're imported unpaid right away and, once
the bank posts them, marked paid with their final amount and date.

- Settling happens once, so later edits stick, and one marked paid by
  hand is left alone. It goes through save() so a transfer's other side
  follows; no rules run again. reference_date moves only if it still
  followed the date.
- Each sync looks back to the oldest unpaid synced transaction, since a
  card purchase can post weeks later when the bill closes. Pending ones
  older than 60 days are taken as cancelled and no longer followed.
2026-11-08 16:09:23 -03:00
Herculino Trotta 9238c48bb5 feat(bank-sync): add SimpleFIN provider
Users paste a one-time setup token. On save it's claimed and the access
URL it returns is stored; credentials go in a header, never the URL, so
errors don't leak them.

- Providers can validate or transform their settings (clean_settings),
  run by the service form once every other field is valid, so a token
  isn't spent on a form that fails anyway.
- Providers can declare a minimum sync interval. SimpleFIN Bridge
  disables tokens that go much past ~24 requests a day, so it's 6 hours.
- Tokens and access URLs are user input pointing the server at a host:
  only SimpleFIN Bridge is allowed unless SIMPLEFIN_ALLOWED_HOSTS says
  otherwise. Checked again on every request, for restored settings.
- Transactions are fetched per account in 90-day windows, Bridge's
  limit per request. Pending ones are skipped.
2026-11-08 16:08:37 -03:00
eitchtee 58e2aa3c8a chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-08 18:27:34 +00:00
Herculino Trotta cc30a1cf37 Merge pull request #651 from eitchtee/feat/investments
feat(investments): investment tracker with asset classes and rebalancing
2026-11-08 15:27:09 -03:00
Herculino Trotta 8ed2c98074 fix(currencies): renumber retention migration after main's 0026 2026-11-08 15:06:14 -03:00
Herculino Trotta 76090ceb4a Merge remote-tracking branch 'origin/main' into feat/investments 2026-11-08 15:01:22 -03:00
Herculino Trotta 6ed0ba4efa feat(api): asset class endpoints
- /api/investment-classes/ to list, create, edit and delete asset classes
- Accounts return their asset class and take investment_class_id, filterable
  by investment_class
- Unique names and targets up to 100% are checked in the model, shared by the
  form and the API
2026-11-08 14:55:42 -03:00
Herculino Trotta 25a2326cf6 feat(export): include bank sync services and account links in export and restore 2026-11-08 12:39:17 -03:00
Herculino Trotta 77ee56f7a3 feat(bank-sync): add sidebar entry 2026-11-08 12:39:17 -03:00
Herculino Trotta 620e7df015 feat(bank-sync): sync bank transactions from external providers
Provider registry modeled after automatic exchange rates: each provider
declares its own settings fields, rendered dynamically in the service form.
Lunch Flow is the first provider.

- Map provider accounts to WYGIWYH accounts with a start date and an
  optional currency filter
- Hourly task syncs due services; manual sync from the list
- Deduplicates through Transaction.internal_id, optionally matching
  transactions already entered by hand
- Per account errors and provider reported connection problems

Refs #498
2026-11-08 12:39:16 -03:00
Herculino Trotta 5964d80e73 Merge pull request #649 from jeonghun917/fix/dcr-token-non-ascii
Compare the DCR initial access token as bytes so a non-ASCII token ge…
2026-11-08 10:29:10 -03:00
eitchtee b0351ae5ac chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-08 13:20:01 +00:00
Herculino Trotta f894464ca7 Merge pull request #648 from eitchtee/feat/transaction-picker
feat(transactions): better transaction picker for DCA and rules
2026-11-08 10:19:26 -03:00
Herculino Trotta 7385c77437 Merge pull request #647 from eitchtee/fix/dca-create-transaction-layout
fix(dca): repair the create transaction section layout
2026-11-08 10:19:13 -03:00
eitchtee d3f652ad87 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-08 13:18:54 +00:00
Herculino Trotta a1718b711c Merge pull request #646 from eitchtee/fix/toast-close-button
fix(toasts): space the close button from the text and edges
2026-11-08 10:18:30 -03:00
Jeonghun Mun 97d5c05a3c Compare the DCR initial access token as bytes so a non-ASCII token gets a 401, not a 500
hmac.compare_digest raises TypeError when two str values hold non-ASCII
characters, and the bearer value comes from the request.
2026-11-08 08:46:34 +00:00
Herculino Trotta 9f9146aaec feat(transactions): search transactions by date, amount, entity and account
Each word of the query must match something, and words can match
different fields: description, notes, category, tags, entities, account
and account group, a date in the user's format (the year is optional),
or an amount in the user's number format, where a whole number like 234
matches 234.00 to 234.99.

Matching through an id subquery also stops transactions with several
matching tags from showing up more than once.
2026-11-08 03:27:21 -03:00
Herculino Trotta 8482c85a76 fix(transactions): keep transaction picker results in monthly view order
Results come back as today, tomorrow, yesterday, upcoming (soonest first)
and then past (newest first); past dates used to be oldest first. The
endpoint is now capped at 100 results, as its docstring already claimed.

tom-select also re-scored and re-sorted the server's results on the
client, which scrambled the order while searching. Load-based selects now
keep the server's order, drop the previous query's results, and preload
recent transactions on focus.
2026-11-08 03:27:15 -03:00
Herculino Trotta aa3ffe9b3e feat(transactions): render the transaction picker as a mini transaction
TransactionSelect options now look like a compact version of the
transaction item: paid state, type colour, date, description, category,
tags, entities, amount and account. The search endpoint returns that data
as JSON, and pre-selected options carry it in data-data so edit forms
render the same way.
2026-11-08 03:27:01 -03:00
Herculino Trotta 48afe0b7b3 fix(dca): repair the create transaction section layout
Nesting Row(Column(Row(Column))) shrank every field to a quarter of the
width, since Column defaults to half, and rounded-3 is a Bootstrap class
that doesn't exist here. Use the same card layout as the transfer form:
account at full width with category and tags side by side.
2026-11-08 03:26:21 -03:00
Herculino Trotta 03f2b0aeb4 fix(toasts): space the close button from the text and edges
Lay the alert out as an icon | text | button grid so the close button gets
a consistent gap from the message and the same visual inset as the icon.
Long messages now wrap instead of pushing the button, and the aria-label
is quoted.
2026-11-08 03:26:11 -03:00
Herculino Trotta 34f6066187 docs(readme): give IMPORT_TEMP_ROOT a literal default and move path vars up
List the Docker path as the default like the other rows do, and move
both rows out from under the dev-only Vite settings.
2026-11-08 03:17:33 -03:00
eitchtee de1df9d13c chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-08 06:13:26 +00:00
Herculino Trotta 56c37d0012 Merge pull request #643 from officialasishkumar/fix/configurable-app-paths
fix(import): stop hardcoding the Docker image's paths
2026-11-08 03:13:03 -03:00
Herculino Trotta e0c1e4d5ff fix(import): resolve symlinks when checking the upload path
The temp root is resolve()d in settings but the file path was only
abspath()ed, so a symlink inside the temp folder could point outside it
and still pass the check. Resolve both sides the same way.

Also drop the unused /temp folder from the prod image.
2026-11-08 03:09:04 -03:00
eitchtee a42265ba49 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-08 06:01:38 +00:00
Herculino Trotta 03900ab4d1 Merge pull request #642 from cahthuranag/feat/allratestoday-provider
feat(automatic-exchange-rates): add central bank tables as a provider
2026-11-08 03:01:13 -03:00
Herculino Trotta 6a335b1a63 fix(automatic-exchange-rates): skip bad AllRatesToday rows individually
A non-numeric, NaN or infinite value raised while parsing and threw away
the whole table. Each bad row is now skipped on its own.

Log a warning when a table has rows but none of a usable rate type, so a
bank with unknown types does not fail silently.
2026-11-08 02:57:07 -03:00
Herculino Trotta 43145be24d feat(automatic-exchange-rates): add provider options field
Add an Options field to exchange rate services. Providers get the raw
text, can validate it on save through clean_options, and describe it
with options_help.

The service form now shows API Key only for providers that require one,
and Options only for providers that read it, with that provider's help.

AllRatesToday reads its bank code from Options instead of API Key.
2026-11-08 02:57:07 -03:00
Herculino Trotta 38118c56d3 Merge remote-tracking branch 'origin/main' into fix/configurable-app-paths 2026-11-08 02:53:53 -03:00
Herculino Trotta a50bdb10ff Merge remote-tracking branch 'origin/main' into feat/allratestoday-provider 2026-11-08 02:53:53 -03:00
Herculino Trotta c17f03775c Merge remote-tracking branch 'origin/main' into feat/investments 2026-11-08 02:53:52 -03:00
Herculino Trotta 23acf85a6b Merge pull request #645 from eitchtee/chore/worktree-dev-setup
chore(dev): run the dev stack in multiple git worktrees
2026-11-08 02:51:05 -03:00
Herculino Trotta a691da7523 chore(dev): run the dev stack in multiple git worktrees
Drop fixed container names so each worktree gets its own containers,
add docker/dev/worktree.sh to set up and tear down a worktree, and
document it under a new Development section in the README.
2026-11-08 02:48:59 -03:00
Herculino Trotta 914c665faf Merge pull request #644 from officialasishkumar/fix/datepicker-mobile-overlay
fix(datepicker): keep the mobile overlay attached after boosted navigation
2026-11-08 02:33:13 -03:00
Herculino Trotta 494937870e fix(export): back up asset classes and investment income links
Restore asset classes before accounts and take a second pass over accounts
once categories, tags and entities exist, so their links survive a restore
into a fresh install.
2026-11-08 02:26:40 -03:00
Herculino Trotta 7038ce8e18 feat(investments): asset classes with target allocation, charts and rebalance
- Asset classes (name, optional target %) managed under Management > Investments,
  chosen per account
- Charts tab: allocation by asset class and currency, value against invested,
  return by investment; off-target classes flagged by the 5/25 rule
- Rebalance tab: what to buy or sell per class to reach the targets
- Investment page: price against average cost with trades, monthly earnings
  and income
2026-11-08 02:26:40 -03:00
Herculino Trotta 7902c4fc4e fix(forms): resolve dynamic select values from the form's queryset
A value the default manager hides, such as another user's object already set
on a shared record, was taken as a new name and created a duplicate named
after its id.
2026-11-08 02:26:40 -03:00
Herculino Trotta 26d0751df6 fix(datepicker): reset a stale overlay before reattaching it
Navigating back while a mobile picker was open reattached its overlay still
active, leaving the page covered by an overlay nothing could close. Also skip
the reattach when the global container doesn't exist yet.
2026-11-08 02:25:31 -03:00
Herculino Trotta a7b8f2c3a9 Merge pull request #641 from eitchtee/dependabot/uv/oauthlib-4.0.1
build(deps): bump oauthlib from 4.0.0 to 4.0.1
2026-11-08 02:13:55 -03:00
Herculino Trotta da7a77a06d docs(readme): update demo url and space out sponsor note 2026-11-07 13:28:27 -03:00
Herculino Trotta fa120bb9f0 docs(readme): credit Simafri for demo and site hosting 2026-11-07 13:25:50 -03:00
Asish Kumar 6cf8cbf786 fix(import): stop hardcoding the Docker image's paths
Import uploads were always written to /usr/src/app/temp, presets were
read from /usr/src/app/import_presets and the caches lived in /var/tmp,
so WYGIWYH only worked from the Docker image's layout. Running it as a
systemd service with a strict sandbox, or on an immutable distro like
NixOS, needs these folders to follow the app or be configurable.

- Presets are read from IMPORT_PRESETS_ROOT, the import_presets folder
  next to the app.
- Uploads wait in IMPORT_TEMP_ROOT, the temp folder next to the app by
  default, and movable with the IMPORT_TEMP_ROOT env var. The web views,
  the API and the import service all use it.
- The file-based caches live under CACHE_ROOT, /var/tmp by default, and
  movable with the CACHE_ROOT env var.

In the Docker image the app sits in /usr/src/app, so every default
resolves to the same folder as before.

The import service checked that a file was inside the temp folder with a
plain string prefix, which also accepted sibling folders such as
/usr/src/app/temp-other. It now compares paths.

CI no longer needs to create /usr/src/app before running the tests.

Fixes #472
2026-11-07 14:56:55 +05:30
Asish Kumar 9f4ea846fd fix(datepicker): keep the mobile overlay attached after boosted navigation
On touch devices AirDatepicker opens pickers in mobile mode, which closes
them through a shared overlay rather than on blur. The library keeps that
overlay in its module scope, inside a global container appended to the
body. Boosted navigation swaps the body, so the container and overlay are
removed: the library notices the missing container and builds a new one
for the next picker, but keeps reusing the detached overlay. From then on
nothing receives the outside click, and an open picker stays on screen
over the form until the page is fully reloaded.

Create every picker through a helper that remembers the overlay and moves
it back into the current global container whenever it was left detached.

Fixes #227
2026-11-07 14:45:49 +05:30
Herculino Trotta 0d7395cee6 fix(investments): tell outside money from yield on value-tracked accounts
Unlinked transactions on value-tracked accounts counted as gains and losses,
so a salary was yield and a bill was a loss. They now count as money in and
out, unless they match the account's new income categories or tags, or its
income entities. Those three define the account's result; only entities also
match income paid into other accounts.
2026-11-07 01:30:24 -03:00
Herculino Trotta 1f9ba8eba8 feat(investments): break converted totals down by currency
Replace the note on converted totals with a collapsible per currency, and
drop the uppercase labels from the summary cards.
2026-11-07 01:30:23 -03:00
cahthuranag 8f361e03d1 feat(automatic-exchange-rates): add central bank tables as a provider 2026-11-06 23:27:53 +00:00
dependabot[bot] b6c2e79031 build(deps): bump oauthlib from 4.0.0 to 4.0.1
Bumps [oauthlib](https://github.com/oauthlib/oauthlib) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/oauthlib/oauthlib/releases)
- [Changelog](https://github.com/oauthlib/oauthlib/blob/v4.0.1/CHANGELOG.rst)
- [Commits](https://github.com/oauthlib/oauthlib/compare/v4.0.0...v4.0.1)

---
updated-dependencies:
- dependency-name: oauthlib
  dependency-version: 4.0.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-11-06 00:25:07 +00:00
Herculino Trotta 99c6ca08a2 feat(currencies): configurable retention for automatic exchange rates
Replace the single exchange rate option with a limit of rates kept per
currency pair (oldest deleted first) and an option to keep one rate per
day. Services that kept a single rate migrate to a limit of 1.
2026-11-05 11:07:47 -03:00
Herculino Trotta 3e4e9f4220 fix(forms): show switch help text below the label 2026-11-05 11:07:47 -03:00
Herculino Trotta 359e5880cc feat(investments): add investment tracker
Investments are accounts flagged as investments; performance is derived from their transactions. Linked transfers are contributions and withdrawals (average cost, realized gains), unlinked transactions are gains or losses, and income comes from the accounts' income entities.

- Unit-priced (exchange currency) or value-tracked accounts
- Money-weighted return (XIRR), annualized after a year
- Totals converted along each currency's exchange chain, with currency effect
- Investments list and detail pages; buy/sell/income use the transfer and transaction forms
- GET /api/accounts/<id>/performance/
2026-11-05 11:07:32 -03:00
Herculino Trotta 1b9ea452d9 feat(transactions): prefill add and transfer forms from query parameters
transaction_add accepts account and entities; transactions_transfer accepts from_account and to_account. Applied only when present.
2026-11-05 00:14:41 -03:00
Herculino Trotta 4d494549b7 feat(dca): link entry transactions as transfers
Transactions created from a DCA entry, or picked from existing ones, are now linked as a transfer, so investment tracking sees the purchase. A data migration links the pairs of existing entries when valid.
2026-11-05 00:14:41 -03:00
Herculino Trotta 2cd9238b18 feat(transactions): link an existing expense and income as a transfer
Adds Transaction.link_transfer() and transfer_link_error(). Only the link changes: dates, amounts and rules are left alone.
2026-11-05 00:14:41 -03:00
Herculino Trotta c05bdcf2bf Merge pull request #640 from eitchtee/fix/update-check
fix(updates): keep update banner across page loads and restarts
2026-11-02 17:36:01 -03:00
Herculino Trotta a5c73ec436 feat(updates): queue an update check on container startup
Fresh installs no longer wait up to 12 hours for the first periodic check.
2026-11-02 17:24:28 -03:00
Herculino Trotta b10897e64b fix(updates): keep update banner across page loads and restarts
The update check shared the file-based default cache with cachalot, whose culling randomly deleted the key as pages loaded. Store only the latest release in a dedicated cache, compare it to APP_VERSION at render time, and drop the startup delete. Hide the banner when CHECK_FOR_UPDATES is off or APP_VERSION isn't a valid version.
2026-11-02 17:24:28 -03:00
eitchtee 47a7fccd39 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-02 19:54:48 +00:00
Herculino Trotta da2085712b Merge pull request #639 from eitchtee/weblate
Translations update from Weblate
2026-11-02 16:54:19 -03:00
Weblate 3719a09e64 Merge remote-tracking branch 'origin/main' 2026-11-02 19:49:54 +00:00
Herculino Trotta 30019a87b2 locale(Portuguese (Brazil)): update translation
Currently translated at 100.0% (832 of 832 strings)

Translation: WYGIWYH/App
Translate-URL: https://translations.herculino.com/projects/wygiwyh/app/pt_BR/
2026-11-02 19:49:54 +00:00
eitchtee 6d45d097b8 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-02 19:49:49 +00:00
Herculino Trotta 4ded47c5f9 Merge pull request #637 from fariskazmi/feat/oidc-disable-local-login
feat(auth): add OIDC_DISABLE_LOCAL_LOGIN to turn off password logins
2026-11-02 16:49:21 -03:00
Herculino Trotta 5b98779070 test(auth): add positive controls for local login and tighten basic auth check
- Add flag-off login page and admin password tests so the refusal tests can't pass for unrelated reasons
- Expect exactly 401 from refused basic auth
- Gate OIDC_DISABLE_LOCAL_LOGIN on an explicit OIDC_CONFIGURED flag
- Realign the README OIDC table, use "email" consistently, and warn about lockout with OIDC_ALLOW_SIGNUP=false
2026-11-02 16:41:52 -03:00
Herculino Trotta 3e2f3236d7 Merge pull request #638 from eitchtee/weblate
Translations update from Weblate
2026-11-02 16:28:40 -03:00
sorcierwax 318be51281 locale(French): update translation
Currently translated at 99.7% (830 of 832 strings)

Translation: WYGIWYH/App
Translate-URL: https://translations.herculino.com/projects/wygiwyh/app/fr/
2026-11-02 18:46:46 +00:00
Dimitri Decrock 91209f49d1 locale(Dutch): update translation
Currently translated at 100.0% (832 of 832 strings)

Translation: WYGIWYH/App
Translate-URL: https://translations.herculino.com/projects/wygiwyh/app/nl/
2026-11-02 17:58:57 +00:00
fariskazmi 161fc19721 feat(auth): add OIDC_DISABLE_LOCAL_LOGIN to turn off password logins
When the flag is on and an OIDC provider is configured, every e-mail and
password check is refused: the login page, the admin and the API's basic
auth. OIDC logins, API tokens, OAuth apps and permission checks keep working.
The login page hides the password form.
2026-11-02 09:08:30 -06:00
eitchtee ec9efccdf2 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-11-02 04:18:58 +00:00
Herculino Trotta b956237390 Merge pull request #636 from eitchtee/feat/linked-transfers
feat(transactions): link transfer sides and show them as a single item
2026-11-02 01:18:33 -03:00
Herculino Trotta 3716b9b0e3 feat(transactions): link transfer sides and show them as a single item
Transfers are still two transactions, now linked to each other:
- shown as one item in lists, with each side's amount, account,
  category and tags ("Combine transfers" user setting, on by default)
- date, reference date, paid and mute stay in sync between sides
- edit opens the transfer form; delete, restore, duplicate and bulk
  actions act on both sides
- transfers made by the transfer form now trigger rules for each side

Recurring and quick transactions gain a Transfer type, with a "To
account" and optional "To amount", generating linked transfers. A
transfer can be converted into either.

Existing transfers are left as they are, since there is no reliable
way to pair them.
2026-11-02 01:14:46 -03:00
Herculino Trotta 7cb2713c6c Merge pull request #635 from eitchtee/feat/relative-demo-data
feat(demo): generate demo data relative to today
2026-11-01 00:16:39 -03:00
Herculino Trotta 1d22c9ff66 feat(demo): generate demo data relative to today
Replace the hard-dated demo fixture with a load_demo_data command that
builds the dataset around the current date, so the demo always has data
in the current month. It also covers more features: multi-currency
accounts with exchange rate history, an ETF and Bitcoin tracked with DCA
strategies, transfers, a credit card using reference dates, installment
plans, recurring transactions, quick transactions, rules and filter
presets.
2026-11-01 00:13:01 -03:00
Herculino Trotta 556542ac9e docs(readme): refresh screenshots 2026-11-01 00:12:33 -03:00
eitchtee 83da4226d9 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-10-31 21:00:46 +00:00
Herculino Trotta 809564a1d3 Merge pull request #634 from eitchtee/feat/reconciliation-cutoff
feat(accounts): reconcile balances against a date or reference month cutoff
2026-10-31 18:00:23 -03:00
Herculino Trotta fcd666946f feat(accounts): reconcile balances against a date or reference month cutoff
Credit cards often have paid transactions dated after the invoice being
closed, which made reconciliation against all paid transactions wrong.

The reconciliation panel now lets you balance against all paid
transactions (previous behavior), paid transactions up to a date, or
paid transactions up to a reference month. The adjustment transaction is
dated at the cutoff so later reconciliations with the same or a later
cutoff include it. An invalid cutoff blocks the reconciliation instead
of silently falling back.
2026-10-31 17:56:36 -03:00
Herculino Trotta 2bb8b4e5d4 Merge pull request #631 from eitchtee/dependabot/uv/djangorestframework-3.18.2
build(deps): bump djangorestframework from 3.17.2 to 3.18.2
2026-10-25 12:11:37 -03:00
Herculino Trotta 948a3426ee Merge pull request #632 from eitchtee/dependabot/uv/brotli-1.2.1
build(deps): bump brotli from 1.2.0 to 1.2.1
2026-10-25 12:11:24 -03:00
Herculino Trotta 02488a1d74 Merge pull request #633 from eitchtee/weblate
Translations update from Weblate
2026-10-25 12:11:12 -03:00
sorcierwax edd87327d0 locale(French): update translation
Currently translated at 100.0% (817 of 817 strings)

Translation: WYGIWYH/App
Translate-URL: https://translations.herculino.com/projects/wygiwyh/app/fr/
2026-10-25 12:58:57 +00:00
dependabot[bot] 785bd054df build(deps): bump brotli from 1.2.0 to 1.2.1
Bumps [brotli](https://github.com/google/brotli) from 1.2.0 to 1.2.1.
- [Release notes](https://github.com/google/brotli/releases)
- [Changelog](https://github.com/google/brotli/blob/go/brotli/v1.2.1/CHANGELOG.md)
- [Commits](https://github.com/google/brotli/compare/v1.2.0...go/brotli/v1.2.1)

---
updated-dependencies:
- dependency-name: brotli
  dependency-version: 1.2.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-24 21:15:51 +00:00
dependabot[bot] b94a6aa4d2 build(deps): bump djangorestframework from 3.17.2 to 3.18.2
Bumps [djangorestframework](https://github.com/encode/django-rest-framework) from 3.17.2 to 3.18.2.
- [Release notes](https://github.com/encode/django-rest-framework/releases)
- [Commits](https://github.com/encode/django-rest-framework/compare/3.17.2...3.18.2)

---
updated-dependencies:
- dependency-name: djangorestframework
  dependency-version: 3.18.2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-24 21:15:42 +00:00
Herculino Trotta aa92aa1926 Merge pull request #630 from eitchtee/ci/test-runner
ci: run Django test suite on pull requests
2026-10-24 14:05:45 -03:00
Herculino Trotta 5468d17e20 ci: provide container paths and a clean TMPDIR for tests
test(transactions): enable soft delete explicitly in the hard-delete attachment test
2026-10-24 14:01:47 -03:00
Herculino Trotta b6833f2b7b ci: run Django test suite on pull requests 2026-10-24 13:57:25 -03:00
eitchtee 2558d601dd chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-10-24 16:57:23 +00:00
Herculino Trotta d7537b54ad Merge pull request #629 from eitchtee/fix/import-object-authorization
fix(import): enforce object-level authorization on import profiles and runs
2026-10-24 13:56:53 -03:00
Herculino Trotta e46e9ad0cf Merge pull request #628 from eitchtee/fix/oidc-tests-allauth-65-20
test(users): fix OIDC tests after allauth 65.20 bump
2026-10-24 13:56:26 -03:00
Herculino Trotta 4446db3249 test(users): stop using SocialLogin.state_from_request removed in allauth 65.20
django-allauth 65.20 removed SocialLogin.state_from_request, breaking the
OIDC email auto-connect tests after the bump in #627.
2026-10-24 13:27:10 -03:00
Herculino Trotta 1e486a608e fix(import): enforce object-level authorization on import profiles and runs
ImportProfile and ImportRun had no owner, and neither the web views nor the
REST viewsets filtered by the requesting user, so any authenticated user could
read, edit and delete every other user's import profiles and runs
(GHSA-q668-h56g-mfp4).

- ImportProfile is now a SharedObject, with share and take-ownership views
  like transaction rules; the global unique constraint on name is dropped.
- ImportRun is now an OwnedObject owned by the uploader, so runs made with a
  shared profile stay private to whoever ran them.
- Existing runs are backfilled with the owner of their imported
  transactions; existing profiles stay unowned and usable by everyone.
2026-10-24 13:26:37 -03:00
Herculino Trotta 8888a850bd Merge pull request #627 from eitchtee/fix/bump-allauth-oauthlib
build(deps): bump django-allauth to 65.20.1 to unblock oauthlib 4.0.0
2026-10-24 12:12:12 -03:00
Herculino Trotta 9941c8fb90 build(deps): bump django-allauth to 65.20.1 to allow oauthlib 4.0.0
django-allauth 65.18 pinned oauthlib<4, preventing Dependabot from
resolving GHSA-hj66-6f7g-4r5v and GHSA-xpv3-w29h-x7cv.
2026-10-24 12:09:19 -03:00
Herculino Trotta 39a67dfb4e Merge pull request #626 from eitchtee/dependabot/npm_and_yarn/frontend/vite-8.3.5
build(deps): bump vite from 8.0.16 to 8.3.5 in /frontend
2026-10-24 12:00:43 -03:00
Herculino Trotta 1b4a2eb3d1 Merge pull request #625 from obervinov/fix/rules-set-mute-writes-to-mute
fix(rules): write set_mute action to mute instead of is_paid
2026-10-24 12:00:30 -03:00
Herculino Trotta 73e29e867f Merge pull request #624 from obervinov/fix/bump-certifi-emudhra-distrust
build(deps): bump certifi to 2026.10.19 (GHSA-6q48-6c7w-9gw2, critical)
2026-10-24 11:59:39 -03:00
eitchtee a6f56c4989 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-10-24 14:59:16 +00:00
Herculino Trotta 601c641bb3 Merge pull request #623 from eitchtee/weblate
Translations update from Weblate
2026-10-24 11:58:49 -03:00
dependabot[bot] 0c9ccf797a build(deps): bump vite from 8.0.16 to 8.3.5 in /frontend
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 8.0.16 to 8.3.5.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.3.5/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 8.3.5
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-10-21 17:56:02 +00:00
obervinov aa95ade7ce fix(rules): write set_mute action to mute instead of is_paid 2026-10-21 21:51:22 +04:00
obervinov 03a27002cc build(deps): bump certifi from 2026.5.20 to 2026.10.19 2026-10-21 19:32:10 +04:00
Dimitri Decrock 565a770130 locale(Dutch): update translation
Currently translated at 100.0% (817 of 817 strings)

Translation: WYGIWYH/App
Translate-URL: https://translations.herculino.com/projects/wygiwyh/app/nl/
2026-10-14 04:57:23 +00:00
Dimitri Decrock a3d75de836 locale(Dutch): update translation
Currently translated at 97.3% (795 of 817 strings)

Translation: WYGIWYH/App
Translate-URL: https://translations.herculino.com/projects/wygiwyh/app/nl/
2026-10-14 03:57:23 +00:00
eitchtee 8794aba1c5 chore(locale): update translation files
[skip ci] Automatically generated by Django makemessages workflow
2026-10-12 05:47:31 +00:00
Herculino Trotta ab0c00f3e0 Merge pull request #621 from eitchtee/fix/convert-applies-form-amount
fix(transactions): apply the form amount when converting a paid transaction
2026-10-12 02:46:54 -03:00
Herculino Trotta 3261e5f266 fix(transactions): apply the form amount to a paid transaction converted into an installment plan 2026-10-12 02:41:14 -03:00
Herculino Trotta 35ef1dd716 fix(transactions): apply the form amount to a paid transaction converted into a recurring transaction 2026-10-12 02:41:14 -03:00
Herculino Trotta 8b896cf31f Merge pull request #619 from eitchtee/weblate
Translations update from Weblate
2026-10-12 02:34:28 -03:00
Weblate 85426392f8 Merge remote-tracking branch 'origin/main' 2026-10-12 05:34:17 +00:00
Herculino Trotta 66e51bb0b4 Merge pull request #620 from eitchtee/fix/dropdown-submenu-hover-delay
Keep dropdown submenu open while moving the pointer into it
2026-10-12 02:34:14 -03:00
Herculino Trotta dfb5cf37b4 fix(dropdown): keep submenu open while the pointer moves into it
Moving the pointer diagonally from a submenu trigger to its flyout crosses
the items below it, which closed the flyout before it could be reached.
Delay hiding the flyout by 300ms, and show it immediately on hover or focus.
2026-10-12 02:33:14 -03:00
Herculino Trotta f52a0f8cd5 locale(Portuguese (Brazil)): update translation
Currently translated at 98.0% (801 of 817 strings)

Translation: WYGIWYH/App
Translate-URL: https://translations.herculino.com/projects/wygiwyh/app/pt_BR/
2026-10-10 21:57:23 +00:00
201 changed files with 31941 additions and 9505 deletions

No files matched your search

+2
View File
@@ -41,6 +41,8 @@ TASK_WORKERS=1 # This only work if you're using the single container option. Inc
# Trust all provider emails, including false or missing email_verified claims.
# Set to false to require provider-verified emails before linking existing accounts.
#OIDC_TRUST_EMAIL=true
# Only allow signing in through OIDC: email and password logins are refused everywhere.
#OIDC_DISABLE_LOCAL_LOGIN=false
# Personal access tokens. How often (seconds) a token's last_used_at is rewritten.
#API_TOKEN_LAST_USED_UPDATE_INTERVAL=600
+2
View File
@@ -0,0 +1,2 @@
# Shell scripts run inside Linux containers and Git Bash, which fail on CRLF
*.sh text eol=lf
Binary file not shown.

Before

Width:  |  Height:  |  Size: 58 KiB

After

Width:  |  Height:  |  Size: 144 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 33 KiB

After

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 57 KiB

After

Width:  |  Height:  |  Size: 136 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 69 KiB

After

Width:  |  Height:  |  Size: 130 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 32 KiB

After

Width:  |  Height:  |  Size: 95 KiB

+73
View File
@@ -0,0 +1,73 @@
name: Tests
on:
pull_request:
push:
branches: [ main ]
workflow_dispatch:
# Cancel superseded runs on the same PR/branch
concurrency:
group: tests-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
django-tests:
runs-on: ubuntu-latest
timeout-minutes: 20
services:
db:
image: postgres:15
env:
POSTGRES_USER: wygiwyh
POSTGRES_PASSWORD: wygiwyh
POSTGRES_DB: wygiwyh
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U wygiwyh"
--health-interval 5s
--health-timeout 5s
--health-retries 10
env:
SECRET_KEY: ci-only-not-secret
SQL_DATABASE: wygiwyh
SQL_USER: wygiwyh
SQL_PASSWORD: wygiwyh
SQL_HOST: localhost
SQL_PORT: 5432
# Emit dev-server script tags instead of reading the Vite manifest,
# so the frontend doesn't need to be built to render templates
DJANGO_VITE_DEV_MODE: "true"
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
- name: Set up Python 3.14
run: uv python install 3.14
- name: Install dependencies
run: uv sync --frozen
- name: Check for missing migrations
working-directory: app
run: uv run python manage.py makemigrations --check --dry-run
- name: Run tests
working-directory: app
env:
# Some tests point STATIC_ROOT at the temp dir; the runner's /tmp
# holds root-only systemd dirs that WhiteNoise can't scan
TMPDIR: ${{ runner.temp }}
run: uv run python manage.py test --parallel auto --noinput
+1
View File
@@ -168,3 +168,4 @@ postgres_data/
# Private local uploads
app/attachments/
app/temp/
+62 -9
View File
@@ -23,6 +23,14 @@
<img src=".github/img/monthly_view.png" width="18%"></img> <img src=".github/img/yearly.png" width="18%"></img> <img src=".github/img/networth.png" width="18%"></img> <img src=".github/img/calendar.png" width="18%"></img> <img src=".github/img/all_transactions.png" width="18%"></img>
<br><br>
<p align="center">
<a href="https://www.simafri.com/en/managed-vps/"><img alt="Simafri" src="./.github/img/simafri-logo.png" height="40" /></a>
<br>
Demo and site hosting sponsored by <a href="https://www.simafri.com/en/managed-vps/">Simafri</a> (managed VPS)
</p>
# Why WYGIWYH?
Managing money can feel unnecessarily complex, but it doesn’t have to be. WYGIWYH (pronounced "wiggy-wih") is based on a simple principle:
@@ -54,7 +62,7 @@ Frustrated by the lack of comprehensive options, I set out to build **WYGIWYH**,
# Demo
You can try WYGIWYH on [wygiwyh-demo.herculino.com](https://wygiwyh-demo.herculino.com/) with the credentials below:
You can try WYGIWYH on [demo.wygiwyh.herculino.com](https://demo.wygiwyh.herculino.com/) with the credentials below:
> [!NOTE]
> E-mail: `demo@demo.com`
@@ -109,6 +117,48 @@ You can now access localhost:OUTBOUND_PORT
> - If you're planning on running this behind Tailscale or other similar service also add your machine given IP to `DJANGO_ALLOWED_HOSTS`
> - If you're going to use another IP that isn't localhost, add it to `DJANGO_ALLOWED_HOSTS`, without `http://`
## Development
The development stack runs from `docker-compose.dev.yml`:
```bash
$ docker compose -f docker-compose.dev.yml up -d
```
Set `COMPOSE_FILE=docker-compose.dev.yml` in your `.env` to drop the `-f` flag.
### Working in multiple git worktrees
You can run one dev stack per [git worktree](https://git-scm.com/docs/git-worktree), side by side. Each worktree gets its own containers, database and volumes, because Compose names them after the worktree's folder.
A new worktree has no `.env` (it's gitignored). After creating one, run this from the worktree's root:
```bash
$ sh docker/dev/worktree.sh
```
It copies `.env` from your main checkout, gives the worktree its own host ports (printed when it finishes) and installs `frontend/node_modules` for tools you run outside Docker. The database starts empty.
To stop the worktree's containers and delete its volumes before removing it:
```bash
$ sh docker/dev/worktree.sh teardown
```
The commands work on Linux, macOS and Windows (Git Bash or WSL). PowerShell and cmd have no `sh`, so on Windows set up git aliases once in your clone; they run through Git's bundled shell from any terminal, and every worktree of the clone shares them:
```bash
git config alias.wt-setup '!sh docker/dev/worktree.sh'
git config alias.wt-teardown '!sh docker/dev/worktree.sh teardown'
```
Run those two lines in Git Bash or PowerShell; cmd doesn't understand the single quotes.
Then use `git wt-setup` and `git wt-teardown`. Tools that create worktrees for you, such as Orca, Conductor or Superset, usually have setup and archive hooks: put these commands there.
> [!NOTE]
> Ports come from the worktree's folder name, so two worktrees can rarely land on the same ones. If `docker compose up` reports a port in use, change `OUTBOUND_PORT`, `WEBPACK_OUTBOUND_PORT` or `SQL_OUTBOUND_PORT` in that worktree's `.env`.
## Latest changes
Features are only added to `main` when ready, if you want to run the latest version, you must build from source or use the `:nightly` tag on docker. Keep in mind that there can be undocumented breaking changes.
@@ -142,6 +192,8 @@ To create the first user, open the container's console using Unraid's UI, by cli
| ENABLE_SOFT_DELETE | true\|false | false | Whether to enable transactions soft delete, if enabled, deleted transactions will remain in the database. Useful for imports and avoiding duplicate entries. |
| KEEP_DELETED_TRANSACTIONS_FOR | int | 365 | Time in days to keep soft deleted transactions for. If 0, will keep all transactions indefinitely. Only works if ENABLE_SOFT_DELETE is true. |
| TASK_WORKERS | int | 1 | How many workers to have for async tasks. One should be enough for most use cases |
| IMPORT_TEMP_ROOT | string | /usr/src/app/temp | Where uploaded import files wait to be processed. Defaults to the `temp` folder inside the app folder. Must be writable and shared between the web and task processes. |
| CACHE_ROOT | string | /var/tmp | The writable folder the file-based caches are stored in. |
| DEMO | true\|false | false | If demo mode is enabled. |
| ADMIN_EMAIL | string | None | Automatically creates an admin account with this email. Must have `ADMIN_PASSWORD` also set. |
| ADMIN_PASSWORD | string | None | Automatically creates an admin account with this password. Must have `ADMIN_EMAIL` also set. |
@@ -168,14 +220,15 @@ WYGIWYH supports login via OpenID Connect (OIDC) through `django-allauth`. This
To configure OIDC, you need to set the following environment variables:
| Variable | Description |
|----------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `OIDC_CLIENT_NAME` | The name of the provider. will be displayed in the login page. Defaults to `OpenID Connect` |
| `OIDC_CLIENT_ID` | The Client ID provided by your OIDC provider. |
| `OIDC_CLIENT_SECRET` | The Client Secret provided by your OIDC provider. |
| `OIDC_SERVER_URL` | The base URL of your OIDC provider's discovery document or authorization server (e.g., `https://your-provider.com/auth/realms/your-realm`). `django-allauth` will use this to discover the necessary endpoints (authorization, token, userinfo, etc.). |
| `OIDC_ALLOW_SIGNUP` | Allow the automatic creation of inexistent accounts on a successfull authentication. Defaults to `true`. |
| `OIDC_TRUST_EMAIL` | Treat every email sent by your provider as verified, linking it to a matching existing local account even without the `email_verified` claim. Set to `false` to only link verified emails. Defaults to `true`. |
| Variable | Description |
|----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `OIDC_CLIENT_NAME` | The name of the provider. will be displayed in the login page. Defaults to `OpenID Connect` |
| `OIDC_CLIENT_ID` | The Client ID provided by your OIDC provider. |
| `OIDC_CLIENT_SECRET` | The Client Secret provided by your OIDC provider. |
| `OIDC_SERVER_URL` | The base URL of your OIDC provider's discovery document or authorization server (e.g., `https://your-provider.com/auth/realms/your-realm`). `django-allauth` will use this to discover the necessary endpoints (authorization, token, userinfo, etc.). |
| `OIDC_ALLOW_SIGNUP` | Allow the automatic creation of inexistent accounts on a successfull authentication. Defaults to `true`. |
| `OIDC_TRUST_EMAIL` | Treat every email sent by your provider as verified, linking it to a matching existing local account even without the `email_verified` claim. Set to `false` to only link verified emails. Defaults to `true`. |
| `OIDC_DISABLE_LOCAL_LOGIN` | Only allow signing in through your OIDC provider. Email and password logins are refused everywhere: the login page, the admin and the API's basic authentication. API tokens and OAuth apps keep working. Only takes effect when OIDC is configured, and changing it signs everyone out once. If `OIDC_ALLOW_SIGNUP` is `false`, make sure an admin's local account email matches their provider account first, or nobody will be able to sign in. Defaults to `false`. |
**Callback URL (Redirect URI):**
+26 -4
View File
@@ -63,6 +63,7 @@ INSTALLED_APPS = [
"procrastinate.contrib.django",
"apps.transactions.apps.TransactionsConfig",
"apps.currencies.apps.CurrenciesConfig",
"apps.bank_sync.apps.BankSyncConfig",
"apps.accounts.apps.AccountsConfig",
"apps.net_worth.apps.NetWorthConfig",
"apps.import_app.apps.ImportConfig",
@@ -77,6 +78,7 @@ INSTALLED_APPS = [
"apps.rules.apps.RulesConfig",
"apps.calendar_view.apps.CalendarViewConfig",
"apps.dca.apps.DcaConfig",
"apps.investments.apps.InvestmentsConfig",
"pwa",
"allauth",
"allauth.account",
@@ -313,6 +315,9 @@ LOCALE_PATHS = [BASE_DIR / "locale"]
STATIC_URL = "static/"
STATIC_ROOT = BASE_DIR / "static_files"
ATTACHMENT_MEDIA_ROOT = BASE_DIR / "attachments"
IMPORT_PRESETS_ROOT = BASE_DIR / "import_presets"
# Web and task workers must see the same folder: uploads wait here to be processed
IMPORT_TEMP_ROOT = Path(os.getenv("IMPORT_TEMP_ROOT") or BASE_DIR / "temp").resolve()
STATICFILES_DIRS = [
ROOT_DIR / "frontend" / "build",
@@ -324,11 +329,18 @@ STATICFILES_FINDERS = [
"django.contrib.staticfiles.finders.AppDirectoriesFinder",
]
CACHE_ROOT = Path(os.getenv("CACHE_ROOT") or "/var/tmp").resolve()
CACHES = {
"default": {
"BACKEND": "django.core.cache.backends.filebased.FileBasedCache",
"LOCATION": "/var/tmp/django_cache",
}
"LOCATION": CACHE_ROOT / "django_cache",
},
# Kept apart from "default": cachalot fills it and FileBasedCache culls random entries past MAX_ENTRIES
"update_check": {
"BACKEND": "django.core.cache.backends.filebased.FileBasedCache",
"LOCATION": CACHE_ROOT / "django_cache_update_check",
},
}
DJANGO_VITE_ASSETS_PATH = STATIC_ROOT
@@ -359,11 +371,12 @@ AUTHENTICATION_BACKENDS = [
SOCIALACCOUNT_PROVIDERS = {"openid_connect": {"APPS": []}}
if (
OIDC_CONFIGURED = bool(
os.getenv("OIDC_CLIENT_ID")
and os.getenv("OIDC_CLIENT_SECRET")
and os.getenv("OIDC_SERVER_URL")
):
)
if OIDC_CONFIGURED:
SOCIALACCOUNT_PROVIDERS["openid_connect"]["APPS"].append(
{
"provider_id": slugify(os.getenv("OIDC_CLIENT_NAME", "OpenID Connect")),
@@ -393,6 +406,15 @@ SOCIALACCOUNT_EMAIL_AUTHENTICATION_AUTO_CONNECT = True
ACCOUNT_ADAPTER = "allauth.account.adapter.DefaultAccountAdapter"
SOCIALACCOUNT_ADAPTER = "apps.users.adapters.AutoConnectSocialAccountAdapter"
# Only takes effect when an OIDC provider is configured, so an instance can't
# be left without any way to sign in.
OIDC_DISABLE_LOCAL_LOGIN = (
OIDC_CONFIGURED
and os.getenv("OIDC_DISABLE_LOCAL_LOGIN", "false").lower() == "true"
)
if OIDC_DISABLE_LOCAL_LOGIN:
AUTHENTICATION_BACKENDS = ["apps.users.backends.SocialOnlyAuthenticationBackend"]
# Personal access tokens. last_used_at is only rewritten once per interval to
# avoid a database write on every authenticated request.
API_TOKEN_LAST_USED_UPDATE_INTERVAL = int(
+2
View File
@@ -90,9 +90,11 @@ urlpatterns = [
path("", include("apps.monthly_overview.urls")),
path("", include("apps.yearly_overview.urls")),
path("", include("apps.currencies.urls")),
path("", include("apps.bank_sync.urls")),
path("", include("apps.rules.urls")),
path("", include("apps.calendar_view.urls")),
path("", include("apps.dca.urls")),
path("", include("apps.investments.urls")),
path("", include("apps.mini_tools.urls")),
path("", include("apps.import_app.urls")),
path("", include("apps.export_app.urls")),
+156 -4
View File
@@ -1,19 +1,28 @@
from apps.accounts.models import Account, AccountGroup
import calendar
from apps.accounts.models import Account, AccountGroup, InvestmentClass
from apps.common.middleware.thread_local import get_current_user
from apps.common.fields.forms.dynamic_select import (
DynamicModelChoiceField,
DynamicModelMultipleChoiceField,
)
from apps.common.widgets.crispy.daisyui import Switch
from apps.common.widgets.crispy.submit import NoClassSubmit
from apps.common.widgets.datepicker import AirDatePickerInput, AirMonthYearPickerInput
from apps.common.widgets.decimal import ArbitraryDecimalDisplayNumberInput
from apps.common.widgets.tom_select import TomSelect
from apps.common.widgets.tom_select import TomSelect, TomSelectMultiple
from apps.currencies.models import Currency
from apps.transactions.models import TransactionCategory, TransactionTag
from apps.transactions.models import (
TransactionCategory,
TransactionEntity,
TransactionTag,
)
from crispy_forms.bootstrap import FormActions
from crispy_forms.helper import FormHelper
from crispy_forms.layout import Column, Field, Layout, Row
from crispy_forms.layout import HTML, Column, Div, Field, Layout, Row
from django import forms
from django.db.models import Q
from django.utils import timezone
from django.utils.translation import gettext_lazy as _
@@ -47,6 +56,30 @@ class AccountGroupForm(forms.ModelForm):
)
class InvestmentClassForm(forms.ModelForm):
class Meta:
model = InvestmentClass
fields = ["name", "target"]
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.helper = FormHelper()
self.helper.form_tag = False
self.helper.form_method = "post"
self.helper.layout = Layout(
"name",
"target",
FormActions(
NoClassSubmit(
"submit",
_("Update") if self.instance.pk else _("Add"),
css_class="btn btn-primary",
),
),
)
class AccountForm(forms.ModelForm):
group = DynamicModelChoiceField(
create_field="name",
@@ -55,6 +88,14 @@ class AccountForm(forms.ModelForm):
required=False,
)
investment_class = DynamicModelChoiceField(
create_field="name",
label=_("Asset class"),
model=InvestmentClass,
required=False,
help_text=Account._meta.get_field("investment_class").help_text,
)
class Meta:
model = Account
fields = [
@@ -62,18 +103,42 @@ class AccountForm(forms.ModelForm):
"group",
"currency",
"exchange_currency",
"show_exchanged_amounts",
"is_asset",
"is_archived",
"is_investment",
"investment_class",
"income_entities",
"income_categories",
"income_tags",
]
widgets = {
"currency": TomSelect(),
"exchange_currency": TomSelect(),
"income_entities": TomSelectMultiple(),
"income_categories": TomSelectMultiple(),
"income_tags": TomSelectMultiple(),
}
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.fields["group"].queryset = AccountGroup.objects.all()
self.fields["investment_class"].queryset = InvestmentClass.all_objects.filter(
Q(owner=get_current_user())
| Q(owner=None)
# Keep a class from the account's owner when it's shared with us
| Q(accounts=self.instance.id or 0)
).distinct()
selectable = Q(active=True)
if self.instance.id:
selectable |= Q(investment_accounts=self.instance.id)
for name, model in (
("income_entities", TransactionEntity),
("income_categories", TransactionCategory),
("income_tags", TransactionTag),
):
self.fields[name].queryset = model.objects.filter(selectable).distinct()
if self.instance.id:
qs = Currency.objects.filter(
@@ -97,6 +162,22 @@ class AccountForm(forms.ModelForm):
Switch("is_archived"),
"currency",
"exchange_currency",
Switch("show_exchanged_amounts"),
Switch("is_investment"),
# Only relevant for investments: shown while the switch above is on
Div(
"investment_class",
"income_entities",
"income_categories",
"income_tags",
css_id="investment_income",
css_class="" if self.instance.is_investment else "hidden",
),
HTML(
'<div class="hidden" _="on change from closest <form/> '
"if #id_is_investment.checked remove .hidden from #investment_income "
'else add .hidden to #investment_income end"></div>'
),
)
if self.instance and self.instance.pk:
@@ -163,4 +244,75 @@ class AccountBalanceForm(forms.Form):
self.fields["tags"].queryset = TransactionTag.objects.filter(active=True)
class ReconciliationCutoffForm(forms.Form):
"""Which transactions the reconciliation balance is computed against."""
MODE_CURRENT = "current"
MODE_DATE = "date"
MODE_MONTH = "month"
mode = forms.ChoiceField(
choices=[
(MODE_CURRENT, _("All paid transactions")),
(MODE_DATE, _("Up to date")),
(MODE_MONTH, _("Up to reference month")),
],
required=False,
initial=MODE_CURRENT,
label=_("Balance against"),
)
date = forms.DateField(
required=False,
label=_("Date"),
widget=AirDatePickerInput(clear_button=False),
)
month = forms.DateField(
required=False,
label=_("Reference month"),
widget=AirMonthYearPickerInput(clear_button=False),
)
def __init__(self, *args, **kwargs):
super().__init__(*args, prefix="cutoff", **kwargs)
today = timezone.localdate(timezone.now())
self.today = today
self.fields["date"].initial = today
self.fields["month"].initial = today.replace(day=1)
def _value(self, name):
# Bound but invalid/empty values fall back to the field's initial.
if self.is_bound and self.is_valid() and self.cleaned_data.get(name):
return self.cleaned_data[name]
return self.fields[name].initial
@property
def selected_mode(self):
return self._value("mode")
def balance_filters(self) -> dict:
"""Extra kwargs for get_account_balance."""
if self.selected_mode == self.MODE_DATE:
return {"date_lte": self._value("date")}
if self.selected_mode == self.MODE_MONTH:
return {"reference_date_lte": self._value("month")}
return {}
def adjustment_dates(self):
"""(date, reference_date) for the reconciliation transaction.
Dated at the cutoff so a later reconcile with the same or later cutoff
includes it, and an earlier one doesn't.
"""
if self.selected_mode == self.MODE_DATE:
cutoff = self._value("date")
return cutoff, cutoff.replace(day=1)
if self.selected_mode == self.MODE_MONTH:
month = self._value("month")
last_day = month.replace(
day=calendar.monthrange(month.year, month.month)[1]
)
return min(self.today, last_day), month
return self.today, self.today.replace(day=1)
AccountBalanceFormSet = forms.formset_factory(AccountBalanceForm, extra=0)
@@ -0,0 +1,25 @@
# Generated by Django 5.2.17 on 2026-11-05 01:12
import django.db.models.deletion
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('accounts', '0016_account_untracked_by'),
('transactions', '0052_transfer_templates'),
]
operations = [
migrations.AddField(
model_name='account',
name='income_entity',
field=models.ForeignKey(blank=True, help_text="Income transactions with this entity (dividends, interest, rent) count towards this investment's return.", null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='investment_accounts', to='transactions.transactionentity', verbose_name='Income entity'),
),
migrations.AddField(
model_name='account',
name='is_investment',
field=models.BooleanField(default=False, help_text='Show this account in the Investments tracker. Set an exchange currency to price it by units (stocks, ETFs, crypto); leave it empty to track it by value (fixed income, property, pensions).', verbose_name='Investment'),
),
]
@@ -0,0 +1,43 @@
import django.db.models.deletion
from django.db import migrations, models
def copy_income_entity(apps, schema_editor):
Account = apps.get_model("accounts", "Account")
for account in Account.objects.exclude(income_entity=None):
account.income_entities.add(account.income_entity_id)
class Migration(migrations.Migration):
dependencies = [
("accounts", "0017_account_income_entity_account_is_investment"),
("transactions", "0052_transfer_templates"),
]
operations = [
# Free the "investment_accounts" reverse name for the new field
migrations.AlterField(
model_name="account",
name="income_entity",
field=models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="+",
to="transactions.transactionentity",
),
),
migrations.AddField(
model_name="account",
name="income_entities",
field=models.ManyToManyField(
blank=True,
help_text="Income transactions with any of these entities (dividends, interest, rent) count towards this investment's return.",
related_name="investment_accounts",
to="transactions.transactionentity",
verbose_name="Income entities",
),
),
migrations.RunPython(copy_income_entity, migrations.RunPython.noop),
migrations.RemoveField(model_name="account", name="income_entity"),
]
@@ -0,0 +1,29 @@
# Generated by Django 5.2.17 on 2026-11-07 04:07
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('accounts', '0018_income_entities'),
('transactions', '0052_transfer_templates'),
]
operations = [
migrations.AddField(
model_name='account',
name='income_categories',
field=models.ManyToManyField(blank=True, help_text="Transactions in this account with any of these categories are its result (interest, taxes, fees). On accounts tracked by value, everything else that isn't a transfer counts as money coming in or going out.", related_name='investment_accounts', to='transactions.transactioncategory', verbose_name='Income categories'),
),
migrations.AddField(
model_name='account',
name='income_tags',
field=models.ManyToManyField(blank=True, help_text='Same as income categories, for transactions with any of these tags.', related_name='investment_accounts', to='transactions.transactiontag', verbose_name='Income tags'),
),
migrations.AlterField(
model_name='account',
name='income_entities',
field=models.ManyToManyField(blank=True, help_text="Transactions with any of these entities count towards this investment's return: in this account, and income paid into other accounts (dividends, rent).", related_name='investment_accounts', to='transactions.transactionentity', verbose_name='Income entities'),
),
]
@@ -0,0 +1,38 @@
# Generated by Django 5.2.17 on 2026-11-08 03:26
import django.core.validators
import django.db.models.deletion
from django.conf import settings
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('accounts', '0019_income_categories_and_tags'),
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.CreateModel(
name='InvestmentClass',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('name', models.CharField(max_length=255, verbose_name='Name')),
('target', models.DecimalField(blank=True, decimal_places=2, help_text='Share of your investments you want in this class. Leave it empty to leave the class out of the targets.', max_digits=5, null=True, validators=[django.core.validators.MinValueValidator(0), django.core.validators.MaxValueValidator(100)], verbose_name='Target allocation (%)')),
('owner', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='%(class)s_owned', to=settings.AUTH_USER_MODEL)),
],
options={
'verbose_name': 'Asset class',
'verbose_name_plural': 'Asset classes',
'db_table': 'investment_classes',
'ordering': ['name', 'id'],
'unique_together': {('owner', 'name')},
},
),
migrations.AddField(
model_name='account',
name='investment_class',
field=models.ForeignKey(blank=True, help_text='Groups investments in the allocation charts, e.g. Fixed income, Stocks, Real estate.', null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='accounts', to='accounts.investmentclass', verbose_name='Asset class'),
),
]
@@ -0,0 +1,19 @@
# Generated by Django 5.2.17 on 2026-11-08 20:59
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('accounts', '0020_investment_classes'),
('transactions', '0052_transfer_templates'),
]
operations = [
migrations.AlterField(
model_name='account',
name='income_categories',
field=models.ManyToManyField(blank=True, help_text="Transactions in this account with any of these categories are its result (interest, taxes, fees). Everything else that isn't a transfer counts as money coming in or going out.", related_name='investment_accounts', to='transactions.transactioncategory', verbose_name='Income categories'),
),
]
@@ -0,0 +1,22 @@
# Generated by Django 5.2.17 on 2026-11-11 09:49
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("accounts", "0021_alter_account_income_categories_help"),
]
operations = [
migrations.AddField(
model_name="account",
name="show_exchanged_amounts",
field=models.BooleanField(
default=True,
help_text="Show amounts converted to this account's exchange currency next to its transactions and totals.",
verbose_name="Show exchanged amounts",
),
),
]
+129 -1
View File
@@ -1,10 +1,17 @@
from django.conf import settings
from django.core.exceptions import ValidationError
from django.core.validators import MaxValueValidator, MinValueValidator
from django.db import models
from django.template.defaultfilters import floatformat
from django.utils.translation import gettext_lazy as _
from apps.common.middleware.thread_local import get_current_user
from apps.common.models import SharedObject, SharedObjectManager
from apps.common.models import (
OwnedObject,
OwnedObjectManager,
SharedObject,
SharedObjectManager,
)
from apps.transactions.models import Transaction
@@ -25,6 +32,54 @@ class AccountGroup(SharedObject):
return self.name
class InvestmentClass(OwnedObject):
name = models.CharField(max_length=255, verbose_name=_("Name"))
target = models.DecimalField(
max_digits=5,
decimal_places=2,
null=True,
blank=True,
validators=[MinValueValidator(0), MaxValueValidator(100)],
verbose_name=_("Target allocation (%)"),
help_text=_(
"Share of your investments you want in this class. Leave it empty to "
"leave the class out of the targets."
),
)
objects = OwnedObjectManager()
all_objects = models.Manager() # Unfiltered manager
def clean(self):
# Shared by the form and the API; owner is set on save, so neither can
# rely on Django's (owner, name) unique check.
others = InvestmentClass.all_objects.filter(
owner=self.owner or get_current_user()
).exclude(pk=self.pk)
errors = {}
if self.name and others.filter(name=self.name).exists():
errors["name"] = _("You already have an asset class with this name.")
if self.target:
taken = others.aggregate(total=models.Sum("target"))["total"] or 0
if taken + self.target > 100:
errors["target"] = _(
"Targets can't add up to more than 100%%. Other classes already "
"take %(taken)s%%."
) % {"taken": floatformat(taken, -2)}
if errors:
raise ValidationError(errors)
class Meta:
verbose_name = _("Asset class")
verbose_name_plural = _("Asset classes")
db_table = "investment_classes"
unique_together = (("owner", "name"),)
ordering = ["name", "id"]
def __str__(self):
return self.name
class Account(SharedObject):
name = models.CharField(max_length=255, verbose_name=_("Name"))
group = models.ForeignKey(
@@ -49,6 +104,14 @@ class Account(SharedObject):
blank=True,
help_text=_("Default currency for exchange calculations"),
)
show_exchanged_amounts = models.BooleanField(
default=True,
verbose_name=_("Show exchanged amounts"),
help_text=_(
"Show amounts converted to this account's exchange currency next to its "
"transactions and totals."
),
)
is_asset = models.BooleanField(
default=False,
@@ -57,6 +120,58 @@ class Account(SharedObject):
"Asset accounts count towards your Net Worth, but not towards your month."
),
)
is_investment = models.BooleanField(
default=False,
verbose_name=_("Investment"),
help_text=_(
"Show this account in the Investments tracker. Set an exchange currency to "
"price it by units (stocks, ETFs, crypto); leave it empty to track it by "
"value (fixed income, property, pensions)."
),
)
investment_class = models.ForeignKey(
InvestmentClass,
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name="accounts",
verbose_name=_("Asset class"),
help_text=_(
"Groups investments in the allocation charts, e.g. Fixed income, Stocks, "
"Real estate."
),
)
income_entities = models.ManyToManyField(
"transactions.TransactionEntity",
blank=True,
related_name="investment_accounts",
verbose_name=_("Income entities"),
help_text=_(
"Transactions with any of these entities count towards this investment's "
"return: in this account, and income paid into other accounts (dividends, "
"rent)."
),
)
income_categories = models.ManyToManyField(
"transactions.TransactionCategory",
blank=True,
related_name="investment_accounts",
verbose_name=_("Income categories"),
help_text=_(
"Transactions in this account with any of these categories are its result "
"(interest, taxes, fees). Everything else that isn't a transfer counts as "
"money coming in or going out."
),
)
income_tags = models.ManyToManyField(
"transactions.TransactionTag",
blank=True,
related_name="investment_accounts",
verbose_name=_("Income tags"),
help_text=_(
"Same as income categories, for transactions with any of these tags."
),
)
is_archived = models.BooleanField(
default=False,
verbose_name=_("Archived"),
@@ -84,6 +199,19 @@ class Account(SharedObject):
user = get_current_user()
return self.untracked_by.filter(pk=user.pk).exists()
@property
def displayed_exchange_currency(self):
"""The exchange currency to show amounts in, or None when they're hidden.
The account's exchange currency takes precedence over its currency's, so
whichever one provides it also decides whether it's shown.
"""
if self.exchange_currency_id:
return self.exchange_currency if self.show_exchanged_amounts else None
if self.currency.show_exchanged_amounts:
return self.currency.exchange_currency
return None
def clean(self):
super().clean()
if self.exchange_currency == self.currency:
+14 -1
View File
@@ -1,3 +1,4 @@
from datetime import date
from decimal import Decimal
from django.db import models
@@ -6,7 +7,12 @@ from apps.accounts.models import Account
from apps.transactions.models import Transaction
def get_account_balance(account: Account, paid_only: bool = True) -> Decimal:
def get_account_balance(
account: Account,
paid_only: bool = True,
date_lte: date | None = None,
reference_date_lte: date | None = None,
) -> Decimal:
"""
Calculate account balance (income - expense).
@@ -14,6 +20,9 @@ def get_account_balance(account: Account, paid_only: bool = True) -> Decimal:
account: Account instance to calculate balance for.
paid_only: If True, only count paid transactions (current balance).
If False, count all transactions (projected balance).
date_lte: If set, only count transactions dated on or before it.
reference_date_lte: If set, only count transactions whose reference
month is on or before it.
Returns:
Decimal: The calculated balance (income - expense).
@@ -21,6 +30,10 @@ def get_account_balance(account: Account, paid_only: bool = True) -> Decimal:
filters = {"account": account}
if paid_only:
filters["is_paid"] = True
if date_lte:
filters["date__lte"] = date_lte
if reference_date_lte:
filters["reference_date__lte"] = reference_date_lte
income = Transaction.objects.filter(
type=Transaction.Type.INCOME, **filters
+153 -1
View File
@@ -1,6 +1,9 @@
from datetime import date
from decimal import Decimal
from django.test import TestCase
from django.contrib.auth import get_user_model
from django.test import TestCase, override_settings
from django.urls import reverse
from apps.accounts.models import Account, AccountGroup
from apps.currencies.models import Currency
@@ -173,3 +176,152 @@ class GetAccountBalanceServiceTests(TestCase):
balance = get_account_balance(self.account) # defaults to paid_only=True
self.assertEqual(balance, Decimal("100.00"))
class GetAccountBalanceCutoffTests(TestCase):
"""get_account_balance with date / reference month cutoffs."""
def setUp(self):
from apps.transactions.models import Transaction
self.Transaction = Transaction
currency = Currency.objects.create(code="BRL", name="Real", decimal_places=2)
self.account = Account.objects.create(name="Card", currency=currency)
# Paid expenses: Oct 10 (ref Oct), Oct 20 (ref Nov), Nov 5 (ref Nov)
for day, ref, amount in [
(date(2026, 10, 10), date(2026, 10, 1), "10"),
(date(2026, 10, 20), date(2026, 11, 1), "20"),
(date(2026, 11, 5), date(2026, 11, 1), "40"),
]:
Transaction.objects.create(
account=self.account,
type=Transaction.Type.EXPENSE,
amount=Decimal(amount),
is_paid=True,
date=day,
reference_date=ref,
description="x",
)
def test_date_cutoff(self):
from apps.accounts.services import get_account_balance
self.assertEqual(
get_account_balance(self.account, date_lte=date(2026, 10, 20)),
Decimal("-30"),
)
def test_reference_month_cutoff(self):
from apps.accounts.services import get_account_balance
self.assertEqual(
get_account_balance(self.account, reference_date_lte=date(2026, 10, 1)),
Decimal("-10"),
)
def test_no_cutoff_counts_everything_paid(self):
from apps.accounts.services import get_account_balance
self.assertEqual(get_account_balance(self.account), Decimal("-70"))
@override_settings(
STORAGES={
"default": {"BACKEND": "django.core.files.storage.FileSystemStorage"},
"staticfiles": {
"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"
},
},
WHITENOISE_AUTOREFRESH=True,
)
class AccountReconciliationCutoffViewTests(TestCase):
HTMX = {"HTTP_HX_REQUEST": "true"}
def setUp(self):
from apps.transactions.models import Transaction
self.Transaction = Transaction
self.user = get_user_model().objects.create_user(
email="u@test.com", password="testpass123"
)
self.client.force_login(self.user)
currency = Currency.objects.create(code="BRL", name="Real", decimal_places=2)
self.account = Account.objects.create(
name="Card", currency=currency, owner=self.user
)
for day, ref, amount in [
(date(2026, 10, 10), date(2026, 10, 1), "10"),
(date(2026, 12, 5), date(2026, 12, 1), "40"), # paid, in the future
]:
Transaction.objects.create(
account=self.account,
type=Transaction.Type.EXPENSE,
amount=Decimal(amount),
is_paid=True,
date=day,
reference_date=ref,
description="x",
owner=self.user,
)
self.url = reverse("account_reconciliation")
def post(self, cutoff, new_balance):
return self.client.post(
self.url,
{
**cutoff,
"form-TOTAL_FORMS": "1",
"form-INITIAL_FORMS": "1",
"form-0-account_id": str(self.account.id),
"form-0-new_balance": new_balance,
},
**self.HTMX,
)
def adjustment(self):
return self.Transaction.objects.get(
account=self.account, description="Balance reconciliation"
)
def test_get_shows_balance_for_month_cutoff(self):
response = self.client.get(
self.url,
{"cutoff-mode": "month", "cutoff-month": "September 2026"},
**self.HTMX,
)
self.assertEqual(response.status_code, 200)
# The current-month fallback would give -10, so 0 proves the month was read.
self.assertEqual(response.context["form"].forms[0].current_balance, 0)
def test_post_date_cutoff_dates_adjustment_at_cutoff(self):
response = self.post(
{"cutoff-mode": "date", "cutoff-date": "2026-10-15"}, "-25"
)
self.assertEqual(response.status_code, 204)
adj = self.adjustment()
# Balance up to Oct 15 is -10, so -15 more is needed.
self.assertEqual(adj.type, self.Transaction.Type.EXPENSE)
self.assertEqual(adj.amount, Decimal("15"))
self.assertEqual(adj.date, date(2026, 10, 15))
self.assertEqual(adj.reference_date, date(2026, 10, 1))
def test_post_month_cutoff_dates_adjustment_in_month(self):
self.post({"cutoff-mode": "month", "cutoff-month": "September 2026"}, "5")
adj = self.adjustment()
self.assertEqual(adj.type, self.Transaction.Type.INCOME)
self.assertEqual(adj.amount, Decimal("5"))
self.assertEqual(adj.date, date(2026, 9, 30))
self.assertEqual(adj.reference_date, date(2026, 9, 1))
def test_post_with_invalid_cutoff_date_does_not_reconcile(self):
response = self.post({"cutoff-mode": "date", "cutoff-date": "garbage"}, "5")
self.assertEqual(response.status_code, 200)
self.assertFalse(
self.Transaction.objects.filter(description="Balance reconciliation").exists()
)
def test_post_without_cutoff_keeps_current_behavior(self):
self.post({}, "-50")
self.assertFalse(
self.Transaction.objects.filter(description="Balance reconciliation").exists()
)
+17 -10
View File
@@ -6,10 +6,9 @@ from django.db import models
from django.db import transaction
from django.http import HttpResponse
from django.shortcuts import render
from django.utils import timezone
from django.utils.translation import gettext_lazy as _
from apps.accounts.forms import AccountBalanceFormSet
from apps.accounts.forms import AccountBalanceFormSet, ReconciliationCutoffForm
from apps.accounts.models import Account, Transaction
from apps.accounts.services import get_account_balance
from apps.common.decorators.htmx import only_htmx
@@ -18,6 +17,11 @@ from apps.common.decorators.htmx import only_htmx
@only_htmx
@login_required
def account_reconciliation(request):
cutoff = ReconciliationCutoffForm(
request.POST if request.method == "POST" else request.GET or None
)
balance_filters = cutoff.balance_filters()
initial_data = [
{
"account_id": account.id,
@@ -26,7 +30,7 @@ def account_reconciliation(request):
"decimal_places": account.currency.decimal_places,
"suffix": account.currency.suffix,
"prefix": account.currency.prefix,
"current_balance": get_account_balance(account),
"current_balance": get_account_balance(account, **balance_filters),
}
for account in Account.objects.filter(is_archived=False)
.select_related("currency", "group")
@@ -35,7 +39,8 @@ def account_reconciliation(request):
if request.method == "POST":
formset = AccountBalanceFormSet(request.POST, initial=initial_data)
if formset.is_valid():
if formset.is_valid() and cutoff.is_valid():
adjustment_date, adjustment_reference_date = cutoff.adjustment_dates()
with transaction.atomic():
for form in formset:
if form.is_valid():
@@ -48,7 +53,9 @@ def account_reconciliation(request):
if new_balance is None:
continue
current_balance = get_account_balance(account)
current_balance = get_account_balance(
account, **balance_filters
)
difference = new_balance - current_balance
if difference != 0:
@@ -60,10 +67,8 @@ def account_reconciliation(request):
else Transaction.Type.EXPENSE
),
amount=abs(difference),
date=timezone.localdate(timezone.now()),
reference_date=timezone.localdate(
timezone.now()
).replace(day=1),
date=adjustment_date,
reference_date=adjustment_reference_date,
description=_("Balance reconciliation"),
is_paid=True,
category=category,
@@ -82,5 +87,7 @@ def account_reconciliation(request):
formset = AccountBalanceFormSet(initial=initial_data)
return render(
request, "accounts/fragments/account_reconciliation.html", {"form": formset}
request,
"accounts/fragments/account_reconciliation.html",
{"form": formset, "cutoff": cutoff},
)
+81 -10
View File
@@ -1,9 +1,13 @@
from copy import copy
from decimal import Decimal
from django.core.exceptions import ValidationError as DjangoValidationError
from django.db.models import Q
from rest_framework import serializers
from rest_framework.permissions import IsAuthenticated
from apps.accounts.models import AccountGroup, Account, InvestmentClass
from apps.api.serializers.currencies import CurrencySerializer
from apps.accounts.models import AccountGroup, Account
from apps.currencies.models import Currency
@@ -15,6 +19,24 @@ class AccountGroupSerializer(serializers.ModelSerializer):
fields = "__all__"
class InvestmentClassSerializer(serializers.ModelSerializer):
permission_classes = [IsAuthenticated]
class Meta:
model = InvestmentClass
fields = ["id", "name", "target"]
def validate(self, attrs):
instance = copy(self.instance) if self.instance else InvestmentClass()
for name, value in attrs.items():
setattr(instance, name, value)
try:
instance.clean() # unique name and targets up to 100%
except DjangoValidationError as e:
raise serializers.ValidationError(e.message_dict)
return attrs
class AccountSerializer(serializers.ModelSerializer):
group = AccountGroupSerializer(read_only=True)
group_id = serializers.PrimaryKeyRelatedField(
@@ -36,6 +58,15 @@ class AccountSerializer(serializers.ModelSerializer):
allow_null=True,
)
investment_class = InvestmentClassSerializer(read_only=True)
investment_class_id = serializers.PrimaryKeyRelatedField(
queryset=InvestmentClass.objects.all(),
source="investment_class",
write_only=True,
allow_null=True,
required=False,
)
permission_classes = [IsAuthenticated]
class Meta:
@@ -49,7 +80,14 @@ class AccountSerializer(serializers.ModelSerializer):
"currency_id",
"exchange_currency",
"exchange_currency_id",
"show_exchanged_amounts",
"is_asset",
"is_investment",
"investment_class",
"investment_class_id",
"income_entities",
"income_categories",
"income_tags",
]
def __init__(self, *args, **kwargs):
@@ -58,15 +96,7 @@ class AccountSerializer(serializers.ModelSerializer):
if request and request.user.is_authenticated:
# Reload the queryset to get an updated version with the requesting user
self.fields["group_id"].queryset = AccountGroup.objects.all()
def create(self, validated_data):
return Account.objects.create(**validated_data)
def update(self, instance, validated_data):
for attr, value in validated_data.items():
setattr(instance, attr, value)
instance.save()
return instance
self.fields["investment_class_id"].queryset = InvestmentClass.objects.all()
class AccountBalanceSerializer(serializers.Serializer):
@@ -76,3 +106,44 @@ class AccountBalanceSerializer(serializers.Serializer):
projected_balance = serializers.DecimalField(max_digits=20, decimal_places=10)
currency = CurrencySerializer()
class InvestmentPerformanceSerializer(serializers.Serializer):
"""Serializer for an investment account's performance."""
currency = CurrencySerializer()
priced = serializers.BooleanField()
quantity = serializers.DecimalField(max_digits=42, decimal_places=30)
price = serializers.DecimalField(max_digits=42, decimal_places=30, allow_null=True)
price_date = serializers.DateTimeField(allow_null=True)
average_cost = serializers.DecimalField(
max_digits=42, decimal_places=30, allow_null=True
)
cost_basis = serializers.DecimalField(max_digits=42, decimal_places=30)
market_value = serializers.DecimalField(
max_digits=42, decimal_places=30, allow_null=True
)
unrealized = serializers.DecimalField(
max_digits=42, decimal_places=30, allow_null=True
)
realized = serializers.DecimalField(max_digits=42, decimal_places=30)
income = serializers.DecimalField(max_digits=42, decimal_places=30)
contributed = serializers.DecimalField(max_digits=42, decimal_places=30)
withdrawn = serializers.DecimalField(max_digits=42, decimal_places=30)
total_return = serializers.DecimalField(max_digits=42, decimal_places=30)
return_percentage = serializers.DecimalField(
max_digits=42, decimal_places=30, allow_null=True
)
estimated = serializers.BooleanField()
money_weighted_return = serializers.SerializerMethodField(
help_text="Return weighted by when money went in and out, in percent: "
"annual (XIRR) when the first contribution is a year old or more, "
"otherwise over the period held."
)
money_weighted_annualized = serializers.SerializerMethodField()
def get_money_weighted_return(self, perf) -> Decimal | None:
return perf.money_weighted and perf.money_weighted["percentage"]
def get_money_weighted_annualized(self, perf) -> bool | None:
return perf.money_weighted and perf.money_weighted["annualized"]
+22
View File
@@ -1,5 +1,7 @@
from django.utils.translation import gettext_lazy as _
from rest_framework import serializers
from apps.accounts.models import Account
from apps.import_app.models import ImportProfile, ImportRun
@@ -21,6 +23,7 @@ class ImportRunSerializer(serializers.ModelSerializer):
"status",
"profile",
"file_name",
"account",
"logs",
"processed_rows",
"total_rows",
@@ -39,3 +42,22 @@ class ImportFileSerializer(serializers.Serializer):
queryset=ImportProfile.objects.all(), source="profile"
)
file = serializers.FileField()
account_id = serializers.PrimaryKeyRelatedField(
queryset=Account.objects.filter(is_archived=False),
source="account",
required=False,
allow_null=True,
)
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
# Reload the querysets to get an updated version with the requesting user
self.fields["profile_id"].queryset = ImportProfile.objects.all()
self.fields["account_id"].queryset = Account.objects.filter(is_archived=False)
def validate(self, data):
if data.get("account") and not data["profile"].imports_transactions:
raise serializers.ValidationError(
{"account_id": _("Only transaction imports can use an account.")}
)
return data
+13
View File
@@ -115,6 +115,8 @@ class RecurringTransactionSerializer(serializers.ModelSerializer):
"account",
"type",
"amount",
"to_account",
"to_amount",
"description",
"category",
"tags",
@@ -130,6 +132,16 @@ class RecurringTransactionSerializer(serializers.ModelSerializer):
]
read_only_fields = ["last_generated_date", "last_generated_reference_date"]
def validate(self, attrs):
# Same rule as TransferTemplate.clean(), which DRF doesn't run
type_ = attrs.get("type", getattr(self.instance, "type", None))
to_account = attrs.get("to_account", getattr(self.instance, "to_account", None))
if type_ == RecurringTransaction.Type.TRANSFER and not to_account:
raise serializers.ValidationError(
{"to_account": _("This field is required.")}
)
return super().validate(attrs)
def create(self, validated_data):
instance = super().create(validated_data)
instance.create_upcoming_transactions()
@@ -170,6 +182,7 @@ class TransactionSerializer(serializers.ModelSerializer):
"id",
"installment_plan",
"recurring_transaction",
"linked_transaction",
"installment_id",
"owner",
"deleted_at",
+38 -1
View File
@@ -8,7 +8,7 @@ from rest_framework.test import APIClient
from apps.accounts.models import Account, AccountGroup
from apps.currencies.models import Currency
from apps.transactions.models import Transaction
from apps.transactions.models import Transaction, TransactionCategory
@override_settings(
@@ -97,3 +97,40 @@ class AccountBalanceAPITests(TestCase):
)
self.assertEqual(response.status_code, status.HTTP_401_UNAUTHORIZED)
def test_patch_and_create_with_income_categories(self):
category = TransactionCategory.objects.create(name="Interest", owner=self.user)
self.account.owner = self.user
self.account.save()
# Writes need Django model permissions
self.user.is_superuser = True
self.user.save()
response = self.client.patch(
f"/api/accounts/{self.account.id}/",
{"income_categories": [category.id]},
format="json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK, response.data)
self.assertEqual(response.data["income_categories"], [category.id])
# The join table: the related manager is per-user and empty without one
self.assertTrue(
Account.income_categories.through.objects.filter(
account=self.account, transactioncategory=category
).exists()
)
response = self.client.post(
"/api/accounts/",
{
"name": "Broker",
"group_id": None,
"currency_id": self.currency.id,
"exchange_currency_id": None,
"is_investment": True,
"income_categories": [category.id],
},
format="json",
)
self.assertEqual(response.status_code, status.HTTP_201_CREATED, response.data)
self.assertEqual(response.data["income_categories"], [category.id])
@@ -0,0 +1,81 @@
from django.contrib.auth import get_user_model
from django.contrib.auth.models import Permission
from django.test import TestCase, override_settings
from rest_framework.test import APIClient
from apps.accounts.models import Account, InvestmentClass
from apps.currencies.models import Currency
@override_settings(
STORAGES={
"default": {"BACKEND": "django.core.files.storage.FileSystemStorage"},
"staticfiles": {
"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"
},
},
WHITENOISE_AUTOREFRESH=True,
DEMO=False,
)
class InvestmentClassAPITests(TestCase):
def setUp(self):
User = get_user_model()
perms = Permission.objects.filter(
codename__in=[
"add_investmentclass",
"change_investmentclass",
"delete_investmentclass",
"change_account",
]
)
self.user = User.objects.create_user(email="u@test.com", password="pw")
self.other = User.objects.create_user(email="o@test.com", password="pw")
for user in (self.user, self.other):
user.user_permissions.set(perms)
self.api = APIClient()
self.api.force_authenticate(User.objects.get(pk=self.user.pk))
def test_crud_and_validation(self):
response = self.api.post(
"/api/investment-classes/", {"name": "Stocks", "target": "60"}, format="json"
)
self.assertEqual(response.status_code, 201, response.data)
stocks_id = response.data["id"]
# Same name again, and targets past 100%: rejected, not a server error
response = self.api.post("/api/investment-classes/", {"name": "Stocks"}, format="json")
self.assertEqual(response.status_code, 400)
self.assertIn("name", response.data)
response = self.api.post(
"/api/investment-classes/", {"name": "Bonds", "target": "50"}, format="json"
)
self.assertEqual(response.status_code, 400)
self.assertIn("target", response.data)
# Editing keeps its own target out of the sum
response = self.api.patch(
f"/api/investment-classes/{stocks_id}/", {"target": "100"}, format="json"
)
self.assertEqual(response.status_code, 200, response.data)
# Another user's names and targets don't count, and they can't see ours
other = APIClient()
other.force_authenticate(get_user_model().objects.get(pk=self.other.pk))
response = other.post(
"/api/investment-classes/", {"name": "Stocks", "target": "100"}, format="json"
)
self.assertEqual(response.status_code, 201, response.data)
self.assertEqual(other.get(f"/api/investment-classes/{stocks_id}/").status_code, 404)
def test_account_takes_a_class(self):
brl = Currency.objects.create(code="BRL", name="Real", decimal_places=2)
stocks = InvestmentClass.all_objects.create(name="Stocks", owner=self.user)
account = Account.all_objects.create(name="PETR4", currency=brl, owner=self.user)
response = self.api.patch(
f"/api/accounts/{account.id}/", {"investment_class_id": stocks.id}, format="json"
)
self.assertEqual(response.status_code, 200, response.data)
self.assertEqual(response.data["investment_class"]["name"], "Stocks")
response = self.api.get("/api/accounts/", {"investment_class": stocks.id})
self.assertEqual(response.data["count"], 1)
+1
View File
@@ -12,6 +12,7 @@ router.register(r"installment-plans", views.InstallmentPlanViewSet)
router.register(r"recurring-transactions", views.RecurringTransactionViewSet)
router.register(r"account-groups", views.AccountGroupViewSet)
router.register(r"accounts", views.AccountViewSet)
router.register(r"investment-classes", views.InvestmentClassViewSet)
router.register(r"currencies", views.CurrencyViewSet)
router.register(r"exchange-rates", views.ExchangeRateViewSet)
router.register(r"dca/strategies", views.DCAStrategyViewSet)
+34 -2
View File
@@ -4,14 +4,17 @@ from rest_framework.decorators import action
from rest_framework.permissions import IsAuthenticated
from rest_framework.response import Response
from apps.accounts.models import AccountGroup, Account
from apps.accounts.models import AccountGroup, Account, InvestmentClass
from apps.accounts.services import get_account_balance
from apps.api.permissions import SHARED_OBJECT_PERMISSIONS
from apps.api.serializers import (
AccountGroupSerializer,
InvestmentClassSerializer,
AccountSerializer,
AccountBalanceSerializer,
InvestmentPerformanceSerializer,
)
from apps.investments.services import get_holding_performance
class AccountGroupViewSet(viewsets.ModelViewSet):
@@ -32,12 +35,31 @@ class AccountGroupViewSet(viewsets.ModelViewSet):
return AccountGroup.objects.all()
class InvestmentClassViewSet(viewsets.ModelViewSet):
"""ViewSet for managing asset classes and their target allocation."""
queryset = InvestmentClass.objects.all()
serializer_class = InvestmentClassSerializer
filterset_fields = {"name": ["exact", "icontains"]}
search_fields = ["name"]
ordering_fields = "__all__"
ordering = ["id"]
def get_queryset(self):
return InvestmentClass.objects.all()
@extend_schema_view(
balance=extend_schema(
summary="Get account balance",
description="Returns the current and projected balance for the account, along with currency data.",
responses={200: AccountBalanceSerializer},
),
performance=extend_schema(
summary="Get investment performance",
description="Returns quantity, cost basis, market value, realized/unrealized gains, income and total return, in the account's exchange currency (or its own currency when it has none).",
responses={200: InvestmentPerformanceSerializer},
),
)
class AccountViewSet(viewsets.ModelViewSet):
"""ViewSet for managing accounts."""
@@ -48,9 +70,11 @@ class AccountViewSet(viewsets.ModelViewSet):
filterset_fields = {
"name": ["exact", "icontains"],
"group": ["exact", "isnull"],
"investment_class": ["exact", "isnull"],
"currency": ["exact"],
"exchange_currency": ["exact", "isnull"],
"is_asset": ["exact"],
"is_investment": ["exact"],
"is_archived": ["exact"],
"owner": ["exact"],
}
@@ -60,7 +84,7 @@ class AccountViewSet(viewsets.ModelViewSet):
def get_queryset(self):
return Account.objects.all().select_related(
"group", "currency", "exchange_currency"
"group", "currency", "exchange_currency", "investment_class"
)
@action(detail=True, methods=["get"], permission_classes=[IsAuthenticated])
@@ -80,3 +104,11 @@ class AccountViewSet(viewsets.ModelViewSet):
)
return Response(serializer.data)
@action(detail=True, methods=["get"], permission_classes=[IsAuthenticated])
def performance(self, request, pk=None):
"""Get investment performance (cost basis, value, returns) for an account."""
account = self.get_object()
return Response(
InvestmentPerformanceSerializer(get_holding_performance(account)).data
)
+13 -3
View File
@@ -1,3 +1,4 @@
from django.conf import settings
from django.core.files.storage import FileSystemStorage
from drf_spectacular.types import OpenApiTypes
from drf_spectacular.utils import OpenApiParameter, extend_schema, extend_schema_view, inline_serializer
@@ -37,6 +38,9 @@ class ImportProfileViewSet(viewsets.ReadOnlyModelViewSet):
ordering_fields = '__all__'
ordering = ['name']
def get_queryset(self):
return ImportProfile.objects.all()
@extend_schema_view(
list=extend_schema(
@@ -81,7 +85,7 @@ class ImportRunViewSet(viewsets.ReadOnlyModelViewSet):
ordering = ['-id']
def get_queryset(self):
queryset = super().get_queryset()
queryset = ImportRun.objects.all().order_by("-id")
profile_id = self.request.query_params.get("profile_id")
if profile_id:
queryset = queryset.filter(profile_id=profile_id)
@@ -98,6 +102,7 @@ class ImportRunViewSet(viewsets.ReadOnlyModelViewSet):
"properties": {
"profile_id": {"type": "integer", "description": "ID of the ImportProfile to use"},
"file": {"type": "string", "format": "binary", "description": "CSV or XLSX file to import"},
"account_id": {"type": "integer", "description": "ID of the Account to import every transaction into. Leave empty to let the profile pick the account. Only for profiles that import transactions"},
},
"required": ["profile_id", "file"],
},
@@ -127,12 +132,17 @@ class ImportViewSet(viewsets.ViewSet):
uploaded_file = serializer.validated_data["file"]
# Save file to temp location
fs = FileSystemStorage(location="/usr/src/app/temp")
fs = FileSystemStorage(location=settings.IMPORT_TEMP_ROOT)
filename = fs.save(uploaded_file.name, uploaded_file)
file_path = fs.path(filename)
# Create ImportRun record
import_run = ImportRun.objects.create(profile=profile, file_name=filename)
import_run = ImportRun.objects.create(
profile=profile,
file_name=filename,
account=serializer.validated_data.get("account"),
owner=request.user,
)
# Queue import task
process_import.defer(
View File
Whitespace-only changes.
+6
View File
@@ -0,0 +1,6 @@
from django.contrib import admin
from apps.bank_sync.models import BankSyncAccount, BankSyncService
admin.site.register(BankSyncService)
admin.site.register(BankSyncAccount)
+6
View File
@@ -0,0 +1,6 @@
from django.apps import AppConfig
class BankSyncConfig(AppConfig):
default_auto_field = "django.db.models.BigAutoField"
name = "apps.bank_sync"
+145
View File
@@ -0,0 +1,145 @@
import copy
from crispy_forms.bootstrap import FormActions
from crispy_forms.helper import FormHelper
from crispy_forms.layout import Layout
from django import forms
from django.db.models import Q
from django.utils.translation import gettext_lazy as _
from apps.accounts.models import Account
from apps.bank_sync.models import BankSyncAccount, BankSyncService
from apps.bank_sync.providers import PROVIDER_MAPPING
from apps.common.widgets.crispy.daisyui import Switch
from apps.common.widgets.crispy.submit import NoClassSubmit
from apps.common.widgets.datepicker import AirDatePickerInput
SETTING_PREFIX = "setting_"
class BankSyncServiceForm(forms.ModelForm):
class Meta:
model = BankSyncService
fields = [
"name",
"service_type",
"is_active",
"sync_interval",
"trigger_rules",
"match_existing",
]
def __init__(self, *args, refresh_url=None, **kwargs):
super().__init__(*args, **kwargs)
if self.instance.pk:
# Synced ids and links belong to the provider, it can't change
self.fields["service_type"].disabled = True
service_type = self.instance.service_type
else:
# The provider's own settings depend on the selected service type
service_type = self.data.get("service_type") or self.initial.get(
"service_type"
)
self.provider = PROVIDER_MAPPING.get(service_type)
self.setting_names = []
for key, field in (
self.provider.settings_fields if self.provider else {}
).items():
name = SETTING_PREFIX + key
self.fields[name] = copy.deepcopy(field)
if key in self.instance.settings:
self.initial.setdefault(name, self.instance.settings[key])
self.setting_names.append(name)
# Re-renders the form with the new provider's fields
self.fields["service_type"].widget.attrs.update(
{
"hx-get": refresh_url,
"hx-include": "closest form",
"hx-target": "#generic-offcanvas",
"hx-trigger": "change",
}
)
self.helper = FormHelper()
self.helper.form_tag = False
self.helper.layout = Layout(
"name",
"service_type",
*self.setting_names,
Switch("is_active"),
Switch("trigger_rules"),
Switch("match_existing"),
"sync_interval",
FormActions(
NoClassSubmit(
"submit",
_("Update") if self.instance.pk else _("Add"),
css_class="btn btn-primary",
),
),
)
def clean_sync_interval(self):
interval = self.cleaned_data["sync_interval"]
if self.provider and interval < self.provider.min_sync_interval:
raise forms.ValidationError(
_("This service can't sync more often than every %(hours)s hours.")
% {"hours": self.provider.min_sync_interval}
)
return interval
def clean(self):
cleaned_data = super().clean()
# Only once the rest is valid: a provider may spend a one-time token here
if self.provider and not self.errors:
try:
self.instance.settings = self.provider.clean_settings(
{
name.removeprefix(SETTING_PREFIX): cleaned_data[name]
for name in self.setting_names
}
)
except forms.ValidationError as e:
self.add_error(None, e)
return cleaned_data
class BankSyncAccountForm(forms.ModelForm):
currencies = forms.MultipleChoiceField(
required=False,
widget=forms.CheckboxSelectMultiple,
label=_("Currencies"),
help_text=_("Only sync transactions in these currencies. Empty syncs all."),
)
class Meta:
model = BankSyncAccount
fields = ["account", "start_date", "currencies"]
widgets = {"start_date": AirDatePickerInput(clear_button=False)}
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
# Keeps an account archived after being linked, or the whole formset fails
self.fields["account"].queryset = Account.objects.filter(
Q(is_archived=False) | Q(pk=self.instance.account_id)
).select_related("currency")
self.fields["account"].empty_label = _("Don't sync")
self.fields["currencies"].choices = [
(code, code)
for code in sorted(
set(self.instance.available_currencies) | set(self.instance.currencies)
)
]
def clean(self):
cleaned_data = super().clean()
if cleaned_data.get("account") and not cleaned_data.get("start_date"):
self.add_error("start_date", _("Required when an account is set."))
return cleaned_data
BankSyncAccountFormSet = forms.modelformset_factory(
BankSyncAccount, form=BankSyncAccountForm, extra=0
)
@@ -0,0 +1,61 @@
# Generated by Django 5.2.17 on 2026-11-08 15:27
import django.core.validators
import django.db.models.deletion
from django.conf import settings
from django.db import migrations, models
class Migration(migrations.Migration):
initial = True
dependencies = [
('accounts', '0016_account_untracked_by'),
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.CreateModel(
name='BankSyncService',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('name', models.CharField(max_length=255, verbose_name='Name')),
('service_type', models.CharField(choices=[('lunchflow', 'Lunch Flow'), ('simplefin', 'SimpleFIN'), ('pluggy', 'Pluggy')], max_length=255, verbose_name='Service Type')),
('is_active', models.BooleanField(default=True, verbose_name='Active')),
('settings', models.JSONField(blank=True, default=dict, verbose_name='Settings')),
('sync_interval', models.PositiveIntegerField(default=24, help_text='Most banks only update once a day.', validators=[django.core.validators.MinValueValidator(1), django.core.validators.MaxValueValidator(168)], verbose_name='Sync every X hours')),
('trigger_rules', models.BooleanField(default=True, help_text='Run your transaction rules on synced transactions.', verbose_name='Trigger transaction rules')),
('match_existing', models.BooleanField(default=False, help_text='Instead of creating a synced transaction, link it to one you already added on the same account, with the same type and amount, dated up to 3 days apart.', verbose_name='Match existing transactions')),
('last_sync', models.DateTimeField(blank=True, null=True, verbose_name='Last Successful Sync')),
('failure_count', models.PositiveIntegerField(default=0)),
('last_error', models.TextField(blank=True, default='')),
('owner', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='%(class)s_owned', to=settings.AUTH_USER_MODEL)),
],
options={
'verbose_name': 'Bank Sync Service',
'verbose_name_plural': 'Bank Sync Services',
'ordering': ['name'],
},
),
migrations.CreateModel(
name='BankSyncAccount',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('external_id', models.CharField(max_length=255)),
('external_name', models.CharField(blank=True, max_length=255)),
('available_currencies', models.JSONField(blank=True, default=list)),
('currencies', models.JSONField(blank=True, default=list, help_text='Only sync transactions in these currencies. Empty syncs all.', verbose_name='Currencies')),
('remote_error', models.TextField(blank=True, default='')),
('last_error', models.TextField(blank=True, default='')),
('start_date', models.DateField(blank=True, help_text='Transactions before this date are never imported.', null=True, verbose_name='Sync from')),
('last_synced_date', models.DateField(blank=True, null=True)),
('account', models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='bank_sync_accounts', to='accounts.account', verbose_name='Account')),
('service', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, related_name='accounts', to='bank_sync.banksyncservice')),
],
options={
'ordering': ['external_name'],
'unique_together': {('service', 'external_id')},
},
),
]
Whitespace-only changes.
+104
View File
@@ -0,0 +1,104 @@
from django.core.validators import MaxValueValidator, MinValueValidator
from django.db import models
from django.utils.translation import gettext_lazy as _
from apps.common.models import OwnedObject, OwnedObjectManager
class BankSyncService(OwnedObject):
class ServiceType(models.TextChoices):
LUNCHFLOW = "lunchflow", "Lunch Flow"
SIMPLEFIN = "simplefin", "SimpleFIN"
PLUGGY = "pluggy", "Pluggy"
name = models.CharField(max_length=255, verbose_name=_("Name"))
service_type = models.CharField(
max_length=255, choices=ServiceType.choices, verbose_name=_("Service Type")
)
is_active = models.BooleanField(default=True, verbose_name=_("Active"))
# Provider specific, keyed by the provider's settings_fields
settings = models.JSONField(default=dict, blank=True, verbose_name=_("Settings"))
sync_interval = models.PositiveIntegerField(
default=24,
validators=[MinValueValidator(1), MaxValueValidator(168)],
verbose_name=_("Sync every X hours"),
help_text=_("Most banks only update once a day."),
)
trigger_rules = models.BooleanField(
default=True,
verbose_name=_("Trigger transaction rules"),
help_text=_("Run your transaction rules on synced transactions."),
)
match_existing = models.BooleanField(
default=False,
verbose_name=_("Match existing transactions"),
help_text=_(
"Instead of creating a synced transaction, link it to one you already "
"added on the same account, with the same type and amount, dated up to "
"3 days apart."
),
)
last_sync = models.DateTimeField(
null=True, blank=True, verbose_name=_("Last Successful Sync")
)
failure_count = models.PositiveIntegerField(default=0)
last_error = models.TextField(blank=True, default="")
objects = OwnedObjectManager()
class Meta:
verbose_name = _("Bank Sync Service")
verbose_name_plural = _("Bank Sync Services")
ordering = ["name"]
def __str__(self):
return self.name
def get_provider(self):
from apps.bank_sync.providers import PROVIDER_MAPPING
return PROVIDER_MAPPING[self.service_type](self.settings)
class BankSyncAccount(models.Model):
"""An account on the provider, optionally linked to a WYGIWYH account."""
service = models.ForeignKey(
BankSyncService, on_delete=models.CASCADE, related_name="accounts"
)
external_id = models.CharField(max_length=255)
external_name = models.CharField(max_length=255, blank=True)
# Currencies seen on the provider, offered as choices for `currencies`
available_currencies = models.JSONField(default=list, blank=True)
currencies = models.JSONField(
default=list,
blank=True,
verbose_name=_("Currencies"),
help_text=_("Only sync transactions in these currencies. Empty syncs all."),
)
# Provider side problem, like an expired connection
remote_error = models.TextField(blank=True, default="")
last_error = models.TextField(blank=True, default="")
account = models.ForeignKey(
"accounts.Account",
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name="bank_sync_accounts",
verbose_name=_("Account"),
)
start_date = models.DateField(
null=True,
blank=True,
verbose_name=_("Sync from"),
help_text=_("Transactions before this date are never imported."),
)
last_synced_date = models.DateField(null=True, blank=True)
class Meta:
unique_together = ("service", "external_id")
ordering = ["external_name"]
def __str__(self):
return self.external_name or self.external_id
+10
View File
@@ -0,0 +1,10 @@
from apps.bank_sync.providers.lunchflow import LunchFlowProvider
from apps.bank_sync.providers.pluggy import PluggyProvider
from apps.bank_sync.providers.simplefin import SimpleFINProvider
# Keys must match BankSyncService.ServiceType
PROVIDER_MAPPING = {
"lunchflow": LunchFlowProvider,
"simplefin": SimpleFINProvider,
"pluggy": PluggyProvider,
}
+62
View File
@@ -0,0 +1,62 @@
from abc import ABC, abstractmethod
from dataclasses import dataclass
from datetime import date
from decimal import Decimal
@dataclass(frozen=True)
class RemoteAccount:
id: str
name: str
# Currency codes as the provider spells them
currencies: tuple = ()
# Why the account can't be synced right now (expired connection...), empty if fine
error: str = ""
@dataclass(frozen=True)
class RemoteTransaction:
id: str
date: date
# Signed: negative is money leaving the account
amount: Decimal
description: str = ""
notes: str = ""
currency: str = ""
# Not settled yet. Imported unpaid and updated when it posts, so only return
# these when the provider keeps the id once they post
pending: bool = False
class BankSyncProvider(ABC):
"""A bank data aggregator.
``settings_fields`` declares the provider's own configuration as Django form
fields. The service form renders them when the provider is selected and the
values end up in ``BankSyncService.settings`` under the same keys.
"""
settings_fields: dict = {}
# In hours, for services that limit how often they can be called
min_sync_interval: int = 1
def __init__(self, settings: dict):
self.settings = settings or {}
@classmethod
def clean_settings(cls, settings: dict) -> dict:
"""Validate or transform the settings before they're saved.
Runs once every other form field is valid. Raise ValidationError to reject.
"""
return settings
@abstractmethod
def list_accounts(self) -> list[RemoteAccount]:
"""Every account the credentials can see."""
@abstractmethod
def get_transactions(
self, account_id: str, start_date: date
) -> list[RemoteTransaction]:
"""Posted transactions of ``account_id`` dated ``start_date`` or later."""
+101
View File
@@ -0,0 +1,101 @@
import os
from datetime import date
from decimal import Decimal
import requests
from django import forms
from django.utils.translation import gettext as _
from django.utils.translation import gettext_lazy
from apps.bank_sync.providers.base import (
BankSyncProvider,
RemoteAccount,
RemoteTransaction,
)
DEFAULT_BASE_URL = "https://www.lunchflow.app/api/v1"
class LunchFlowProvider(BankSyncProvider):
"""https://www.lunchflow.app/docs/api/personal-api-overview"""
settings_fields = {
"api_key": forms.CharField(
label=gettext_lazy("API Key"),
# Not a PasswordInput: browsers autofill those with saved logins
widget=forms.TextInput(attrs={"autocomplete": "off"}),
help_text=gettext_lazy(
"Create a REST API destination on Lunch Flow and give it access to "
"the accounts you want to sync."
),
),
}
def __init__(self, settings: dict):
super().__init__(settings)
# Server side only, users must not point the server at arbitrary hosts
self.base_url = os.getenv("LUNCHFLOW_API_URL", DEFAULT_BASE_URL).rstrip("/")
self.session = requests.Session()
self.session.headers["x-api-key"] = self.settings.get("api_key", "")
def _get(self, path: str, **params) -> dict:
response = self.session.get(f"{self.base_url}{path}", params=params, timeout=30)
response.raise_for_status()
return response.json()
def _balance_currency(self, account_id) -> str:
# The account list has no currency, the balance does
try:
return self._get(f"/accounts/{account_id}/balance")["balance"]["currency"]
except (requests.RequestException, ValueError, KeyError, TypeError):
return ""
def list_accounts(self) -> list[RemoteAccount]:
accounts = []
for account in self._get("/accounts")["accounts"]:
currency = account.get("currency") or self._balance_currency(account["id"])
status = account.get("status") or "ACTIVE"
accounts.append(
RemoteAccount(
id=str(account["id"]),
name=" - ".join(
filter(None, [account.get("institution_name"), account["name"]])
),
currencies=(currency.upper(),) if currency else (),
error=(
""
if status == "ACTIVE"
else _(
"Lunch Flow reports this account as %(status)s, "
"check the connection on Lunch Flow."
)
% {"status": status}
),
)
)
return accounts
def get_transactions(
self, account_id: str, start_date: date
) -> list[RemoteTransaction]:
data = self._get(
f"/accounts/{account_id}/transactions", **{"from": start_date.isoformat()}
)
transactions = []
for tx in data["transactions"]:
# Pending ones are excluded by default, but they change ids when posted
if tx.get("isPending"):
continue
merchant = (tx.get("merchant") or "").strip()
description = (tx.get("description") or "").strip()
transactions.append(
RemoteTransaction(
id=str(tx["id"]),
date=date.fromisoformat(tx["date"][:10]),
amount=Decimal(str(tx["amount"])),
description=merchant or description,
notes=description if merchant and description != merchant else "",
currency=(tx.get("currency") or "").upper(),
)
)
return transactions
+147
View File
@@ -0,0 +1,147 @@
import re
from datetime import date
from decimal import Decimal
from urllib.parse import parse_qs, urlsplit
import requests
from django import forms
from django.utils import timezone
from django.utils.translation import gettext as _
from django.utils.translation import gettext_lazy
from apps.bank_sync.providers.base import (
BankSyncProvider,
RemoteAccount,
RemoteTransaction,
)
BASE_URL = "https://api.pluggy.ai"
# Item statuses that need the user to reconnect on Pluggy
NEEDS_ACTION = {"LOGIN_ERROR", "WAITING_USER_INPUT"}
class PluggyProvider(BankSyncProvider):
"""https://docs.pluggy.ai"""
settings_fields = {
"client_id": forms.CharField(
label=gettext_lazy("Client ID"),
widget=forms.TextInput(attrs={"autocomplete": "off"}),
),
"client_secret": forms.CharField(
label=gettext_lazy("Client Secret"),
# Not a PasswordInput: browsers autofill those with saved logins
widget=forms.TextInput(attrs={"autocomplete": "off"}),
),
"item_ids": forms.CharField(
label=gettext_lazy("Item IDs"),
widget=forms.TextInput(attrs={"autocomplete": "off"}),
help_text=gettext_lazy(
"The IDs of your Pluggy connections (items), separated by commas. "
"Meu Pluggy shows them for each connected institution."
),
),
}
def __init__(self, settings: dict):
super().__init__(settings)
self.session = requests.Session()
self.item_ids = [
item_id
for item_id in re.split(r"[\s,]+", self.settings.get("item_ids", ""))
if item_id
]
def _get(self, path: str, **params) -> dict:
if "X-API-KEY" not in self.session.headers:
# Lasts 2 hours, longer than any sync
response = self.session.post(
f"{BASE_URL}/auth",
json={
"clientId": self.settings.get("client_id", ""),
"clientSecret": self.settings.get("client_secret", ""),
},
timeout=30,
)
response.raise_for_status()
self.session.headers["X-API-KEY"] = response.json()["apiKey"]
response = self.session.get(f"{BASE_URL}{path}", params=params, timeout=30)
response.raise_for_status()
return response.json()
def list_accounts(self) -> list[RemoteAccount]:
accounts = []
for item_id in self.item_ids:
item = self._get(f"/items/{item_id}")
institution = (item.get("connector") or {}).get("name")
status = item.get("status") or ""
error = (
_(
"Pluggy reports this connection as %(status)s, reconnect it "
"on Pluggy."
)
% {"status": status}
if status in NEEDS_ACTION
else ""
)
for account in self._get("/accounts", itemId=item_id)["results"]:
currency = account.get("currencyCode") or ""
accounts.append(
RemoteAccount(
id=str(account["id"]),
name=" - ".join(
filter(
None,
[
institution,
account.get("marketingName") or account["name"],
account.get("number"),
],
)
),
currencies=(currency.upper(),) if currency else (),
error=error,
)
)
return accounts
def get_transactions(
self, account_id: str, start_date: date
) -> list[RemoteTransaction]:
params = {"accountId": account_id, "dateFrom": start_date.isoformat()}
transactions = []
while True:
data = self._get("/v2/transactions", **params)
for tx in data["results"]:
day = date.fromisoformat(tx["date"][:10])
# Pending keeps its id once it posts (a card's open bill). Future
# ones (installments, scheduled payments) wait for their date
pending = tx.get("status") == "PENDING"
if pending and day > timezone.localdate():
continue
converted = tx.get("amountInAccountCurrency")
amount = abs(Decimal(str(tx["amount"] if converted is None else converted)))
notes = ""
if converted is not None:
# Foreign purchase: keep the account's amount, note the original
notes = f"{tx.get('currencyCode', '')} {abs(Decimal(str(tx['amount'])))}"
transactions.append(
RemoteTransaction(
id=str(tx["id"]),
# Installments already come dated in their own month
date=day,
# Not the amount's sign: cards keep theirs, purchases are
# positive. `type` is normalized for every account
amount=amount if tx.get("type") == "CREDIT" else -amount,
description=(tx.get("description") or "").strip(),
notes=notes,
currency=""
if converted is not None
else (tx.get("currencyCode") or "").upper(),
pending=pending,
)
)
if not data.get("next"):
return transactions
# Send our own filters again with the cursor, `next` may drop them
params["after"] = parse_qs(urlsplit(data["next"]).query)["after"][0]
+182
View File
@@ -0,0 +1,182 @@
import base64
import binascii
import os
from datetime import date, datetime, time, timedelta
from decimal import Decimal
from urllib.parse import unquote, urlsplit, urlunsplit
import requests
from django import forms
from django.utils import timezone
from django.utils.translation import gettext as _
from django.utils.translation import gettext_lazy
from apps.bank_sync.providers.base import (
BankSyncProvider,
RemoteAccount,
RemoteTransaction,
)
DEFAULT_ALLOWED_HOSTS = "bridge.simplefin.org,beta-bridge.simplefin.org"
# SimpleFIN Bridge refuses longer ranges in a single request
MAX_RANGE_DAYS = 90
def _check_host(url: str) -> None:
# Tokens are user input: without this they could point the server at any host
allowed = {
host.strip().lower()
for host in os.getenv("SIMPLEFIN_ALLOWED_HOSTS", DEFAULT_ALLOWED_HOSTS).split(",")
if host.strip()
}
parts = urlsplit(url)
if parts.scheme not in ("http", "https") or (parts.hostname or "") not in allowed:
raise ValueError(
_("SimpleFIN server %(host)s isn't allowed on this instance.")
% {"host": parts.hostname or url}
)
def _timestamp(day: date) -> int:
# Local midnight, so windows and transaction dates agree on what a day is
return int(
datetime.combine(day, time(), tzinfo=timezone.get_current_timezone()).timestamp()
)
class SimpleFINProvider(BankSyncProvider):
"""https://www.simplefin.org/protocol.html"""
# SimpleFIN Bridge disables tokens that go much past ~24 requests a day, and a
# sync costs one plus one per account
min_sync_interval = 6
settings_fields = {
"access_url": forms.CharField(
label=gettext_lazy("Setup Token"),
widget=forms.TextInput(attrs={"autocomplete": "off"}),
help_text=gettext_lazy(
"Paste a setup token from your SimpleFIN server. It can only be "
"used once: on save it's exchanged for an access URL, which is "
"what you'll see here afterwards."
),
),
}
@classmethod
def clean_settings(cls, settings: dict) -> dict:
try:
return cls._claim(settings)
except ValueError as e:
raise forms.ValidationError(str(e))
@staticmethod
def _claim(settings: dict) -> dict:
value = settings["access_url"].strip()
# Already claimed: the form is being saved again
if not value.startswith(("http://", "https://")):
try:
claim_url = base64.b64decode(
value + "=" * (-len(value) % 4), validate=True
).decode()
except (binascii.Error, UnicodeDecodeError):
raise ValueError(_("This isn't a valid SimpleFIN setup token."))
_check_host(claim_url)
try:
response = requests.post(claim_url, timeout=30)
response.raise_for_status()
except requests.RequestException as e:
raise ValueError(
_(
"Couldn't claim the setup token, it may have been used "
"already: %(error)s"
)
% {"error": e}
)
value = response.text.strip()
_check_host(value)
return {**settings, "access_url": value}
def __init__(self, settings: dict):
super().__init__(settings)
parts = urlsplit(self.settings.get("access_url", ""))
self.session = requests.Session()
# Credentials go in a header, not the URL, so errors never show them
self.session.auth = (unquote(parts.username or ""), unquote(parts.password or ""))
netloc = (parts.hostname or "") + (f":{parts.port}" if parts.port else "")
self.base_url = urlunsplit((parts.scheme, netloc, parts.path.rstrip("/"), "", ""))
def _get_accounts(self, **params) -> dict:
# Settings can also arrive through a backup restore, skipping the form
_check_host(self.base_url)
response = self.session.get(
f"{self.base_url}/accounts",
params={"version": "2", **params},
# Slow: the server can fetch from every bank on each request
timeout=60,
)
response.raise_for_status()
return response.json()
def list_accounts(self) -> list[RemoteAccount]:
data = self._get_accounts(**{"balances-only": "1"})
connections = {c["conn_id"]: c.get("name", "") for c in data.get("connections", [])}
account_errors, connection_errors = {}, {}
for error in data.get("errlist", []):
if error.get("account_id"):
account_errors.setdefault(error["account_id"], error.get("msg", ""))
elif error.get("conn_id"):
connection_errors.setdefault(error["conn_id"], error.get("msg", ""))
accounts = []
for account in data["accounts"]:
institution = connections.get(account.get("conn_id")) or (
account.get("org") or {}
).get("name")
currency = account.get("currency") or ""
accounts.append(
RemoteAccount(
id=str(account["id"]),
name=" - ".join(filter(None, [institution, account["name"]])),
# Custom currencies are URLs, nothing to match them against
currencies=(currency.upper(),) if currency and "/" not in currency else (),
error=account_errors.get(account["id"])
or connection_errors.get(account.get("conn_id"), ""),
)
)
return accounts
def get_transactions(
self, account_id: str, start_date: date
) -> list[RemoteTransaction]:
transactions = []
start, stop = start_date, timezone.localdate() + timedelta(days=1)
while start < stop:
end = min(start + timedelta(days=MAX_RANGE_DAYS), stop)
data = self._get_accounts(
account=account_id,
**{"start-date": _timestamp(start), "end-date": _timestamp(end)},
)
for account in data["accounts"]:
if str(account["id"]) != account_id:
continue
currency = account.get("currency") or ""
for tx in account.get("transactions", []):
if tx.get("pending"):
continue
payee = (tx.get("payee") or "").strip()
description = (tx.get("description") or "").strip()
transactions.append(
RemoteTransaction(
id=str(tx["id"]),
date=datetime.fromtimestamp(
tx["posted"], tz=timezone.get_current_timezone()
).date(),
amount=Decimal(str(tx["amount"])),
description=payee or description,
notes=description if payee and description != payee else "",
currency=currency.upper() if "/" not in currency else "",
)
)
start = end
return transactions
+211
View File
@@ -0,0 +1,211 @@
import logging
from datetime import timedelta
from django.db.models import Min
from django.utils import timezone
from apps.bank_sync.models import BankSyncAccount, BankSyncService
from apps.common.functions.decimals import truncate_decimal
from apps.common.middleware.thread_local import delete_current_user, write_current_user
from apps.rules.signals import transaction_created
from apps.transactions.models import Transaction
logger = logging.getLogger(__name__)
# Banks post late and back-date transactions, so every sync looks this far back
OVERLAP_DAYS = 7
# How far apart a synced transaction and the one it matches can be dated
MATCH_DAYS = 3
# Pending ones older than this are taken as cancelled and no longer followed. A
# card's bill cycle is at most ~40 days
PENDING_MAX_DAYS = 60
def refresh_remote_accounts(service: BankSyncService, provider=None) -> None:
"""Mirror the provider's account list, keeping existing links.
Accounts the provider no longer returns are dropped: their ids are gone for good.
"""
provider = provider or service.get_provider()
remote_accounts = {remote.id: remote for remote in provider.list_accounts()}
service.accounts.exclude(external_id__in=remote_accounts).delete()
for remote in remote_accounts.values():
link, _ = BankSyncAccount.objects.get_or_create(
service=service, external_id=remote.id
)
link.external_name = remote.name
link.remote_error = remote.error
link.available_currencies = sorted(
set(link.available_currencies) | set(remote.currencies)
)
link.save()
def _match_existing(link: BankSyncAccount, internal_id, type_, amount, date) -> bool:
"""Tag a transaction the user already added instead of creating a new one."""
candidates = Transaction.userless_all_objects.filter(
account=link.account,
type=type_,
amount=truncate_decimal(amount, link.account.currency.decimal_places),
date__range=(date - timedelta(days=MATCH_DAYS), date + timedelta(days=MATCH_DAYS)),
internal_id__isnull=True,
deleted=False,
)
best = min(candidates, key=lambda t: abs((t.date - date).days), default=None)
if best is None:
return False
# Not a user edit, so no transaction_updated (it would run "on update" rules)
Transaction.userless_all_objects.filter(pk=best.pk).update(
emit_signal=False, internal_id=internal_id
)
return True
def _settle(transaction: Transaction, amount, date) -> None:
"""The bank posted a transaction that was still unpaid: take its final values.
Happens once, so later edits stick. One marked paid by hand is left alone.
"""
if transaction.is_paid or transaction.deleted:
return
# Only when it still follows the date, so one picked by the user is kept
if transaction.reference_date == transaction.date.replace(day=1):
transaction.reference_date = date.replace(day=1)
transaction.is_paid = True
transaction.amount = amount
transaction.date = date
# save() keeps a transfer's other side in step. It sends no signal, and as a
# bank update rather than a user edit, rules don't run again
transaction.save()
def _sync_account(service: BankSyncService, link: BankSyncAccount, provider) -> int:
prefix = f"{service.service_type}:{link.external_id}:"
start = link.start_date
if link.last_synced_date:
start = max(start, link.last_synced_date - timedelta(days=OVERLAP_DAYS))
# Pending ones can post weeks later (a card's bill closing): look back far
# enough to see it happen
oldest_unpaid = Transaction.userless_all_objects.filter(
internal_id__startswith=prefix,
is_paid=False,
deleted=False,
date__gte=timezone.localdate() - timedelta(days=PENDING_MAX_DAYS),
).aggregate(Min("date"))["date__min"]
if oldest_unpaid:
start = min(start, oldest_unpaid)
created = 0
seen_currencies = set()
for remote in provider.get_transactions(link.external_id, start):
if remote.date < link.start_date:
continue
if remote.currency:
seen_currencies.add(remote.currency)
if link.currencies and remote.currency not in link.currencies:
continue
internal_id = prefix + remote.id
type_ = (
Transaction.Type.INCOME if remote.amount > 0 else Transaction.Type.EXPENSE
)
amount = abs(remote.amount)
# userless_all_objects: also sees soft deleted ones, so deleting a synced
# transaction doesn't bring it back on the next run
existing = Transaction.userless_all_objects.filter(
internal_id=internal_id
).first()
if existing:
if not remote.pending:
_settle(existing, amount, remote.date)
continue
if service.match_existing and _match_existing(
link, internal_id, type_, amount, remote.date
):
continue
transaction = Transaction.objects.create(
account=link.account,
owner=service.owner,
type=type_,
amount=amount,
date=remote.date,
description=remote.description[:500],
notes=remote.notes,
is_paid=not remote.pending,
internal_id=internal_id,
)
created += 1
if service.trigger_rules:
transaction_created.send(sender=transaction)
link.last_synced_date = timezone.localdate()
link.available_currencies = sorted(set(link.available_currencies) | seen_currencies)
link.save(update_fields=["last_synced_date", "available_currencies"])
return created
def sync_service(service: BankSyncService) -> int:
"""Import new transactions for every linked account. Returns how many.
A failing account doesn't stop the others. Any failure keeps last_sync as is,
so the service is retried on the next run.
"""
created = 0
errors = []
# Rules and the owned-object managers read the user from the thread
write_current_user(service.owner)
try:
provider = service.get_provider()
refresh_remote_accounts(service, provider)
links = service.accounts.filter(
account__isnull=False, start_date__isnull=False
).select_related("account__currency")
for link in links:
error = link.remote_error
if not error:
try:
created += _sync_account(service, link, provider)
except Exception as e:
logger.exception(f"Bank sync failed for {service.name} / {link}")
error = str(e)
link.last_error = error[:1000]
link.save(update_fields=["last_error"])
if error:
errors.append(f"{link}: {error}")
except Exception as e:
logger.exception(f"Bank sync failed for {service.name}")
errors.append(str(e))
finally:
delete_current_user()
if errors:
service.failure_count += 1
service.last_error = "\n".join(errors)[:1000]
service.save(update_fields=["failure_count", "last_error"])
else:
service.last_sync = timezone.now()
service.failure_count = 0
service.last_error = ""
service.save(update_fields=["last_sync", "failure_count", "last_error"])
logger.info(f"Bank sync for {service.name} created {created} transactions")
return created
def is_due(service: BankSyncService) -> bool:
if service.last_sync is None:
return True
# Hour granularity, the scheduler runs on the hour
elapsed = timezone.now().replace(minute=0, second=0, microsecond=0) - (
service.last_sync.replace(minute=0, second=0, microsecond=0)
)
return elapsed >= timedelta(hours=service.sync_interval)
def sync_due_services(force: bool = False) -> None:
for service in BankSyncService.objects.filter(is_active=True):
if force or is_due(service):
sync_service(service)
+17
View File
@@ -0,0 +1,17 @@
from procrastinate.contrib.django import app
from apps.bank_sync.models import BankSyncService
from apps.bank_sync.sync import sync_due_services, sync_service
@app.periodic(cron="0 * * * *")
@app.task(lock="bank_sync", name="automatic_bank_sync")
def automatic_bank_sync(timestamp=None):
sync_due_services()
@app.task(lock="bank_sync", name="manual_bank_sync")
def manual_bank_sync(service_id: int):
service = BankSyncService.objects.filter(id=service_id).first()
if service:
sync_service(service)
Whitespace-only changes.
+897
View File
@@ -0,0 +1,897 @@
import base64
import io
import os
import zipfile
from datetime import date, timedelta
from decimal import Decimal
from unittest.mock import MagicMock, patch
from django.contrib.auth import get_user_model
from django.core.exceptions import ValidationError
from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import TestCase
from django.urls import reverse
from django.utils import timezone
from apps.accounts.models import Account
from apps.bank_sync.forms import BankSyncAccountForm, BankSyncServiceForm
from apps.bank_sync.models import BankSyncAccount, BankSyncService
from apps.bank_sync.providers.base import RemoteAccount, RemoteTransaction
from apps.bank_sync.providers.lunchflow import LunchFlowProvider
from apps.bank_sync.providers.pluggy import PluggyProvider
from apps.bank_sync.providers.simplefin import SimpleFINProvider
from apps.bank_sync.providers.simplefin import _timestamp as _ts
from apps.bank_sync.sync import is_due, refresh_remote_accounts, sync_service
from apps.common.middleware.thread_local import delete_current_user, write_current_user
from apps.currencies.models import Currency
from apps.transactions.models import Transaction
def _response(payload):
response = MagicMock()
response.json.return_value = payload
return response
class LunchFlowProviderTests(TestCase):
def setUp(self):
# Independent of whatever the test machine has set
self.enterContext(patch.dict("os.environ"))
os.environ.pop("LUNCHFLOW_API_URL", None)
self.provider = LunchFlowProvider({"api_key": "key"})
def test_sends_api_key_and_uses_default_url(self):
self.assertEqual(self.provider.session.headers["x-api-key"], "key")
self.assertEqual(self.provider.base_url, "https://www.lunchflow.app/api/v1")
def test_url_comes_from_the_environment_not_settings(self):
with patch.dict("os.environ", {"LUNCHFLOW_API_URL": "https://x.test/v1/"}):
provider = LunchFlowProvider({"api_key": "k", "base_url": "http://db:5432"})
self.assertEqual(provider.base_url, "https://x.test/v1")
def test_list_accounts(self):
def get(url, **kwargs):
if url.endswith("/accounts"):
return _response(
{
"accounts": [
{"id": 7, "name": "Checking", "institution_name": "Bank", "status": "ACTIVE"},
{"id": 8, "name": "Card", "institution_name": None, "status": "EXPIRED"},
]
}
)
if url.endswith("/7/balance"):
return _response({"balance": {"amount": 1, "currency": "eur"}})
raise __import__("requests").HTTPError("expired")
with patch.object(self.provider.session, "get", side_effect=get):
accounts = self.provider.list_accounts()
self.assertEqual(accounts[0], RemoteAccount("7", "Bank - Checking", ("EUR",), ""))
self.assertEqual(accounts[1].name, "Card")
self.assertEqual(accounts[1].currencies, ())
self.assertIn("EXPIRED", accounts[1].error)
def test_get_transactions(self):
payload = {
"transactions": [
{"id": "a", "amount": -12.5, "currency": "EUR", "date": "2026-10-01", "merchant": "Coffee", "description": "CARD 123 COFFEE", "isPending": False},
{"id": "b", "amount": 1000, "currency": "usd", "date": "2026-10-02", "merchant": None, "description": "Salary"},
{"id": "c", "amount": -1, "date": "2026-10-03", "merchant": "Pending", "isPending": True},
{"id": "d", "amount": -3, "date": "2026-10-04", "merchant": "Same", "description": "Same"},
]
}
with patch.object(
self.provider.session, "get", return_value=_response(payload)
) as mock_get:
transactions = self.provider.get_transactions("7", date(2026, 10, 1))
mock_get.assert_called_once_with(
"https://www.lunchflow.app/api/v1/accounts/7/transactions",
params={"from": "2026-10-01"},
timeout=30,
)
self.assertEqual(
transactions,
[
RemoteTransaction("a", date(2026, 10, 1), Decimal("-12.5"), "Coffee", "CARD 123 COFFEE", "EUR"),
RemoteTransaction("b", date(2026, 10, 2), Decimal("1000"), "Salary", "", "USD"),
RemoteTransaction("d", date(2026, 10, 4), Decimal("-3"), "Same", "", ""),
],
)
class SimpleFINProviderTests(TestCase):
def setUp(self):
self.enterContext(patch.dict("os.environ"))
os.environ.pop("SIMPLEFIN_ALLOWED_HOSTS", None)
self.provider = SimpleFINProvider(
{"access_url": "https://us%40r:p%3Ass@bridge.simplefin.org/simplefin/"}
)
def test_credentials_are_kept_out_of_the_url(self):
self.assertEqual(self.provider.session.auth, ("us@r", "p:ss"))
self.assertEqual(self.provider.base_url, "https://bridge.simplefin.org/simplefin")
def test_claims_setup_token(self):
token = base64.b64encode(b"https://bridge.simplefin.org/simplefin/claim/abc").decode()
response = MagicMock(text="https://u:p@bridge.simplefin.org/simplefin\n")
with patch("requests.post", return_value=response) as post:
settings = SimpleFINProvider.clean_settings({"access_url": token})
post.assert_called_once_with(
"https://bridge.simplefin.org/simplefin/claim/abc", timeout=30
)
self.assertEqual(settings, {"access_url": "https://u:p@bridge.simplefin.org/simplefin"})
def test_claimed_url_is_not_claimed_again(self):
with patch("requests.post") as post:
SimpleFINProvider.clean_settings({"access_url": "https://u:p@bridge.simplefin.org/x"})
post.assert_not_called()
def test_rejects_hosts_not_allowed(self):
token = base64.b64encode(b"http://db:5432/claim").decode()
with patch("requests.post") as post:
for value in (token, "https://u:p@evil.test/x", "not base64!"):
with self.assertRaises(ValidationError):
SimpleFINProvider.clean_settings({"access_url": value})
post.assert_not_called()
with patch.dict("os.environ", {"SIMPLEFIN_ALLOWED_HOSTS": "sfin.local"}):
SimpleFINProvider.clean_settings({"access_url": "http://u:p@sfin.local/x"})
# Restored settings skip the form, the request itself is checked too
provider = SimpleFINProvider({"access_url": "https://u:p@evil.test/x"})
with self.assertRaises(ValueError):
provider.list_accounts()
def test_list_accounts(self):
payload = {
"errlist": [
{"code": "con.auth", "msg": "Reconnect", "conn_id": "c2"},
{"code": "gen.api", "msg": "Slow down"},
],
"connections": [{"conn_id": "c1", "name": "Bank"}, {"conn_id": "c2", "name": "Other"}],
"accounts": [
{"id": "a1", "name": "Checking", "conn_id": "c1", "currency": "usd"},
{"id": "a2", "name": "Card", "conn_id": "c2", "currency": "https://x.test/pts"},
],
}
with patch.object(self.provider.session, "get", return_value=_response(payload)) as get:
accounts = self.provider.list_accounts()
get.assert_called_once_with(
"https://bridge.simplefin.org/simplefin/accounts",
params={"version": "2", "balances-only": "1"},
timeout=60,
)
self.assertEqual(
accounts,
[
RemoteAccount("a1", "Bank - Checking", ("USD",), ""),
RemoteAccount("a2", "Other - Card", (), "Reconnect"),
],
)
def test_get_transactions_in_90_day_windows(self):
today = timezone.localdate()
start = today - timedelta(days=100)
def get(url, params, timeout):
if params["start-date"] != _ts(start):
return _response({"accounts": []})
return _response(
{
"accounts": [
{
"id": "a1",
"currency": "USD",
"transactions": [
{"id": "t1", "posted": _ts(start) + 3600, "amount": "-5.10", "description": "CARD COFFEE", "payee": "Coffee"},
{"id": "t2", "posted": 0, "amount": "-1", "description": "Pending", "pending": True},
],
},
{"id": "other", "transactions": [{"id": "x", "posted": 0, "amount": "1"}]},
]
}
)
with patch.object(self.provider.session, "get", side_effect=get) as mock_get:
transactions = self.provider.get_transactions("a1", start)
windows = [(c.kwargs["params"]["start-date"], c.kwargs["params"]["end-date"]) for c in mock_get.call_args_list]
self.assertEqual(
windows,
[
(_ts(start), _ts(start + timedelta(days=90))),
(_ts(start + timedelta(days=90)), _ts(today + timedelta(days=1))),
],
)
self.assertEqual(mock_get.call_args.kwargs["params"]["account"], "a1")
self.assertEqual(
transactions,
[RemoteTransaction("t1", start, Decimal("-5.10"), "Coffee", "CARD COFFEE", "USD")],
)
class PluggyProviderTests(TestCase):
def setUp(self):
self.provider = PluggyProvider(
{"client_id": "id", "client_secret": "secret", "item_ids": "i1, i2\n"}
)
auth = _response({"apiKey": "key"})
self.post = self.enterContext(
patch.object(self.provider.session, "post", return_value=auth)
)
def test_list_accounts(self):
def get(url, params, timeout):
self.assertEqual(self.provider.session.headers["X-API-KEY"], "key")
if url.endswith("/items/i1"):
return _response({"status": "UPDATED", "connector": {"name": "Nubank"}})
if url.endswith("/items/i2"):
return _response({"status": "LOGIN_ERROR", "connector": {"name": "Itaú"}})
return _response(
{
"results": [
{"id": f"{params['itemId']}-a", "name": "Conta", "marketingName": None, "number": "0001", "currencyCode": "BRL"}
]
}
)
with patch.object(self.provider.session, "get", side_effect=get):
accounts = self.provider.list_accounts()
self.post.assert_called_once_with(
"https://api.pluggy.ai/auth",
json={"clientId": "id", "clientSecret": "secret"},
timeout=30,
)
self.assertEqual(accounts[0], RemoteAccount("i1-a", "Nubank - Conta - 0001", ("BRL",), ""))
self.assertEqual(accounts[1].id, "i2-a")
self.assertIn("LOGIN_ERROR", accounts[1].error)
def _get_transactions(self, pages):
responses = [_response(p) for p in pages]
with patch.object(self.provider.session, "get", side_effect=responses) as get:
transactions = self.provider.get_transactions("a", date(2026, 10, 1))
return transactions, get.call_args_list
def test_bank_account_follows_cursor_with_own_filters(self):
pages = [
{
"results": [
{"id": "t1", "date": "2026-10-01T03:00:00.000Z", "amount": -12.5, "type": "DEBIT", "status": "POSTED", "description": "Padaria", "currencyCode": "BRL"},
{"id": "t2", "date": "2026-10-02T03:00:00.000Z", "amount": -1, "type": "DEBIT", "status": "PENDING", "description": "Pending"},
],
"next": "?accountId=a&after=MjAy%3D",
},
{
"results": [
{"id": "t3", "date": "2026-10-04T03:00:00.000Z", "amount": 100, "type": "CREDIT", "status": "POSTED", "description": "Pix", "currencyCode": "BRL"},
],
"next": None,
},
]
transactions, calls = self._get_transactions(pages)
self.assertEqual(
[c.kwargs["params"] for c in calls],
[
{"accountId": "a", "dateFrom": "2026-10-01"},
{"accountId": "a", "dateFrom": "2026-10-01", "after": "MjAy="},
],
)
self.assertEqual(
transactions,
[
RemoteTransaction("t1", date(2026, 10, 1), Decimal("-12.5"), "Padaria", "", "BRL"),
RemoteTransaction("t2", date(2026, 10, 2), Decimal("-1"), "Pending", "", "", pending=True),
RemoteTransaction("t3", date(2026, 10, 4), Decimal("100"), "Pix", "", "BRL"),
],
)
def test_credit_card_follows_type_not_sign(self):
# Shapes from a real Meu Pluggy card: purchases are positive DEBITs,
# the bill payment a negative CREDIT
pages = [
{
"results": [
{"id": "t1", "date": "2026-10-03T03:00:00.000Z", "amount": 10, "amountInAccountCurrency": 55.3, "type": "DEBIT", "status": "POSTED", "description": "Steam", "currencyCode": "USD"},
{"id": "t2", "date": "2026-10-05T03:00:00.000Z", "amount": -500, "type": "CREDIT", "status": "POSTED", "description": "Pagamento recebido", "currencyCode": "BRL"},
{"id": "t3", "date": "2026-10-13T03:00:00.000Z", "amount": 18.72, "type": "DEBIT", "status": "POSTED", "description": "Loja 2/10", "currencyCode": "BRL",
"creditCardMetadata": {"installmentNumber": 2, "totalInstallments": 10, "purchaseDate": "2026-09-12T03:00:00.000Z"}},
# Future installment, waits for its date
{"id": "t4", "date": "2027-07-13T03:00:00.000Z", "amount": 18.72, "type": "DEBIT", "status": "PENDING", "description": "Loja 10/10", "currencyCode": "BRL"},
],
"next": None,
},
]
transactions, _ = self._get_transactions(pages)
self.assertEqual(
transactions,
[
RemoteTransaction("t1", date(2026, 10, 3), Decimal("-55.3"), "Steam", "USD 10", ""),
RemoteTransaction("t2", date(2026, 10, 5), Decimal("500"), "Pagamento recebido", "", "BRL"),
RemoteTransaction("t3", date(2026, 10, 13), Decimal("-18.72"), "Loja 2/10", "", "BRL"),
],
)
class FakeProvider:
def __init__(self, transactions=(), accounts=()):
self.transactions = list(transactions)
self.accounts = list(accounts)
self.calls = []
def list_accounts(self):
return self.accounts
def get_transactions(self, account_id, start_date):
self.calls.append((account_id, start_date))
return [t for t in self.transactions if t.date >= start_date]
class SyncTests(TestCase):
def setUp(self):
self.user = get_user_model().objects.create_user(
email="sync@example.com", password="x"
)
self.eur = Currency.objects.create(code="EUR", name="Euro", decimal_places=2)
self.account = Account.objects.create(
name="Checking", currency=self.eur, owner=self.user
)
self.service = BankSyncService.objects.create(
name="LF", service_type="lunchflow", owner=self.user
)
self.link = BankSyncAccount.objects.create(
service=self.service,
external_id="7",
account=self.account,
start_date=date(2026, 10, 1),
)
self.provider = FakeProvider(
[
RemoteTransaction("old", date(2026, 9, 30), Decimal("-1"), "Before start"),
RemoteTransaction("a", date(2026, 10, 1), Decimal("-12.555"), "Coffee", "note", "EUR"),
RemoteTransaction("b", date(2026, 10, 2), Decimal("1000"), "Salary", "", "EUR"),
],
accounts=[RemoteAccount("7", "Bank - Checking", ("EUR",))],
)
def _sync(self):
with patch.object(BankSyncService, "get_provider", return_value=self.provider):
created = sync_service(self.service)
self.service.refresh_from_db()
self.link.refresh_from_db()
return created
def test_creates_transactions(self):
self.assertEqual(self._sync(), 2)
coffee = Transaction.userless_all_objects.get(internal_id="lunchflow:7:a")
self.assertEqual(coffee.type, Transaction.Type.EXPENSE)
self.assertEqual(coffee.amount, Decimal("12.55")) # truncated to currency
self.assertEqual(coffee.description, "Coffee")
self.assertEqual(coffee.notes, "note")
self.assertEqual(coffee.owner, self.user)
self.assertEqual(coffee.account, self.account)
self.assertTrue(coffee.is_paid)
salary = Transaction.userless_all_objects.get(internal_id="lunchflow:7:b")
self.assertEqual(salary.type, Transaction.Type.INCOME)
self.assertEqual(salary.amount, Decimal("1000"))
self.assertIsNotNone(self.service.last_sync)
self.assertEqual(self.link.last_synced_date, timezone.localdate())
self.assertEqual(self.link.external_name, "Bank - Checking")
def test_second_run_does_not_duplicate(self):
self._sync()
self.assertEqual(self._sync(), 0)
self.assertEqual(Transaction.userless_all_objects.count(), 2)
def test_deleted_transactions_are_not_reimported(self):
self._sync()
Transaction.userless_all_objects.filter(internal_id="lunchflow:7:a").update(
deleted=True
)
self.assertEqual(self._sync(), 0)
def test_later_runs_only_look_back_the_overlap(self):
self.link.last_synced_date = date(2026, 11, 1)
self.link.save()
self._sync()
self.assertEqual(self.provider.calls, [("7", date(2026, 10, 25))])
def _pending(self, days_ago, amount="-20", pending=True):
day = timezone.localdate() - timedelta(days=days_ago)
return RemoteTransaction("p", day, Decimal(amount), "Hotel", pending=pending)
def _hotel(self):
return Transaction.userless_all_objects.get(internal_id="lunchflow:7:p")
def test_pending_is_imported_unpaid_and_settled_once_when_it_posts(self):
self.link.start_date = timezone.localdate() - timedelta(days=90)
self.link.save()
self.provider.transactions = [self._pending(30)]
self.assertEqual(self._sync(), 1)
self.assertFalse(self._hotel().is_paid)
# Still pending with another amount: left alone, edits stick
self.provider.transactions = [self._pending(30, "-22")]
self._sync()
self.assertEqual(self._hotel().amount, Decimal("20"))
# And it's still looked for, way past the overlap
self.assertEqual(self.provider.calls[-1][1], timezone.localdate() - timedelta(days=30))
# Posts with its final amount and date
self.provider.transactions = [self._pending(29, "-25.509", pending=False)]
self.assertEqual(self._sync(), 0)
hotel = self._hotel()
self.assertTrue(hotel.is_paid)
self.assertEqual(hotel.amount, Decimal("25.50"))
self.assertEqual(hotel.date, timezone.localdate() - timedelta(days=29))
self.assertEqual(hotel.reference_date, hotel.date.replace(day=1))
# Settled: later changes on the bank don't touch it, look back is normal
self.provider.transactions = [self._pending(29, "-99", pending=False)]
self._sync()
self.assertEqual(self._hotel().amount, Decimal("25.50"))
self.assertEqual(self.provider.calls[-1][1], timezone.localdate() - timedelta(days=7))
def test_settling_keeps_a_reference_date_picked_by_the_user(self):
self.link.start_date = timezone.localdate() - timedelta(days=90)
self.link.save()
self.provider.transactions = [self._pending(10)]
self._sync()
Transaction.userless_all_objects.filter(internal_id="lunchflow:7:p").update(
emit_signal=False, reference_date=date(2030, 1, 1)
)
self.provider.transactions = [self._pending(10, pending=False)]
self._sync()
self.assertEqual(self._hotel().reference_date, date(2030, 1, 1))
self.assertTrue(self._hotel().is_paid)
def test_unpaid_marked_paid_by_hand_is_left_alone(self):
self.link.start_date = timezone.localdate() - timedelta(days=90)
self.link.save()
self.provider.transactions = [self._pending(10)]
self._sync()
Transaction.userless_all_objects.filter(internal_id="lunchflow:7:p").update(
emit_signal=False, is_paid=True
)
self.provider.transactions = [self._pending(10, "-99", pending=False)]
self._sync()
self.assertEqual(self._hotel().amount, Decimal("20"))
def test_old_pending_is_no_longer_followed(self):
self.link.start_date = timezone.localdate() - timedelta(days=200)
self.link.save()
self.provider.transactions = [self._pending(61)]
self._sync()
self._sync()
# Taken as cancelled: back to the normal overlap
self.assertEqual(self.provider.calls[-1][1], timezone.localdate() - timedelta(days=7))
def test_unlinked_accounts_are_skipped(self):
self.link.account = None
self.link.save()
self.assertEqual(self._sync(), 0)
self.assertEqual(self.provider.calls, [])
def test_currency_filter(self):
self.provider.transactions.append(
RemoteTransaction("usd", date(2026, 10, 3), Decimal("-5"), "Abroad", "", "USD")
)
self.link.currencies = ["USD"]
self.link.save()
self.assertEqual(self._sync(), 1)
self.assertTrue(
Transaction.userless_all_objects.filter(internal_id="lunchflow:7:usd").exists()
)
# Every currency seen becomes a choice, even filtered out ones
self.assertEqual(self.link.available_currencies, ["EUR", "USD"])
def test_no_currency_filter_syncs_all(self):
self.provider.transactions.append(
RemoteTransaction("usd", date(2026, 10, 3), Decimal("-5"), "Abroad", "", "USD")
)
self.assertEqual(self._sync(), 3)
def test_match_existing_is_off_by_default(self):
Transaction.objects.create(
account=self.account, owner=self.user, type=Transaction.Type.INCOME,
amount=Decimal("1000"), date=date(2026, 10, 2), description="Mine",
)
self.assertEqual(self._sync(), 2)
@patch("apps.rules.signals.check_for_transaction_rules.defer")
def test_matching_does_not_trigger_rules(self, defer):
self.service.match_existing = True
self.service.save()
self.provider.transactions = self.provider.transactions[2:] # salary only
Transaction.objects.create(
account=self.account, owner=self.user, type=Transaction.Type.INCOME,
amount=Decimal("1000"), date=date(2026, 10, 2), description="Mine",
)
defer.reset_mock()
self.assertEqual(self._sync(), 0)
defer.assert_not_called()
def test_match_existing(self):
self.service.match_existing = True
self.service.save()
far = Transaction.objects.create(
account=self.account, owner=self.user, type=Transaction.Type.INCOME,
amount=Decimal("1000"), date=date(2026, 9, 28), description="Too far",
)
close = Transaction.objects.create(
account=self.account, owner=self.user, type=Transaction.Type.INCOME,
amount=Decimal("1000"), date=date(2026, 10, 3), description="Mine",
)
expense = Transaction.objects.create(
account=self.account, owner=self.user, type=Transaction.Type.EXPENSE,
amount=Decimal("12.55"), date=date(2026, 10, 1), description="Coffee by hand",
)
self.assertEqual(self._sync(), 0)
close.refresh_from_db()
expense.refresh_from_db()
far.refresh_from_db()
self.assertEqual(close.internal_id, "lunchflow:7:b")
self.assertEqual(close.description, "Mine") # left as the user wrote it
self.assertEqual(expense.internal_id, "lunchflow:7:a")
self.assertIsNone(far.internal_id)
# Matched once, never again
self.assertEqual(self._sync(), 0)
def test_match_existing_takes_one_per_synced_transaction(self):
self.service.match_existing = True
self.service.save()
self.provider.transactions = [
RemoteTransaction("x1", date(2026, 10, 5), Decimal("-3"), "Coffee", "", "EUR"),
RemoteTransaction("x2", date(2026, 10, 5), Decimal("-3"), "Coffee", "", "EUR"),
]
Transaction.objects.create(
account=self.account, owner=self.user, type=Transaction.Type.EXPENSE,
amount=Decimal("3"), date=date(2026, 10, 5), description="One coffee",
)
self.assertEqual(self._sync(), 1)
self.assertEqual(Transaction.userless_all_objects.count(), 2)
def test_failing_account_does_not_stop_the_others(self):
other = Account.objects.create(name="Other", currency=self.eur, owner=self.user)
other_link = BankSyncAccount.objects.create(
service=self.service, external_id="8", account=other, start_date=date(2026, 10, 1)
)
self.provider.accounts.append(RemoteAccount("8", "Broken"))
get_transactions = self.provider.get_transactions
def flaky(account_id, start_date):
if account_id == "8":
raise Exception("boom")
return get_transactions(account_id, start_date)
self.provider.get_transactions = flaky
self.assertEqual(self._sync(), 2)
other_link.refresh_from_db()
self.assertEqual(other_link.last_error, "boom")
self.assertEqual(self.link.last_error, "")
self.assertEqual(self.service.failure_count, 1)
self.assertIn("boom", self.service.last_error)
self.assertIsNone(self.service.last_sync) # retried next run
self.provider.get_transactions = get_transactions
self._sync()
other_link.refresh_from_db()
self.assertEqual(other_link.last_error, "")
self.assertEqual(self.service.failure_count, 0)
self.assertEqual(self.service.last_error, "")
self.assertIsNotNone(self.service.last_sync)
def test_provider_reported_problem_skips_the_account(self):
self.provider.accounts = [RemoteAccount("7", "Bank", (), "Connection expired")]
self.assertEqual(self._sync(), 0)
self.assertEqual(self.provider.calls, [])
self.assertEqual(self.link.remote_error, "Connection expired")
self.assertEqual(self.link.last_error, "Connection expired")
self.assertEqual(self.service.failure_count, 1)
def test_listing_failure_is_recorded(self):
self.provider.list_accounts = MagicMock(side_effect=Exception("401"))
self.assertEqual(self._sync(), 0)
self.assertEqual(self.service.failure_count, 1)
self.assertEqual(self.service.last_error, "401")
self.assertTrue(BankSyncAccount.objects.filter(pk=self.link.pk).exists())
@patch("apps.rules.signals.check_for_transaction_rules.defer")
def test_rules_are_triggered(self, defer):
self._sync()
self.assertEqual(defer.call_count, 2)
self.assertEqual(defer.call_args.kwargs["user_id"], self.user.id)
@patch("apps.rules.signals.check_for_transaction_rules.defer")
def test_rules_can_be_disabled(self, defer):
self.service.trigger_rules = False
self.service.save()
self._sync()
defer.assert_not_called()
def test_refresh_remote_accounts(self):
self.link.available_currencies = ["GBP"]
self.link.save()
gone = BankSyncAccount.objects.create(service=self.service, external_id="gone")
self.provider.accounts = [
RemoteAccount("7", "Renamed", ("EUR",)),
RemoteAccount("9", "New", ("USD",)),
]
refresh_remote_accounts(self.service, self.provider)
self.link.refresh_from_db()
self.assertEqual(self.link.external_name, "Renamed")
self.assertEqual(self.link.account, self.account)
self.assertEqual(self.link.available_currencies, ["EUR", "GBP"])
self.assertTrue(self.service.accounts.filter(external_id="9", account=None).exists())
self.assertFalse(BankSyncAccount.objects.filter(pk=gone.pk).exists())
def test_is_due(self):
self.service.sync_interval = 24
self.assertTrue(is_due(self.service))
self.service.last_sync = timezone.now() - timedelta(hours=23)
self.assertFalse(is_due(self.service))
self.service.last_sync = timezone.now() - timedelta(hours=25)
self.assertTrue(is_due(self.service))
class FormTests(TestCase):
def setUp(self):
# Account choices are scoped to the logged in user
self.user = get_user_model().objects.create_user(email="f@example.com", password="x")
write_current_user(self.user)
self.addCleanup(delete_current_user)
def test_provider_settings_fields_are_added(self):
form = BankSyncServiceForm(initial={"service_type": "lunchflow"})
self.assertIn("setting_api_key", form.fields)
self.assertNotIn("setting_base_url", form.fields)
self.assertNotIn("setting_api_key", BankSyncServiceForm().fields)
def test_settings_are_saved_and_loaded(self):
form = BankSyncServiceForm(
{
"name": "LF",
"service_type": "lunchflow",
"is_active": "on",
"sync_interval": "24",
"setting_api_key": "secret",
}
)
self.assertTrue(form.is_valid(), form.errors)
service = form.save()
self.assertEqual(service.settings, {"api_key": "secret"})
edit = BankSyncServiceForm(instance=service)
self.assertEqual(edit.initial["setting_api_key"], "secret")
def test_service_type_is_locked_on_edit(self):
service = BankSyncService.objects.create(
name="LF", service_type="lunchflow", settings={"api_key": "k"}
)
form = BankSyncServiceForm(
{"name": "LF", "service_type": "other", "sync_interval": "24", "setting_api_key": "k"},
instance=service,
)
self.assertTrue(form.fields["service_type"].disabled)
self.assertTrue(form.is_valid(), form.errors)
self.assertEqual(form.save().service_type, "lunchflow")
def test_provider_settings_are_validated(self):
form = BankSyncServiceForm(
{"name": "LF", "service_type": "lunchflow", "sync_interval": "24"}
)
self.assertFalse(form.is_valid())
self.assertIn("setting_api_key", form.errors)
def test_provider_minimum_sync_interval(self):
data = {"name": "SF", "service_type": "simplefin", "setting_access_url": "x"}
with patch.object(SimpleFINProvider, "clean_settings", side_effect=lambda s: s):
form = BankSyncServiceForm({**data, "sync_interval": "5"})
self.assertIn("sync_interval", form.errors)
self.assertTrue(BankSyncServiceForm({**data, "sync_interval": "6"}).is_valid())
@patch.object(SimpleFINProvider, "clean_settings")
def test_clean_settings_runs_last_and_its_errors_show(self, clean_settings):
data = {"service_type": "simplefin", "sync_interval": "24", "setting_access_url": "tok"}
# Another field is invalid: a one-time token must not be spent
self.assertFalse(BankSyncServiceForm(data).is_valid())
clean_settings.assert_not_called()
clean_settings.side_effect = ValidationError("Token used")
form = BankSyncServiceForm({**data, "name": "SF"})
self.assertFalse(form.is_valid())
self.assertEqual(form.non_field_errors(), ["Token used"])
clean_settings.side_effect = None
clean_settings.return_value = {"access_url": "https://u:p@bridge.simplefin.org/x"}
form = BankSyncServiceForm({**data, "name": "SF"})
self.assertTrue(form.is_valid())
self.assertEqual(form.save().settings, clean_settings.return_value)
def test_account_link_form(self):
eur = Currency.objects.create(code="€", name="Euro")
service = BankSyncService.objects.create(name="LF", service_type="lunchflow")
link = BankSyncAccount.objects.create(
service=service, external_id="7", available_currencies=["EUR", "USD"]
)
account = Account.objects.create(name="EUR", currency=eur, owner=self.user)
form = BankSyncAccountForm({"account": account.id}, instance=link)
self.assertFalse(form.is_valid())
self.assertIn("start_date", form.errors)
# Currency codes don't need to match: WYGIWYH's are free form
form = BankSyncAccountForm(
{"account": account.id, "start_date": "2026-10-01", "currencies": ["USD"]},
instance=link,
)
self.assertTrue(form.is_valid(), form.errors)
self.assertEqual(form.save().currencies, ["USD"])
form = BankSyncAccountForm({"currencies": ["GBP"]}, instance=link)
self.assertIn("currencies", form.errors)
self.assertTrue(BankSyncAccountForm({}, instance=link).is_valid())
def test_linked_archived_account_stays_valid(self):
eur = Currency.objects.create(code="EUR", name="Euro")
service = BankSyncService.objects.create(name="LF", service_type="lunchflow")
archived = Account.objects.create(name="Old", currency=eur, owner=self.user, is_archived=True)
other_archived = Account.objects.create(name="Older", currency=eur, owner=self.user, is_archived=True)
link = BankSyncAccount.objects.create(
service=service, external_id="7", account=archived, start_date=date(2026, 10, 1)
)
form = BankSyncAccountForm(
{"account": archived.id, "start_date": "2026-10-01"}, instance=link
)
self.assertTrue(form.is_valid(), form.errors)
self.assertNotIn(other_archived, form.fields["account"].queryset)
class ViewTests(TestCase):
def setUp(self):
self.user = get_user_model().objects.create_superuser(
email="admin@example.com", password="x"
)
self.client.force_login(self.user)
self.htmx = {"HTTP_HX_REQUEST": "true"}
def test_switching_provider_renders_its_fields(self):
response = self.client.get(
reverse("bank_sync_add"), {"service_type": "lunchflow", "name": "Typed"}, **self.htmx
)
self.assertContains(response, 'name="setting_api_key"')
self.assertContains(response, 'value="Typed"')
def test_add_and_map_accounts(self):
response = self.client.post(
reverse("bank_sync_add"),
{"name": "LF", "service_type": "lunchflow", "sync_interval": "24", "setting_api_key": "k"},
**self.htmx,
)
self.assertEqual(response.status_code, 204)
service = BankSyncService.objects.get()
self.assertEqual(service.owner, self.user)
provider = FakeProvider(
accounts=[
RemoteAccount("7", "Bank - Checking", ("EUR",)),
RemoteAccount("8", "Card", (), "Connection expired"),
]
)
with patch.object(BankSyncService, "get_provider", return_value=provider):
response = self.client.get(reverse("bank_sync_accounts", args=[service.id]), **self.htmx)
self.assertContains(response, "Bank - Checking")
self.assertContains(response, "Connection expired")
self.assertContains(response, 'value="EUR"')
account = Account.objects.create(
name="Checking", currency=Currency.objects.create(code="EUR", name="Euro")
)
link = service.accounts.get(external_id="7")
card = service.accounts.get(external_id="8")
response = self.client.post(
reverse("bank_sync_accounts", args=[service.id]),
{
"form-TOTAL_FORMS": "2",
"form-INITIAL_FORMS": "2",
"form-0-id": card.id,
"form-1-id": link.id,
"form-1-account": account.id,
"form-1-start_date": "2026-10-01",
"form-1-currencies": "EUR",
},
**self.htmx,
)
self.assertEqual(response.status_code, 204)
link.refresh_from_db()
self.assertEqual(link.account, account)
self.assertEqual(link.currencies, ["EUR"])
def test_accounts_view_shows_provider_errors(self):
service = BankSyncService.objects.create(name="LF", service_type="lunchflow")
with patch.object(BankSyncService, "get_provider", side_effect=Exception("401")):
response = self.client.get(reverse("bank_sync_accounts", args=[service.id]), **self.htmx)
self.assertContains(response, "401")
def test_other_users_services_are_hidden(self):
other = get_user_model().objects.create_user(email="o@example.com", password="x")
service = BankSyncService.objects.create(name="LF", service_type="lunchflow", owner=other)
response = self.client.get(reverse("bank_sync_edit", args=[service.id]), **self.htmx)
self.assertEqual(response.status_code, 404)
class ExportRestoreTests(TestCase):
def setUp(self):
self.admin = get_user_model().objects.create_superuser(
email="admin@example.com", password="x"
)
self.other = get_user_model().objects.create_user(
email="other@example.com", password="x"
)
self.client.force_login(self.admin)
eur = Currency.objects.create(code="EUR", name="Euro")
# Same name for both users: links must not mix them up
self.account = Account.objects.create(name="Checking", currency=eur, owner=self.other)
Account.objects.create(name="Checking", currency=eur, owner=self.admin)
self.service = BankSyncService.objects.create(
name="LF",
service_type="lunchflow",
owner=self.other,
settings={"api_key": "secret"},
sync_interval=12,
trigger_rules=False,
match_existing=True,
)
BankSyncAccount.objects.create(
service=self.service,
external_id="7",
external_name="Bank - Checking",
available_currencies=["EUR", "USD"],
currencies=["EUR"],
account=self.account,
start_date=date(2026, 10, 1),
last_synced_date=date(2026, 11, 1),
)
BankSyncAccount.objects.create(service=self.service, external_id="8")
def test_round_trip(self):
response = self.client.post(
reverse("export_form"),
{"users": "on", "accounts": "on", "currencies": "on", "bank_sync": "on"},
HTTP_HX_REQUEST="true",
)
self.assertEqual(response["Content-Type"], "application/zip")
with zipfile.ZipFile(io.BytesIO(response.content)) as z:
self.assertIn("bank_sync_services.csv", z.namelist())
self.assertIn("bank_sync_accounts.csv", z.namelist())
BankSyncService._base_manager.all().delete()
upload = SimpleUploadedFile("export.zip", response.content)
response = self.client.post(
reverse("restore_form"), {"zip_file": upload}, HTTP_HX_REQUEST="true"
)
self.assertEqual(response.status_code, 204)
service = BankSyncService._base_manager.get()
self.assertEqual(service.owner, self.other)
self.assertEqual(service.settings, {"api_key": "secret"})
self.assertEqual(service.sync_interval, 12)
self.assertFalse(service.trigger_rules)
self.assertTrue(service.match_existing)
linked = service.accounts.get(external_id="7")
self.assertEqual(linked.account, self.account)
self.assertEqual(linked.start_date, date(2026, 10, 1))
self.assertEqual(linked.last_synced_date, date(2026, 11, 1))
self.assertEqual(linked.available_currencies, ["EUR", "USD"])
self.assertEqual(linked.currencies, ["EUR"])
self.assertIsNone(service.accounts.get(external_id="8").account)
+21
View File
@@ -0,0 +1,21 @@
from django.urls import path
from . import views
urlpatterns = [
path("bank-sync/", views.bank_sync_index, name="bank_sync_index"),
path("bank-sync/list/", views.bank_sync_list, name="bank_sync_list"),
path("bank-sync/add/", views.bank_sync_add, name="bank_sync_add"),
path("bank-sync/<int:pk>/edit/", views.bank_sync_edit, name="bank_sync_edit"),
path(
"bank-sync/<int:pk>/delete/", views.bank_sync_delete, name="bank_sync_delete"
),
path(
"bank-sync/<int:pk>/accounts/",
views.bank_sync_accounts,
name="bank_sync_accounts",
),
path(
"bank-sync/<int:pk>/sync/", views.bank_sync_sync_now, name="bank_sync_sync_now"
),
]
+135
View File
@@ -0,0 +1,135 @@
import logging
from django.contrib import messages
from django.contrib.auth.decorators import login_required
from django.http import HttpResponse
from django.shortcuts import get_object_or_404, render
from django.urls import reverse
from django.utils.translation import gettext_lazy as _
from django.views.decorators.http import require_http_methods
from apps.bank_sync.forms import BankSyncAccountFormSet, BankSyncServiceForm
from apps.bank_sync.models import BankSyncService
from apps.bank_sync.sync import refresh_remote_accounts
from apps.bank_sync.tasks import manual_bank_sync
from apps.common.decorators.demo import disabled_on_demo
from apps.common.decorators.htmx import only_htmx
logger = logging.getLogger(__name__)
_CLOSE = {"HX-Trigger": "updated, hide_offcanvas"}
@login_required
@disabled_on_demo
@require_http_methods(["GET"])
def bank_sync_index(request):
return render(request, "bank_sync/pages/index.html")
@only_htmx
@login_required
@disabled_on_demo
@require_http_methods(["GET"])
def bank_sync_list(request):
services = BankSyncService.objects.prefetch_related("accounts")
return render(request, "bank_sync/fragments/list.html", {"services": services})
def _service_form(request, instance=None):
url = request.path
if request.method == "POST":
return BankSyncServiceForm(request.POST, instance=instance, refresh_url=url)
# A GET with params comes from switching provider: keep what was typed
return BankSyncServiceForm(
instance=instance, initial=request.GET.dict() or None, refresh_url=url
)
@only_htmx
@login_required
@disabled_on_demo
@require_http_methods(["GET", "POST"])
def bank_sync_add(request):
form = _service_form(request)
if request.method == "POST" and form.is_valid():
form.save()
messages.success(request, _("Service added successfully"))
return HttpResponse(status=204, headers=_CLOSE)
return render(
request,
"bank_sync/fragments/form.html",
{"form": form, "title": _("Add bank sync service")},
)
@only_htmx
@login_required
@disabled_on_demo
@require_http_methods(["GET", "POST"])
def bank_sync_edit(request, pk):
service = get_object_or_404(BankSyncService, id=pk)
form = _service_form(request, instance=service)
if request.method == "POST" and form.is_valid():
form.save()
messages.success(request, _("Service updated successfully"))
return HttpResponse(status=204, headers=_CLOSE)
return render(
request,
"bank_sync/fragments/form.html",
{"form": form, "title": _("Edit bank sync service")},
)
@only_htmx
@login_required
@disabled_on_demo
@require_http_methods(["DELETE"])
def bank_sync_delete(request, pk):
get_object_or_404(BankSyncService, id=pk).delete()
messages.success(request, _("Service deleted successfully"))
return HttpResponse(status=204, headers=_CLOSE)
@only_htmx
@login_required
@disabled_on_demo
@require_http_methods(["GET", "POST"])
def bank_sync_accounts(request, pk):
service = get_object_or_404(BankSyncService, id=pk)
error = None
if request.method == "POST":
formset = BankSyncAccountFormSet(request.POST, queryset=service.accounts.all())
if formset.is_valid():
formset.save()
messages.success(request, _("Accounts updated successfully"))
return HttpResponse(status=204, headers=_CLOSE)
else:
try:
refresh_remote_accounts(service)
except Exception as e:
logger.warning(f"Could not list accounts for {service.name}: {e}")
error = _("Couldn't load the accounts from the provider: %(error)s") % {
"error": e
}
formset = BankSyncAccountFormSet(queryset=service.accounts.all())
return render(
request,
"bank_sync/fragments/accounts.html",
{"service": service, "formset": formset, "error": error},
)
@only_htmx
@login_required
@disabled_on_demo
@require_http_methods(["GET"])
def bank_sync_sync_now(request, pk):
service = get_object_or_404(BankSyncService, id=pk)
manual_bank_sync.defer(service_id=service.id)
messages.success(request, _("Sync queued successfully"))
return HttpResponse(status=204, headers={"HX-Trigger": "toasts"})
+1 -1
View File
@@ -66,7 +66,7 @@ def calendar_list(request, month: int, year: int):
@require_http_methods(["GET"])
def calendar_transactions_list(request, day: int, month: int, year: int):
date = datetime.date(year=year, month=month, day=day)
transactions = Transaction.objects.filter(date=date).order_by(
transactions = Transaction.objects.filter(date=date).collapse_transfers().order_by(
"date",
"-type",
"-is_paid",
-5
View File
@@ -1,5 +1,4 @@
from django.apps import AppConfig
from django.core.cache import cache
class CommonConfig(AppConfig):
@@ -20,9 +19,5 @@ class CommonConfig(AppConfig):
admin.site.unregister(SocialApp)
admin.site.unregister(SocialToken)
# Delete the cache for update checks to prevent false-positives when the app is restarted
# this will be recreated by the check_for_updates task
cache.delete("update_check")
# Register system checks for required environment variables
from apps.common import checks # noqa: F401
+10 -2
View File
@@ -21,11 +21,19 @@ class DynamicModelChoiceField(forms.ModelChoiceField):
super().__init__(queryset=self.queryset, *args, **kwargs)
self._created_instance = None
def _get(self, **lookup):
# The form's queryset may hold choices the default manager hides, like
# another user's object already set on a shared record
try:
return self.queryset.get(**lookup)
except self.model.DoesNotExist:
return self.model.objects.get(**lookup)
def to_python(self, value):
if value in self.empty_values:
return None
try:
return self.model.objects.get(**{self.to_field_name: value})
return self._get(**{self.to_field_name: value})
except (ValueError, TypeError, self.model.DoesNotExist):
return value # Return the raw value; we'll handle creation in clean()
@@ -49,7 +57,7 @@ class DynamicModelChoiceField(forms.ModelChoiceField):
try:
if value.isdigit():
return self.model.objects.get(id=value)
return self._get(id=value)
else:
raise self.model.DoesNotExist
except self.model.DoesNotExist:
@@ -0,0 +1,536 @@
"""Builds the demo dataset with every date relative to today.
Run on an empty database (the demo reset flushes first). Covers every feature:
multi-currency accounts, exchange rates, an ETF and Bitcoin tracked as
currencies with DCA strategies, transfers, a credit card using reference
dates, installment plans, recurring transactions, quick transactions, rules,
filter presets, muted/unpaid/archived/deactivated examples, and investments:
unit-priced (ETF, crypto, a stock with dividends, a sale and gifted shares, gold)
and value-tracked (savings, a retirement fund) holdings in asset classes.
"""
import calendar
import random
from datetime import date, datetime, time
from decimal import ROUND_HALF_UP, Decimal
from dateutil.relativedelta import relativedelta
from django.contrib.auth import get_user_model
from django.core.management.base import BaseCommand
from django.db import transaction as db_transaction
from django.db.models import Q
from django.utils import timezone
from apps.accounts.models import Account, AccountGroup, InvestmentClass
from apps.common.middleware.thread_local import delete_current_user, write_current_user
from apps.currencies.models import Currency, ExchangeRate
from apps.dca.models import DCAEntry, DCAStrategy
from apps.rules.models import TransactionRule, TransactionRuleAction
from apps.transactions.models import (
FilterPreset,
InstallmentPlan,
QuickTransaction,
RecurringTransaction,
Transaction,
TransactionCategory,
TransactionEntity,
TransactionTag,
)
DEMO_EMAIL = "demo@demo.com"
DEMO_PASSWORD = "wygiwyhdemo"
HISTORY_MONTHS = 12 # months of history before the current one
IN = Transaction.Type.INCOME
EX = Transaction.Type.EXPENSE
def q(value, places=2):
return Decimal(str(value)).quantize(Decimal(1).scaleb(-places), ROUND_HALF_UP)
class Command(BaseCommand):
help = "Loads demo data with dates relative to today. Expects an empty database."
def handle(self, *args, **options):
User = get_user_model()
if User.objects.filter(email=DEMO_EMAIL).exists():
if options["verbosity"]:
self.stdout.write("Demo user already exists, skipping.")
return
self.today = timezone.localdate()
self.rng = random.Random(42) # same demo every day, only shifted in time
with db_transaction.atomic():
user = User.objects.create_user(
email=DEMO_EMAIL,
password=DEMO_PASSWORD,
first_name="Demo",
last_name="User",
)
# SharedObject/OwnedObject.save() pick the owner from here
write_current_user(user)
try:
self._build(user)
finally:
delete_current_user()
if options["verbosity"]:
self.stdout.write(self.style.SUCCESS("Demo data loaded."))
# ---------- helpers ----------
def day(self, months, day):
"""Date `months` away from the current month, day clamped to the month."""
first = self.today.replace(day=1) + relativedelta(months=months)
last = calendar.monthrange(first.year, first.month)[1]
return first.replace(day=min(day, last))
def tx(self, account, kind, amount, when, description, category=None,
tags=(), entities=(), reference=None, paid=None, mute=False, notes=""):
t = Transaction.objects.create(
account=account,
type=kind,
amount=amount,
date=when,
reference_date=(reference or when).replace(day=1),
description=description,
category=self.cats[category] if category else None,
is_paid=(when <= self.today) if paid is None else paid,
mute=mute,
notes=notes,
)
if tags:
t.tags.set([self.tags[n] for n in tags])
if entities:
t.entities.set([self.ents[n] for n in entities])
return t
def transfer(self, source, target, amount, when, description,
to_amount=None, mute=True, category=None):
"""Same shape as TransferForm.save(): an expense and an income."""
out = self.tx(source, EX, amount, when, description, category, mute=mute)
inc = self.tx(target, IN, to_amount or amount, when, description, category, mute=mute)
inc.linked_transaction = out
inc.save()
out.linked_transaction = inc
out.save()
return out, inc
def rate(self, series, months):
return series[months + HISTORY_MONTHS]
# ---------- data ----------
def _build(self, user):
today = self.today
rng = self.rng
# Currencies, including an ETF and airline miles tracked as currencies
usd = Currency.objects.create(code="USD", name="US Dollar", decimal_places=2, prefix="$ ")
eur = Currency.objects.create(
code="EUR", name="Euro", decimal_places=2, prefix="€ ", exchange_currency=usd
)
btc = Currency.objects.create(
code="BTC", name="Bitcoin", decimal_places=8, prefix="₿ ", exchange_currency=usd
)
vwce = Currency.objects.create(
code="VWCE", name="Vanguard FTSE All-World ETF", decimal_places=4,
suffix=" VWCE", exchange_currency=eur,
)
miles = Currency.objects.create(
code="MILES", name="Airline Miles", decimal_places=0, suffix=" mi",
exchange_currency=usd,
)
aapl = Currency.objects.create(
code="AAPL", name="Apple Inc. shares", decimal_places=4, suffix=" AAPL",
exchange_currency=usd,
)
gold = Currency.objects.create(
code="XAU", name="Gold (grams)", decimal_places=2, suffix=" g",
exchange_currency=usd,
)
# Monthly rate history so net worth charts move; index 0 = oldest month
n = HISTORY_MONTHS + 1
eur_usd = [q(1.06 + 0.04 * i / n + rng.uniform(-0.01, 0.01), 4) for i in range(n)]
btc_usd = [q(58000 + 40000 * i / n + rng.uniform(-4000, 4000)) for i in range(n)]
vwce_eur = [q(112 + 20 * i / n + rng.uniform(-2, 2)) for i in range(n)]
# Own generator, so the investment examples don't shift the rest of the demo
inv = random.Random(7)
aapl_usd = [q(185 + 45 * i / n + inv.uniform(-9, 9)) for i in range(n)]
gold_usd = [q(76 + 18 * i / n + inv.uniform(-2, 2)) for i in range(n)]
for i in range(n):
months = i - HISTORY_MONTHS
first = self.day(months, 1)
when = timezone.make_aware(datetime.combine(first, time(12)))
if months == 0:
when = timezone.now() # latest rate is "today"
for src, dst, series in [
(eur, usd, eur_usd), (btc, usd, btc_usd), (vwce, eur, vwce_eur),
(aapl, usd, aapl_usd), (gold, usd, gold_usd),
]:
ExchangeRate.objects.create(
from_currency=src, to_currency=dst, rate=series[i], date=when
)
ExchangeRate.objects.create(
from_currency=miles, to_currency=usd, rate=Decimal("0.012"),
date=timezone.make_aware(datetime.combine(self.day(-HISTORY_MONTHS, 1), time(12))),
)
# Asset classes: targets add up to 100%; Commodities has none, so the
# rebalance view leaves it out
stocks = InvestmentClass.objects.create(name="Stocks", target=Decimal("60"))
fixed_income = InvestmentClass.objects.create(name="Fixed income", target=Decimal("25"))
crypto = InvestmentClass.objects.create(name="Crypto", target=Decimal("15"))
commodities = InvestmentClass.objects.create(name="Commodities")
# Accounts
everyday = AccountGroup.objects.create(name="Everyday")
investing = AccountGroup.objects.create(name="Savings & Investing")
rewards = AccountGroup.objects.create(name="Rewards")
checking = Account.objects.create(name="Checking", group=everyday, currency=usd)
card = Account.objects.create(name="Credit Card", group=everyday, currency=usd)
cash = Account.objects.create(name="Cash Wallet", group=everyday, currency=usd)
# No exchange currency on an investment: tracked by value
savings = Account.objects.create(
name="High-Yield Savings", group=investing, currency=usd, is_asset=True,
is_investment=True, investment_class=fixed_income,
)
euro = Account.objects.create(
name="Euro Account", group=investing, currency=eur, exchange_currency=usd
)
# An exchange currency on an investment: priced by units
brokerage = Account.objects.create(
name="Brokerage (VWCE)", group=investing, currency=vwce, exchange_currency=eur,
is_asset=True, is_investment=True, investment_class=stocks,
)
btc_wallet = Account.objects.create(
name="Bitcoin Wallet", group=investing, currency=btc, exchange_currency=usd,
is_asset=True, is_investment=True, investment_class=crypto,
)
stock_acc = Account.objects.create(
name="Brokerage (Apple shares)", group=investing, currency=aapl,
exchange_currency=usd, is_asset=True, is_investment=True, investment_class=stocks,
)
retirement = Account.objects.create(
name="Retirement Fund", group=investing, currency=usd, is_asset=True,
is_investment=True, investment_class=stocks,
)
gold_acc = Account.objects.create(
name="Gold Bars", group=investing, currency=gold, exchange_currency=usd,
is_asset=True, is_investment=True, investment_class=commodities,
)
miles_acc = Account.objects.create(name="Airline Miles", group=rewards, currency=miles)
old_bank = Account.objects.create(
name="Old Bank (closed)", group=everyday, currency=usd, is_archived=True
)
self.cats = {
name: TransactionCategory.objects.create(name=name)
for name in [
"Salary", "Freelance", "Interest", "Rent", "Groceries", "Dining",
"Transport", "Utilities", "Subscriptions", "Health", "Travel",
"Shopping", "Investments", "Fees", "Rewards", "Gifts", "Transfers",
"Dividends", "Market result",
]
}
self.cats["Cable TV"] = TransactionCategory.objects.create(
name="Cable TV", active=False # deactivated: hidden from new transactions
)
self.tags = {
name: TransactionTag.objects.create(name=name)
for name in ["essentials", "fun", "work", "home", "travel", "reimbursable"]
}
self.ents = {
name: TransactionEntity.objects.create(name=name)
for name in [
"Acme Corp", "Northwind (client)", "Landlord", "Power & Light Co.",
"FiberNet", "Fresh Market", "Corner Bistro", "Sushi House", "Uber",
"City Transit", "StreamFlix", "Iron Gym", "Pharma Plus", "BuyMore",
"Alex", "Sky Airlines", "Apple Inc.",
]
}
# What counts as an investment's result rather than money in or out
savings.income_categories.set([self.cats["Interest"]])
retirement.income_categories.set([self.cats["Market result"], self.cats["Fees"]])
# Dividends paid into checking count towards the Apple shares' return
stock_acc.income_entities.set([self.ents["Apple Inc."]])
btc_dca = DCAStrategy.objects.create(
name="Bitcoin monthly", target_currency=btc, payment_currency=usd,
notes="Buy $100 of Bitcoin every month, no matter the price.",
)
etf_dca = DCAStrategy.objects.create(
name="World ETF (VWCE)", target_currency=vwce, payment_currency=eur,
notes="€250 into a global index ETF every month.",
)
# Installment plans and recurring transactions first, so card statements
# below include them
laptop = InstallmentPlan.objects.create(
account=card, type=EX, description="New laptop", number_of_installments=12,
start_date=self.day(-4, 14), recurrence=InstallmentPlan.Recurrence.MONTHLY,
installment_amount=Decimal("108.25"), category=self.cats["Shopping"],
notes="12 interest-free installments on the credit card.",
)
laptop.entities.set([self.ents["BuyMore"]])
laptop.create_transactions()
dental = InstallmentPlan.objects.create(
account=checking, type=EX, description="Dental treatment",
number_of_installments=3, start_date=self.day(-1, 16),
recurrence=InstallmentPlan.Recurrence.MONTHLY,
installment_amount=Decimal("240.00"), category=self.cats["Health"],
)
dental.create_transactions()
# Recurring transactions
def recurring(account, amount, description, category, start, entity=None,
recurrence=RecurringTransaction.RecurrenceType.MONTH, paused=False):
r = RecurringTransaction.objects.create(
account=account, type=EX, amount=amount, description=description,
category=self.cats[category], start_date=start, recurrence_type=recurrence,
recurrence_interval=1, keep_at_most=3, is_paused=paused,
)
if entity:
r.entities.set([self.ents[entity]])
if not paused:
r.create_upcoming_transactions()
return r
recurring(card, Decimal("15.99"), "StreamFlix", "Subscriptions",
self.day(-HISTORY_MONTHS, 3), "StreamFlix")
recurring(checking, Decimal("59.99"), "Internet", "Utilities",
self.day(-HISTORY_MONTHS, 15), "FiberNet")
recurring(checking, Decimal("39.90"), "Gym membership", "Health",
self.day(-HISTORY_MONTHS, 2), "Iron Gym")
recurring(card, Decimal("14.99"), "Domain renewal", "Subscriptions",
self.day(-10, 9), recurrence=RecurringTransaction.RecurrenceType.YEAR)
recurring(card, Decimal("6.99"), "Magazine subscription", "Subscriptions",
self.day(-3, 11), paused=True)
# Work trip paid on the card, muted so it stays out of monthly totals
self.tx(card, EX, Decimal("230.00"), self.day(-2, 9), "Client visit (reimbursable)",
"Travel", ["work", "reimbursable"], mute=True,
notes="Muted: Acme pays this back, so it shouldn't count as spending.")
self.tx(checking, IN, Decimal("230.00"), self.day(-1, 5), "Expense reimbursement",
tags=["work", "reimbursable"], entities=["Acme Corp"], mute=True)
# ---- month by month ----
for m in range(-HISTORY_MONTHS, 2):
past_or_current = m <= 0
self.tx(checking, IN, Decimal("5400.00") if m >= -6 else Decimal("5100.00"),
self.day(m, 5), "Salary", "Salary", ["work"], ["Acme Corp"])
self.tx(checking, EX, Decimal("1650.00"), self.day(m, 1), "Rent", "Rent",
["home", "essentials"], ["Landlord"])
self.tx(checking, EX, q(rng.uniform(70, 135)), self.day(m, 12), "Electricity bill",
"Utilities", ["home", "essentials"], ["Power & Light Co."])
self.transfer(checking, savings, Decimal("600.00"), self.day(m, 6),
"Monthly savings", category="Transfers")
if not past_or_current:
continue
# Credit card: closes on the 25th, later purchases count for next month
for _ in range(rng.randint(5, 7)):
d = self.day(m, rng.randint(1, 28))
if d > today:
continue
kind = rng.choice(["groceries", "groceries", "dining", "uber", "shopping"])
ref = self.day(m + 1, 1) if d.day > 25 else d
if kind == "groceries":
self.tx(card, EX, q(rng.uniform(45, 160)), d, "Groceries", "Groceries",
["essentials"], ["Fresh Market"], reference=ref)
elif kind == "dining":
place = rng.choice(["Corner Bistro", "Sushi House"])
self.tx(card, EX, q(rng.uniform(28, 95)), d, f"Dinner at {place}", "Dining",
["fun"], [place], reference=ref)
elif kind == "uber":
self.tx(card, EX, q(rng.uniform(9, 32)), d, "Uber ride", "Transport",
entities=["Uber"], reference=ref)
else:
self.tx(card, EX, q(rng.uniform(25, 140)), d, "Household stuff", "Shopping",
["home"], ["BuyMore"], reference=ref)
# Pay last month's card statement (a transfer between your own accounts)
statement = sum(
Transaction.objects.filter(
account=card, type=EX, reference_date=self.day(m - 1, 1)
).values_list("amount", flat=True),
Decimal("0"),
)
if statement:
self.transfer(checking, card, statement, self.day(m, 10),
"Credit card payment", category="Transfers")
# Miles earned on card spending; small cash expenses
if self.day(m, 27) <= today:
self.tx(miles_acc, IN, Decimal(rng.randint(400, 900)), self.day(m, 27),
"Miles earned on card spending", "Rewards", entities=["Sky Airlines"])
for _ in range(2):
d = self.day(m, rng.randint(1, 28))
if d <= today:
self.tx(cash, EX, q(rng.choice([3.5, 4.25, 4.5, 2.75])), d,
rng.choice(["Coffee", "Bus ticket"]),
rng.choice(["Dining", "Transport"]))
# Interest on savings
if self.day(m, 28) <= today:
self.tx(savings, IN, q(rng.uniform(18, 32)), self.day(m, 28),
"Savings interest", "Interest")
# Bitcoin DCA: $100 per month at that month's price
d = self.day(m, 22)
if d <= today:
price = self.rate(btc_usd, m)
sats = q(Decimal(100) / price, 8)
out, inc = self.transfer(checking, btc_wallet, Decimal("100.00"), d,
"Bitcoin DCA", to_amount=sats, mute=False,
category="Investments")
DCAEntry.objects.create(strategy=btc_dca, date=d, amount_paid=Decimal("100.00"),
amount_received=sats, expense_transaction=out,
income_transaction=inc)
# ETF DCA: €250 per month from the Euro account
d = self.day(m, 20)
if d <= today:
price = self.rate(vwce_eur, m)
shares = q(Decimal(250) / price, 4)
out, inc = self.transfer(euro, brokerage, Decimal("250.00"), d,
"Buy VWCE", to_amount=shares, mute=False,
category="Investments")
DCAEntry.objects.create(strategy=etf_dca, date=d, amount_paid=Decimal("250.00"),
amount_received=shares, expense_transaction=out,
income_transaction=inc)
self.tx(euro, EX, Decimal("1.50"), d, "Broker fee", "Fees")
# Apple shares every other month; quarterly dividends land in checking
d = self.day(m, 18)
if m % 2 == 0 and d <= today:
self.transfer(checking, stock_acc, Decimal("500.00"), d, "Buy AAPL",
to_amount=q(Decimal(500) / self.rate(aapl_usd, m), 4),
mute=False, category="Investments")
d = self.day(m, 14)
if m % 3 == 1 and d <= today:
self.tx(checking, IN, q(inv.uniform(4, 7)), d, "AAPL dividend", "Dividends",
entities=["Apple Inc."])
# Retirement fund, tracked by value: contributions plus the quarterly
# statement's gain or loss
d = self.day(m, 6)
if d <= today:
self.transfer(checking, retirement, Decimal("300.00"), d,
"Retirement contribution", category="Investments")
d = self.day(m, 28)
result = {-12: 310, -9: -180, -6: 420, -3: 265, 0: 190}.get(m)
if result and d <= today:
self.tx(retirement, IN if result > 0 else EX, Decimal(abs(result)), d,
"Quarterly statement result", "Market result",
notes="The fund's gain or loss from its statement.")
# Every 3 months: convert dollars to euros at that month's rate
if m % 3 == 0 and self.day(m, 7) <= today:
usd_amount = Decimal("1000.00")
self.transfer(checking, euro, usd_amount, self.day(m, 7),
"Currency exchange USD → EUR",
to_amount=q(usd_amount / self.rate(eur_usd, m)),
category="Transfers")
# Freelance paid in euros, every other month
if m % 2 == 0 and self.day(m, 15) <= today:
self.tx(euro, IN, q(rng.choice([650, 800, 1200])), self.day(m, 15),
"Freelance project", "Freelance", ["work"], ["Northwind (client)"])
# ATM withdrawal every other month
if m % 2 == 1 and self.day(m, 3) <= today:
self.transfer(checking, cash, Decimal("100.00"), self.day(m, 3),
"ATM withdrawal", category="Transfers")
# ---- one-offs ----
# Investments: gifted shares (no known cost, so estimated), a partial sale,
# a yearly fund fee, and gold bought twice
self.tx(stock_acc, IN, Decimal("5"), self.day(-7, 9), "Shares gifted by grandma",
"Gifts", notes="No cost is known, so it's valued at that day's price.")
self.transfer(stock_acc, checking, Decimal("3"), self.day(-2, 21), "Sell AAPL",
to_amount=q(3 * self.rate(aapl_usd, -2)), mute=False,
category="Investments")
self.tx(retirement, EX, Decimal("45.00"), self.day(-6, 30), "Annual management fee",
"Fees")
for m, spent in ((-9, 1500), (-3, 800)):
self.transfer(checking, gold_acc, Decimal(spent), self.day(m, 12), "Buy gold",
to_amount=q(Decimal(spent) / self.rate(gold_usd, m)), mute=False,
category="Investments")
self.tx(miles_acc, IN, Decimal("30000"), self.day(-HISTORY_MONTHS, 4),
"Credit card sign-up bonus", "Rewards", entities=["Sky Airlines"])
trip = -5
self.tx(miles_acc, EX, Decimal("25000"), self.day(trip, 2), "Flight to Lisbon",
"Travel", ["travel"], ["Sky Airlines"], notes="Paid entirely with miles!")
self.tx(euro, EX, Decimal("420.00"), self.day(trip, 14), "Hotel in Lisbon", "Travel",
["travel", "fun"])
self.tx(euro, EX, Decimal("96.40"), self.day(trip, 15), "Pastéis & dinners", "Dining",
["travel", "fun"])
# Someone owes you: unpaid income in the past shows up as late
self.tx(checking, IN, Decimal("85.00"), self.day(-1, 20),
"Alex owes me for concert tickets", "Gifts", entities=["Alex"], paid=False,
notes="Unpaid income is a projection until you mark it as paid.")
# Upcoming expense that isn't paid yet
self.tx(checking, EX, Decimal("612.00"), self.day(1, 18), "Car insurance (yearly)",
"Transport", ["essentials"])
# Cancelled service on a deactivated category
self.tx(checking, EX, Decimal("49.90"), self.day(-HISTORY_MONTHS, 8), "Cable TV",
"Cable TV", ["home"])
# Archived account with its history and the move out of it
self.tx(old_bank, IN, Decimal("1200.00"), date(2015, 3, 2), "Opening balance",
notes="Very old transactions are no problem.")
self.transfer(old_bank, checking, Decimal("1200.00"), self.day(-HISTORY_MONTHS, 3),
"Closing old bank account", category="Transfers")
# Balance reconciliation: the wallet had less cash than recorded
self.tx(cash, EX, Decimal("12.40"), self.day(-2, 28), "Balance reconciliation")
# Generated occurrences start unpaid; everything up to today happened already
Transaction.objects.filter(date__lte=today, is_paid=False).filter(
Q(installment_plan__isnull=False) | Q(recurring_transaction__isnull=False)
).update(is_paid=True)
# Quick transactions: one-click templates
QuickTransaction.objects.create(
name="Coffee", account=cash, type=EX, amount=Decimal("4.50"),
description="Coffee", category=self.cats["Dining"],
)
QuickTransaction.objects.create(
name="Bus ticket", account=cash, type=EX, amount=Decimal("2.75"),
description="Bus ticket", category=self.cats["Transport"],
)
lunch = QuickTransaction.objects.create(
name="Lunch near the office", account=card, type=EX, amount=Decimal("13.90"),
description="Lunch", category=self.cats["Dining"],
)
lunch.tags.set([self.tags["work"]])
# Rules run on transactions you create or edit
ride = TransactionRule.objects.create(
name="Ride-hailing is Transport",
description="Categorize Uber and Lyft rides automatically.",
trigger="'uber' in description.lower() or 'lyft' in description.lower()",
)
TransactionRuleAction.objects.create(rule=ride, field="category", value="'Transport'")
TransactionRuleAction.objects.create(rule=ride, field="entities", value="['Uber']")
big = TransactionRule.objects.create(
name="Flag big expenses",
description="Add a reminder to expenses of $500 or more.",
trigger="is_expense and amount >= 500",
)
TransactionRuleAction.objects.create(
rule=big, field="notes", value="'Big expense: keep the receipt!'"
)
# Saved filters for the transactions page
FilterPreset.objects.create(owner=user, name="Unpaid", parameters={"is_paid": ["0"]})
FilterPreset.objects.create(
owner=user, name="Credit card", parameters={"account": [str(card.id)]}
)
+1 -1
View File
@@ -88,7 +88,7 @@ def _dcr_initial_access_token_ok(request):
scheme, _, value = header.partition(" ")
if scheme.lower() != "bearer" or not value:
return False
return hmac.compare_digest(value, expected)
return hmac.compare_digest(value.encode(), expected.encode())
@require_http_methods(["GET"])
+11 -28
View File
@@ -5,7 +5,7 @@ from asgiref.sync import sync_to_async
from django.conf import settings
from django.core import management
from django.db import DEFAULT_DB_ALIAS
from django.core.cache import cache
from django.core.cache import caches
from procrastinate import builtin_tasks
from procrastinate.contrib.django import app
@@ -78,14 +78,10 @@ def reset_demo_data(timestamp=None):
)
logger.info("Database flushed successfully.")
# 2. Load data from the fixture
# TO-DO: Roll dates over based on today's date
fixture_name = "fixtures/demo_data.json"
logger.info(f"Loading data from fixture: {fixture_name}...")
management.call_command(
"loaddata", fixture_name, database=DEFAULT_DB_ALIAS, verbosity=1
)
logger.info(f"Data loaded successfully from {fixture_name}.")
# 2. Load demo data, with dates relative to today
logger.info("Loading demo data...")
management.call_command("load_demo_data", verbosity=1)
logger.info("Demo data loaded successfully.")
logger.info("Daily demo data reset completed.")
@@ -111,26 +107,13 @@ def check_for_updates(timestamp=None):
if latest_version:
try:
current_v = parse_version(settings.APP_VERSION)
latest_version = str(parse_version(latest_version))
except InvalidVersion:
current_v = parse_version("0.0.0")
try:
latest_v = parse_version(latest_version)
except InvalidVersion:
latest_v = parse_version("0.0.0")
update_info = {
"update_available": False,
"current_version": str(current_v),
"latest_version": str(latest_v),
}
if latest_v > current_v:
update_info["update_available"] = True
# Cache the entire dictionary
cache.set("update_check", update_info, 60 * 60 * 25)
logger.info(f"Update check complete. Result: {update_info}")
logger.warning(f"Invalid version tag from GitHub: {latest_version}")
return f"Invalid version tag: {latest_version}"
# Only the latest tag is stored; it's compared to APP_VERSION at render time so upgrades never leave it stale
caches["update_check"].set("latest_version", latest_version, None)
logger.info(f"Update check complete. Latest version: {latest_version}")
else:
logger.warning("Could not find 'tag_name' in GitHub API response.")
+17 -8
View File
@@ -1,6 +1,7 @@
# core/templatetags/update_tags.py
from django import template
from django.core.cache import cache
from django.conf import settings
from django.core.cache import caches
from packaging.version import InvalidVersion, parse as parse_version
register = template.Library()
@@ -8,10 +9,18 @@ register = template.Library()
@register.simple_tag
def get_update_check():
"""
Retrieves the update status dictionary from the cache.
Returns a default dictionary if nothing is found.
Compares the latest GitHub release (cached by the check_for_updates task)
against the running APP_VERSION.
"""
return cache.get("update_check") or {
"update_available": False,
"latest_version": "N/A",
}
latest_version = caches["update_check"].get("latest_version")
# The cached tag never expires, so opting out must also hide what was found before
if not settings.CHECK_FOR_UPDATES or not latest_version:
return {"update_available": False, "latest_version": "N/A"}
try:
update_available = parse_version(latest_version) > parse_version(settings.APP_VERSION)
except InvalidVersion:
# Builds without a real version (dev, "unknown") can't be compared, so don't claim an update
update_available = False
return {"update_available": update_available, "latest_version": latest_version}
+116
View File
@@ -0,0 +1,116 @@
from datetime import date
from unittest import mock
from django.contrib.auth import get_user_model
from django.core.management import call_command
from django.test import TestCase, override_settings
from django.urls import reverse
from apps.accounts.models import Account
from apps.common.middleware.thread_local import delete_current_user, write_current_user
from apps.dca.models import DCAStrategy
from apps.investments.services import get_holding_performance
from apps.transactions.models import Transaction
@override_settings(
STORAGES={
"default": {"BACKEND": "django.core.files.storage.FileSystemStorage"},
"staticfiles": {
"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"
},
},
WHITENOISE_AUTOREFRESH=True,
)
class LoadDemoDataTests(TestCase):
def load(self, today):
with mock.patch("django.utils.timezone.localdate", return_value=today):
call_command("load_demo_data", verbosity=0)
user = get_user_model().objects.get(email="demo@demo.com")
write_current_user(user)
self.addCleanup(delete_current_user)
return user
def test_dates_follow_today(self):
"""Demo data must be visible on first load, whatever day it is."""
today = date(2031, 7, 1) # the 1st is the edge case for "late"
user = self.load(today)
this_month = Transaction.objects.filter(reference_date=today.replace(day=1))
self.assertTrue(this_month.filter(is_paid=True).exists())
self.assertTrue(this_month.filter(description="Salary").exists())
# Upcoming, unpaid items exist so projections have something to show
self.assertTrue(
Transaction.objects.filter(date__gt=today, is_paid=False).exists()
)
# Past unpaid items show up on the late transactions page
self.assertTrue(
Transaction.objects.filter(date__lt=today, is_paid=False).exists()
)
# Nothing in the past is left as an unpaid installment/recurring occurrence
self.assertFalse(
Transaction.objects.filter(
date__lte=today, is_paid=False, installment_plan__isnull=False
).exists()
)
self.assertTrue(user.check_password("wygiwyhdemo"))
def test_investments_cover_each_kind(self):
self.load(date.today()) # the tracker values holdings as of the real today
perfs = {
a.name: get_holding_performance(a)
for a in Account.objects.filter(is_investment=True)
}
self.assertTrue(any(p.priced for p in perfs.values()))
self.assertTrue(any(not p.priced for p in perfs.values()))
for name, perf in perfs.items():
with self.subTest(name=name):
self.assertGreater(perf.quantity, 0)
self.assertIsNotNone(perf.market_value)
apple = perfs["Brokerage (Apple shares)"]
self.assertTrue(apple.estimated) # gifted shares
self.assertGreater(apple.income, 0) # dividends paid into checking
self.assertNotEqual(apple.realized, 0) # partial sale
self.assertTrue(perfs["Retirement Fund"].results) # gains and losses
def test_running_twice_is_a_no_op(self):
self.load(date(2031, 7, 14))
count = Transaction.all_objects.count()
call_command("load_demo_data", verbosity=0)
self.assertEqual(Transaction.all_objects.count(), count)
def test_pages_render_with_demo_data(self):
today = date.today()
user = self.load(today)
self.client.force_login(user)
htmx = {"HTTP_HX_REQUEST": "true"}
month = {"month": today.month, "year": today.year}
urls = [
reverse("monthly_overview", kwargs=month),
reverse("monthly_summary", kwargs=month),
reverse("monthly_account_summary", kwargs=month),
reverse("monthly_currency_summary", kwargs=month),
reverse("monthly_transactions_list", kwargs=month),
reverse("net_worth_current"),
reverse("net_worth_projected"),
reverse("yearly_index_account"),
reverse("transactions_all_list"),
reverse("insights_late_transactions"),
reverse("insights_emergency_fund"),
reverse("quick_transactions_index"),
reverse("rules_index"),
reverse("investments_list"),
reverse("investments_charts"),
reverse("investments_rebalance"),
reverse("investment_classes_list"),
] + [
reverse("dca_strategy_detail", kwargs={"strategy_id": s.id})
for s in DCAStrategy.objects.all()
] + [
reverse("investment_detail", kwargs={"account_id": a.id})
for a in Account.objects.filter(is_investment=True)
]
for url in urls:
with self.subTest(url=url):
response = self.client.get(url, **htmx)
self.assertLess(response.status_code, 400)
+12
View File
@@ -224,6 +224,18 @@ class DynamicClientRegistrationInitialAccessTokenTests(TestCase):
self.assertEqual(response.json()["error"], "invalid_token")
self.assertEqual(Application.objects.count(), 0)
def test_rejects_registration_with_non_ascii_initial_access_token(self):
response = self.client.post(
reverse("oauth-dynamic-client-registration"),
data=json.dumps({"redirect_uris": ["http://127.0.0.1:8765/callback"]}),
content_type="application/json",
HTTP_AUTHORIZATION="Bearer s3cret-iat\u00e9",
)
self.assertEqual(response.status_code, 401)
self.assertEqual(response.json()["error"], "invalid_token")
self.assertEqual(Application.objects.count(), 0)
def test_allows_registration_with_initial_access_token(self):
response = self.client.post(
reverse("oauth-dynamic-client-registration"),
@@ -0,0 +1,67 @@
from unittest import mock
from django.core.cache import caches
from django.test import SimpleTestCase, override_settings
from apps.common.tasks import check_for_updates
from apps.common.templatetags.cache_access import get_update_check
# Distinct LOCATIONs: LocMemCache aliases with the same LOCATION share one store and would hide wrong-cache bugs
CACHES = {
"default": {"BACKEND": "django.core.cache.backends.locmem.LocMemCache", "LOCATION": "default"},
"update_check": {"BACKEND": "django.core.cache.backends.locmem.LocMemCache", "LOCATION": "update_check"},
}
@override_settings(CACHES=CACHES)
class GetUpdateCheckTests(SimpleTestCase):
def setUp(self):
caches["update_check"].clear()
def render_with(self, app_version, latest):
if latest:
caches["update_check"].set("latest_version", latest)
with self.settings(APP_VERSION=app_version):
return get_update_check()
def test_newer_release_is_available(self):
self.assertEqual(
self.render_with("1.0.0", "1.1.0"),
{"update_available": True, "latest_version": "1.1.0"},
)
def test_after_upgrade_no_update(self):
self.assertFalse(self.render_with("1.1.0", "1.1.0")["update_available"])
self.assertFalse(self.render_with("1.2.0", "1.1.0")["update_available"])
def test_nothing_cached(self):
self.assertEqual(
self.render_with("1.0.0", None),
{"update_available": False, "latest_version": "N/A"},
)
def test_unknown_app_version_hides_update(self):
self.assertFalse(self.render_with("unknown", "1.1.0")["update_available"])
@override_settings(CHECK_FOR_UPDATES=False)
def test_disabled_hides_cached_update(self):
self.assertFalse(self.render_with("1.0.0", "1.1.0")["update_available"])
@override_settings(CACHES=CACHES, CHECK_FOR_UPDATES=True)
class CheckForUpdatesTaskTests(SimpleTestCase):
def setUp(self):
caches["update_check"].clear()
def run_task(self, tag):
with mock.patch("apps.common.tasks.requests.get") as get:
get.return_value.json.return_value = {"tag_name": tag}
check_for_updates.func()
return caches["update_check"].get("latest_version")
def test_stores_normalized_version(self):
self.assertEqual(self.run_task("v1.8.0"), "1.8.0")
def test_invalid_tag_keeps_previous_value(self):
caches["update_check"].set("latest_version", "1.7.0")
self.assertEqual(self.run_task("not-a-version"), "1.7.0")
+19
View File
@@ -1,3 +1,6 @@
import json
from urllib.parse import urlencode
from django.forms import SelectMultiple, widgets
from django.urls import reverse
from django.utils.translation import gettext_lazy as _
@@ -139,9 +142,13 @@ class TransactionSelect(TomSelect):
self.load_income = income
self.load_expense = expense
self.create = False
# Extra search filters sent as query params, e.g. {"currency": 1};
# keys must be in SEARCH_FILTERS of the transactions_search view
self.filters = {}
def build_attrs(self, base_attrs, extra_attrs=None):
attrs = super().build_attrs(base_attrs, extra_attrs)
attrs["data-render"] = "transaction"
if self.load_income and self.load_expense:
attrs["data-load"] = reverse("transactions_search")
@@ -153,5 +160,17 @@ class TransactionSelect(TomSelect):
attrs["data-load"] = reverse(
"transactions_search", kwargs={"filter_type": "expenses"}
)
if self.filters:
attrs["data-load"] += "?" + urlencode(self.filters)
return attrs
def create_option(self, name, value, *args, **kwargs):
option = super().create_option(name, value, *args, **kwargs)
instance = getattr(value, "instance", None)
if instance is not None:
from apps.transactions.utils.select_search import transaction_option
# tom-select reads this to render the selected transaction like the search results
option["attrs"]["data-data"] = json.dumps(transaction_option(instance))
return option
+9 -1
View File
@@ -8,9 +8,12 @@ from apps.currencies.models import Currency
class ExchangeRateProvider(ABC):
rates_inverted = False
# Help shown under the service's Options field. None hides the field
options_help = None
def __init__(self, api_key: Optional[str] = None):
def __init__(self, api_key: Optional[str] = None, options: str = ""):
self.api_key = api_key
self.options = options or ""
@abstractmethod
def get_rates(
@@ -24,6 +27,11 @@ class ExchangeRateProvider(ABC):
"""Return True if the service requires an API key"""
return True
@classmethod
def clean_options(cls, options: str) -> str:
"""Validate the raw Options text on save. Raise ValidationError if invalid"""
return options
@staticmethod
def invert_rate(rate: Decimal) -> Decimal:
"""Invert the given rate."""
+32 -62
View File
@@ -19,6 +19,7 @@ PROVIDER_MAPPING = {
"twelvedatamarkets": providers.TwelveDataMarketsProvider,
"yfinance": providers.YFinanceMarketsProvider,
"fxmacrodata": providers.FXMacroDataProvider,
"allratestoday": providers.AllRatesTodayProvider,
}
@@ -165,6 +166,31 @@ class ExchangeRateFetcher:
return target_currencies_qs, exchange_currencies
@staticmethod
def _store_rate(service, from_currency, to_currency, rate):
now = timezone.now()
rates = ExchangeRate.objects.filter(
automatic=True, from_currency=from_currency, to_currency=to_currency
)
today = rates.filter(date__date=timezone.localdate(now)).order_by("-date")
existing = today.first() if service.one_per_day else None
if existing:
existing.rate, existing.date = rate, now
existing.save()
else:
ExchangeRate.objects.create(
automatic=True,
from_currency=from_currency,
to_currency=to_currency,
rate=rate,
date=now,
)
if service.max_rates:
stale = rates.order_by("-date").values_list("id", flat=True)[
service.max_rates :
]
ExchangeRate.objects.filter(id__in=list(stale)).delete()
@staticmethod
def _fetch_service_rates(service: ExchangeRateService) -> None:
"""Fetch rates for a specific service"""
@@ -192,71 +218,15 @@ class ExchangeRateFetcher:
processed_pairs = set()
for from_currency, to_currency, rate in rates:
# Create a unique identifier for this currency pair
if provider.rates_inverted:
from_currency, to_currency = to_currency, from_currency
pair_key = (from_currency.id, to_currency.id)
if pair_key in processed_pairs:
continue
if provider.rates_inverted:
# If rates are inverted, we need to swap currencies
if service.singleton:
# Try to get the last automatically created exchange rate
exchange_rate = (
ExchangeRate.objects.filter(
automatic=True,
from_currency=to_currency,
to_currency=from_currency,
)
.order_by("-date")
.first()
)
else:
exchange_rate = None
if not exchange_rate:
ExchangeRate.objects.create(
automatic=True,
from_currency=to_currency,
to_currency=from_currency,
rate=rate,
date=timezone.now(),
)
else:
exchange_rate.rate = rate
exchange_rate.date = timezone.now()
exchange_rate.save()
processed_pairs.add((to_currency.id, from_currency.id))
else:
# If rates are not inverted, we can use them as is
if service.singleton:
# Try to get the last automatically created exchange rate
exchange_rate = (
ExchangeRate.objects.filter(
automatic=True,
from_currency=from_currency,
to_currency=to_currency,
)
.order_by("-date")
.first()
)
else:
exchange_rate = None
if not exchange_rate:
ExchangeRate.objects.create(
automatic=True,
from_currency=from_currency,
to_currency=to_currency,
rate=rate,
date=timezone.now(),
)
else:
exchange_rate.rate = rate
exchange_rate.date = timezone.now()
exchange_rate.save()
processed_pairs.add((from_currency.id, to_currency.id))
ExchangeRateFetcher._store_rate(
service, from_currency, to_currency, rate
)
processed_pairs.add(pair_key)
service.last_fetch = timezone.now()
service.failure_count = 0
@@ -1,11 +1,14 @@
import logging
import re
import time
import requests
from decimal import Decimal
from typing import Tuple, List, Optional, Dict
from django.core.exceptions import ValidationError
from django.db.models import QuerySet
from django.utils.translation import gettext_lazy as _
from apps.currencies.models import Currency, ExchangeRate
from apps.currencies.exchange_rates.base import ExchangeRateProvider
@@ -681,3 +684,166 @@ class FXMacroDataProvider(ExchangeRateProvider):
)
return results
class AllRatesTodayProvider(ExchangeRateProvider):
"""
Implementation for the central bank tables served by allratestoday.com.
Each service reads the latest published table of one central bank, picked by
the bank code in the Options field (e.g. "bcb" or "rbi"). An empty field
reads the ECB table. The endpoint needs no API key.
"""
BASE_URL = "https://allratestoday.com/api/open/central-bank"
DEFAULT_BANK = "ecb"
BANK_CODE_RE = re.compile(r"[a-z0-9_-]{2,20}")
rates_inverted = False # A base/quote row says how many quote one base buys
options_help = _(
"Central bank code, e.g. 'bcb' or 'rbi'. Leave empty to use the ECB table."
)
# A pair can have several rows. Prefer a published mid, else the mean of a buy
# and sell quote. Period averages are left out, they are not a current rate
MID_TYPES = (
"reference",
"middle",
"official",
"indicative",
"close",
"spot",
"informational",
)
BUY_SELL_TYPES = (("buy", "sell"), ("tt_buy", "tt_sell"))
def __init__(self, api_key: str = None, options: str = ""):
super().__init__(api_key, options)
self.session = requests.Session()
@property
def bank(self) -> str:
return self.options.strip().lower() or self.DEFAULT_BANK
@classmethod
def requires_api_key(cls) -> bool:
return False
@classmethod
def clean_options(cls, options: str) -> str:
bank = options.strip().lower()
if bank and not cls.BANK_CODE_RE.fullmatch(bank):
raise ValidationError(_("Invalid bank code, it should look like 'bcb'."))
return bank
def _parse_table(self, rows: list) -> Dict[Tuple[str, str], Decimal]:
"""Reduce the table to one rate per pair, in both directions"""
grouped = {}
for row in rows:
if not isinstance(row, dict):
continue
try:
value = Decimal(str(row.get("value")))
except ArithmeticError:
continue
# A bad cell skips its own row, not the whole table
if not value.is_finite() or value <= 0:
continue
pair = (row.get("base"), row.get("quote"))
grouped.setdefault(pair, {})[row.get("type")] = value
rates = {}
for (base, quote), values in grouped.items():
rate = next((values[t] for t in self.MID_TYPES if t in values), None)
if rate is None:
rate = next(
(
(values[buy] + values[sell]) / 2
for buy, sell in self.BUY_SELL_TYPES
if buy in values and sell in values
),
None,
)
if rate is None:
logger.debug(
f"AllRatesToday {self.bank}: no usable rate type for "
f"{base}/{quote}, got {sorted(map(str, values))}"
)
continue
rates[(base, quote)] = rate
rates.setdefault((quote, base), self.invert_rate(rate))
if grouped and not rates:
types = sorted({str(t) for values in grouped.values() for t in values})
logger.warning(
f"AllRatesToday table for {self.bank} has no usable rate types, "
f"got {types}"
)
return rates
@staticmethod
def _find_rate(
rates: Dict[Tuple[str, str], Decimal], base: str, quote: str
) -> Optional[Decimal]:
"""Look up a pair, crossing it through a shared currency if needed"""
if (base, quote) in rates:
return rates[(base, quote)]
for (from_code, via), rate in rates.items():
if from_code == base and (via, quote) in rates:
return rate * rates[(via, quote)]
return None
def get_rates(
self, target_currencies: QuerySet, exchange_currencies: set
) -> List[Tuple[Currency, Currency, Decimal]]:
results = []
if not self.BANK_CODE_RE.fullmatch(self.bank):
logger.error("Invalid AllRatesToday bank code, it should look like 'bcb'")
return results
try:
response = self.session.get(f"{self.BASE_URL}/{self.bank}", timeout=30)
response.raise_for_status()
data = response.json()
if data.get("stale"):
logger.warning(
f"AllRatesToday table for {self.bank} is stale, "
f"last published on {data.get('rate_date')}"
)
rates = self._parse_table(data["rates"])
except requests.RequestException as e:
logger.error(f"Error fetching AllRatesToday table for {self.bank}: {e}")
return results
except (KeyError, TypeError, ValueError, AttributeError, ArithmeticError) as e:
logger.error(
f"Unexpected response structure from AllRatesToday "
f"for {self.bank}: {e}"
)
return results
for target_currency in target_currencies:
if target_currency.exchange_currency not in exchange_currencies:
continue
base_currency = target_currency.exchange_currency
if base_currency.code == target_currency.code:
results.append((base_currency, target_currency, Decimal("1")))
continue
rate = self._find_rate(rates, base_currency.code, target_currency.code)
if rate is None:
logger.error(
f"No rate for {base_currency.code}/{target_currency.code} "
f"in the AllRatesToday table for {self.bank}"
)
continue
results.append((base_currency, target_currency, rate))
return results
+48 -2
View File
@@ -1,13 +1,17 @@
import json
from apps.common.widgets.crispy.daisyui import Switch
from apps.common.widgets.crispy.submit import NoClassSubmit
from apps.common.widgets.datepicker import AirDateTimePickerInput
from apps.common.widgets.decimal import ArbitraryDecimalDisplayNumberInput
from apps.common.widgets.tom_select import TomSelect
from apps.currencies.exchange_rates.fetcher import PROVIDER_MAPPING
from apps.currencies.models import Currency, ExchangeRate, ExchangeRateService
from crispy_forms.bootstrap import FormActions
from crispy_forms.helper import FormHelper
from crispy_forms.layout import Column, Layout, Row
from django import forms
from django.core.exceptions import ValidationError
from django.forms import CharField
from django.utils.translation import gettext_lazy as _
@@ -25,6 +29,7 @@ class CurrencyForm(forms.ModelForm):
"suffix",
"code",
"exchange_currency",
"show_exchanged_amounts",
"is_archived",
]
widgets = {
@@ -45,6 +50,7 @@ class CurrencyForm(forms.ModelForm):
"prefix",
"suffix",
"exchange_currency",
Switch("show_exchanged_amounts"),
)
if self.instance and self.instance.pk:
@@ -103,16 +109,42 @@ class ExchangeRateServiceForm(forms.ModelForm):
"service_type",
"is_active",
"api_key",
"options",
"interval_type",
"fetch_interval",
"target_currencies",
"target_accounts",
"singleton",
"one_per_day",
"max_rates",
]
widgets = {"options": forms.Textarea(attrs={"rows": 2})}
# Shows API Key and Options only for providers that use them
SERVICE_TYPE_SCRIPT = """
init send change to me end
on change js(me)
const p = JSON.parse(me.dataset.providers)[me.value] || {};
document.getElementById('div_id_api_key').classList.toggle('hidden', !p.api_key);
document.getElementById('div_id_options').classList.toggle('hidden', !p.options);
const help = document.getElementById('id_options_helptext');
if (help) help.textContent = p.options || '';
end
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
providers = {
key: {
"api_key": provider.requires_api_key(),
"options": str(provider.options_help or ""),
}
for key, provider in PROVIDER_MAPPING.items()
}
self.fields["service_type"].widget.attrs.update(
{"data-providers": json.dumps(providers), "_": self.SERVICE_TYPE_SCRIPT}
)
self.helper = FormHelper()
self.helper.form_tag = False
self.helper.form_method = "post"
@@ -120,14 +152,16 @@ class ExchangeRateServiceForm(forms.ModelForm):
"name",
"service_type",
Switch("is_active"),
Switch("singleton"),
"api_key",
"options",
Row(
Column("interval_type"),
Column("fetch_interval"),
),
"target_currencies",
"target_accounts",
Switch("one_per_day"),
"max_rates",
)
if self.instance and self.instance.pk:
@@ -142,3 +176,15 @@ class ExchangeRateServiceForm(forms.ModelForm):
NoClassSubmit("submit", _("Add"), css_class="btn btn-primary"),
),
)
def clean(self):
cleaned_data = super().clean()
provider = PROVIDER_MAPPING.get(cleaned_data.get("service_type"))
if provider:
try:
cleaned_data["options"] = provider.clean_options(
cleaned_data.get("options") or ""
)
except ValidationError as e:
self.add_error("options", e)
return cleaned_data
@@ -0,0 +1,41 @@
# Generated by Django 5.2.17 on 2026-11-08 05:52
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("currencies", "0025_alter_exchangerateservice_service_type"),
]
operations = [
migrations.AddField(
model_name="exchangerateservice",
name="options",
field=models.TextField(
blank=True,
default="",
help_text="Provider specific settings",
verbose_name="Options",
),
),
migrations.AlterField(
model_name="exchangerateservice",
name="service_type",
field=models.CharField(
choices=[
("coingecko_free", "CoinGecko (Demo/Free)"),
("coingecko_pro", "CoinGecko (Pro)"),
("transitive", "Transitive (Calculated from Existing Rates)"),
("frankfurter", "Frankfurter"),
("twelvedata", "TwelveData"),
("twelvedatamarkets", "TwelveData Markets"),
("yfinance", "Yahoo Finance"),
("fxmacrodata", "FXMacroData"),
("allratestoday", "AllRatesToday Central Banks"),
],
max_length=255,
verbose_name="Service Type",
),
),
]
@@ -0,0 +1,43 @@
import django.core.validators
from django.db import migrations, models
def singleton_to_max_rates(apps, schema_editor):
Service = apps.get_model("currencies", "ExchangeRateService")
Service.objects.filter(singleton=True).update(max_rates=1)
def max_rates_to_singleton(apps, schema_editor):
Service = apps.get_model("currencies", "ExchangeRateService")
Service.objects.filter(max_rates=1).update(singleton=True)
class Migration(migrations.Migration):
dependencies = [
("currencies", "0026_exchangerateservice_options"),
]
operations = [
migrations.AddField(
model_name="exchangerateservice",
name="one_per_day",
field=models.BooleanField(
default=False,
help_text="Update the day's exchange rate on each fetch instead of adding a new one.",
verbose_name="One exchange rate per day",
),
),
migrations.AddField(
model_name="exchangerateservice",
name="max_rates",
field=models.PositiveIntegerField(
blank=True,
null=True,
validators=[django.core.validators.MinValueValidator(1)],
help_text="Keep at most this many automatic exchange rates per currency pair, deleting the oldest first. Leave empty to keep all.",
verbose_name="Exchange rates to keep",
),
),
migrations.RunPython(singleton_to_max_rates, max_rates_to_singleton),
migrations.RemoveField(model_name="exchangerateservice", name="singleton"),
]
@@ -0,0 +1,22 @@
# Generated by Django 5.2.17 on 2026-11-11 09:53
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("currencies", "0027_exchangerateservice_retention"),
]
operations = [
migrations.AddField(
model_name="currency",
name="show_exchanged_amounts",
field=models.BooleanField(
default=True,
help_text="Show amounts converted to the exchange currency next to transactions and totals in this currency. Accounts with their own exchange currency use their own setting, and consolidated totals are calculated either way.",
verbose_name="Show exchanged amounts",
),
),
]
+31 -4
View File
@@ -31,6 +31,15 @@ class Currency(models.Model):
blank=True,
help_text=_("Default currency for exchange calculations"),
)
show_exchanged_amounts = models.BooleanField(
default=True,
verbose_name=_("Show exchanged amounts"),
help_text=_(
"Show amounts converted to the exchange currency next to transactions and "
"totals in this currency. Accounts with their own exchange currency use "
"their own setting, and consolidated totals are calculated either way."
),
)
is_archived = models.BooleanField(
default=False,
@@ -107,6 +116,7 @@ class ExchangeRateService(models.Model):
TWELVEDATA_MARKETS = "twelvedatamarkets", "TwelveData Markets"
YFINANCE = "yfinance", "Yahoo Finance"
FXMACRODATA = "fxmacrodata", "FXMacroData"
ALLRATESTODAY = "allratestoday", "AllRatesToday Central Banks"
class IntervalType(models.TextChoices):
ON = "on", _("On")
@@ -125,6 +135,12 @@ class ExchangeRateService(models.Model):
verbose_name=_("API Key"),
help_text=_("API key for the service (if required)"),
)
options = models.TextField(
blank=True,
default="",
verbose_name=_("Options"),
help_text=_("Provider specific settings"),
)
interval_type = models.CharField(
max_length=255,
choices=IntervalType.choices,
@@ -160,11 +176,20 @@ class ExchangeRateService(models.Model):
blank=True,
)
singleton = models.BooleanField(
verbose_name=_("Single exchange rate"),
one_per_day = models.BooleanField(
verbose_name=_("One exchange rate per day"),
default=False,
help_text=_(
"Create one exchange rate and keep updating it. Avoids database clutter."
"Update the day's exchange rate on each fetch instead of adding a new one."
),
)
max_rates = models.PositiveIntegerField(
verbose_name=_("Exchange rates to keep"),
null=True,
blank=True,
validators=[MinValueValidator(1)],
help_text=_(
"Keep at most this many automatic exchange rates per currency pair, deleting the oldest first. Leave empty to keep all."
),
)
@@ -180,7 +205,9 @@ class ExchangeRateService(models.Model):
from apps.currencies.exchange_rates.fetcher import PROVIDER_MAPPING
provider_class = PROVIDER_MAPPING[self.service_type]
return provider_class(self.api_key)
provider = provider_class(self.api_key)
provider.options = self.options or ""
return provider
@staticmethod
def _parse_hour_ranges(interval_str: str) -> Set[int]:
@@ -0,0 +1,280 @@
import json
from decimal import Decimal
from unittest import TestCase
from unittest.mock import MagicMock, patch
import requests
from django.core.exceptions import ValidationError
from django.test import TestCase as DjangoTestCase
from apps.currencies.exchange_rates.fetcher import PROVIDER_MAPPING
from apps.currencies.exchange_rates.providers import AllRatesTodayProvider
from apps.currencies.forms import ExchangeRateServiceForm
from apps.currencies.models import ExchangeRateService
class _FakeCurrency:
def __init__(self, code, exchange_currency=None):
self.code = code
self.exchange_currency = exchange_currency
def _row(base, quote, value, type_="reference"):
return {"base": base, "quote": quote, "type": type_, "value": value}
def _response(payload):
response = MagicMock()
response.status_code = 200
response.json.return_value = payload
return response
class AllRatesTodayProviderTests(TestCase):
def setUp(self):
self.brl = _FakeCurrency("BRL")
self.usd = _FakeCurrency("USD")
self.eur = _FakeCurrency("EUR")
self.provider = AllRatesTodayProvider(options="bcb")
def _get_rates(self, rows, target, base):
target.exchange_currency = base
with patch.object(
self.provider.session, "get", return_value=_response({"rates": rows})
):
return self.provider.get_rates([target], {base})
def test_requests_the_configured_bank(self):
self.usd.exchange_currency = self.brl
payload = {"rates": [_row("USD", "BRL", 5.4)]}
with patch.object(
self.provider.session, "get", return_value=_response(payload)
) as mock_get:
self.provider.get_rates([self.usd], {self.brl})
mock_get.assert_called_once_with(
"https://allratestoday.com/api/open/central-bank/bcb", timeout=30
)
def test_fetches_one_table_for_all_pairs(self):
self.usd.exchange_currency = self.brl
self.eur.exchange_currency = self.brl
payload = {"rates": [_row("USD", "BRL", 5.4), _row("EUR", "BRL", 6.2)]}
with patch.object(
self.provider.session, "get", return_value=_response(payload)
) as mock_get:
rates = self.provider.get_rates([self.usd, self.eur], {self.brl})
mock_get.assert_called_once()
self.assertEqual(len(rates), 2)
def test_empty_field_reads_the_ecb_table(self):
for options in (None, "", " "):
self.assertEqual(AllRatesTodayProvider(options=options).bank, "ecb")
def test_normalizes_the_bank_code(self):
self.assertEqual(AllRatesTodayProvider(options=" BCB ").bank, "bcb")
def test_ignores_the_api_key(self):
self.assertEqual(AllRatesTodayProvider("bcb").bank, "ecb")
def test_clean_options_validates_the_bank_code(self):
self.assertEqual(AllRatesTodayProvider.clean_options(" BCB "), "bcb")
self.assertEqual(AllRatesTodayProvider.clean_options(""), "")
with self.assertRaises(ValidationError):
AllRatesTodayProvider.clean_options("../latest/USD")
def test_rejects_malformed_bank_code_without_a_request(self):
provider = AllRatesTodayProvider(options="../latest/USD")
self.usd.exchange_currency = self.brl
with patch.object(provider.session, "get") as mock_get:
rates = provider.get_rates([self.usd], {self.brl})
mock_get.assert_not_called()
self.assertEqual(rates, [])
def test_uses_a_row_in_the_requested_direction(self):
rates = self._get_rates([_row("EUR", "USD", 1.16)], self.usd, self.eur)
self.assertEqual(rates, [(self.eur, self.usd, Decimal("1.16"))])
def test_inverts_a_row_in_the_opposite_direction(self):
# bcb quotes foreign currencies in BRL, so BRL -> USD has to be inverted
rates = self._get_rates([_row("USD", "BRL", 5)], self.brl, self.usd)
self.assertEqual(rates, [(self.usd, self.brl, Decimal("5"))])
rates = self._get_rates([_row("USD", "BRL", 5)], self.usd, self.brl)
self.assertEqual(rates, [(self.brl, self.usd, Decimal("0.2"))])
def test_crosses_pairs_through_the_table_home_currency(self):
rows = [_row("USD", "BRL", 5), _row("EUR", "BRL", 6)]
rates = self._get_rates(rows, self.eur, self.usd)
self.assertEqual(len(rates), 1)
self.assertAlmostEqual(rates[0][2], Decimal("0.8333333"), places=6)
def test_same_currency_is_one(self):
rates = self._get_rates([_row("USD", "BRL", 5)], self.usd, self.usd)
self.assertEqual(rates, [(self.usd, self.usd, Decimal("1"))])
def test_prefers_the_published_mid_over_buy_and_sell(self):
rows = [
_row("USD", "BRL", 5.3, "buy"),
_row("USD", "BRL", 5.4, "middle"),
_row("USD", "BRL", 5.7, "sell"),
]
rates = self._get_rates(rows, self.brl, self.usd)
self.assertEqual(rates, [(self.usd, self.brl, Decimal("5.4"))])
def test_averages_buy_and_sell_without_a_mid(self):
rows = [_row("USD", "BRL", 4, "buy"), _row("USD", "BRL", 6, "sell")]
rates = self._get_rates(rows, self.brl, self.usd)
self.assertEqual(rates, [(self.usd, self.brl, Decimal("5"))])
def test_averages_tt_buy_and_sell_without_a_mid(self):
rows = [
_row("USD", "MUR", 44.0, "tt_buy"),
_row("USD", "MUR", 46.0, "tt_sell"),
]
result = self._get_rates(rows, self.usd, _FakeCurrency("MUR"))
self.assertEqual(result[0][2], Decimal("1") / Decimal("45.0"))
def test_ignores_rows_of_other_types(self):
rows = [_row("USD", "BRL", 5, "monthly_average")]
self.assertEqual(self._get_rates(rows, self.brl, self.usd), [])
def test_logs_a_table_without_usable_types(self):
self.usd.exchange_currency = self.brl
payload = {"rates": [_row("USD", "BRL", 5, "monthly_average")]}
with patch.object(
self.provider.session, "get", return_value=_response(payload)
):
with self.assertLogs(
"apps.currencies.exchange_rates.providers", level="WARNING"
) as logs:
self.provider.get_rates([self.usd], {self.brl})
self.assertIn("monthly_average", logs.output[0])
def test_skips_pair_missing_from_the_table(self):
rows = [_row("USD", "BRL", 5)]
self.assertEqual(self._get_rates(rows, self.eur, self.usd), [])
def test_ignores_zero_and_malformed_rows(self):
rows = [
"nope",
_row("USD", "BRL", 0),
_row("USD", "BRL", "5,4"),
_row("USD", "BRL", None),
_row("USD", "BRL", "Infinity"),
_row("USD", "BRL", "NaN"),
_row("EUR", "BRL", 6),
]
rates = self._get_rates(rows, self.brl, self.eur)
self.assertEqual(rates, [(self.eur, self.brl, Decimal("6"))])
def test_logs_a_stale_table_and_still_uses_it(self):
self.usd.exchange_currency = self.brl
payload = {
"stale": True,
"rate_date": "2026-10-01",
"rates": [_row("BRL", "USD", 0.2)],
}
with patch.object(
self.provider.session, "get", return_value=_response(payload)
):
with self.assertLogs(
"apps.currencies.exchange_rates.providers", level="WARNING"
) as logs:
rates = self.provider.get_rates([self.usd], {self.brl})
self.assertIn("2026-10-01", logs.output[0])
self.assertEqual(rates, [(self.brl, self.usd, Decimal("0.2"))])
def test_returns_nothing_on_request_error(self):
self.usd.exchange_currency = self.brl
with patch.object(
self.provider.session,
"get",
side_effect=requests.RequestException("404 Not Found"),
):
rates = self.provider.get_rates([self.usd], {self.brl})
self.assertEqual(rates, [])
def test_returns_nothing_on_malformed_body(self):
self.usd.exchange_currency = self.brl
for payload in ({}, [], {"rates": None}):
with patch.object(
self.provider.session, "get", return_value=_response(payload)
):
self.assertEqual(self.provider.get_rates([self.usd], {self.brl}), [])
def test_returns_nothing_on_non_json_body(self):
self.usd.exchange_currency = self.brl
response = _response({})
response.json.side_effect = ValueError("not json")
with patch.object(self.provider.session, "get", return_value=response):
self.assertEqual(self.provider.get_rates([self.usd], {self.brl}), [])
def test_is_registered_without_an_api_key(self):
self.assertFalse(AllRatesTodayProvider.requires_api_key())
self.assertIs(PROVIDER_MAPPING["allratestoday"], AllRatesTodayProvider)
self.assertEqual(ExchangeRateService.ServiceType.ALLRATESTODAY, "allratestoday")
def test_service_passes_its_options_to_the_provider(self):
service = ExchangeRateService(service_type="allratestoday", options="rbi")
self.assertEqual(service.get_provider().bank, "rbi")
class ExchangeRateServiceFormOptionsTests(DjangoTestCase):
def _form(self, **data):
return ExchangeRateServiceForm(
data={
"name": "Rates",
"service_type": "allratestoday",
"interval_type": "every",
"fetch_interval": "24",
**data,
}
)
def test_saves_cleaned_options(self):
form = self._form(options=" BCB ")
self.assertTrue(form.is_valid(), form.errors)
self.assertEqual(form.save().options, "bcb")
def test_rejects_invalid_options_on_save(self):
form = self._form(options="../latest/USD")
self.assertFalse(form.is_valid())
self.assertIn("options", form.errors)
def test_tells_the_page_which_fields_each_provider_uses(self):
attrs = self._form().fields["service_type"].widget.attrs
providers = json.loads(attrs["data-providers"])
self.assertEqual(providers["allratestoday"]["api_key"], False)
self.assertIn("bank code", providers["allratestoday"]["options"])
self.assertEqual(providers["twelvedata"], {"api_key": True, "options": ""})
@@ -1,10 +1,11 @@
import datetime
from decimal import Decimal
from unittest.mock import patch, MagicMock
from django.test import TestCase
from django.utils import timezone
from apps.currencies.models import Currency, ExchangeRateService
from apps.currencies.models import Currency, ExchangeRate, ExchangeRateService
from apps.currencies.exchange_rates.fetcher import ExchangeRateFetcher
@@ -107,3 +108,42 @@ class ExchangeRateServiceFailureTrackingTests(TestCase):
self.service.refresh_from_db()
self.assertIsNotNone(self.service.last_fetch)
self.assertEqual(self.service.failure_count, 0)
class ExchangeRateRetentionTests(TestCase):
def setUp(self):
self.usd = Currency.objects.create(code="USD", name="Dollar", decimal_places=2)
self.eur = Currency.objects.create(
code="EUR", name="Euro", decimal_places=2, exchange_currency=self.usd
)
self.service = ExchangeRateService.objects.create(
name="S", service_type=ExchangeRateService.ServiceType.FRANKFURTER
)
def fetch(self, rate, when):
with patch("django.utils.timezone.now", return_value=when):
ExchangeRateFetcher._store_rate(self.service, self.eur, self.usd, Decimal(rate))
def rates(self):
return list(
ExchangeRate.objects.order_by("date").values_list("rate", flat=True)
)
def test_keeps_every_fetch_by_default(self):
for hour in (1, 2, 3):
self.fetch(hour, timezone.now().replace(hour=hour))
self.assertEqual(len(self.rates()), 3)
def test_max_rates_deletes_the_oldest(self):
self.service.max_rates = 2
for hour in (1, 2, 3):
self.fetch(hour, timezone.now().replace(hour=hour))
self.assertEqual(self.rates(), [Decimal(2), Decimal(3)])
def test_one_per_day_updates_the_days_rate(self):
self.service.one_per_day = True
day = timezone.localtime().replace(hour=10)
self.fetch(1, day)
self.fetch(2, day.replace(hour=11))
self.fetch(3, day + datetime.timedelta(days=1))
self.assertEqual(self.rates(), [Decimal(2), Decimal(3)])
+13
View File
@@ -0,0 +1,13 @@
from django.db.models import Case, F, When
def shown_exchange_currency(path):
"""Query expression for the exchange currency id at ``path`` (an account or a
currency lookup, such as ``"account__currency__"``), or NULL when that object
hides its exchanged amounts."""
return Case(
When(
**{f"{path}show_exchanged_amounts": True},
then=F(f"{path}exchange_currency"),
)
)
+97 -51
View File
@@ -9,10 +9,15 @@ from apps.common.widgets.datepicker import AirDatePickerInput
from apps.common.widgets.decimal import ArbitraryDecimalDisplayNumberInput
from apps.common.widgets.tom_select import TomSelect, TransactionSelect
from apps.dca.models import DCAEntry, DCAStrategy
from apps.transactions.models import Transaction, TransactionCategory, TransactionTag
from apps.transactions.models import (
Transaction,
TransactionCategory,
TransactionEntity,
TransactionTag,
)
from crispy_forms.bootstrap import AccordionGroup, FormActions, Accordion
from crispy_forms.helper import FormHelper
from crispy_forms.layout import HTML, Column, Layout, Row
from crispy_forms.layout import HTML, Column, Div, Layout, Row
from django import forms
from django.utils.translation import gettext_lazy as _
@@ -104,6 +109,23 @@ class DCAEntryForm(forms.ModelForm):
queryset=TransactionTag.objects.filter(active=True),
)
from_entities = DynamicModelMultipleChoiceField(
model=TransactionEntity,
to_field_name="name",
create_field="name",
required=False,
label=_("Entities"),
queryset=TransactionEntity.objects.filter(active=True),
)
to_entities = DynamicModelMultipleChoiceField(
model=TransactionEntity,
to_field_name="name",
create_field="name",
required=False,
label=_("Entities"),
queryset=TransactionEntity.objects.filter(active=True),
)
expense_transaction = DynamicModelChoiceField(
model=Transaction,
to_field_name="id",
@@ -157,34 +179,23 @@ class DCAEntryForm(forms.ModelForm):
AccordionGroup(
_("Create transaction"),
Switch("create_transaction"),
Row(
Column(
Row(
Column(
"from_account",
),
),
Row(
Column("from_category"),
Column("from_tags"),
),
Div(
"from_account",
Row(
Column("from_category"),
Column("from_tags"),
),
css_class="p-1 mx-1 my-3 border rounded-3",
"from_entities",
css_class="bg-base-100 rounded-box p-4 border-base-content/60 border my-3",
),
Row(
Column(
Row(
Column(
"to_account",
css_class="form-group",
),
),
Row(
Column("to_category"),
Column("to_tags"),
),
Div(
"to_account",
Row(
Column("to_category"),
Column("to_tags"),
),
css_class="p-1 mx-1 my-3 border rounded-3",
"to_entities",
css_class="bg-base-100 rounded-box p-4 border-base-content/60 border",
),
active=False,
),
@@ -216,36 +227,30 @@ class DCAEntryForm(forms.ModelForm):
self.fields["amount_received"].widget = ArbitraryDecimalDisplayNumberInput()
self.fields["date"].widget = AirDatePickerInput(clear_button=False)
expense_transaction = None
income_transaction = None
if self.instance and self.instance.pk:
# Edit mode - get from instance
expense_transaction = self.instance.expense_transaction
income_transaction = self.instance.income_transaction
elif self.data.get("expense_transaction"):
# Form validation - get from submitted data
try:
expense_transaction = Transaction.objects.get(
id=self.data["expense_transaction"]
)
income_transaction = Transaction.objects.get(
id=self.data["income_transaction"]
)
except Transaction.DoesNotExist:
pass
# Amounts can come from the linked transactions instead
self.fields["amount_paid"].required = False
self.fields["amount_received"].required = False
# If we have a current transaction, ensure it's in the queryset
if income_transaction:
self.fields["income_transaction"].queryset = Transaction.objects.filter(
id=income_transaction.id
# The picker only searches the currency each side expects (enforced in clean)
for name, currency in (
("expense_transaction", self.strategy.payment_currency),
("income_transaction", self.strategy.target_currency),
):
field = self.fields[name]
field.widget.filters = {"currency": currency.id}
selected = (
self.data.get(name)
if self.is_bound
else getattr(self.instance, f"{name}_id", None)
)
if expense_transaction:
self.fields["expense_transaction"].queryset = Transaction.objects.filter(
id=expense_transaction.id
# Only the selected one: the picker loads the rest by search
field.queryset = Transaction.objects.filter(
id=selected if str(selected or "").isdigit() else None
)
self.fields["from_account"].queryset = Account.objects.filter(
is_archived=False,
currency=self.strategy.payment_currency,
)
self.fields["from_category"].queryset = TransactionCategory.objects.filter(
@@ -255,16 +260,43 @@ class DCAEntryForm(forms.ModelForm):
self.fields["to_account"].queryset = Account.objects.filter(
is_archived=False,
currency=self.strategy.target_currency,
)
self.fields["to_category"].queryset = TransactionCategory.objects.filter(
active=True
)
self.fields["to_tags"].queryset = TransactionTag.objects.filter(active=True)
self.fields["from_entities"].queryset = TransactionEntity.objects.filter(
active=True
)
self.fields["to_entities"].queryset = TransactionEntity.objects.filter(
active=True
)
def clean(self):
cleaned_data = super().clean()
creating = cleaned_data.get("create_transaction")
for amount_field, link_field, currency in (
("amount_paid", "expense_transaction", self.strategy.payment_currency),
("amount_received", "income_transaction", self.strategy.target_currency),
):
linked = cleaned_data.get(link_field)
if linked and linked.account.currency_id != currency.id:
self.add_error(
link_field,
_("Transaction must be in %(currency)s.")
% {"currency": currency.name},
)
linked = cleaned_data[link_field] = None
if cleaned_data.get(amount_field) is not None:
continue
if linked and not creating:
cleaned_data[amount_field] = linked.amount
elif amount_field not in self.errors:
self.add_error(amount_field, _("This field is required."))
if cleaned_data.get("create_transaction"):
from_account = cleaned_data.get("from_account")
to_account = cleaned_data.get("to_account")
@@ -289,6 +321,14 @@ class DCAEntryForm(forms.ModelForm):
raise forms.ValidationError(
_("From and To accounts must be different.")
)
else:
# Both sides of a purchase are linked as a transfer on save
expense = cleaned_data.get("expense_transaction")
income = cleaned_data.get("income_transaction")
if expense and income:
error = expense.transfer_link_error(income)
if error:
self.add_error("income_transaction", error)
return cleaned_data
@@ -320,6 +360,7 @@ class DCAEntryForm(forms.ModelForm):
notes=notes,
)
from_transaction.tags.set(self.cleaned_data.get("from_tags", []))
from_transaction.entities.set(self.cleaned_data.get("from_entities", []))
# Create "To" transaction
to_transaction = Transaction.objects.create(
@@ -333,6 +374,9 @@ class DCAEntryForm(forms.ModelForm):
notes=notes,
)
to_transaction.tags.set(self.cleaned_data.get("to_tags", []))
to_transaction.entities.set(self.cleaned_data.get("to_entities", []))
from_transaction.link_transfer(to_transaction)
instance.expense_transaction = from_transaction
instance.income_transaction = to_transaction
@@ -343,6 +387,8 @@ class DCAEntryForm(forms.ModelForm):
if instance.income_transaction:
instance.income_transaction.amount = instance.amount_received
instance.income_transaction.save()
if instance.expense_transaction and instance.income_transaction:
instance.expense_transaction.link_transfer(instance.income_transaction)
instance.strategy = self.strategy
instance.save()
@@ -0,0 +1,39 @@
from django.db import migrations
def link_entry_transactions(apps, schema_editor):
"""Joins each DCA entry's expense and income into a transfer, so investment
tracking sees the purchase. Mirrors Transaction.transfer_link_error()."""
DCAEntry = apps.get_model("dca", "DCAEntry")
Transaction = apps.get_model("transactions", "Transaction")
entries = DCAEntry.objects.filter(
expense_transaction__isnull=False, income_transaction__isnull=False
).select_related("expense_transaction", "income_transaction")
for entry in entries:
expense, income = entry.expense_transaction, entry.income_transaction
if (
expense.type != "EX"
or income.type != "IN"
or expense.account_id == income.account_id
or expense.linked_transaction_id
or income.linked_transaction_id
# a legacy one-way link already points at one of them
or Transaction.objects.filter(
linked_transaction_id__in=(expense.pk, income.pk)
).exists()
):
continue
Transaction.objects.filter(pk=expense.pk).update(linked_transaction=income)
Transaction.objects.filter(pk=income.pk).update(linked_transaction=expense)
class Migration(migrations.Migration):
dependencies = [
("dca", "0004_alter_dcastrategy_owner_and_more"),
("transactions", "0052_transfer_templates"),
]
operations = [
migrations.RunPython(link_entry_transactions, migrations.RunPython.noop),
]
+166
View File
@@ -0,0 +1,166 @@
from datetime import date
from decimal import Decimal
from django.contrib.auth import get_user_model
from django.core.exceptions import ValidationError
from django.test import TestCase
from apps.accounts.models import Account
from apps.common.middleware.thread_local import delete_current_user, write_current_user
from apps.currencies.models import Currency
from apps.dca.forms import DCAEntryForm
from apps.dca.models import DCAStrategy
from apps.transactions.models import Transaction, TransactionEntity
IN, EX = Transaction.Type.INCOME, Transaction.Type.EXPENSE
class DCATransferLinkingTests(TestCase):
def setUp(self):
# Form querysets are scoped to the current user
write_current_user(get_user_model().objects.create_user(email="d@test.com"))
self.addCleanup(delete_current_user)
brl = Currency.objects.create(code="BRL", name="Real", decimal_places=2)
btc = Currency.objects.create(code="BTC", name="Bitcoin", decimal_places=8)
self.cash = Account.objects.create(name="Cash", currency=brl)
self.wallet = Account.objects.create(name="Wallet", currency=btc)
self.strategy = DCAStrategy.objects.create(
name="BTC", target_currency=btc, payment_currency=brl
)
def tx(self, account, type_, amount):
return Transaction.objects.create(
account=account, type=type_, amount=Decimal(amount), date=date(2026, 1, 5)
)
def entry_data(self, **extra):
return {
"date": "2026-01-05",
"amount_paid": "500",
"amount_received": "0.001",
**extra,
}
def test_created_transactions_are_linked(self):
form = DCAEntryForm(
data=self.entry_data(
create_transaction="on",
from_account=self.cash.id,
to_account=self.wallet.id,
),
strategy=self.strategy,
)
self.assertTrue(form.is_valid(), form.errors)
entry = form.save()
self.assertEqual(
entry.expense_transaction.linked_transaction, entry.income_transaction
)
self.assertEqual(
entry.income_transaction.linked_transaction, entry.expense_transaction
)
def test_existing_transactions_are_linked(self):
expense, income = self.tx(self.cash, EX, "500"), self.tx(self.wallet, IN, "0.001")
form = DCAEntryForm(
data=self.entry_data(
expense_transaction=str(expense.id), income_transaction=str(income.id)
),
strategy=self.strategy,
)
self.assertTrue(form.is_valid(), form.errors)
form.save()
expense.refresh_from_db()
income.refresh_from_db()
self.assertEqual(expense.linked_transaction, income)
self.assertEqual(income.linked_transaction, expense)
def test_transaction_in_another_transfer_is_rejected(self):
expense, income = self.tx(self.cash, EX, "500"), self.tx(self.wallet, IN, "0.001")
other = self.tx(self.wallet, IN, "0.002")
expense.link_transfer(other)
form = DCAEntryForm(
data=self.entry_data(
expense_transaction=str(expense.id), income_transaction=str(income.id)
),
strategy=self.strategy,
)
self.assertFalse(form.is_valid())
self.assertIn("income_transaction", form.errors)
def test_link_transfer_rules(self):
expense = self.tx(self.cash, EX, "500")
with self.assertRaises(ValidationError):
expense.link_transfer(self.tx(self.cash, IN, "500")) # same account
with self.assertRaises(ValidationError):
expense.link_transfer(self.tx(self.wallet, EX, "1")) # two expenses
income = self.tx(self.wallet, IN, "0.001")
expense.link_transfer(income)
expense.link_transfer(income) # already linked to each other: no-op
self.assertEqual(Transaction.objects.get(pk=income.pk).linked_transaction, expense)
def test_amounts_come_from_linked_transactions(self):
expense, income = self.tx(self.cash, EX, "500"), self.tx(self.wallet, IN, "0.001")
form = DCAEntryForm(
data=self.entry_data(
amount_paid="",
amount_received="",
expense_transaction=str(expense.id),
income_transaction=str(income.id),
),
strategy=self.strategy,
)
self.assertTrue(form.is_valid(), form.errors)
entry = form.save()
self.assertEqual(entry.amount_paid, Decimal("500"))
self.assertEqual(entry.amount_received, Decimal("0.001"))
def test_amount_required_without_link(self):
form = DCAEntryForm(
data=self.entry_data(amount_received=""), strategy=self.strategy
)
self.assertFalse(form.is_valid())
self.assertIn("amount_received", form.errors)
def test_link_in_wrong_currency_is_rejected(self):
# Expense must be in the payment currency, income in the target currency
expense, income = self.tx(self.wallet, EX, "1"), self.tx(self.cash, IN, "500")
form = DCAEntryForm(
data=self.entry_data(
expense_transaction=str(expense.id), income_transaction=str(income.id)
),
strategy=self.strategy,
)
self.assertFalse(form.is_valid())
self.assertIn("expense_transaction", form.errors)
self.assertIn("income_transaction", form.errors)
def test_created_transactions_get_entities(self):
broker = TransactionEntity.objects.create(name="Broker")
form = DCAEntryForm(
data=self.entry_data(
create_transaction="on",
from_account=self.cash.id,
to_account=self.wallet.id,
from_entities=["Broker"],
to_entities=["Broker"],
),
strategy=self.strategy,
)
self.assertTrue(form.is_valid(), form.errors)
entry = form.save()
self.assertEqual(list(entry.expense_transaction.entities.all()), [broker])
self.assertEqual(list(entry.income_transaction.entities.all()), [broker])
def test_create_accounts_must_hold_strategy_currency(self):
# Swapped: paying from the BTC wallet into the BRL cash account
form = DCAEntryForm(
data=self.entry_data(
create_transaction="on",
from_account=self.wallet.id,
to_account=self.cash.id,
),
strategy=self.strategy,
)
self.assertFalse(form.is_valid())
self.assertIn("from_account", form.errors)
self.assertIn("to_account", form.errors)
+18
View File
@@ -73,6 +73,13 @@ class ExportForm(forms.Form):
label=_("Automatic Exchange Rates"),
initial=False,
)
bank_sync = forms.BooleanField(
required=False,
widget=forms.CheckboxInput(),
label=_("Bank Sync"),
help_text=_("Includes the providers' API keys"),
initial=False,
)
rules = forms.BooleanField(
required=False,
widget=forms.CheckboxInput(),
@@ -101,6 +108,7 @@ class ExportForm(forms.Form):
self.helper.layout = Layout(
"users",
"accounts",
"investment_classes",
"currencies",
"transactions",
"categories",
@@ -110,6 +118,7 @@ class ExportForm(forms.Form):
"recurring_transactions",
"exchange_rates_services",
"exchange_rates",
"bank_sync",
"rules",
"dca",
"import_profiles",
@@ -127,6 +136,7 @@ class RestoreForm(forms.Form):
)
users = forms.FileField(required=False, label=_("Users"))
accounts = forms.FileField(required=False, label=_("Accounts"))
investment_classes = forms.FileField(required=False, label=_("Asset classes"))
currencies = forms.FileField(required=False, label=_("Currencies"))
transactions_categories = forms.FileField(required=False, label=_("Categories"))
transactions_tags = forms.FileField(required=False, label=_("Tags"))
@@ -140,6 +150,12 @@ class RestoreForm(forms.Form):
required=False, label=_("Automatic Exchange Rates")
)
exchange_rates = forms.FileField(required=False, label=_("Exchange Rates"))
bank_sync_services = forms.FileField(
required=False, label=_("Bank Sync Services")
)
bank_sync_accounts = forms.FileField(
required=False, label=_("Bank Sync Accounts")
)
transaction_rules = forms.FileField(required=False, label=_("Transaction rules"))
transaction_rules_actions = forms.FileField(
required=False, label=_("Edit transaction action")
@@ -171,6 +187,8 @@ class RestoreForm(forms.Form):
"recurring_transactions",
"automatic_exchange_rates",
"exchange_rates",
"bank_sync_services",
"bank_sync_accounts",
"transaction_rules",
"transaction_rules_actions",
"transaction_rules_update_or_create",
+21 -1
View File
@@ -1,10 +1,24 @@
from import_export import fields, resources, widgets
from apps.accounts.models import Account, AccountGroup
from apps.accounts.models import Account, AccountGroup, InvestmentClass
from apps.export_app.widgets.foreign_key import AutoCreateForeignKeyWidget
from apps.currencies.models import Currency
class InvestmentClassResource(resources.ModelResource):
class Meta:
model = InvestmentClass
def get_queryset(self):
return InvestmentClass.all_objects.all()
class AllClassesWidget(widgets.ForeignKeyWidget):
# By id, across owners: a shared account may carry another user's class
def get_queryset(self, value, row, *args, **kwargs):
return InvestmentClass.all_objects.all()
class AccountResource(resources.ModelResource):
group = fields.Field(
attribute="group",
@@ -22,6 +36,12 @@ class AccountResource(resources.ModelResource):
widget=widgets.ForeignKeyWidget(Currency, "name"),
)
investment_class = fields.Field(
attribute="investment_class",
column_name="investment_class",
widget=AllClassesWidget(InvestmentClass),
)
class Meta:
model = Account
@@ -0,0 +1,39 @@
from import_export import fields, resources, widgets
from apps.accounts.models import Account
from apps.bank_sync.models import BankSyncAccount, BankSyncService
class _AnyOwnerWidget(widgets.ForeignKeyWidget):
"""By id, across every user.
The default managers only see the restoring user's rows, and names can clash
between users. Ids survive a restore.
"""
def get_queryset(self, value, row, *args, **kwargs):
return self.model._base_manager.all()
class BankSyncServiceResource(resources.ModelResource):
class Meta:
model = BankSyncService
def get_queryset(self):
return BankSyncService._base_manager.all()
class BankSyncAccountResource(resources.ModelResource):
service = fields.Field(
attribute="service",
column_name="service",
widget=_AnyOwnerWidget(BankSyncService),
)
account = fields.Field(
attribute="account",
column_name="account",
widget=_AnyOwnerWidget(Account),
)
class Meta:
model = BankSyncAccount
+51 -1
View File
@@ -1,3 +1,53 @@
import io
import zipfile
from decimal import Decimal
from django.contrib.auth import get_user_model
from django.test import TestCase
# Create your tests here.
from apps.accounts.models import Account, InvestmentClass
from apps.common.middleware.thread_local import delete_current_user, write_current_user
from apps.currencies.models import Currency
from apps.export_app.resources.accounts import AccountResource, InvestmentClassResource
from apps.export_app.resources.transactions import TransactionEntityResource
from apps.export_app.views import process_imports
from apps.transactions.models import TransactionEntity
class InvestmentRestoreTests(TestCase):
def setUp(self):
self.user = get_user_model().objects.create_user(email="u@test.com", password="pw")
write_current_user(self.user)
def tearDown(self):
delete_current_user()
def test_round_trip_keeps_asset_class_and_income_links(self):
brl = Currency.objects.create(code="BRL", name="Real", decimal_places=2)
stocks = InvestmentClass.objects.create(name="Stocks", target=Decimal("60"))
broker = TransactionEntity.objects.create(name="Broker")
account = Account.objects.create(
name="PETR4", currency=brl, is_investment=True, investment_class=stocks
)
account.income_entities.add(broker)
backup = io.BytesIO()
with zipfile.ZipFile(backup, "w") as z:
for resource, name in (
(AccountResource, "accounts"),
(InvestmentClassResource, "investment_classes"),
(TransactionEntityResource, "transactions_entities"),
):
z.writestr(f"{name}.csv", resource().export().csv)
Account.all_objects.all().delete()
InvestmentClass.all_objects.all().delete()
TransactionEntity.all_objects.all().delete()
backup.seek(0)
process_imports(None, {"zip_file": backup})
account = Account.all_objects.get(name="PETR4")
self.assertEqual(account.investment_class.name, "Stocks")
self.assertEqual(account.investment_class.target, Decimal("60"))
self.assertEqual([e.name for e in account.income_entities.all()], ["Broker"])
+19 -1
View File
@@ -14,7 +14,14 @@ from tablib import Dataset
from apps.common.decorators.htmx import only_htmx
from apps.export_app.forms import ExportForm, RestoreForm
from apps.export_app.resources.accounts import AccountResource
from apps.export_app.resources.accounts import (
AccountResource,
InvestmentClassResource,
)
from apps.export_app.resources.bank_sync import (
BankSyncAccountResource,
BankSyncServiceResource,
)
from apps.export_app.resources.currencies import (
CurrencyResource,
ExchangeRateResource,
@@ -91,6 +98,9 @@ def export_form(request):
exports.append((UserResource().export(), "users"))
if export_accounts:
exports.append((AccountResource().export(), "accounts"))
exports.append(
(InvestmentClassResource().export(), "investment_classes")
)
if export_currencies:
exports.append((CurrencyResource().export(), "currencies"))
if export_transactions:
@@ -119,6 +129,9 @@ def export_form(request):
)
if export_exchange_rates:
exports.append((ExchangeRateResource().export(), "exchange_rates"))
if form.cleaned_data.get("bank_sync"):
exports.append((BankSyncServiceResource().export(), "bank_sync_services"))
exports.append((BankSyncAccountResource().export(), "bank_sync_accounts"))
if export_rules:
exports.append(
(TransactionRuleResource().export(), "transaction_rules")
@@ -226,15 +239,20 @@ def process_imports(request, cleaned_data):
"currencies",
CurrencyResource,
), # We do a double pass because exchange_currency may not exist when currency is initially created
("investment_classes", InvestmentClassResource),
("accounts", AccountResource),
("transactions_categories", TransactionCategoyResource),
("transactions_tags", TransactionTagResource),
("transactions_entities", TransactionEntityResource),
# Again: income categories, tags and entities don't exist on the first pass
("accounts", AccountResource),
("automatic_exchange_rates", ExchangeRateServiceResource),
("exchange_rates", ExchangeRateResource),
("installment_plans", InstallmentPlanResource),
("recurring_transactions", RecurringTransactionResource),
("transactions", TransactionResource),
("bank_sync_services", BankSyncServiceResource),
("bank_sync_accounts", BankSyncAccountResource),
("dca_strategies", DCAStrategyResource),
("dca_entries", DCAEntryResource),
("import_profiles", ImportProfileResource),
+23 -3
View File
@@ -1,4 +1,6 @@
from apps.accounts.models import Account
from apps.common.widgets.crispy.submit import NoClassSubmit
from apps.common.widgets.tom_select import TomSelect
from apps.import_app.models import ImportProfile
from crispy_forms.bootstrap import FormActions
from crispy_forms.helper import FormHelper
@@ -42,15 +44,33 @@ class ImportProfileForm(forms.ModelForm):
class ImportRunFileUploadForm(forms.Form):
file = forms.FileField(label=_("Select a file"))
account = forms.ModelChoiceField(
queryset=Account.objects.filter(is_archived=False),
required=False,
label=_("Account"),
help_text=_(
"Import every transaction into this account. Leave empty to let the "
"profile pick the account."
),
widget=TomSelect(group_by="group"),
)
def __init__(self, *args, **kwargs):
def __init__(self, *args, profile, **kwargs):
super().__init__(*args, **kwargs)
self.helper = FormHelper()
self.helper.form_tag = False
self.helper.form_method = "post"
self.helper.layout = Layout(
"file",
self.helper.layout = Layout("file")
if profile.imports_transactions:
self.fields["account"].queryset = Account.objects.filter(is_archived=False)
self.helper.layout.append("account")
else:
# Only transactions belong to an account
del self.fields["account"]
self.helper.layout.append(
FormActions(
NoClassSubmit("submit", _("Import"), css_class="btn btn-primary"),
),
@@ -0,0 +1,64 @@
# Generated by Django 5.2.17 on 2026-10-24 16:17
import django.db.models.deletion
from django.conf import settings
from django.db import migrations, models
def backfill_import_run_owner(apps, schema_editor):
"""Assign existing runs to the user who owns the transactions they imported.
Runs whose transactions belong to several users, or that imported nothing,
are left without an owner and stay visible to everyone.
"""
ImportRun = apps.get_model("import_app", "ImportRun")
for run in ImportRun.objects.filter(owner__isnull=True):
owner_ids = set(
run.transactions.exclude(owner__isnull=True).values_list(
"owner_id", flat=True
)
)
if len(owner_ids) == 1:
run.owner_id = owner_ids.pop()
run.save(update_fields=["owner"])
class Migration(migrations.Migration):
dependencies = [
('import_app', '0002_alter_importprofile_name_and_more'),
('transactions', '0038_transaction_owner'),
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.AddField(
model_name='importprofile',
name='owner',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='%(class)s_owned', to=settings.AUTH_USER_MODEL, verbose_name='Owner'),
),
migrations.AddField(
model_name='importprofile',
name='shared_with',
field=models.ManyToManyField(blank=True, related_name='%(class)s_shared', to=settings.AUTH_USER_MODEL, verbose_name='Shared with users'),
),
migrations.AddField(
model_name='importprofile',
name='visibility',
field=models.CharField(choices=[('private', 'Private'), ('public', 'Public')], default='private', max_length=10, verbose_name='Visibility'),
),
migrations.AddField(
model_name='importrun',
name='owner',
field=models.ForeignKey(blank=True, null=True, on_delete=django.db.models.deletion.CASCADE, related_name='%(class)s_owned', to=settings.AUTH_USER_MODEL),
),
migrations.AlterField(
model_name='importprofile',
name='name',
field=models.CharField(max_length=100, verbose_name='Name'),
),
migrations.RunPython(
backfill_import_run_owner, migrations.RunPython.noop
),
]
@@ -0,0 +1,20 @@
# Generated by Django 5.2.17 on 2026-11-11 17:04
import django.db.models.deletion
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('accounts', '0022_account_show_exchanged_amounts'),
('import_app', '0003_owner_and_sharing'),
]
operations = [
migrations.AddField(
model_name='importrun',
name='account',
field=models.ForeignKey(blank=True, help_text='Import every transaction into this account. Leave empty to let the profile pick the account.', null=True, on_delete=django.db.models.deletion.SET_NULL, related_name='import_runs', to='accounts.account', verbose_name='Account'),
),
]
+36 -3
View File
@@ -4,14 +4,20 @@ from django.core.exceptions import ValidationError
from django.db import models
from django.utils.translation import gettext_lazy as _
from apps.common.models import (
OwnedObject,
OwnedObjectManager,
SharedObject,
SharedObjectManager,
)
from apps.import_app.schemas import version_1
class ImportProfile(models.Model):
class ImportProfile(SharedObject):
class Versions(models.IntegerChoices):
VERSION_1 = 1, "Version 1"
name = models.CharField(max_length=100, verbose_name=_("Name"), unique=True)
name = models.CharField(max_length=100, verbose_name=_("Name"))
yaml_config = models.TextField(verbose_name=_("YAML Configuration"))
version = models.IntegerField(
choices=Versions,
@@ -19,6 +25,9 @@ class ImportProfile(models.Model):
verbose_name=_("Version"),
)
objects = SharedObjectManager()
all_objects = models.Manager() # Unfiltered manager
def __str__(self):
return self.name
@@ -29,6 +38,15 @@ class ImportProfile(models.Model):
version_number = self.Versions(self.version).name.split("_")[1]
return _("Version {number}").format(number=version_number)
@property
def imports_transactions(self):
"""Whether this profile creates transactions, as opposed to accounts, tags, etc."""
try:
config = version_1.ImportProfileSchema(**yaml.safe_load(self.yaml_config))
except Exception:
return False
return config.settings.importing == "transactions"
def clean(self):
if self.version and self.version == self.Versions.VERSION_1:
try:
@@ -40,7 +58,7 @@ class ImportProfile(models.Model):
)
class ImportRun(models.Model):
class ImportRun(OwnedObject):
class Status(models.TextChoices):
QUEUED = "QUEUED", _("Queued")
PROCESSING = "PROCESSING", _("Processing")
@@ -61,6 +79,18 @@ class ImportRun(models.Model):
max_length=10000,
help_text=_("File name"),
)
account = models.ForeignKey(
"accounts.Account",
on_delete=models.SET_NULL,
null=True,
blank=True,
related_name="import_runs",
verbose_name=_("Account"),
help_text=_(
"Import every transaction into this account. Leave empty to let the "
"profile pick the account."
),
)
transactions = models.ManyToManyField(
"transactions.Transaction", related_name="import_runs"
)
@@ -85,3 +115,6 @@ class ImportRun(models.Model):
failed_rows = models.IntegerField(default=0)
started_at = models.DateTimeField(null=True)
finished_at = models.DateTimeField(null=True)
objects = OwnedObjectManager()
all_objects = models.Manager() # Unfiltered manager
+15
View File
@@ -75,6 +75,20 @@ class SubtractTransformationRule(BaseModel):
)
class LineTransformationRule(BaseModel):
type: Literal["line"]
line: int = Field(
...,
ge=1,
description="Line of the file to read, counted from 1 at the top of the file, including skipped lines",
)
column: int | None = Field(
default=None,
ge=1,
description="Column of that line to read, counted from 1. Empty to read the whole line (Excel cells are joined with commas).",
)
class CSVImportSettings(BaseModel):
skip_errors: bool = Field(
default=False,
@@ -133,6 +147,7 @@ class ColumnMapping(BaseModel):
| SplitTransformationRule
| AddTransformationRule
| SubtractTransformationRule
| LineTransformationRule
]
] = Field(default_factory=list)
+3 -3
View File
@@ -1,17 +1,17 @@
import json
from pathlib import Path
from django.conf import settings
from apps.import_app.models import ImportProfile
class PresetService:
PRESET_PATH = "/usr/src/app/import_presets"
@classmethod
def get_all_presets(cls):
presets = []
for folder in Path(cls.PRESET_PATH).iterdir():
for folder in settings.IMPORT_PRESETS_ROOT.iterdir():
if folder.is_dir():
manifest_path = folder / "manifest.json"
config_path = folder / "config.yml"
+76 -11
View File
@@ -7,12 +7,14 @@ import zipfile
from django.db import transaction
from datetime import datetime, date
from decimal import Decimal, InvalidOperation
from pathlib import Path
from typing import Dict, Any, Literal, Union
import openpyxl
import xlrd
import yaml
from cachalot.api import cachalot_disabled
from django.conf import settings
from django.core.exceptions import FieldDoesNotExist
from django.utils import timezone
from openpyxl.utils.exceptions import InvalidFileException
@@ -39,8 +41,6 @@ logger = logging.getLogger(__name__)
class ImportService:
TEMP_DIR = "/usr/src/app/temp"
def __init__(self, import_run: ImportRun):
self.import_run: ImportRun = import_run
self.profile: ImportProfile = import_run.profile
@@ -53,8 +53,17 @@ class ImportService:
)
self.mapping: Dict[str, version_1.ColumnMapping] = self.config.mapping
# Lines read by `line` transformations, captured as (text, cells) while the file is read
self._wanted_lines = {
transform.line
for mapping in self.mapping.values()
for transform in mapping.transformations or []
if transform.type == "line"
}
self._file_lines: Dict[int, tuple[str, list[str]]] = {}
# Ensure temp directory exists
os.makedirs(self.TEMP_DIR, exist_ok=True)
os.makedirs(settings.IMPORT_TEMP_ROOT, exist_ok=True)
def _load_config(self) -> version_1.ImportProfileSchema:
yaml_data = yaml.safe_load(self.profile.yaml_config)
@@ -189,6 +198,15 @@ class ImportService:
transform.pattern, transform.replacement, transformed
)
elif transform.type == "line":
text, cells = self._file_lines.get(transform.line, ("", []))
if transform.column is None:
transformed = text
elif transform.column <= len(cells):
transformed = cells[transform.column - 1]
else:
transformed = ""
elif transform.type == "date_format":
transformed = datetime.strptime(
transformed, transform.original_format
@@ -289,7 +307,8 @@ class ImportService:
# Ignore if category doesn't exist and create is False or not set
data["category"] = None
if "account" in data:
# Look the account up, unless the run already picked one
if "account" in data and not isinstance(data["account"], Account):
account_id = data.pop("account")
account_mapping = next(
(
@@ -710,6 +729,10 @@ class ImportService:
if mapped_data:
# Handle different import types
if self.settings.importing == "transactions":
if self.import_run.account:
# The run's account wins over the profile's account mapping
mapped_data["account"] = self.import_run.account
if self.deduplication and self._check_duplicate_transaction(
mapped_data
):
@@ -744,14 +767,31 @@ class ImportService:
logger.error(f"Fatal error processing row {row_number}", exc_info=e)
def _capture_csv_lines(self, csv_file):
"""Pass the file's lines through, keeping the ones `line` transformations read."""
for number, line in enumerate(csv_file, start=1):
if number in self._wanted_lines:
text = line.rstrip("\r\n")
cells = next(csv.reader([text], delimiter=self.settings.delimiter), [])
self._file_lines[number] = (text, cells)
yield line
def _capture_excel_lines(self, read_row):
"""Keep the sheet rows `line` transformations read. read_row(n) returns row n's values."""
self._file_lines = {}
for number in self._wanted_lines:
cells = ["" if value is None else str(value) for value in read_row(number)]
self._file_lines[number] = (",".join(cells), cells)
def _process_csv(self, file_path):
# First pass: count rows
# First pass: count rows, and capture the lines `line` transformations read
with open(file_path, "r", encoding=self.settings.encoding) as csv_file:
lines = self._capture_csv_lines(csv_file)
# Skip specified number of rows
for _ in range(self.settings.skip_lines):
next(csv_file)
next(lines)
reader = csv.DictReader(csv_file, delimiter=self.settings.delimiter)
reader = csv.DictReader(lines, delimiter=self.settings.delimiter)
self._update_totals("total", value=sum(1 for _ in reader))
with open(file_path, "r", encoding=self.settings.encoding) as csv_file:
@@ -803,6 +843,14 @@ class ImportService:
sheet = workbook[sheet_name]
self._log("info", f"Processing sheet: {sheet_name}")
self._capture_excel_lines(
lambda number: next(
sheet.iter_rows(
min_row=number, max_row=number, values_only=True
),
(),
)
)
headers = [
str(cell.value or "") for cell in sheet[self.settings.start_row]
]
@@ -863,6 +911,13 @@ class ImportService:
continue
sheet = workbook.sheet_by_name(sheet_name)
self._log("info", f"Processing sheet: {sheet_name}")
self._capture_excel_lines(
lambda number: (
sheet.row_values(number - 1)
if number <= sheet.nrows
else []
)
)
headers = [
str(sheet.cell_value(self.settings.start_row - 1, col) or "")
for col in range(sheet.ncols)
@@ -920,7 +975,9 @@ class ImportService:
current_transaction = {}
raw_lines_buffer = []
account = Account.objects.filter(name=account_name).first()
account = (
self.import_run.account or Account.objects.filter(name=account_name).first()
)
if not account:
raise ValueError(f"Account '{account_name}' not found.")
@@ -1131,9 +1188,12 @@ class ImportService:
Validates that the file path is within the allowed temporary directory.
Returns the absolute path.
"""
abs_path = os.path.abspath(file_path)
if not abs_path.startswith(self.TEMP_DIR):
raise ValueError(f"Invalid file path. File must be in {self.TEMP_DIR}")
# resolve() like IMPORT_TEMP_ROOT, so symlinks can't point outside it
abs_path = str(Path(file_path).resolve())
if not Path(abs_path).is_relative_to(settings.IMPORT_TEMP_ROOT):
raise ValueError(
f"Invalid file path. File must be in {settings.IMPORT_TEMP_ROOT}"
)
return abs_path
def process_file(self, file_path: str):
@@ -1146,6 +1206,11 @@ class ImportService:
self.import_run.save(update_fields=["started_at"])
self._log("info", "Starting import process")
if self.import_run.account:
self._log(
"info",
f"Importing transactions into account '{self.import_run.account.name}'",
)
try:
if isinstance(self.settings, version_1.CSVImportSettings):
@@ -0,0 +1,167 @@
"""Tests for the `line` transformation, which reads a line outside the table, like a header cell."""
import shutil
import tempfile
from pathlib import Path
import openpyxl
from django.contrib.auth import get_user_model
from django.test import TestCase, override_settings
from apps.accounts.models import Account
from apps.common.middleware.thread_local import delete_current_user, write_current_user
from apps.currencies.models import Currency
from apps.import_app.models import ImportProfile, ImportRun
from apps.import_app.services.v1 import ImportService
from apps.transactions.models import Transaction
MAPPING = """
mapping:
date:
target: date
source: date
format: "%Y-%m-%d"
amount:
target: amount
source: amount
type:
target: type
source: amount
detection_method: sign
description:
target: description
source: description
account:
target: account
transformations:
{account_transformations}
"""
# The account name sits in the header, above the table
CSV_CONTENT = """Bank statement
Account:;Savings
Account number:;Savings 1234
date;amount;description
2026-01-05;-50.00;Groceries
2026-01-06;1000.00;Salary
"""
class LineTransformationTests(TestCase):
def setUp(self):
self.temp_root = Path(tempfile.mkdtemp()).resolve()
self.addCleanup(shutil.rmtree, self.temp_root, ignore_errors=True)
temp_root_override = override_settings(IMPORT_TEMP_ROOT=self.temp_root)
temp_root_override.enable()
self.addCleanup(temp_root_override.disable)
self.user = get_user_model().objects.create_user(
email="test@example.com", password="password"
)
write_current_user(self.user)
self.addCleanup(delete_current_user)
currency = Currency.objects.create(
code="EUR", name="Euro", decimal_places=2, prefix="€ "
)
Account.objects.create(name="Checking", currency=currency, owner=self.user)
self.savings = Account.objects.create(
name="Savings", currency=currency, owner=self.user
)
def _import(self, settings, account_transformations, file_name, write):
yaml_config = settings + MAPPING.format(
account_transformations=account_transformations
)
profile = ImportProfile.objects.create(
name="Profile", yaml_config=yaml_config, owner=self.user
)
run = ImportRun.objects.create(
profile=profile, file_name=file_name, owner=self.user
)
file_path = self.temp_root / file_name
write(file_path)
ImportService(run).process_file(str(file_path))
run.refresh_from_db()
return run
def _import_csv(self, account_transformations):
settings = """
settings:
file_type: csv
importing: transactions
delimiter: ";"
skip_lines: 4
trigger_transaction_rules: false
"""
return self._import(
settings,
account_transformations,
"statement.csv",
lambda path: path.write_text(CSV_CONTENT, encoding="utf-8"),
)
def assert_imported_into_savings(self, run):
self.assertEqual(run.status, ImportRun.Status.FINISHED, run.logs)
self.assertEqual(run.successful_rows, 2)
self.assertEqual(Transaction.objects.filter(account=self.savings).count(), 2)
def test_csv_reads_a_cell_from_a_skipped_line(self):
run = self._import_csv(
"""
- type: line
line: 2
column: 2
"""
)
self.assert_imported_into_savings(run)
def test_csv_whole_line_cleaned_with_regex(self):
run = self._import_csv(
"""
- type: line
line: 3
- type: regex
pattern: '^Account number:;(\\w+) \\d+$'
replacement: '\\1'
"""
)
self.assert_imported_into_savings(run)
def test_xlsx_reads_a_cell_above_the_header(self):
def write(path):
workbook = openpyxl.Workbook()
sheet = workbook.active
sheet["A1"] = "Bank statement"
sheet["A4"] = "Account:"
sheet["B4"] = "Savings"
sheet.append([])
sheet.append(["date", "amount", "description"])
sheet.append(["2026-01-05", "-50.00", "Groceries"])
sheet.append(["2026-01-06", "1000.00", "Salary"])
workbook.save(path)
settings = """
settings:
file_type: xlsx
importing: transactions
start_row: 6
trigger_transaction_rules: false
"""
run = self._import(
settings,
"""
- type: line
line: 4
column: 2
""",
"statement.xlsx",
write,
)
self.assert_imported_into_savings(run)
+161
View File
@@ -0,0 +1,161 @@
"""Tests for the folders import code reads from and writes to."""
import json
import os
import runpy
import shutil
import tempfile
from pathlib import Path
from unittest.mock import patch
from django.conf import settings
from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import SimpleTestCase, override_settings
from django.urls import reverse
from rest_framework import status
from rest_framework.test import APIClient
from apps.import_app.models import ImportRun
from apps.import_app.services import PresetService
from apps.import_app.services.v1 import ImportService
from apps.import_app.tests.test_view_permissions import (
HTMX,
ImportObjectPermissionTestCase,
)
def load_settings(**env):
"""Evaluates the settings module from scratch under the given env vars."""
with patch.dict(os.environ, env):
return runpy.run_path(str(Path(settings.BASE_DIR) / "WYGIWYH" / "settings.py"))
class PathSettingsTests(SimpleTestCase):
def test_defaults_match_the_docker_image_layout(self):
loaded = load_settings(IMPORT_TEMP_ROOT="", CACHE_ROOT="")
base_dir = loaded["BASE_DIR"]
self.assertEqual(loaded["IMPORT_TEMP_ROOT"], base_dir / "temp")
self.assertEqual(loaded["IMPORT_PRESETS_ROOT"], base_dir / "import_presets")
self.assertEqual(
loaded["CACHES"]["default"]["LOCATION"],
Path("/var/tmp/django_cache").resolve(),
)
self.assertEqual(
loaded["CACHES"]["update_check"]["LOCATION"],
Path("/var/tmp/django_cache_update_check").resolve(),
)
def test_env_vars_move_temp_and_cache_folders(self):
with tempfile.TemporaryDirectory() as root:
root = Path(root).resolve()
loaded = load_settings(
IMPORT_TEMP_ROOT=str(root / "uploads"), CACHE_ROOT=str(root / "cache")
)
self.assertEqual(loaded["IMPORT_TEMP_ROOT"], root / "uploads")
self.assertEqual(
loaded["CACHES"]["default"]["LOCATION"], root / "cache" / "django_cache"
)
self.assertEqual(
loaded["CACHES"]["update_check"]["LOCATION"],
root / "cache" / "django_cache_update_check",
)
class PresetServiceTests(SimpleTestCase):
def test_lists_the_presets_shipped_with_the_app(self):
folder_names = {
preset["folder_name"] for preset in PresetService.get_all_presets()
}
self.assertIn("qif_standard", folder_names)
def test_reads_presets_from_configured_folder(self):
with tempfile.TemporaryDirectory() as root:
preset = Path(root) / "my_bank"
preset.mkdir()
(preset / "manifest.json").write_text(json.dumps({"name": "My Bank"}))
(preset / "config.yml").write_text("settings: {}")
with override_settings(IMPORT_PRESETS_ROOT=Path(root)):
presets = PresetService.get_all_presets()
self.assertEqual([p["name"] for p in presets], ["My Bank"])
class ImportTempRootTests(ImportObjectPermissionTestCase):
def setUp(self):
super().setUp()
self.temp_root = Path(tempfile.mkdtemp()).resolve()
self.addCleanup(shutil.rmtree, self.temp_root, ignore_errors=True)
temp_root_override = override_settings(IMPORT_TEMP_ROOT=self.temp_root)
temp_root_override.enable()
self.addCleanup(temp_root_override.disable)
@patch("apps.import_app.views.process_import.defer")
def test_web_upload_is_stored_in_temp_root(self, mock_defer):
self.login(self.owner)
response = self.client.post(
reverse("import_run_add", kwargs={"profile_id": self.private_profile.id}),
{"file": SimpleUploadedFile("web.csv", b"date,amount\n")},
**HTMX,
)
self.assertEqual(response.status_code, 204)
self.assertEqual(
mock_defer.call_args.kwargs["file_path"], str(self.temp_root / "web.csv")
)
self.assertTrue((self.temp_root / "web.csv").is_file())
@patch("apps.api.views.imports.process_import.defer")
def test_api_upload_is_stored_in_temp_root(self, mock_defer):
client = APIClient()
client.force_authenticate(user=self.owner)
response = client.post(
"/api/import/import/",
{
"profile_id": self.private_profile.id,
"file": SimpleUploadedFile("api.csv", b"date,amount\n"),
},
format="multipart",
)
self.assertEqual(response.status_code, status.HTTP_202_ACCEPTED)
self.assertEqual(
mock_defer.call_args.kwargs["file_path"], str(self.temp_root / "api.csv")
)
self.assertTrue((self.temp_root / "api.csv").is_file())
def _service(self):
run = ImportRun.objects.create(
profile=self.private_profile, file_name="file.csv", owner=self.owner
)
return ImportService(run)
def test_service_accepts_files_inside_temp_root(self):
path = str(self.temp_root / "file.csv")
self.assertEqual(self._service()._validate_file_path(path), path)
def test_service_rejects_files_outside_temp_root(self):
service = self._service()
for path in (
"/etc/passwd",
self.temp_root / ".." / "file.csv",
# Shares the temp root as a string prefix, but is a different folder
f"{self.temp_root}-other/file.csv",
):
with self.subTest(path=path), self.assertRaises(ValueError):
service._validate_file_path(str(path))
def test_service_rejects_symlinks_pointing_outside_temp_root(self):
link = self.temp_root / "link.csv"
try:
link.symlink_to("/etc/passwd")
except OSError:
self.skipTest("symlinks not available")
with self.assertRaises(ValueError):
self._service()._validate_file_path(str(link))
+5 -5
View File
@@ -1,7 +1,7 @@
from decimal import Decimal
import os
import shutil
from django.test import TestCase
from django.test import TestCase, override_settings
from django.contrib.auth import get_user_model
from apps.accounts.models import Account, AccountGroup
from apps.currencies.models import Currency
@@ -15,10 +15,11 @@ from apps.transactions.models import (
class QIFImportTests(TestCase):
def setUp(self):
# Patch TEMP_DIR for testing
self.original_temp_dir = ImportService.TEMP_DIR
# Point the import temp folder somewhere disposable for testing
self.test_dir = os.path.abspath("temp_test_import")
ImportService.TEMP_DIR = self.test_dir
temp_root_override = override_settings(IMPORT_TEMP_ROOT=self.test_dir)
temp_root_override.enable()
self.addCleanup(temp_root_override.disable)
os.makedirs(self.test_dir, exist_ok=True)
# Create user and set context
@@ -41,7 +42,6 @@ class QIFImportTests(TestCase):
def tearDown(self):
delete_current_user()
ImportService.TEMP_DIR = self.original_temp_dir
if os.path.exists(self.test_dir):
shutil.rmtree(self.test_dir)
@@ -0,0 +1,380 @@
"""Tests for importing a file into an account picked for the import run."""
import shutil
import tempfile
from decimal import Decimal
from pathlib import Path
from unittest.mock import patch
from django.contrib.auth import get_user_model
from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import TestCase, override_settings
from django.urls import reverse
from rest_framework import status
from rest_framework.test import APIClient
from apps.accounts.models import Account
from apps.common.middleware.thread_local import delete_current_user, write_current_user
from apps.currencies.models import Currency
from apps.import_app.models import ImportProfile, ImportRun
from apps.import_app.services.v1 import ImportService
from apps.import_app.tests.test_view_permissions import (
HTMX,
ImportObjectPermissionTestCase,
)
from apps.transactions.models import Transaction
CSV_SETTINGS = """
settings:
file_type: csv
importing: transactions
trigger_transaction_rules: false
"""
CSV_MAPPING = """
mapping:
date:
target: date
source: date
format: "%Y-%m-%d"
amount:
target: amount
source: amount
type:
target: type
source: amount
detection_method: sign
description:
target: description
source: description
"""
CSV_ACCOUNT_MAPPING = """
account:
target: account
default: Checking
"""
CSV_ACCOUNT_DEDUPLICATION = """
deduplication:
- type: compare
fields:
- date
- amount
- account
match_type: strict
"""
CSV_CONTENT = """date,amount,description
2026-01-05,-50.00,Groceries
2026-01-06,1000.00,Salary
"""
TAGS_YAML_CONFIG = """
settings:
file_type: csv
importing: tags
mapping:
name:
target: tag_name
source: name
"""
class ImportRunAccountServiceTests(TestCase):
def setUp(self):
self.temp_root = Path(tempfile.mkdtemp()).resolve()
self.addCleanup(shutil.rmtree, self.temp_root, ignore_errors=True)
temp_root_override = override_settings(IMPORT_TEMP_ROOT=self.temp_root)
temp_root_override.enable()
self.addCleanup(temp_root_override.disable)
self.user = get_user_model().objects.create_user(
email="test@example.com", password="password"
)
write_current_user(self.user)
self.addCleanup(delete_current_user)
currency = Currency.objects.create(
code="EUR", name="Euro", decimal_places=2, prefix="€ "
)
self.checking = Account.objects.create(
name="Checking", currency=currency, owner=self.user
)
self.savings = Account.objects.create(
name="Savings", currency=currency, owner=self.user
)
def _import(self, yaml_config, file_name, content, account=None):
profile = ImportProfile.objects.create(
name="Profile", yaml_config=yaml_config, owner=self.user
)
run = ImportRun.objects.create(
profile=profile, file_name=file_name, account=account, owner=self.user
)
file_path = self.temp_root / file_name
file_path.write_text(content, encoding="utf-8")
ImportService(run).process_file(str(file_path))
run.refresh_from_db()
return run
def test_run_account_overrides_the_profile_account(self):
run = self._import(
CSV_SETTINGS + CSV_MAPPING + CSV_ACCOUNT_MAPPING,
"statement.csv",
CSV_CONTENT,
account=self.savings,
)
self.assertEqual(run.status, ImportRun.Status.FINISHED)
self.assertEqual(run.successful_rows, 2)
self.assertEqual(Transaction.objects.filter(account=self.savings).count(), 2)
self.assertFalse(Transaction.objects.filter(account=self.checking).exists())
self.assertIn("Importing transactions into account 'Savings'", run.logs)
def test_profile_account_is_used_without_a_run_account(self):
run = self._import(
CSV_SETTINGS + CSV_MAPPING + CSV_ACCOUNT_MAPPING,
"statement.csv",
CSV_CONTENT,
)
self.assertEqual(run.successful_rows, 2)
self.assertEqual(Transaction.objects.filter(account=self.checking).count(), 2)
def test_run_account_imports_with_a_profile_without_account_mapping(self):
run = self._import(
CSV_SETTINGS + CSV_MAPPING,
"statement.csv",
CSV_CONTENT,
account=self.checking,
)
self.assertEqual(run.successful_rows, 2)
expense = Transaction.objects.get(description="Groceries")
self.assertEqual(expense.account, self.checking)
self.assertEqual(expense.type, Transaction.Type.EXPENSE)
self.assertEqual(expense.amount, Decimal("50.00"))
def test_deduplication_by_account_uses_the_run_account(self):
yaml_config = CSV_SETTINGS + CSV_MAPPING + CSV_ACCOUNT_DEDUPLICATION
self._import(yaml_config, "first.csv", CSV_CONTENT, account=self.checking)
# Same rows, other account: not duplicates
into_savings = self._import(
yaml_config, "second.csv", CSV_CONTENT, account=self.savings
)
# Same rows, same account: duplicates
into_savings_again = self._import(
yaml_config, "third.csv", CSV_CONTENT, account=self.savings
)
self.assertEqual(into_savings.successful_rows, 2)
self.assertEqual(into_savings.skipped_rows, 0)
self.assertEqual(into_savings_again.successful_rows, 0)
self.assertEqual(into_savings_again.skipped_rows, 2)
self.assertEqual(Transaction.objects.filter(account=self.savings).count(), 2)
def test_qif_run_account_is_used_instead_of_the_file_name(self):
yaml_config = """
settings:
file_type: qif
importing: transactions
date_format: "%d/%m/%Y"
mapping: {}
"""
content = """!Type:Bank
D05/01/2026
T-100.00
PSupermarket
^
"""
run = self._import(
yaml_config, "export-2026.qif", content, account=self.savings
)
self.assertEqual(run.status, ImportRun.Status.FINISHED)
transaction = Transaction.objects.get()
self.assertEqual(transaction.account, self.savings)
self.assertEqual(transaction.amount, Decimal("100.00"))
class ImportRunAccountTestCase(ImportObjectPermissionTestCase):
def setUp(self):
super().setUp()
self.temp_root = Path(tempfile.mkdtemp()).resolve()
self.addCleanup(shutil.rmtree, self.temp_root, ignore_errors=True)
temp_root_override = override_settings(IMPORT_TEMP_ROOT=self.temp_root)
temp_root_override.enable()
self.addCleanup(temp_root_override.disable)
currency = Currency.objects.create(
code="EUR", name="Euro", decimal_places=2, prefix="€ "
)
self.account = Account.all_objects.create(
name="Checking", currency=currency, owner=self.owner
)
self.archived_account = Account.all_objects.create(
name="Old checking", currency=currency, owner=self.owner, is_archived=True
)
self.strangers_account = Account.all_objects.create(
name="Stranger's checking", currency=currency, owner=self.stranger
)
self.tags_profile = ImportProfile.all_objects.create(
name="Tags profile", yaml_config=TAGS_YAML_CONFIG, owner=self.owner
)
class ImportRunAccountViewTests(ImportRunAccountTestCase):
def _upload_form(self, profile):
self.login(self.owner)
response = self.client.get(
reverse("import_run_add", kwargs={"profile_id": profile.id}), **HTMX
)
self.assertEqual(response.status_code, 200)
return response.context["form"]
def test_upload_form_offers_active_accounts_for_transaction_profiles(self):
form = self._upload_form(self.private_profile)
self.assertIn("account", form.fields)
self.assertFalse(form.fields["account"].required)
self.assertEqual(list(form.fields["account"].queryset), [self.account])
def test_upload_form_has_no_account_for_other_profiles(self):
form = self._upload_form(self.tags_profile)
self.assertNotIn("account", form.fields)
@patch("apps.import_app.views.process_import.defer")
def test_web_upload_stores_the_account_on_the_run(self, mock_defer):
self.login(self.owner)
response = self.client.post(
reverse("import_run_add", kwargs={"profile_id": self.private_profile.id}),
{
"file": SimpleUploadedFile("web.csv", b"date,amount\n"),
"account": self.account.id,
},
**HTMX,
)
self.assertEqual(response.status_code, 204)
run = ImportRun.all_objects.get(file_name="web.csv")
self.assertEqual(run.account, self.account)
mock_defer.assert_called_once()
@patch("apps.import_app.views.process_import.defer")
def test_web_upload_without_account_keeps_the_profile_account(self, mock_defer):
self.login(self.owner)
response = self.client.post(
reverse("import_run_add", kwargs={"profile_id": self.private_profile.id}),
{"file": SimpleUploadedFile("web.csv", b"date,amount\n")},
**HTMX,
)
self.assertEqual(response.status_code, 204)
self.assertIsNone(ImportRun.all_objects.get(file_name="web.csv").account)
@patch("apps.import_app.views.process_import.defer")
def test_web_upload_rejects_another_users_account(self, mock_defer):
self.login(self.owner)
response = self.client.post(
reverse("import_run_add", kwargs={"profile_id": self.private_profile.id}),
{
"file": SimpleUploadedFile("web.csv", b"date,amount\n"),
"account": self.strangers_account.id,
},
**HTMX,
)
self.assertEqual(response.status_code, 200)
self.assertIn("account", response.context["form"].errors)
self.assertFalse(ImportRun.all_objects.filter(file_name="web.csv").exists())
mock_defer.assert_not_called()
def test_runs_list_shows_the_run_account(self):
ImportRun.all_objects.create(
profile=self.private_profile,
file_name="statement.csv",
account=self.account,
owner=self.owner,
)
self.login(self.owner)
response = self.client.get(
reverse(
"import_profile_runs_list",
kwargs={"profile_id": self.private_profile.id},
),
**HTMX,
)
self.assertContains(response, "Checking")
class ImportRunAccountAPITests(ImportRunAccountTestCase):
def setUp(self):
super().setUp()
self.api = APIClient()
self.api.force_authenticate(user=self.owner)
def _post(self, profile, **data):
return self.api.post(
"/api/import/import/",
{
"profile_id": profile.id,
"file": SimpleUploadedFile("api.csv", b"date,amount\n"),
**data,
},
format="multipart",
)
@patch("apps.api.views.imports.process_import.defer")
def test_api_upload_stores_the_account_on_the_run(self, mock_defer):
response = self._post(self.private_profile, account_id=self.account.id)
self.assertEqual(response.status_code, status.HTTP_202_ACCEPTED)
run = ImportRun.all_objects.get(id=response.data["import_run_id"])
self.assertEqual(run.account, self.account)
response = self.api.get(f"/api/import/runs/{run.id}/")
self.assertEqual(response.data["account"], self.account.id)
@patch("apps.api.views.imports.process_import.defer")
def test_api_upload_without_account_keeps_the_profile_account(self, mock_defer):
response = self._post(self.private_profile)
self.assertEqual(response.status_code, status.HTTP_202_ACCEPTED)
run = ImportRun.all_objects.get(id=response.data["import_run_id"])
self.assertIsNone(run.account)
@patch("apps.api.views.imports.process_import.defer")
def test_api_rejects_another_users_account(self, mock_defer):
response = self._post(
self.private_profile, account_id=self.strangers_account.id
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertIn("account_id", response.data)
mock_defer.assert_not_called()
@patch("apps.api.views.imports.process_import.defer")
def test_api_rejects_an_archived_account(self, mock_defer):
response = self._post(
self.private_profile, account_id=self.archived_account.id
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertIn("account_id", response.data)
mock_defer.assert_not_called()
@patch("apps.api.views.imports.process_import.defer")
def test_api_rejects_an_account_for_profiles_not_importing_transactions(
self, mock_defer
):
response = self._post(self.tags_profile, account_id=self.account.id)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertIn("account_id", response.data)
mock_defer.assert_not_called()
@@ -0,0 +1,440 @@
"""Object-level authorization tests for import profiles and import runs.
Regression coverage for GHSA-q668-h56g-mfp4: ImportProfile and ImportRun had no
owner, and neither the web views nor the REST viewsets filtered by the
requesting user, so any authenticated user could read, edit and delete every
other user's profiles and runs.
"""
from unittest.mock import patch
from django.contrib.auth import get_user_model
from django.core.files.uploadedfile import SimpleUploadedFile
from django.test import TestCase, override_settings
from django.urls import reverse
from rest_framework import status
from rest_framework.test import APIClient
from apps.import_app.models import ImportProfile, ImportRun
HTMX = {"HTTP_HX_REQUEST": "true"}
YAML_CONFIG = """
settings:
file_type: csv
importing: transactions
trigger_transaction_rules: false
mapping:
date_field:
source: date
target: date
format: "%Y-%m-%d"
amount_field:
source: amount
target: amount
description_field:
source: description
target: description
account_field:
source: account
target: account
type: id
"""
@override_settings(
STORAGES={
"default": {"BACKEND": "django.core.files.storage.FileSystemStorage"},
"staticfiles": {
"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"
},
},
WHITENOISE_AUTOREFRESH=True,
DEMO=False,
)
class ImportObjectPermissionTestCase(TestCase):
def setUp(self):
User = get_user_model()
self.owner = User.objects.create_user(
email="owner@test.com", password="testpass123"
)
self.shared_user = User.objects.create_user(
email="shared@test.com", password="testpass123"
)
self.stranger = User.objects.create_user(
email="stranger@test.com", password="testpass123"
)
# Private and unshared: invisible to everyone but the owner.
self.private_profile = ImportProfile.all_objects.create(
name="Private profile",
yaml_config=YAML_CONFIG,
owner=self.owner,
visibility="private",
)
# Private but shared with shared_user: usable by them, not by stranger.
self.shared_profile = ImportProfile.all_objects.create(
name="Shared profile",
yaml_config=YAML_CONFIG,
owner=self.owner,
visibility="private",
)
self.shared_profile.shared_with.add(self.shared_user)
# Legacy profile created before profiles had owners.
self.unowned_profile = ImportProfile.all_objects.create(
name="Unowned profile", yaml_config=YAML_CONFIG, owner=None
)
self.owner_run = ImportRun.all_objects.create(
profile=self.shared_profile,
file_name="owner-statement.csv",
logs="owner log",
owner=self.owner,
)
self.shared_user_run = ImportRun.all_objects.create(
profile=self.shared_profile,
file_name="shared-user-statement.csv",
logs="shared user log",
owner=self.shared_user,
)
def login(self, user):
self.client.force_login(user)
class ImportProfileViewPermissionTests(ImportObjectPermissionTestCase):
def test_list_only_shows_visible_profiles(self):
self.login(self.stranger)
response = self.client.get(reverse("import_profiles_list"), **HTMX)
self.assertEqual(response.status_code, 200)
self.assertNotContains(response, "Private profile")
self.assertNotContains(response, "Shared profile")
self.assertContains(response, "Unowned profile")
def test_list_shows_profiles_shared_with_user(self):
self.login(self.shared_user)
response = self.client.get(reverse("import_profiles_list"), **HTMX)
self.assertContains(response, "Shared profile")
self.assertNotContains(response, "Private profile")
def test_stranger_cannot_edit_private_profile(self):
self.login(self.stranger)
response = self.client.post(
reverse(
"import_profile_edit", kwargs={"profile_id": self.private_profile.id}
),
{"name": "Hijacked", "version": 1, "yaml_config": YAML_CONFIG},
**HTMX,
)
self.assertEqual(response.status_code, 404)
self.private_profile.refresh_from_db()
self.assertEqual(self.private_profile.name, "Private profile")
def test_shared_user_cannot_edit_shared_profile(self):
self.login(self.shared_user)
response = self.client.post(
reverse(
"import_profile_edit", kwargs={"profile_id": self.shared_profile.id}
),
{"name": "Hijacked", "version": 1, "yaml_config": YAML_CONFIG},
**HTMX,
)
self.assertEqual(response.status_code, 403)
self.shared_profile.refresh_from_db()
self.assertEqual(self.shared_profile.name, "Shared profile")
def test_owner_can_edit_profile(self):
self.login(self.owner)
response = self.client.post(
reverse(
"import_profile_edit", kwargs={"profile_id": self.private_profile.id}
),
{
"name": "Renamed",
"version": 1,
"yaml_config": YAML_CONFIG,
"submit": "Update",
},
**HTMX,
)
self.assertEqual(response.status_code, 204)
self.private_profile.refresh_from_db()
self.assertEqual(self.private_profile.name, "Renamed")
def test_stranger_cannot_delete_private_profile(self):
self.login(self.stranger)
response = self.client.delete(
reverse(
"import_profile_delete",
kwargs={"profile_id": self.private_profile.id},
),
**HTMX,
)
self.assertEqual(response.status_code, 404)
self.assertTrue(
ImportProfile.all_objects.filter(id=self.private_profile.id).exists()
)
def test_shared_user_delete_only_removes_their_access(self):
self.login(self.shared_user)
response = self.client.delete(
reverse(
"import_profile_delete", kwargs={"profile_id": self.shared_profile.id}
),
**HTMX,
)
self.assertEqual(response.status_code, 204)
self.assertTrue(
ImportProfile.all_objects.filter(id=self.shared_profile.id).exists()
)
self.assertFalse(
self.shared_profile.shared_with.filter(pk=self.shared_user.pk).exists()
)
self.assertTrue(
ImportRun.all_objects.filter(id=self.owner_run.id).exists()
)
def test_owner_can_delete_profile(self):
self.login(self.owner)
response = self.client.delete(
reverse(
"import_profile_delete",
kwargs={"profile_id": self.private_profile.id},
),
**HTMX,
)
self.assertEqual(response.status_code, 204)
self.assertFalse(
ImportProfile.all_objects.filter(id=self.private_profile.id).exists()
)
def test_shared_user_cannot_change_share_settings(self):
self.login(self.shared_user)
response = self.client.post(
reverse(
"import_profile_share_settings", kwargs={"pk": self.shared_profile.id}
),
{"owner": self.shared_user.id, "visibility": "public"},
**HTMX,
)
self.assertEqual(response.status_code, 403)
self.shared_profile.refresh_from_db()
self.assertEqual(self.shared_profile.owner, self.owner)
def test_take_ownership_of_unowned_profile(self):
self.login(self.stranger)
response = self.client.get(
reverse(
"import_profile_take_ownership",
kwargs={"profile_id": self.unowned_profile.id},
),
**HTMX,
)
self.assertEqual(response.status_code, 204)
self.unowned_profile.refresh_from_db()
self.assertEqual(self.unowned_profile.owner, self.stranger)
def test_cannot_take_ownership_of_owned_profile(self):
self.login(self.shared_user)
response = self.client.get(
reverse(
"import_profile_take_ownership",
kwargs={"profile_id": self.shared_profile.id},
),
**HTMX,
)
self.assertEqual(response.status_code, 403)
self.shared_profile.refresh_from_db()
self.assertEqual(self.shared_profile.owner, self.owner)
class ImportRunViewPermissionTests(ImportObjectPermissionTestCase):
def test_runs_list_only_shows_own_runs(self):
self.login(self.shared_user)
response = self.client.get(
reverse(
"import_profile_runs_list",
kwargs={"profile_id": self.shared_profile.id},
),
**HTMX,
)
self.assertEqual(response.status_code, 200)
self.assertContains(response, "shared-user-statement.csv")
self.assertNotContains(response, "owner-statement.csv")
def test_stranger_cannot_list_runs_of_private_profile(self):
self.login(self.stranger)
response = self.client.get(
reverse(
"import_profile_runs_list",
kwargs={"profile_id": self.shared_profile.id},
),
**HTMX,
)
self.assertEqual(response.status_code, 404)
def test_cannot_read_another_users_run_log(self):
self.login(self.shared_user)
response = self.client.get(
reverse(
"import_run_log",
kwargs={
"profile_id": self.shared_profile.id,
"run_id": self.owner_run.id,
},
),
**HTMX,
)
self.assertEqual(response.status_code, 404)
def test_cannot_delete_another_users_run(self):
self.login(self.shared_user)
response = self.client.delete(
reverse(
"import_run_delete",
kwargs={
"profile_id": self.shared_profile.id,
"run_id": self.owner_run.id,
},
),
**HTMX,
)
self.assertEqual(response.status_code, 404)
self.assertTrue(ImportRun.all_objects.filter(id=self.owner_run.id).exists())
def test_owner_can_delete_own_run(self):
self.login(self.shared_user)
response = self.client.delete(
reverse(
"import_run_delete",
kwargs={
"profile_id": self.shared_profile.id,
"run_id": self.shared_user_run.id,
},
),
**HTMX,
)
self.assertEqual(response.status_code, 204)
self.assertFalse(
ImportRun.all_objects.filter(id=self.shared_user_run.id).exists()
)
def test_stranger_cannot_import_with_private_profile(self):
self.login(self.stranger)
response = self.client.get(
reverse("import_run_add", kwargs={"profile_id": self.private_profile.id}),
**HTMX,
)
self.assertEqual(response.status_code, 404)
@patch("apps.import_app.views.process_import.defer")
@patch("apps.import_app.views.FileSystemStorage.save", return_value="file.csv")
@patch("apps.import_app.views.FileSystemStorage.path", return_value="/tmp/file.csv")
def test_import_with_shared_profile_is_owned_by_uploader(
self, mock_path, mock_save, mock_defer
):
self.login(self.shared_user)
response = self.client.post(
reverse("import_run_add", kwargs={"profile_id": self.shared_profile.id}),
{"file": SimpleUploadedFile("file.csv", b"date,amount\n")},
**HTMX,
)
self.assertEqual(response.status_code, 204)
run = ImportRun.all_objects.latest("id")
self.assertEqual(run.owner, self.shared_user)
self.assertEqual(run.profile, self.shared_profile)
class ImportAPIPermissionTests(ImportObjectPermissionTestCase):
def setUp(self):
super().setUp()
self.api = APIClient()
def test_profiles_list_only_returns_visible_profiles(self):
self.api.force_authenticate(user=self.stranger)
response = self.api.get("/api/import/profiles/")
self.assertEqual(response.status_code, status.HTTP_200_OK)
names = {p["name"] for p in response.data["results"]}
self.assertEqual(names, {"Unowned profile"})
def test_cannot_retrieve_private_profile(self):
self.api.force_authenticate(user=self.stranger)
response = self.api.get(f"/api/import/profiles/{self.private_profile.id}/")
self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND)
def test_runs_list_only_returns_own_runs(self):
self.api.force_authenticate(user=self.stranger)
response = self.api.get("/api/import/runs/")
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["count"], 0)
def test_runs_list_on_shared_profile_hides_other_users_runs(self):
self.api.force_authenticate(user=self.shared_user)
response = self.api.get(
"/api/import/runs/", {"profile_id": self.shared_profile.id}
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
ids = {r["id"] for r in response.data["results"]}
self.assertEqual(ids, {self.shared_user_run.id})
def test_cannot_retrieve_another_users_run(self):
self.api.force_authenticate(user=self.stranger)
response = self.api.get(f"/api/import/runs/{self.owner_run.id}/")
self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND)
def test_cannot_import_with_private_profile(self):
self.api.force_authenticate(user=self.stranger)
response = self.api.post(
"/api/import/import/",
{
"profile_id": self.private_profile.id,
"file": SimpleUploadedFile("file.csv", b"date,amount\n"),
},
format="multipart",
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertIn("profile_id", response.data)
@patch("apps.import_app.tasks.process_import.defer")
@patch("django.core.files.storage.FileSystemStorage.save", return_value="file.csv")
@patch("django.core.files.storage.FileSystemStorage.path", return_value="/tmp/file.csv")
def test_import_with_shared_profile_is_owned_by_uploader(
self, mock_path, mock_save, mock_defer
):
self.api.force_authenticate(user=self.shared_user)
response = self.api.post(
"/api/import/import/",
{
"profile_id": self.shared_profile.id,
"file": SimpleUploadedFile("file.csv", b"date,amount\n"),
},
format="multipart",
)
self.assertEqual(response.status_code, status.HTTP_202_ACCEPTED)
run = ImportRun.all_objects.get(id=response.data["import_run_id"])
self.assertEqual(run.owner, self.shared_user)
+10
View File
@@ -32,6 +32,16 @@ urlpatterns = [
views.import_profile_edit,
name="import_profile_edit",
),
path(
"import/profiles/<int:profile_id>/take-ownership/",
views.import_profile_take_ownership,
name="import_profile_take_ownership",
),
path(
"import/profiles/<int:pk>/share/",
views.import_profile_share,
name="import_profile_share_settings",
),
path(
"import/profiles/<int:profile_id>/runs/list/",
views.import_runs_list,
+102 -14
View File
@@ -1,7 +1,9 @@
import shutil
from django.conf import settings
from django.contrib import messages
from django.contrib.auth.decorators import login_required
from django.core.exceptions import PermissionDenied
from django.core.files.storage import FileSystemStorage
from django.http import HttpResponse
from django.shortcuts import render, get_object_or_404
@@ -9,6 +11,13 @@ from django.utils.translation import gettext_lazy as _
from django.views.decorators.http import require_http_methods
from apps.common.decorators.htmx import only_htmx
from apps.common.forms import SharedObjectForm
from apps.common.functions.permissions import (
EDIT,
READ,
get_shared_object_or_error,
)
from apps.common.models import SharedObject
from apps.import_app.forms import ImportRunFileUploadForm, ImportProfileForm
from apps.import_app.models import ImportRun, ImportProfile
from apps.import_app.services import PresetService
@@ -93,7 +102,9 @@ def import_profile_add(request):
@disabled_on_demo
@require_http_methods(["GET", "POST"])
def import_profile_edit(request, profile_id):
profile = get_object_or_404(ImportProfile, id=profile_id)
profile = get_shared_object_or_error(
ImportProfile, request, id=profile_id, level=EDIT
)
if request.method == "POST":
form = ImportProfileForm(request.POST, instance=profile)
@@ -123,11 +134,20 @@ def import_profile_edit(request, profile_id):
@disabled_on_demo
@require_http_methods(["DELETE"])
def import_profile_delete(request, profile_id):
profile = ImportProfile.objects.get(id=profile_id)
profile = get_shared_object_or_error(
ImportProfile, request, id=profile_id, level=READ
)
profile.delete()
messages.success(request, _("Import Profile deleted successfully"))
if profile.is_editable_by(request.user):
profile.delete()
messages.success(request, _("Import Profile deleted successfully"))
elif profile.shared_with.filter(pk=request.user.pk).exists():
# Someone else's profile shared with us: we can drop our own access to
# it, but never delete it.
profile.shared_with.remove(request.user)
messages.success(request, _("Item no longer shared with you"))
else:
raise PermissionDenied
return HttpResponse(
status=204,
@@ -142,9 +162,17 @@ def import_profile_delete(request, profile_id):
@disabled_on_demo
@require_http_methods(["GET", "POST"])
def import_runs_list(request, profile_id):
profile = ImportProfile.objects.get(id=profile_id)
profile = get_shared_object_or_error(
ImportProfile, request, id=profile_id, level=READ
)
runs = ImportRun.objects.filter(profile=profile).order_by("-id")
# ImportRun.objects only returns the requesting user's runs, so a shared
# profile does not expose the runs other users made with it.
runs = (
ImportRun.objects.filter(profile=profile)
.select_related("account")
.order_by("-id")
)
return render(
request,
@@ -158,7 +186,7 @@ def import_runs_list(request, profile_id):
@disabled_on_demo
@require_http_methods(["GET", "POST"])
def import_run_log(request, profile_id, run_id):
run = ImportRun.objects.get(profile__id=profile_id, id=run_id)
run = get_object_or_404(ImportRun, profile__id=profile_id, id=run_id)
return render(
request,
@@ -172,18 +200,25 @@ def import_run_log(request, profile_id, run_id):
@disabled_on_demo
@require_http_methods(["GET", "POST"])
def import_run_add(request, profile_id):
profile = ImportProfile.objects.get(id=profile_id)
profile = get_shared_object_or_error(
ImportProfile, request, id=profile_id, level=READ
)
if request.method == "POST":
form = ImportRunFileUploadForm(request.POST, request.FILES)
form = ImportRunFileUploadForm(request.POST, request.FILES, profile=profile)
if form.is_valid():
uploaded_file = request.FILES["file"]
fs = FileSystemStorage(location="/usr/src/app/temp")
fs = FileSystemStorage(location=settings.IMPORT_TEMP_ROOT)
filename = fs.save(uploaded_file.name, uploaded_file)
file_path = fs.path(filename)
import_run = ImportRun.objects.create(profile=profile, file_name=filename)
import_run = ImportRun.objects.create(
profile=profile,
file_name=filename,
account=form.cleaned_data.get("account"),
owner=request.user,
)
# Defer the procrastinate task
process_import.defer(
@@ -201,7 +236,7 @@ def import_run_add(request, profile_id):
},
)
else:
form = ImportRunFileUploadForm()
form = ImportRunFileUploadForm(profile=profile)
return render(
request,
@@ -215,7 +250,7 @@ def import_run_add(request, profile_id):
@disabled_on_demo
@require_http_methods(["DELETE"])
def import_run_delete(request, profile_id, run_id):
run = ImportRun.objects.get(profile__id=profile_id, id=run_id)
run = get_object_or_404(ImportRun, profile__id=profile_id, id=run_id)
run.delete()
@@ -227,3 +262,56 @@ def import_run_delete(request, profile_id, run_id):
"HX-Trigger": "updated",
},
)
@only_htmx
@login_required
@disabled_on_demo
@require_http_methods(["GET"])
def import_profile_take_ownership(request, profile_id):
profile = get_shared_object_or_error(
ImportProfile, request, id=profile_id, level=EDIT
)
if not profile.owner:
profile.owner = request.user
profile.visibility = SharedObject.Visibility.private
profile.save()
messages.success(request, _("Ownership taken successfully"))
return HttpResponse(
status=204,
headers={
"HX-Trigger": "updated, hide_offcanvas",
},
)
@only_htmx
@login_required
@disabled_on_demo
@require_http_methods(["GET", "POST"])
def import_profile_share(request, pk):
obj = get_shared_object_or_error(ImportProfile, request, id=pk, level=EDIT)
if request.method == "POST":
form = SharedObjectForm(request.POST, instance=obj, user=request.user)
if form.is_valid():
form.save()
messages.success(request, _("Configuration saved successfully"))
return HttpResponse(
status=204,
headers={
"HX-Trigger": "updated, hide_offcanvas",
},
)
else:
form = SharedObjectForm(instance=obj, user=request.user)
return render(
request,
"import_app/fragments/profiles/share.html",
{"form": form, "object": obj},
)
+6 -6
View File
@@ -8,6 +8,7 @@ from django.utils import timezone
from apps.currencies.models import Currency
from apps.currencies.utils.convert import convert
from apps.currencies.utils.display import shown_exchange_currency
from apps.transactions.models import Transaction
@@ -94,7 +95,7 @@ def get_month_by_month_data(year=None, group_by="categories"):
"account__currency__decimal_places",
"account__currency__prefix",
"account__currency__suffix",
"account__currency__exchange_currency",
exchange_currency_id=shown_exchange_currency("account__currency__"),
)
.annotate(
expense_total=Coalesce(
@@ -139,6 +140,7 @@ def get_month_by_month_data(year=None, group_by="categories"):
item_name = metric[name_field]
month = metric["reference_date__month"]
currency_id = metric["account__currency"]
exchange_currency_id = metric["exchange_currency_id"]
# Use a consistent key for None (uncategorized/untagged/no entity)
item_key = item_id if item_id is not None else "__none__"
@@ -164,7 +166,7 @@ def get_month_by_month_data(year=None, group_by="categories"):
"decimal_places": metric["account__currency__decimal_places"],
"prefix": metric["account__currency__prefix"],
"suffix": metric["account__currency__suffix"],
"exchange_currency_id": metric["account__currency__exchange_currency"],
"exchange_currency_id": exchange_currency_id,
}
currency_data = {
@@ -181,11 +183,9 @@ def get_month_by_month_data(year=None, group_by="categories"):
}
# Handle currency conversion if exchange currency is set
if metric["account__currency__exchange_currency"]:
if exchange_currency_id:
from_currency = Currency.objects.get(id=currency_id)
exchange_currency = Currency.objects.get(
id=metric["account__currency__exchange_currency"]
)
exchange_currency = Currency.objects.get(id=exchange_currency_id)
converted_amount, prefix, suffix, decimal_places = convert(
amount=final_total,
+16 -7
View File
@@ -6,6 +6,7 @@ from django.db.models.functions import Coalesce
from apps.currencies.models import Currency
from apps.currencies.utils.convert import convert
from apps.currencies.utils.display import shown_exchange_currency
from apps.transactions.models import Transaction
# Grouping levels the overview can be built from. Any ordering of these keys is
@@ -25,11 +26,11 @@ CURRENCY_FIELDS = (
"account__currency__decimal_places",
"account__currency__prefix",
"account__currency__suffix",
"account__currency__exchange_currency",
)
# Aggregated amounts, plus the totals derived from them. Every one of these is
# converted when the account's currency has an exchange currency.
# converted when the account's currency has an exchange currency and shows
# exchanged amounts.
AGGREGATED_FIELDS = (
"expense_current",
"expense_projected",
@@ -78,7 +79,12 @@ def _sum_of(transaction_type, is_paid):
def _aggregate(transactions_queryset, group_fields, name_field):
"""Sum income/expense per currency, grouped by the given fields."""
return (
transactions_queryset.values(*group_fields, name_field, *CURRENCY_FIELDS)
transactions_queryset.values(
*group_fields,
name_field,
*CURRENCY_FIELDS,
exchange_currency_id=shown_exchange_currency("account__currency__"),
)
.annotate(
expense_current=_sum_of(Transaction.Type.EXPENSE, is_paid=True),
expense_projected=_sum_of(Transaction.Type.EXPENSE, is_paid=False),
@@ -120,11 +126,9 @@ def _build_currency_data(metric, showing):
"total": currency_data[total],
}
if metric["account__currency__exchange_currency"]:
if metric["exchange_currency_id"]:
from_currency = Currency.objects.get(id=metric["account__currency"])
exchange_currency = Currency.objects.get(
id=metric["account__currency__exchange_currency"]
)
exchange_currency = Currency.objects.get(id=metric["exchange_currency_id"])
exchanged = {}
for field in AGGREGATED_FIELDS + TOTAL_FIELDS:
@@ -144,6 +148,11 @@ def _build_currency_data(metric, showing):
"name": exchange_currency.name,
}
if exchanged:
exchanged["shown"] = {
"income": exchanged.get(income),
"expense": exchanged.get(expense),
"total": exchanged.get(total),
}
currency_data["exchanged"] = exchanged
return currency_data
+6 -6
View File
@@ -7,6 +7,7 @@ from django.db.models.functions import Coalesce
from apps.currencies.models import Currency
from apps.currencies.utils.convert import convert
from apps.currencies.utils.display import shown_exchange_currency
from apps.transactions.models import Transaction
@@ -83,7 +84,7 @@ def get_year_by_year_data(group_by="categories"):
"account__currency__decimal_places",
"account__currency__prefix",
"account__currency__suffix",
"account__currency__exchange_currency",
exchange_currency_id=shown_exchange_currency("account__currency__"),
)
.annotate(
expense_total=Coalesce(
@@ -126,6 +127,7 @@ def get_year_by_year_data(group_by="categories"):
item_name = metric[name_field]
year = metric["reference_date__year"]
currency_id = metric["account__currency"]
exchange_currency_id = metric["exchange_currency_id"]
# Use a consistent key for None (uncategorized/untagged/no entity)
item_key = item_id if item_id is not None else "__none__"
@@ -151,7 +153,7 @@ def get_year_by_year_data(group_by="categories"):
"decimal_places": metric["account__currency__decimal_places"],
"prefix": metric["account__currency__prefix"],
"suffix": metric["account__currency__suffix"],
"exchange_currency_id": metric["account__currency__exchange_currency"],
"exchange_currency_id": exchange_currency_id,
}
currency_data = {
@@ -168,11 +170,9 @@ def get_year_by_year_data(group_by="categories"):
}
# Handle currency conversion if exchange currency is set
if metric["account__currency__exchange_currency"]:
if exchange_currency_id:
from_currency = Currency.objects.get(id=currency_id)
exchange_currency = Currency.objects.get(
id=metric["account__currency__exchange_currency"]
)
exchange_currency = Currency.objects.get(id=exchange_currency_id)
converted_amount, prefix, suffix, decimal_places = convert(
amount=final_total,
+8 -4
View File
@@ -305,9 +305,11 @@ def overview_results(request):
@require_http_methods(["GET"])
def latest_transactions(request):
limit = timezone.now() - relativedelta(days=3)
transactions = Transaction.objects.filter(created_at__gte=limit).order_by("-id")[
:30
]
transactions = (
Transaction.objects.filter(created_at__gte=limit)
.collapse_transfers()
.order_by("-id")[:30]
)
return render(
request,
@@ -321,7 +323,9 @@ def latest_transactions(request):
@require_http_methods(["GET"])
def late_transactions(request):
now = timezone.localdate(timezone.now())
transactions = Transaction.objects.filter(is_paid=False, date__lt=now)
transactions = Transaction.objects.filter(
is_paid=False, date__lt=now
).collapse_transfers()
return render(
request,
View File
Whitespace-only changes.
+6
View File
@@ -0,0 +1,6 @@
from django.apps import AppConfig
class InvestmentsConfig(AppConfig):
default_auto_field = "django.db.models.BigAutoField"
name = "apps.investments"
Loaded 100 of 201 files, more files were not shown because too many files have changed in this diff. Show more