reference_date does not seem to be optional when importing #71

New Issue

Closed

opened 2025-12-28 23:25:12 +01:00 by adam · 3 comments

adam commented 2025-12-28 23:25:12 +01:00

Owner

Copy Link

Copy Source

Originally created by @zkutasi on GitHub (Nov 26, 2025).

I see this txt in the Wiki

Reference Date

One of WYGIWYH's key features. The reference date determines which month a transaction should count towards. For example, you can have a transaction that occurred on January 26th count towards February's finances.

Optional - defaults to the transaction date's month if not specified.

But when I did not set this at all in my importer's YAML config, I got this error:

django.core.exceptions.ValidationError: {'reference_date': ['This field cannot be null.']}

The input file is this:

"Felhasználónév","Számlaszám","Könyvelés dátuma","Összeg","Devizanem","Partner név","Partner IBAN száma","Partner számlaszáma","Partner bankkódja","Könyvelési információk","Tranzakcióazonosító","Tranzakció dátuma és ideje"
"Erste FőSzámla","11600006-00000000-12616657","2025.11.26","-9 965","HUF","ORTOREX-STORE","","","","478736xxxxxx9618 BPTE8SUM ORTOREX-STORE WROCLAW 25112122:27 vásár.","","2025.11.21 22:27:09"

And I have this importer config now:

settings:
  file_type: csv
  delimiter: ","
  encoding: utf-16
  skip_lines: 0
  importing: transactions
  trigger_transaction_rules: true
  skip_errors: false
mapping:
  account:
    target: account
    source: "Felhasználónév"
    required: true
  type:
    target: type
    source: "Összeg"
    detection_method: sign
  is_paid:
    target: is_paid
    detection_method: always_paid
  date:
    target: date
    required: true
    format: "%Y.%m.%d"
    transformations:
      - type: merge
        fields:
          - "Tranzakció dátuma és ideje"
          - "Könyvelés dátuma"
        separator: " "
      - type: split
        separator: " "
        index: 0
  reference_date:
    target: reference_date
    required: true
    format: "%Y.%m.%d"
    transformations:
      - type: merge
        fields:
          - "Tranzakció dátuma és ideje"
          - "Könyvelés dátuma"
        separator: " "
      - type: split
        separator: " "
        index: 0
  amount:
    target: amount
    source: "Összeg"
    required: true
    transformations:
      - type: replace
        pattern: " "
        replacement: ""
  description:
    target: description
    source: "Könyvelési információk"
    default:
    required: false
  internal_id:
    target: internal_id
    source: "Tranzakcióazonosító"
    default:
    required: false
deduplication:
  - type: compare
    fields:
      - internal_id
      - date
      - amount
    match_type: strict

Originally created by @zkutasi on GitHub (Nov 26, 2025). I see this txt in the Wiki ``` Reference Date One of WYGIWYH's key features. The reference date determines which month a transaction should count towards. For example, you can have a transaction that occurred on January 26th count towards February's finances. Optional - defaults to the transaction date's month if not specified. ``` But when I did not set this at all in my importer's YAML config, I got this error: ``` django.core.exceptions.ValidationError: {'reference_date': ['This field cannot be null.']} ``` The input file is this: ``` "Felhasználónév","Számlaszám","Könyvelés dátuma","Összeg","Devizanem","Partner név","Partner IBAN száma","Partner számlaszáma","Partner bankkódja","Könyvelési információk","Tranzakcióazonosító","Tranzakció dátuma és ideje" "Erste FőSzámla","11600006-00000000-12616657","2025.11.26","-9 965","HUF","ORTOREX-STORE","","","","478736xxxxxx9618 BPTE8SUM ORTOREX-STORE WROCLAW 25112122:27 vásár.","","2025.11.21 22:27:09" ``` And I have this importer config now: ``` settings: file_type: csv delimiter: "," encoding: utf-16 skip_lines: 0 importing: transactions trigger_transaction_rules: true skip_errors: false mapping: account: target: account source: "Felhasználónév" required: true type: target: type source: "Összeg" detection_method: sign is_paid: target: is_paid detection_method: always_paid date: target: date required: true format: "%Y.%m.%d" transformations: - type: merge fields: - "Tranzakció dátuma és ideje" - "Könyvelés dátuma" separator: " " - type: split separator: " " index: 0 reference_date: target: reference_date required: true format: "%Y.%m.%d" transformations: - type: merge fields: - "Tranzakció dátuma és ideje" - "Könyvelés dátuma" separator: " " - type: split separator: " " index: 0 amount: target: amount source: "Összeg" required: true transformations: - type: replace pattern: " " replacement: "" description: target: description source: "Könyvelési információk" default: required: false internal_id: target: internal_id source: "Tranzakcióazonosító" default: required: false deduplication: - type: compare fields: - internal_id - date - amount match_type: strict ```

adam added the bug label 2025-12-28 23:25:12 +01:00

adam closed this issue 2025-12-28 23:25:12 +01:00

adam commented 2025-12-28 23:25:13 +01:00

Author

Owner

Copy Link

Copy Source

@zkutasi commented on GitHub (Nov 26, 2025):

Also, please help me how could I state in a transformation this:

• If the row contains the transaction date&time (last column), use that
• If it does not, use the always existing 3rd column
• Since I also need to cut off the exact time from the last column, I tried to use the merge-split trick.... but it did not work out

For this row:

"Erste FőSzámla","11600006-00000000-12616657","2025.11.21","25 590","HUF","ABOUT YOU SE + CO. KGDOMSTRASSE 10","HU45137000160797601000000000","13700016-07976010-00000000","","0036510000ZV258231Z Trn: F0HO211120250231903 Oth.bank: 13700016 Oth.acct: 0797601000000000 Oth.acct.desc: ABOUT YOU SE + CO. KGDOMSTRASSE Ref.: -","F0HO211120250231903",""

I got a missing date error.

@zkutasi commented on GitHub (Nov 26, 2025): Also, please help me how could I state in a transformation this: - If the row contains the transaction date&time (last column), use that - If it does not, use the always existing 3rd column - Since I also need to cut off the exact time from the last column, I tried to use the merge-split trick.... but it did not work out For this row: ``` "Erste FőSzámla","11600006-00000000-12616657","2025.11.21","25 590","HUF","ABOUT YOU SE + CO. KGDOMSTRASSE 10","HU45137000160797601000000000","13700016-07976010-00000000","","0036510000ZV258231Z Trn: F0HO211120250231903 Oth.bank: 13700016 Oth.acct: 0797601000000000 Oth.acct.desc: ABOUT YOU SE + CO. KGDOMSTRASSE Ref.: -","F0HO211120250231903","" ``` I got a missing date error.

adam commented 2025-12-28 23:25:14 +01:00

Author

Owner

Copy Link

Copy Source

@eitchtee commented on GitHub (Nov 26, 2025):

This is indeed a bug! Thanks for bringing it to my attention, until a fix is released you can keep doing what you're doing (duplicate the definition for date as reference_date).

---

Also, please help me how could I state in a transformation this:

First, you don't need to remove the time from the last column, just use a proper format for it like %Y.%m.%d %H:%M:%S. WYGIWYH should be smart enough to convert this to a date when needed.

This is a nice brain-teaser, I don't think there's an easy way to do this right now because merge will add the empty string to the result, creating 2 or 3 different possible strings that don't have matching indexes:

2025.11.21 22:27:09____
2025.11.21 22:27:09____2025.11.21
____2025.11.21

Instead you can leverage merge and regex with a lookbehind. Don't ask too much about how the dark sacred art of regex works, but this should work.

date:
    target: date
    required: true
    format: 
      - "%Y.%m.%d"
      - "%Y.%m.%d %H:%M:%S"
    transformations:
      - type: merge
        fields:
          - "Tranzakció dátuma és ideje"
          - "Könyvelés dátuma"
        separator: "____"
      - type: regex
        pattern: "(?<=^\\d{4}\\.\\d{2}\\.\\d{2} \\d{2}:\\d{2}:\\d{2})____.*|^____"
        replacement: ""
        exclusive: false

(duplicate this for the reference_date)

@eitchtee commented on GitHub (Nov 26, 2025): This is indeed a bug! Thanks for bringing it to my attention, until a fix is released you can keep doing what you're doing (duplicate the definition for date as reference_date). --- > Also, please help me how could I state in a transformation this: First, you don't need to remove the time from the last column, just use a proper format for it like `%Y.%m.%d %H:%M:%S`. WYGIWYH should be smart enough to convert this to a date when needed. This is a nice brain-teaser, I don't think there's an easy way to do this right now because merge will add the empty string to the result, creating 2 or 3 different possible strings that don't have matching indexes: ``` 2025.11.21 22:27:09____ 2025.11.21 22:27:09____2025.11.21 ____2025.11.21 ``` Instead you can leverage merge and regex with a lookbehind. Don't ask too much about how the dark sacred art of regex works, but this should work. ``` date: target: date required: true format: - "%Y.%m.%d" - "%Y.%m.%d %H:%M:%S" transformations: - type: merge fields: - "Tranzakció dátuma és ideje" - "Könyvelés dátuma" separator: "____" - type: regex pattern: "(?<=^\\d{4}\\.\\d{2}\\.\\d{2} \\d{2}:\\d{2}:\\d{2})____.*|^____" replacement: "" exclusive: false ``` _(duplicate this for the reference_date)_

adam commented 2025-12-28 23:25:14 +01:00

Author

Owner

Copy Link

Copy Source

@eitchtee commented on GitHub (Nov 26, 2025):

A fix for this will be available on the next release (hopefully soon).

@eitchtee commented on GitHub (Nov 26, 2025): A fix for this will be available on the next release (hopefully soon).

adam referenced this issue 2025-12-28 23:26:35 +01:00

[PR #71] [MERGED] feat: tidy up transactions action bar #155

Sign in to join this conversation.

No Branch/Tag Specified

Branches Tags

main

dependabot/uv/mistune-3.3.3

dependabot/npm_and_yarn/frontend/browserslist-4.28.8

dependabot/uv/djangorestframework-3.17.2

weblate

feat/filter-presets

dev

copilot/fix-code-scanning-alerts

feat/cuelume

feat/tom-select-improvements

chore/bump-deps

feat/attatchments

fix-procrastinate

feat/replace-webpack+bootstrap

internal_port

add-mcp-server-mention

feat/networth-chart

feat/better-rules

feature/untrack-account

feat/setup-screen

feat/sidebar

feat/auto-refresh

fix/amount-input-doesnt-update-on-delete

feature/add-django-tests

tests

0.23.0

0.22.1

0.22.0

0.21.0

0.20.2

0.20.1

0.20.0

0.19.1

0.19.0

0.18.3

0.18.2

0.18.1

0.18.0

0.17.5

0.17.4

0.17.3

0.17.2

0.17.1

0.17.0

0.16.2

0.16.1

0.16.0

0.15.1

0.15.0

0.14.1

0.14.0

0.13.1

0.13.0

0.12.11

0.12.10

0.12.9

0.12.8

0.12.7

0.12.6

0.12.5

0.12.4

0.12.3

0.12.2

0.12.1

0.12.0

0.11.8

0.11.7

0.11.6

0.11.5

0.11.4

0.11.3

0.11.2

0.11.1

0.11.0

0.10.1

0.10.0

0.9.4

0.9.3

0.9.2

0.9.1

0.9.0

0.8.6

0.8.5

0.8.4

0.8.3

0.8.2

0.8.1

0.8.0

0.7.8

0.7.7

0.7.6

0.7.5

0.7.4

0.7.3

0.7.2

0.7.1

0.7.0

0.6.0

0.5.0

0.4.0

0.3.6

0.3.5

0.3.4

0.3.3

0.3.2

0.3.1

0.3.0

0.2.0

0.1.9

0.1.8

0.1.7

0.1.6

0.1.5

0.1.4

0.1.3

0.1.2

0.1.1

0.1.0

Labels

Clear labels

bug

bug

documentation

enhancement

help wanted

pull-request

Mirrored from GitHub Pull Request

question

No labels bug

Milestone

No items

No Milestone

Projects

Clear projects

No projects

Assignees

Clear assignees

adam (Adam Melkus)

No Assignees

1 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/WYGIWYH#71