reference_date does not seem to be optional when importing #71

New Issue

adam · 2025-12-28T23:25:12+01:00

adam commented

2025-12-28 23:25:12 +01:00

Originally created by @zkutasi on GitHub (Nov 26, 2025).

I see this txt in the Wiki

Reference Date

One of WYGIWYH's key features. The reference date determines which month a transaction should count towards. For example, you can have a transaction that occurred on January 26th count towards February's finances.

Optional - defaults to the transaction date's month if not specified.

But when I did not set this at all in my importer's YAML config, I got this error:

django.core.exceptions.ValidationError: {'reference_date': ['This field cannot be null.']}

The input file is this:

"Felhasználónév","Számlaszám","Könyvelés dátuma","Összeg","Devizanem","Partner név","Partner IBAN száma","Partner számlaszáma","Partner bankkódja","Könyvelési információk","Tranzakcióazonosító","Tranzakció dátuma és ideje"
"Erste FőSzámla","11600006-00000000-12616657","2025.11.26","-9 965","HUF","ORTOREX-STORE","","","","478736xxxxxx9618 BPTE8SUM ORTOREX-STORE WROCLAW 25112122:27 vásár.","","2025.11.21 22:27:09"

And I have this importer config now:

settings:
  file_type: csv
  delimiter: ","
  encoding: utf-16
  skip_lines: 0
  importing: transactions
  trigger_transaction_rules: true
  skip_errors: false
mapping:
  account:
    target: account
    source: "Felhasználónév"
    required: true
  type:
    target: type
    source: "Összeg"
    detection_method: sign
  is_paid:
    target: is_paid
    detection_method: always_paid
  date:
    target: date
    required: true
    format: "%Y.%m.%d"
    transformations:
      - type: merge
        fields:
          - "Tranzakció dátuma és ideje"
          - "Könyvelés dátuma"
        separator: " "
      - type: split
        separator: " "
        index: 0
  reference_date:
    target: reference_date
    required: true
    format: "%Y.%m.%d"
    transformations:
      - type: merge
        fields:
          - "Tranzakció dátuma és ideje"
          - "Könyvelés dátuma"
        separator: " "
      - type: split
        separator: " "
        index: 0
  amount:
    target: amount
    source: "Összeg"
    required: true
    transformations:
      - type: replace
        pattern: " "
        replacement: ""
  description:
    target: description
    source: "Könyvelési információk"
    default:
    required: false
  internal_id:
    target: internal_id
    source: "Tranzakcióazonosító"
    default:
    required: false
deduplication:
  - type: compare
    fields:
      - internal_id
      - date
      - amount
    match_type: strict

Originally created by @zkutasi on GitHub (Nov 26, 2025). I see this txt in the Wiki ``` Reference Date One of WYGIWYH's key features. The reference date determines which month a transaction should count towards. For example, you can have a transaction that occurred on January 26th count towards February's finances. Optional - defaults to the transaction date's month if not specified. ``` But when I did not set this at all in my importer's YAML config, I got this error: ``` django.core.exceptions.ValidationError: {'reference_date': ['This field cannot be null.']} ``` The input file is this: ``` "Felhasználónév","Számlaszám","Könyvelés dátuma","Összeg","Devizanem","Partner név","Partner IBAN száma","Partner számlaszáma","Partner bankkódja","Könyvelési információk","Tranzakcióazonosító","Tranzakció dátuma és ideje" "Erste FőSzámla","11600006-00000000-12616657","2025.11.26","-9 965","HUF","ORTOREX-STORE","","","","478736xxxxxx9618 BPTE8SUM ORTOREX-STORE WROCLAW 25112122:27 vásár.","","2025.11.21 22:27:09" ``` And I have this importer config now: ``` settings: file_type: csv delimiter: "," encoding: utf-16 skip_lines: 0 importing: transactions trigger_transaction_rules: true skip_errors: false mapping: account: target: account source: "Felhasználónév" required: true type: target: type source: "Összeg" detection_method: sign is_paid: target: is_paid detection_method: always_paid date: target: date required: true format: "%Y.%m.%d" transformations: - type: merge fields: - "Tranzakció dátuma és ideje" - "Könyvelés dátuma" separator: " " - type: split separator: " " index: 0 reference_date: target: reference_date required: true format: "%Y.%m.%d" transformations: - type: merge fields: - "Tranzakció dátuma és ideje" - "Könyvelés dátuma" separator: " " - type: split separator: " " index: 0 amount: target: amount source: "Összeg" required: true transformations: - type: replace pattern: " " replacement: "" description: target: description source: "Könyvelési információk" default: required: false internal_id: target: internal_id source: "Tranzakcióazonosító" default: required: false deduplication: - type: compare fields: - internal_id - date - amount match_type: strict ```

adam added the bug label 2025-12-28 23:25:12 +01:00

adam closed this issue

2025-12-28 23:25:12 +01:00

adam commented

2025-12-28 23:25:13 +01:00

@zkutasi commented on GitHub (Nov 26, 2025):

Also, please help me how could I state in a transformation this:

If the row contains the transaction date&time (last column), use that
If it does not, use the always existing 3rd column
Since I also need to cut off the exact time from the last column, I tried to use the merge-split trick.... but it did not work out

For this row:

"Erste FőSzámla","11600006-00000000-12616657","2025.11.21","25 590","HUF","ABOUT YOU SE + CO. KGDOMSTRASSE 10","HU45137000160797601000000000","13700016-07976010-00000000","","0036510000ZV258231Z Trn: F0HO211120250231903 Oth.bank: 13700016 Oth.acct: 0797601000000000 Oth.acct.desc: ABOUT YOU SE + CO. KGDOMSTRASSE Ref.: -","F0HO211120250231903",""

I got a missing date error.

@zkutasi commented on GitHub (Nov 26, 2025): Also, please help me how could I state in a transformation this: - If the row contains the transaction date&time (last column), use that - If it does not, use the always existing 3rd column - Since I also need to cut off the exact time from the last column, I tried to use the merge-split trick.... but it did not work out For this row: ``` "Erste FőSzámla","11600006-00000000-12616657","2025.11.21","25 590","HUF","ABOUT YOU SE + CO. KGDOMSTRASSE 10","HU45137000160797601000000000","13700016-07976010-00000000","","0036510000ZV258231Z Trn: F0HO211120250231903 Oth.bank: 13700016 Oth.acct: 0797601000000000 Oth.acct.desc: ABOUT YOU SE + CO. KGDOMSTRASSE Ref.: -","F0HO211120250231903","" ``` I got a missing date error.

adam commented

2025-12-28 23:25:14 +01:00

@eitchtee commented on GitHub (Nov 26, 2025):

This is indeed a bug! Thanks for bringing it to my attention, until a fix is released you can keep doing what you're doing (duplicate the definition for date as reference_date).

Also, please help me how could I state in a transformation this:

First, you don't need to remove the time from the last column, just use a proper format for it like %Y.%m.%d %H:%M:%S. WYGIWYH should be smart enough to convert this to a date when needed.

This is a nice brain-teaser, I don't think there's an easy way to do this right now because merge will add the empty string to the result, creating 2 or 3 different possible strings that don't have matching indexes:

2025.11.21 22:27:09____
2025.11.21 22:27:09____2025.11.21
____2025.11.21

Instead you can leverage merge and regex with a lookbehind. Don't ask too much about how the dark sacred art of regex works, but this should work.

date:
    target: date
    required: true
    format: 
      - "%Y.%m.%d"
      - "%Y.%m.%d %H:%M:%S"
    transformations:
      - type: merge
        fields:
          - "Tranzakció dátuma és ideje"
          - "Könyvelés dátuma"
        separator: "____"
      - type: regex
        pattern: "(?<=^\\d{4}\\.\\d{2}\\.\\d{2} \\d{2}:\\d{2}:\\d{2})____.*|^____"
        replacement: ""
        exclusive: false

(duplicate this for the reference_date)

@eitchtee commented on GitHub (Nov 26, 2025): This is indeed a bug! Thanks for bringing it to my attention, until a fix is released you can keep doing what you're doing (duplicate the definition for date as reference_date). --- > Also, please help me how could I state in a transformation this: First, you don't need to remove the time from the last column, just use a proper format for it like `%Y.%m.%d %H:%M:%S`. WYGIWYH should be smart enough to convert this to a date when needed. This is a nice brain-teaser, I don't think there's an easy way to do this right now because merge will add the empty string to the result, creating 2 or 3 different possible strings that don't have matching indexes: ``` 2025.11.21 22:27:09____ 2025.11.21 22:27:09____2025.11.21 ____2025.11.21 ``` Instead you can leverage merge and regex with a lookbehind. Don't ask too much about how the dark sacred art of regex works, but this should work. ``` date: target: date required: true format: - "%Y.%m.%d" - "%Y.%m.%d %H:%M:%S" transformations: - type: merge fields: - "Tranzakció dátuma és ideje" - "Könyvelés dátuma" separator: "____" - type: regex pattern: "(?<=^\\d{4}\\.\\d{2}\\.\\d{2} \\d{2}:\\d{2}:\\d{2})____.*|^____" replacement: "" exclusive: false ``` _(duplicate this for the reference_date)_

adam commented

2025-12-28 23:25:14 +01:00

@eitchtee commented on GitHub (Nov 26, 2025):

A fix for this will be available on the next release (hopefully soon).

@eitchtee commented on GitHub (Nov 26, 2025): A fix for this will be available on the next release (hopefully soon).

adam referenced this issue

2025-12-28 23:26:35 +01:00

[PR #71] [MERGED] feat: tidy up transactions action bar #155

Sign in to join this conversation.