godoxy-yusing/internal/serialization

Serialization Package

A Go package for flexible, type-safe serialization/deserialization with validation support. It provides robust handling of YAML/JSON input, environment variable substitution, and field-level validation with case-insensitive matching.

Architecture Overview

---
config:
    theme: redux-dark-color
---
flowchart TB
    subgraph Input Processing
        YAML[YAML Bytes] --> EnvSub[Env Substitution]
        EnvSub --> YAMLParse[YAML Parse]
        YAMLParse --> Map[map<string,any>]
    end

    subgraph Type Inspection
        Map --> TypeInfo[Type Info Cache]
        TypeInfo -.-> FieldLookup[Field Lookup]
    end

    subgraph Conversion
        FieldLookup --> Convert[Convert Function]
        Convert --> StringConvert[String Conversion]
        Convert --> NumericConvert[Numeric Conversion]
        Convert --> MapConvert[Map/Struct Conversion]
        Convert --> SliceConvert[Slice Conversion]
    end

    subgraph Validation
        Convert --> Validate[ValidateWithFieldTags]
        Convert --> CustomValidate[Custom Validator]
        CustomValidate --> CustomValidator[CustomValidator Interface]
    end

    subgraph Output
        Validate --> Result[Typed Struct/Map]
    end

File Structure

File	Purpose
serialization.go	Core serialization/deserialization logic
validation.go	Field tag validation and custom validator support
time.go	Duration unit extensions (d, w, M)
serialization_test.go	Core functionality tests
validation_*_test.go	Validation-specific tests

Core Types

type SerializedObject = map[string]any

The SerializedObject is the intermediate representation used throughout deserialization.

Interfaces

// For custom map unmarshaling logic
type MapUnmarshaller interface {
    UnmarshalMap(m map[string]any) gperr.Error
}

// For custom validation logic
type CustomValidator interface {
    Validate() gperr.Error
}

Key Features

1. Case-Insensitive Field Matching

Fields are matched using FNV-1a hash with case-insensitive comparison:

type Config struct {
    AuthToken string `json:"auth_token"`
}

// Matches: "auth_token", "AUTH_TOKEN", "AuthToken", "Auth_Token"

2. Field Tags

type Config struct {
    Name    string `json:"name"`           // JSON/deserialize field name
    Port    int    `validate:"required"`   // Validation tag
    Secret  string `json:"-"`              // Exclude from deserialization
    Token   string `aliases:"key,api_key"` // Aliases for matching
}

Tag	Purpose
json	Field name for serialization; - to exclude
deserialize	Explicit deserialize name; - to exclude
validate	go-playground/validator tags
aliases	Comma-separated alternative field names

3. Environment Variable Substitution

Supports ${VAR} syntax with prefix-aware lookup:

autocert:
  auth_token: ${CLOUDFLARE_AUTH_TOKEN}

Prefix resolution order: GODOXY_VAR, GOPROXY_VAR, VAR

4. String Conversions

Converts strings to various types:

// Duration: "1h30m", "2d" (d=day, w=week, M=month)
ConvertString("2d", reflect.ValueOf(&duration))

// Numeric: "123", "0xFF"
ConvertString("123", reflect.ValueOf(&intVal))

// Slice: "a,b,c" or YAML list format
ConvertString("a,b,c", reflect.ValueOf(&slice))

// Map/Struct: YAML format
ConvertString("key: value", reflect.ValueOf(&mapVal))

5. Custom Convertor Pattern

Types can implement a Parse method for custom string conversion:

type Duration struct {
    Value int
    Unit  string
}

func (d *Duration) Parse(v string) error {
    // custom parsing logic
}

Main Functions

Deserialization

// YAML with validation
func UnmarshalValidateYAML[T any](data []byte, target *T) gperr.Error

// YAML with interceptor
func UnmarshalValidateYAMLIntercept[T any](
    data []byte,
    target *T,
    intercept func(m map[string]any) gperr.Error,
) gperr.Error

// Direct map deserialization
func MapUnmarshalValidate(src SerializedObject, dst any) gperr.Error

// To xsync.Map
func UnmarshalValidateYAMLXSync[V any](data []byte) (*xsync.Map[string, V], gperr.Error)

Conversion

// Convert any value to target reflect.Value
func Convert(src reflect.Value, dst reflect.Value, checkValidateTag bool) gperr.Error

// String to target type
func ConvertString(src string, dst reflect.Value) (convertible bool, convErr gperr.Error)

Validation

// Validate using struct tags
func ValidateWithFieldTags(s any) gperr.Error

// Register custom validator
func MustRegisterValidation(tag string, fn validator.Func)

// Validate using CustomValidator interface
func ValidateWithCustomValidator(v reflect.Value) gperr.Error

Default Values

// Register factory for default values
func RegisterDefaultValueFactory[T any](factory func() *T)

Usage Example

package main

import (
    "os"
    "github.com/yusing/godoxy/internal/serialization"
)

type ServerConfig struct {
    Host        string `json:"host" validate:"required,hostname_port"`
    Port        int    `json:"port" validate:"required,min=1,max=65535"`
    MaxConns    int    `json:"max_conns"`
    TLSEnabled  bool   `json:"tls_enabled"`
}

func main() {
    yamlData := []byte(`
host: localhost
port: 8080
max_conns: 100
tls_enabled: true
`)

    var config ServerConfig
    if err := serialization.UnmarshalValidateYAML(yamlData, &config); err != nil {
        panic(err)
    }
    // config is now populated and validated
}

Deserialization Flow

sequenceDiagram
    participant C as Caller
    participant U as UnmarshalValidateYAML
    participant E as Env Substitution
    participant Y as YAML Parser
    participant M as MapUnmarshalValidate
    participant T as Type Info Cache
    participant CV as Convert
    participant V as Validator

    C->>U: YAML bytes + target struct
    U->>E: Substitute ${ENV} vars
    E-->>U: Substituted bytes
    U->>Y: Parse YAML
    Y-->>U: map[string]any
    U->>M: Map + target
    M->>T: Get type info
    loop For each field in map
        M->>T: Lookup field by name (case-insensitive)
        T-->>M: Field reflect.Value
        M->>CV: Convert value to field type
        CV-->>M: Converted value or error
    end
    M->>V: Validate struct tags
    V-->>M: Validation errors
    M-->>U: Combined errors
    U-->>C: Result

Error Handling

Errors use gperr (goutils error package) with structured error subjects:

// Unknown field
ErrUnknownField.Subject("field_name").With(gperr.DoYouMeanField("field_name", ["fieldName"]))

// Validation error
ErrValidationError.Subject("Namespace").Withf("required")

// Unsupported conversion
ErrUnsupportedConversion.Subjectf("string to int")

Performance Optimizations

1. Type Info Caching: Uses xsync.Map to cache field metadata per type
2. Hash-based Lookup: FNV-1a hash for O(1) field matching
3. Lazy Pointer Init: Pointers initialized only when first set
4. Presized Collections: Initial capacity hints for maps/slices

Testing

go test ./internal/serialization/... -v

Test categories:

• Basic deserialization
• Anonymous struct handling
• Pointer primitives
• String conversions
• Environment substitution
• Custom validators