The accesslog package captures HTTP request/response information and writes it to files or stdout. It includes configurable log formats, filtering rules, and automatic log rotation with retention policies.

Primary Consumers

internal/route - Middleware for logging proxied requests
internal/acl - ACL decision logging
internal/api - Request audit trails

Non-goals

Does not provide log parsing or analysis
Does not implement log aggregation across services
Does not provide log shipping to external systems
Does not implement access control (use internal/acl)

Stability

Internal package. Public interfaces are stable.

Public API

Exported Types

AccessLogger Interface

type AccessLogger interface {
    // Log records an HTTP request and response
    Log(req *http.Request, res *http.Response)

    // LogError logs a request with an error status code
    LogError(req *http.Request, err error)

    // LogACL logs an ACL block/allow event
    LogACL(info *maxmind.IPInfo, blocked bool)

    // Config returns the logger configuration
    Config() *Config

    // Flush forces any buffered log data to be written
    Flush()

    // Close closes the logger and releases resources
    Close() error
}

Main interface for logging HTTP requests and ACL events.

Writer Interface

type Writer interface {
    io.WriteCloser
    ShouldBeBuffered() bool
    Name() string // file name or path
}

Output destination interface.

Format Type

type Format string

const (
    FormatCommon   Format = "common"
    FormatCombined Format = "combined"
    FormatJSON     Format = "json"
)

Log format constants.

Configuration Types

RequestLoggerConfig

type RequestLoggerConfig struct {
    ConfigBase
    Format  Format  `json:"format" validate:"oneof=common combined json"`
    Filters Filters `json:"filters"`
    Fields  Fields  `json:"fields"`
}

Configuration for request/response logging.

ACLLoggerConfig

type ACLLoggerConfig struct {
    ConfigBase
    LogAllowed bool `json:"log_allowed"`
}

Configuration for ACL event logging.

ConfigBase

type ConfigBase struct {
    B              int           `json:"buffer_size"` // Deprecated: buffer size is adjusted dynamically
    Path           string        `json:"path"`
    Stdout         bool          `json:"stdout"`
    Retention      *Retention    `json:"retention" aliases:"keep"`
    RotateInterval time.Duration `json:"rotate_interval,omitempty" swaggertype:"primitive,integer"`
}

Common configuration for all loggers.

Filters

type Filters struct {
    StatusCodes LogFilter[*StatusCodeRange] `json:"status_codes"`
    Method      LogFilter[HTTPMethod]       `json:"method"`
    Host        LogFilter[Host]             `json:"host"`
    Headers     LogFilter[*HTTPHeader]      `json:"headers"`
    CIDR        LogFilter[*CIDR]            `json:"cidr"`
}

Filtering rules for what to log.

Fields

type Fields struct {
    Headers FieldConfig `json:"headers" aliases:"header"`
    Query   FieldConfig `json:"query" aliases:"queries"`
    Cookies FieldConfig `json:"cookies" aliases:"cookie"`
}

Field configuration for what data to include.

Exported Functions

Constructor

func NewAccessLogger(parent task.Parent, cfg AnyConfig) (AccessLogger, error)
func NewMockAccessLogger(parent task.Parent, cfg *RequestLoggerConfig) AccessLogger
func NewAccessLoggerWithIO(parent task.Parent, writer Writer, anyCfg AnyConfig) AccessLogger

Create access loggers from configurations.

Default Configurations

func DefaultRequestLoggerConfig() *RequestLoggerConfig
func DefaultACLLoggerConfig() *ACLLoggerConfig

Returns default configurations.

Architecture

Core Components

graph TD
    subgraph Request Flow
        Req[HTTP Request] -->|Passed to| Log[AccessLogger.Log]
        Res[HTTP Response] -->|Passed to| Log
        Log -->|Formats| Fmt[RequestFormatter]
        Fmt -->|Writes to| Writer[BufferedWriter]
        Writer -->|Outputs to| Output[File/Stdout]
    end

    subgraph Background Tasks
        Rotator[Rotation Task] -->|Triggers| Rotate[ShouldRotate]
        Adjuster[Buffer Adjuster] -->|Adjusts| Buffer[Buffer Size]
    end

Component	Responsibility
`AccessLogger`	Main logging interface
`RequestFormatter`	Formats request/response logs
`ACLFormatter`	Formats ACL decision logs
`Writer`	Output destination (file/stdout)
`BufferedWriter`	Efficient I/O with dynamic buffering

Log Flow

sequenceDiagram
    participant Request
    participant AccessLogger
    participant Formatter
    participant BufferedWriter
    participant File

    Request->>AccessLogger: Log(req, res)
    AccessLogger->>AccessLogger: shouldLog() filter check
    alt Passes filters
        AccessLogger->>Formatter: AppendRequestLog(line, req, res)
        Formatter->>AccessLogger: Formatted line
        AccessLogger->>BufferedWriter: Write(line)
        BufferedWriter->>BufferedWriter: Buffer if needed
        BufferedWriter->>File: Flush when full/rotating
    else Fails filters
        AccessLogger->>Request: Skip logging
    end

Buffer Management

The logger dynamically adjusts buffer size based on write throughput:

Parameter	Value
Initial Buffer Size	4 KB
Maximum Buffer Size	8 MB
Adjustment Interval	5 seconds

Buffer size adjustment formula:

newBufSize = origBufSize +/- step
step = max(|wps - origBufSize|/2, wps/2)

Rotation Logic

stateDiagram-v2
    [*] --> Logging
    Logging --> Logging: Normal writes
    Logging --> Rotating: Interval reached
    Rotating --> Logging: New file created
    Rotating --> [*]: Logger closed

Rotation checks:

Is rotation enabled (supportRotate + valid retention)?
Is retention period valid?
Create new file with timestamp suffix
Delete old files beyond retention

Log Formats

Common Format

127.0.0.1 - - [10/Jan/2024:12:00:00 +0000] "GET /api HTTP/1.1" 200 1234

Combined Format

127.0.0.1 - - [10/Jan/2024:12:00:00 +0000] "GET /api HTTP/1.1" 200 1234 "https://example.com" "Mozilla/5.0"

JSON Format

{
  "level": "info",
  "time": "10/Jan/2024:12:00:00 +0000",
  "ip": "127.0.0.1",
  "method": "GET",
  "scheme": "http",
  "host": "example.com",
  "path": "/api",
  "protocol": "HTTP/1.1",
  "status": 200,
  "type": "application/json",
  "size": 1234,
  "referer": "https://example.com",
  "useragent": "Mozilla/5.0"
}

Configuration Surface

YAML Configuration

access_log:
  path: /var/log/godoxy/access.log
  stdout: false
  rotate_interval: 1h
  retention:
    days: 30
  format: combined
  filters:
    status_codes:
      keep:
        - min: 200
          max: 599
    method:
      keep:
        - GET
        - POST
    headers:
      keep:
        - name: Authorization

Configuration Fields

Field	Type	Default	Description
`path`	string	-	Log file path
`stdout`	bool	false	Also log to stdout
`rotate_interval`	duration	1h	Rotation interval
`retention.days`	int	30	Days to retain logs
`format`	string	combined	Log format
`filters.status_codes`	range[]	all	Status code filter
`filters.method`	string[]	all	HTTP method filter
`filters.cidr`	CIDR[]	none	IP range filter

Reloading

Configuration is fixed at construction time. Create a new logger to apply changes.

Dependency and Integration Map

Internal Dependencies

Package	Purpose
`internal/maxmind/types`	IP geolocation for ACL logs
`internal/serialization`	Default value factory registration

External Dependencies

Dependency	Purpose
`github.com/rs/zerolog`	JSON formatting and logging
`github.com/yusing/goutils/task`	Lifetime management
`github.com/puzpuzpuz/xsync/v4`	Concurrent map operations
`golang.org/x/time/rate`	Error rate limiting

Observability

Logs

Level	When
Debug	Buffer size adjustments
Info	Log file rotation
Error	Write failures (rate limited)

Metrics

None exposed directly. Write throughput tracked internally.

Security Considerations

Log files should have appropriate permissions (644)
Sensitive headers can be filtered via Filters.Headers
Query parameters and cookies are configurable via Fields
Rate limiting prevents error log flooding

Failure Modes and Recovery

Failure	Detection	Recovery
Write error	`Write()` returns error	Rate-limited logging, then task finish
File deleted while open	Write failure	Logger continues with error
Disk full	Write failure	Error logged, may terminate
Rotation error	`Rotate()` returns error	Continue with current file

Error Rate Limiting

const (
    errRateLimit = 200 * time.Millisecond
    errBurst     = 5
)

Errors are rate-limited to prevent log flooding. After burst exceeded, task is finished.

Usage Examples

Basic Request Logger

import "github.com/yusing/godoxy/internal/logging/accesslog"

cfg := accesslog.DefaultRequestLoggerConfig()
cfg.Path = "/var/log/godoxy/access.log"
cfg.RotateInterval = time.Hour
cfg.Retention = &accesslog.Retention{Days: 30}

logger, err := accesslog.NewAccessLogger(parent, cfg)
if err != nil {
    log.Fatal(err)
}
defer logger.Close()

// Log a request
logger.Log(req, res)

JSON Format with Filters

cfg := accesslog.RequestLoggerConfig{
    ConfigBase: accesslog.ConfigBase{
        Path:      "/var/log/godoxy/requests.json.log",
        Retention: &accesslog.Retention{Days: 7},
    },
    Format: accesslog.FormatJSON,
    Filters: accesslog.Filters{
        StatusCodes: accesslog.LogFilter[*accesslog.StatusCodeRange]{
            Keep: []accesslog.StatusCodeRange{{Min: 400, Max: 599}},
        },
    },
}

logger := accesslog.NewAccessLogger(parent, &cfg)

ACL Logger

aclCfg := accesslog.DefaultACLLoggerConfig()
aclCfg.Path = "/var/log/godoxy/acl.log"
aclCfg.LogAllowed = false // Only log blocked requests

aclLogger, err := accesslog.NewAccessLogger(parent, aclCfg)
if err != nil {
    log.Fatal(err)
}

// Log ACL decision
aclLogger.LogACL(ipInfo, true) // blocked
aclLogger.LogACL(ipInfo, false) // allowed (if LogAllowed is true)

Custom Writer

type customWriter struct {
    *os.File
}

func (w *customWriter) ShouldBeBuffered() bool { return true }
func (w *customWriter) Name() string          { return "custom" }

writer := &customWriter{File: myFile}
logger := accesslog.NewAccessLoggerWithIO(parent, writer, cfg)

Integration with Route Middleware

func accessLogMiddleware(logger accesslog.AccessLogger) gin.HandlerFunc {
    return func(c *gin.Context) {
        c.Next()
        logger.Log(c.Request, c.Writer.Result())
    }
}

Performance Characteristics

Buffered writes reduce I/O operations
Dynamic buffer sizing adapts to throughput
Per-writer locks allow parallel writes to different files
Byte pools reduce GC pressure
Efficient log rotation with back scanning

Testing Notes

NewMockAccessLogger for testing without file I/O
Mock file implementation via NewMockFile
Filter tests verify predicate logic
Rotation tests verify retention cleanup

internal/route - Route middleware integration
internal/acl - ACL decision logging
internal/maxmind - IP geolocation for ACL logs

README.md

Access Logging

Overview