godoxy-yusing/internal/net/gphttp/middleware/README.md

internal/net/gphttp/middleware

HTTP middleware framework providing request/response processing, middleware chaining, and composition from YAML files.

Overview

This package implements a flexible HTTP middleware system for GoDoxy. Middleware can modify requests before they reach the backend and modify responses before they return to the client. The system supports:

• Request Modifiers: Process requests before forwarding
• Response Modifiers: Modify responses before returning to client
• Middleware Chaining: Compose multiple middleware in priority order
• YAML Composition: Define middleware chains in configuration files
• Bypass Rules: Skip middleware based on request properties
• Entrypoint Overlay Promotion: Promote route-local middleware entries with bypass into matching entrypoint middleware for HTTP routes
• Dynamic Loading: Load middleware definitions from files at runtime

Response body rewriting is only applied to unencoded, text-like content types (for example text/*, JSON, YAML, XML). Response status and headers can always be modified.

Architecture

graph TD
    A[HTTP Request] --> B[Middleware Chain]

    subgraph Chain [Middleware Pipeline]
        direction LR
        B1[RedirectHTTP] --> B2[RealIP]
        B2 --> B3[RateLimit]
        B3 --> B4[OIDC]
        B4 --> B5[CustomErrorPage]
    end

    Chain --> C[Backend Handler]
    C --> D[Response Modifier]

    subgraph ResponseChain [Response Pipeline]
        direction LR
        D1[CustomErrorPage] --> D2[ModifyResponse]
        D2 --> D3[ModifyHTML]
    end

    ResponseChain --> E[HTTP Response]

Middleware Flow

sequenceDiagram
    participant C as Client
    participant M as Middleware Chain
    participant B as Backend
    participant R as Response Chain
    participant C2 as Client

    C->>M: HTTP Request
    M->>M: before() - RequestModifier
    M->>M: Check Bypass Rules
    M->>M: Sort by Priority

    par Request Modifiers
        M->>M: Middleware 1 (before)
        M->>M: Middleware 2 (before)
    end

    M->>B: Forward Request

    B-->>M: HTTP Response

    par Response Modifiers
        M->>R: ResponseModifier 1
        M->>R: ResponseModifier 2
    end

    R-->>C2: Modified Response

Core Components

Middleware

type Middleware struct {
    name      string
    construct ImplNewFunc
    impl      any
    commonOptions
}

type commonOptions struct {
    Priority int    `json:"priority"` // Default: 10, 0 is highest
    Bypass   Bypass `json:"bypass"`
}

Interfaces:

// RequestModifier - modify or filter requests
type RequestModifier interface {
    before(w http.ResponseWriter, r *http.Request) (proceed bool)
}

// ResponseModifier - modify responses
type ResponseModifier interface {
    modifyResponse(r *http.Response) error
}

// MiddlewareWithSetup - one-time setup after construction
type MiddlewareWithSetup interface {
    setup()
}

// MiddlewareFinalizer - finalize after options applied
type MiddlewareFinalizer interface {
    finalize()
}

// MiddlewareFinalizerWithError - finalize with error handling
type MiddlewareFinalizerWithError interface {
    finalize() error
}

Middleware Chain

type middlewareChain struct {
    beforess  []RequestModifier
    modResps []ResponseModifier
}

func NewMiddlewareChain(name string, chain []*Middleware) *Middleware

Bypass Rules

type Bypass []rules.RuleOn

// ShouldBypass checks if request should skip middleware
func (b Bypass) ShouldBypass(w http.ResponseWriter, r *http.Request) bool

For HTTP routes, any route-local middleware entry that sets bypass and matches an existing entrypoint middleware name contributes an overlay: its bypass rules are promoted into the effective entrypoint middleware for that route.

Semantics:

• route-local middleware entries may be promoted when they include bypass; only the bypass portion is promoted in v1
• promoted rules are qualified as route <alias> & <rule>
• existing entrypoint bypass rules are preserved and the route rules are appended
• if the route-local middleware entry is bypass-only, it is consumed so the same middleware is not evaluated twice
• if the route-local middleware entry contains additional options, only the bypass portion is consumed; the rest of the route-local middleware still executes normally
• if no matching entrypoint middleware exists, route-local middleware behavior is unchanged

Example:

entrypoint:
  middlewares:
    - use: oidc

routes:
  app:
    middlewares:
      oidc:
        bypass:
          - path glob("/public/*")

Effective behavior for route app is equivalent to:

entrypoint:
  middlewares:
    - use: oidc
      bypass:
        - route app & path glob("/public/*")

Available Middleware

Name	Type	Description
redirecthttp	Request	Redirect HTTP to HTTPS
oidc	Request	OIDC authentication
forwardauth	Request	Forward authentication to external service
modifyrequest / request	Request	Modify request headers and path
modifyresponse / response	Response	Modify response headers
setxforwarded	Request	Set X-Forwarded headers
hidexforwarded	Request	Remove X-Forwarded headers
modifyhtml	Response	Inject HTML into responses
themed	Response	Apply theming to HTML
errorpage / customerrorpage	Response	Serve custom error pages
realip	Request	Extract real client IP from headers
cloudflarerealip	Request	Cloudflare-specific real IP extraction
cidrwhitelist	Request	Allow only specific IP ranges
ratelimit	Request	Rate limiting by IP
hcaptcha	Request	hCAPTCHA verification

Usage Examples

Creating a Middleware

import "github.com/yusing/godoxy/internal/net/gphttp/middleware"

type myMiddleware struct {
    SomeOption string `json:"some_option"`
}

func (m *myMiddleware) before(w http.ResponseWriter, r *http.Request) bool {
    // Process request
    r.Header.Set("X-Custom", m.SomeOption)
    return true // false would block the request
}

var MyMiddleware = middleware.NewMiddleware[myMiddleware]()

Building Middleware from Map

middlewaresMap := map[string]middleware.OptionsRaw{
    "realip": {
        "priority": 5,
        "header":   "X-Real-IP",
        "from":     []string{"10.0.0.0/8"},
    },
    "ratelimit": {
        "priority": 10,
        "average":  10,
        "burst":    20,
    },
}

mid, err := middleware.BuildMiddlewareFromMap("my-chain", middlewaresMap)
if err != nil {
    log.Fatal(err)
}

YAML Composition

# config/middlewares/my-chain.yml
- use: realip
  header: X-Real-IP
  from:
    - 10.0.0.0/8
    - 172.16.0.0/12
  bypass:
    - path glob("/public/*")

- use: ratelimit
  average: 100
  burst: 200

- use: oidc
  allowed_users:
    - user@example.com

// Load from file
eb := &gperr.Builder{}
middlewares := middleware.BuildMiddlewaresFromComposeFile(
    "config/middlewares/my-chain.yml",
    eb,
)

Applying Middleware to Reverse Proxy

import "github.com/yusing/goutils/http/reverseproxy"

rp := &reverseproxy.ReverseProxy{
    Target: backendURL,
}

err := middleware.PatchReverseProxy(rp, middlewaresMap)
if err != nil {
    log.Fatal(err)
}

PatchReverseProxy still handles route-local middleware in the normal way. Entrypoint overlay promotion happens earlier, at entrypoint request dispatch time, where the server has both the resolved route and the raw entrypoint middleware definitions available.

Bypass Rules

bypassRules := middleware.Bypass{
    {
        Type:  rules.RuleOnTypePathPrefix,
        Value: "/public",
    },
    {
        Type:  rules.RuleOnTypePath,
        Value: "/health",
    },
}

mid, _ := middleware.RateLimiter.New(middleware.OptionsRaw{
    "bypass": bypassRules,
    "average": 10,
    "burst":  20,
})

Priority

Middleware are executed in priority order (lower number = higher priority):

graph LR
    A[Priority 0] --> B[Priority 5]
    B --> C[Priority 10]
    C --> D[Priority 20]

    style A fill:#14532d,stroke:#fff,color:#fff
    style B fill:#14532d,stroke:#fff,color:#fff
    style C fill:#44403c,stroke:#fff,color:#fff
    style D fill:#44403c,stroke:#fff,color:#fff

Request Processing

flowchart TD
    A[Request] --> B{Has Bypass Rules?}
    B -->|Yes| C{Match Bypass?}
    B -->|No| D[Execute before#40;#41;]

    C -->|Match| E[Skip Middleware<br/>Proceed to Next]
    C -->|No Match| D

    D --> F{before#40;#41; Returns?}
    F -->|true| G[Continue to Next]
    F -->|false| H[Stop Pipeline]

    G --> I[Backend Handler]
    I --> J[Response]
    J --> K{Has Response Modifier?}
    K -->|Yes| L[Execute modifyResponse]
    K -->|No| M[Return Response]
    L --> M

Integration Points

• Error Pages: Uses errorpage package for custom error responses
• Authentication: Integrates with internal/auth for OIDC
• Rate Limiting: Uses golang.org/x/time/rate
• IP Processing: Uses internal/net/types for CIDR handling

Error Handling

Errors during middleware construction are collected and reported:

var errs gperr.Builder
for name, opts := range middlewaresMap {
    m, err := middleware.Get(name)
    if err != nil {
        errs.Add(err)
        continue
    }
    mid, err := m.New(opts)
    if err != nil {
        errs.AddSubjectf(err, "middlewares.%s", name)
        continue
    }
}
if errs.HasError() {
    log.Error().Err(errs.Error()).Msg("middleware compilation failed")
}