starred/godoxy-yusing
1
0
Fork 0
mirror of https://github.com/yusing/godoxy.git synced 2026-04-21 16:01:22 +02:00
Code Issues 18 Packages Projects Releases 10 Wiki Activity

docs: add per package README for implementation details (AI generated with human review)

Browse Source
This commit is contained in:
yusing
2026-01-08 23:39:19 +08:00
parent 13441286d1
commit e9d7edef12
54 changed files with 13431 additions and 1519 deletions
Download Patch File Download Diff File Expand all files Collapse all files

325
internal/route/README.md Normal file
View File

@@ -0,0 +1,325 @@
# Route
Provides HTTP routing, reverse proxy, file serving, and TCP/UDP stream proxying for GoDoxy.
## Overview
The `internal/route` package implements the core routing system for GoDoxy. It handles reverse proxying HTTP requests, serving static files, and proxying TCP/UDP streams. Routes can be discovered from Docker containers, YAML files, or remote agents.
### Primary Consumers
- **Route providers**: Create and manage route instances
- **HTTP server**: Dispatches requests to route handlers
- **Configuration layer**: Validates and loads route configs
### Non-goals
- Does not implement container runtime operations (delegates to providers)
- Does not handle authentication (delegates to middleware/rules)
- Does not manage health checks (delegates to `internal/health/monitor`)
### Stability
Internal package with stable core types. Route configuration schema is versioned.
## Public API
### Exported Types
```go
type Route struct {
Alias string // Unique route identifier
Scheme Scheme // http, https, h2c, tcp, udp, fileserver
Host string // Virtual host
Port Port // Listen and target ports
// File serving
Root string // Document root
SPA bool // Single-page app mode
Index string // Index file
// Route rules and middleware
HTTPConfig
PathPatterns []string
Rules rules.Rules
RuleFile string
// Health and load balancing
HealthCheck types.HealthCheckConfig
LoadBalance *types.LoadBalancerConfig
// Additional features
Middlewares map[string]types.LabelMap
Homepage *homepage.ItemConfig
AccessLog *accesslog.RequestLoggerConfig
Agent string
Idlewatcher *types.IdlewatcherConfig
Metadata
}
```
```go
type Scheme string
const (
SchemeHTTP Scheme = "http"
SchemeHTTPS Scheme = "https"
SchemeH2C Scheme = "h2c"
SchemeTCP Scheme = "tcp"
SchemeUDP Scheme = "udp"
SchemeFileServer Scheme = "fileserver"
)
```
```go
type ExcludedReason int
const (
ExcludedReasonNone ExcludedReason = iota
ExcludedReasonError
ExcludedReasonManual
ExcludedReasonNoPortContainer
ExcludedReasonNoPortSpecified
ExcludedReasonBlacklisted
ExcludedReasonBuildx
ExcludedReasonOld
)
```
### Exported Functions/Methods
```go
// Validation and lifecycle
func (r *Route) Validate() gperr.Error
func (r *Route) Start(parent task.Parent) gperr.Error
func (r *Route) Finish(reason any)
func (r *Route) Started() <-chan struct{}
// Route queries
func (r *Route) Impl() types.Route
func (r *Route) Task() *task.Task
func (r *Route) ProviderName() string
func (r *Route) TargetURL() *nettypes.URL
func (r *Route) References() []string
// Status queries
func (r *Route) ShouldExclude() bool
func (r *Route) UseLoadBalance() bool
func (r *Route) UseIdleWatcher() bool
func (r *Route) UseHealthCheck() bool
```
## Architecture
### Core Components
```mermaid
classDiagram
class Route {
+Validate() gperr.Error
+Start(parent) gperr.Error
+Finish(reason)
+Started() &lt;-chan struct#123;#125;
}
class Metadata {
+impl types.Route
+task *task.Task
+started chan struct#123;#125;
}
class HealthMonitor {
+Start(parent) error
+Healthy() bool
+URL() string
}
Route --> Metadata : contains
Route --> HealthMonitor : optional
```
### Route Types
```mermaid
graph TD
Route --> HTTPRoute
Route --> StreamRoute
HTTPRoute --> ReverseProxyRoute
HTTPRoute --> FileServer
StreamRoute --> TCPStream
StreamRoute --> UDPStream
```
### Request Processing Pipeline
```mermaid
flowchart LR
A[Request] --> B[Route Matching]
B --> C{Route Type}
C -->|HTTP| D[Middleware]
C -->|FileServer| E[File System]
C -->|Stream| F[TCP/UDP Proxy]
D --> G[Rules Engine]
G --> H[Upstream]
H --> I[Response]
E --> I
F --> I
```
### Reverse Proxy Flow
```mermaid
sequenceDiagram
participant C as Client
participant P as Proxy
participant L as Load Balancer
participant B as Backend
C->>P: GET /
P->>L: Select Backend
L-->>P: Backend1
P->>B: Forward Request
B-->>P: 200 OK
P-->>C: Response
```
## Configuration Surface
### Route Configuration
```go
type Route struct {
Alias string `json:"alias"`
Scheme Scheme `json:"scheme"`
Host string `json:"host,omitempty"`
Port Port `json:"port"`
Root string `json:"root,omitempty"`
SPA bool `json:"spa,omitempty"`
Index string `json:"index,omitempty"`
// ... additional fields
}
```
### Docker Labels
```yaml
labels:
proxy.aliases: myapp
proxy.myapp.port: 3000
```
### YAML Configuration
```yaml
routes:
myapp:
scheme: http
root: /var/www/myapp
spa: true
```
## Dependency and Integration Map
| Dependency | Purpose |
| -------------------------------- | -------------------------------- |
| `internal/route/routes` | Route registry and lookup |
| `internal/route/rules` | Request/response rule processing |
| `internal/route/stream` | TCP/UDP stream proxying |
| `internal/route/provider` | Route discovery and loading |
| `internal/health/monitor` | Health checking |
| `internal/idlewatcher` | Idle container management |
| `internal/logging/accesslog` | Request logging |
| `internal/homepage` | Dashboard integration |
| `github.com/yusing/goutils/errs` | Error handling |
## Observability
### Logs
- **INFO**: Route start/stop, validation results
- **DEBUG**: Request processing details
- **ERROR**: Proxy failures, health check failures
Log context includes: `alias`, `host`, `method`, `path`, `status`
### Metrics
Health check metrics via `internal/health/monitor`:
- `health_check_total`
- `health_check_failure_total`
- `health_check_duration_seconds`
## Security Considerations
- Route matching validates host and path patterns
- Upstream URL validation prevents SSRF attacks
- Rules engine can enforce authentication/authorization
- ACL integration available for IP-based access control
## Failure Modes and Recovery
| Failure | Behavior | Recovery |
| ------------------------- | -------------------------- | --------------------------- |
| Backend unavailable | Returns 502 error | Fix backend service |
| Health check fails | Route marked unhealthy | Fix backend health endpoint |
| Route validation fails | Route excluded with reason | Fix configuration |
| TLS handshake fails | Connection error | Fix certificates |
| Load balancer no backends | Returns 503 error | Add healthy backends |
## Usage Examples
### Creating a Basic HTTP Route
```go
route := &route.Route{
Alias: "myapp",
Scheme: route.SchemeHTTP,
Host: "myapp.local",
Port: route.Port{Proxy: 8080, Target: 3000},
}
if err := route.Validate(); err != nil {
return err
}
if err := route.Start(parent); err != nil {
return err
}
```
### Route with Health Check
```go
route := &route.Route{
Alias: "myservice",
HealthCheck: types.HealthCheckConfig{
Path: "/health",
Interval: 30 * time.Second,
Timeout: 5 * time.Second,
},
}
```
### File Server Route
```go
route := &route.Route{
Alias: "files",
Scheme: route.SchemeFileServer,
Root: "/var/www/files",
SPA: false,
Index: "index.html",
}
```
## Testing Notes
- Unit tests for validation logic
- Integration tests with real backends
- Mock health monitors for testing
- Route exclusion tests cover all reason codes

318
internal/route/provider/README.md Normal file
View File

@@ -0,0 +1,318 @@
# Route Provider
Discovers and loads routes from Docker containers, YAML files, and remote agents.
## Overview
The `internal/route/provider` package implements route discovery and loading for GoDoxy. It supports multiple provider types (Docker, File, Agent) and manages route lifecycle including validation, start/stop, and event handling.
### Primary Consumers
- **Main entry point**: Creates providers during startup
- **Route layer**: Provides route instances to registry
- **Watcher system**: Receives container/config change events
### Non-goals
- Does not implement route execution (handled by route package)
- Does not manage global configuration
- Does not provide authentication
### Stability
Internal package with stable provider interface.
## Public API
### Exported Types
```go
type Provider struct {
ProviderImpl
t provider.Type
routes route.Routes
routesMu sync.RWMutex
watcher W.Watcher
}
type ProviderImpl interface {
fmt.Stringer
ShortName() string
IsExplicitOnly() bool
loadRoutesImpl() (route.Routes, gperr.Error)
NewWatcher() W.Watcher
Logger() *zerolog.Logger
}
```
### Exported Functions
```go
// Create a file-based provider
func NewFileProvider(filename string) (p *Provider, err error)
// Create a Docker-based provider
func NewDockerProvider(name string, dockerCfg types.DockerProviderConfig) *Provider
// Create an agent-based provider
func NewAgentProvider(cfg *agent.AgentConfig) *Provider
```
### Provider Methods
```go
func (p *Provider) GetType() provider.Type
func (p *Provider) Start(parent task.Parent) gperr.Error
func (p *Provider) LoadRoutes() gperr.Error
func (p *Provider) IterRoutes(yield func(string, types.Route) bool)
func (p *Provider) GetRoute(alias string) (types.Route, bool)
func (p *Provider) FindService(project, service string) (types.Route, bool)
```
## Architecture
### Core Components
```mermaid
classDiagram
class Provider {
+ProviderImpl
+t provider.Type
+routes route.Routes
+watcher W.Watcher
+Start(parent) gperr.Error
+LoadRoutes() gperr.Error
+IterRoutes(yield)
}
class ProviderImpl {
<<interface>>
+String() string
+ShortName() string
+IsExplicitOnly() bool
+loadRoutesImpl() (route.Routes, gperr.Error)
+NewWatcher() W.Watcher
+Logger() *zerolog.Logger
}
class DockerProviderImpl {
+name string
+dockerCfg types.DockerProviderConfig
+ShortName() string
+loadRoutesImpl() (route.Routes, gperr.Error)
}
class FileProviderImpl {
+filename string
+ShortName() string
+loadRoutesImpl() (route.Routes, gperr.Error)
}
class AgentProviderImpl {
+*agent.AgentConfig
+docker DockerProviderImpl
+ShortName() string
+loadRoutesImpl() (route.Routes, gperr.Error)
}
Provider --> ProviderImpl : wraps
ProviderImpl <|-- DockerProviderImpl
ProviderImpl <|-- FileProviderImpl
ProviderImpl <|-- AgentProviderImpl
```
### Provider Types
```mermaid
graph TD
A[Provider] --> B{Docker}
A --> C{File}
A --> D{Agent}
B --> E[DockerWatcher]
C --> F[ConfigFileWatcher]
D --> G[DockerWatcher]
E --> H[Container Labels]
F --> I[YAML Files]
G --> J[Remote Agent]
```
### Route Loading Flow
```mermaid
sequenceDiagram
participant Main as Application
participant Provider as Provider
participant Impl as ProviderImpl
participant Watcher as Watcher
participant Routes as Route Registry
Main->>Provider: Start()
Provider->>Provider: LoadRoutes()
Provider->>Impl: loadRoutesImpl()
Impl-->>Provider: Routes
Provider->>Routes: Validate & Add Routes
loop Event Loop
Watcher->>Provider: Container/Config Changed
Provider->>Provider: Reload Routes
Provider->>Impl: loadRoutesImpl()
Provider->>Routes: Update Registry
end
```
### Docker Provider Features
- Lists all containers and reads their labels
- Supports Docker Compose project/service discovery
- Handles host network mode with port detection
- Supports alias references (`#1`, `#2` for container aliases)
- Explicit-only mode for providers ending with `!`
### File Provider Features
- Reads YAML files from the config directory
- Validates routes on load
- Supports embedded presets
### Agent Provider Features
- Connects to a remote agent via Unix socket or TCP
- Delegates to a Docker provider internally
- Supports the same Docker label-based route discovery
## Configuration Surface
### Docker Provider Labels
```yaml
labels:
proxy.aliases: app1,app2
proxy.app1.listen: http://0.0.0.0:8080
proxy.app1.target: http://app:3000
proxy.app1.rules: |
- name: default
do: pass
```
### File Provider Configuration
```yaml
# config/routes/myapp.yml
routes:
myapp:
target: http://localhost:8080
rules:
- name: default
do: pass
```
### Agent Provider Configuration
```yaml
agents:
remote:
socket: /run/godoxy-agent.sock
name: remote-agent
```
## Dependency and Integration Map
| Dependency | Purpose |
| -------------------------------- | -------------------------- |
| `internal/route` | Route types and validation |
| `internal/route/routes` | Route registry |
| `internal/docker` | Docker API integration |
| `internal/serialization` | YAML parsing |
| `internal/watcher` | Container/config watching |
| `internal/watcher/events` | Event queue handling |
| `agent/pkg/agent` | Agent configuration |
| `github.com/yusing/goutils/errs` | Error handling |
## Observability
### Logs
- **INFO**: Provider start/stop, route count
- **DEBUG**: Route loading details, label parsing
- **ERROR**: Container fetch errors, parse failures
Log context includes: `provider`, `alias`, `route_count`
### Metrics
- `routes_loaded_total` by provider
- `provider_events_total` by type
## Security Considerations
- Docker provider requires socket access
- Agent provider uses Unix socket or TCP with auth
- Route validation prevents SSRF via URL validation
- Container labels are validated before use
## Failure Modes and Recovery
| Failure | Behavior | Recovery |
| ------------------------- | ---------------------------- | ----------------------- |
| Docker socket unavailable | Provider fails to start | Fix socket permissions |
| Container not found | Route excluded with error | Verify container exists |
| YAML parse error | Route excluded, error logged | Fix configuration file |
| Agent connection lost | Routes removed, reconnection | Fix agent connectivity |
| Watcher error | Provider finishes with error | Check watcher logs |
## Usage Examples
### Creating a Docker Provider
```go
provider := provider.NewDockerProvider("default", types.DockerProviderConfig{
URL: "unix:///var/run/docker.sock",
})
if err := provider.LoadRoutes(); err != nil {
return err
}
if err := provider.Start(parentTask); err != nil {
return err
}
```
### Creating a File Provider
```go
provider, err := provider.NewFileProvider("routes/myapp.yml")
if err != nil {
return err
}
if err := provider.Start(parentTask); err != nil {
return err
}
```
### Iterating Over Routes
```go
for alias, r := range provider.IterRoutes {
log.Printf("Route: %s -> %s", alias, r.Name())
}
```
### Finding a Service
```go
route, ok := provider.FindService("myproject", "myservice")
if ok {
log.Printf("Found service: %s", route.Name())
}
```
## Testing Notes
- Docker provider tests use test containers
- File provider tests use temp directories
- Agent provider tests use mock agents
- Integration tests cover event handling

269
internal/route/routes/README.md Normal file
View File

@@ -0,0 +1,269 @@
# Route Registry
Provides centralized route registry with O(1) lookups and route context management for HTTP handlers.
## Overview
The `internal/route/routes` package maintains the global route registry for GoDoxy. It provides thread-safe route lookups by alias, route iteration, and utilities for propagating route context through HTTP request handlers.
### Primary Consumers
- **HTTP handlers**: Lookup routes and extract request context
- **Route providers**: Register and unregister routes
- **Health system**: Query route health status
- **WebUI**: Display route information
### Non-goals
- Does not create or modify routes
- Does not handle route validation
- Does not implement routing logic (matching)
### Stability
Internal package with stable public API.
## Public API
### Route Pools
```go
var (
HTTP = pool.New[types.HTTPRoute]("http_routes")
Stream = pool.New[types.StreamRoute]("stream_routes")
Excluded = pool.New[types.Route]("excluded_routes")
)
```
Pool methods:
- `Get(alias string) (T, bool)` - O(1) lookup
- `Add(r T)` - Register route
- `Del(r T)` - Unregister route
- `Size() int` - Route count
- `Clear()` - Remove all routes
- `Iter` - Channel-based iteration
### Exported Functions
```go
// Iterate over active routes (HTTP + Stream)
func IterActive(yield func(r types.Route) bool)
// Iterate over all routes (HTTP + Stream + Excluded)
func IterAll(yield func(r types.Route) bool)
// Get route count
func NumActiveRoutes() int
func NumAllRoutes() int
// Clear all routes
func Clear()
// Lookup functions
func Get(alias string) (types.Route, bool)
func GetHTTPRouteOrExact(alias, host string) (types.HTTPRoute, bool)
```
### Route Context
```go
type RouteContext struct {
context.Context
Route types.HTTPRoute
}
// Attach route to request context (uses unsafe pointer for performance)
func WithRouteContext(r *http.Request, route types.HTTPRoute) *http.Request
// Extract route from request context
func TryGetRoute(r *http.Request) types.HTTPRoute
```
### Upstream Information
```go
func TryGetUpstreamName(r *http.Request) string
func TryGetUpstreamScheme(r *http.Request) string
func TryGetUpstreamHost(r *http.Request) string
func TryGetUpstreamPort(r *http.Request) string
func TryGetUpstreamHostPort(r *http.Request) string
func TryGetUpstreamAddr(r *http.Request) string
func TryGetUpstreamURL(r *http.Request) string
```
### Health Information
```go
type HealthInfo struct {
HealthInfoWithoutDetail
Detail string
}
type HealthInfoWithoutDetail struct {
Status types.HealthStatus
Uptime time.Duration
Latency time.Duration
}
func GetHealthInfo() map[string]HealthInfo
func GetHealthInfoWithoutDetail() map[string]HealthInfoWithoutDetail
func GetHealthInfoSimple() map[string]types.HealthStatus
```
### Provider Grouping
```go
func ByProvider() map[string][]types.Route
```
## Architecture
### Core Components
```mermaid
classDiagram
class HTTP
class Stream
class Excluded
class RouteContext
HTTP : +Get(alias) T
HTTP : +Add(r)
HTTP : +Del(r)
HTTP : +Size() int
HTTP : +Iter chan
Stream : +Get(alias) T
Stream : +Add(r)
Stream : +Del(r)
Excluded : +Get(alias) T
Excluded : +Add(r)
Excluded : +Del(r)
```
### Route Lookup Flow
```mermaid
flowchart TD
A[Lookup Request] --> B{HTTP Pool}
B -->|Found| C[Return Route]
B -->|Not Found| D{Stream Pool}
D -->|Found| C
D -->|Not Found| E[Return nil]
```
### Context Propagation
```mermaid
sequenceDiagram
participant H as HTTP Handler
participant R as Registry
participant C as RouteContext
H->>R: WithRouteContext(req, route)
R->>C: Attach route via unsafe pointer
C-->>H: Modified request
H->>R: TryGetRoute(req)
R->>C: Extract route from context
C-->>R: Route
R-->>H: Route
```
## Dependency and Integration Map
| Dependency | Purpose |
| -------------------------------- | --------------------------------- |
| `internal/types` | Route and health type definitions |
| `github.com/yusing/goutils/pool` | Thread-safe pool implementation |
## Observability
### Logs
Registry operations logged at DEBUG level:
- Route add/remove
- Pool iteration
- Context operations
### Performance
- `WithRouteContext` uses `unsafe.Pointer` to avoid request cloning
- Route lookups are O(1) using internal maps
- Iteration uses channels for memory efficiency
## Security Considerations
- Route context propagation is internal to the process
- No sensitive data exposed in context keys
- Routes are validated before registration
## Failure Modes and Recovery
| Failure | Behavior | Recovery |
| ---------------------------------------- | ------------------------------ | -------------------- |
| Route not found | Returns (nil, false) | Verify route alias |
| Context extraction on non-route request | Returns nil | Check request origin |
| Concurrent modification during iteration | Handled by pool implementation | N/A |
## Usage Examples
### Basic Route Lookup
```go
route, ok := routes.Get("myapp")
if !ok {
return fmt.Errorf("route not found")
}
```
### Iterating Over All Routes
```go
for r := range routes.IterActive {
log.Printf("Route: %s", r.Name())
}
```
### Getting Health Status
```go
healthMap := routes.GetHealthInfo()
for name, health := range healthMap {
log.Printf("Route %s: %s (uptime: %v)", name, health.Status, health.Uptime)
}
```
### Using Route Context in Handler
```go
func MyHandler(w http.ResponseWriter, r *http.Request) {
route := routes.TryGetRoute(r)
if route == nil {
http.Error(w, "Route not found", http.StatusNotFound)
return
}
upstreamHost := routes.TryGetUpstreamHost(r)
log.Printf("Proxying to: %s", upstreamHost)
}
```
### Grouping Routes by Provider
```go
byProvider := routes.ByProvider()
for providerName, routeList := range byProvider {
log.Printf("Provider %s: %d routes", providerName, len(routeList))
}
```
## Testing Notes
- Unit tests for pool thread safety
- Context propagation tests
- Health info aggregation tests
- Provider grouping tests

360
internal/route/rules/README.md Normal file
View File

@@ -0,0 +1,360 @@
# Route Rules
Implements a rule engine for HTTP request/response processing, enabling conditional routing, header manipulation, authentication, and more.
## Overview
The `internal/route/rules` package provides a powerful rule engine for GoDoxy. Rules allow conditional processing of HTTP requests and responses based on various matchers (headers, path, method, IP, etc.). Matching rules can modify requests, route to different backends, or terminate processing.
### Primary Consumers
- **Route layer**: Applies rules during request processing
- **Configuration system**: Parses rule YAML
- **Middleware integration**: Extends rule capabilities
### Non-goals
- Does not implement proxy transport (delegates to reverse proxy)
- Does not handle TLS/SSL (handled at entrypoint)
- Does not perform health checking
### Stability
Internal package with stable YAML schema. Backward-compatible additions to rule types are allowed.
## Public API
### Exported Types
```go
type Rules []Rule
type Rule struct {
Name string // Rule identifier for debugging
On RuleOn // Condition matcher
Do Command // Action to execute
}
type RuleOn struct {
raw string
checker Checker
isResponseChecker bool
}
type Command struct {
raw string
exec CommandHandler
isResponseHandler bool
}
```
### Exported Functions
```go
// BuildHandler converts rules to an HTTP handler
func (rules Rules) BuildHandler(up http.HandlerFunc) http.HandlerFunc
// ParseRules parses rule configuration
func ParseRules(config string) (Rules, error)
// ValidateRules validates rule syntax
func ValidateRules(config string) error
```
## Architecture
### Core Components
```mermaid
classDiagram
class Rules {
+BuildHandler(up) http.HandlerFunc
}
class Rule {
+Name string
+On RuleOn
+Do Command
+IsResponseRule() bool
}
class RuleOn {
+raw string
+checker Checker
+isResponseChecker bool
}
class Command {
+raw string
+exec CommandHandler
+isResponseHandler bool
}
class Checker {
<<interface>>
+Check(r *http.Request) bool
+CheckResponse(w ResponseWriter, r *http.Request) bool
}
class CommandHandler {
<<interface>>
+Execute(w ResponseWriter, r *http.Request, rm *ResponseModifier) gperr.Error
}
Rules --> Rule
Rule --> RuleOn
Rule --> Command
RuleOn --> Checker
Command --> CommandHandler
```
### Request Processing Flow
```mermaid
sequenceDiagram
participant Req as Request
participant Pre as Pre Rules
participant Proxy as Upstream
participant Post as Post Rules
Req->>Pre: Check pre-rules
alt Rule matches
Pre->>Pre: Execute handler
alt Terminating action
Pre-->>Req: Response
Return-->>Req: Return immediately
end
end
Req->>Proxy: Forward request
Proxy-->>Req: Response
Req->>Post: Check post-rules
Post->>Post: Execute handlers
Post-->>Req: Modified response
```
### Condition Matchers
| Matcher | Type | Description |
| ------------- | -------- | ---------------------------- |
| `header` | Request | Match request header value |
| `query` | Request | Match query parameter |
| `cookie` | Request | Match cookie value |
| `form` | Request | Match form field |
| `method` | Request | Match HTTP method |
| `host` | Request | Match virtual host |
| `path` | Request | Match request path |
| `proto` | Request | Match protocol (http/https) |
| `remote` | Request | Match remote IP/CIDR |
| `basic_auth` | Request | Match basic auth credentials |
| `route` | Request | Match route name |
| `resp_header` | Response | Match response header |
| `status` | Response | Match status code range |
### Matcher Types
```sh
# String: exact match (default)
# Glob: shell-style wildcards (*, ?)
# Regex: regular expressions
path /api/users // exact match
path glob("/api/*") // glob pattern
path regex("/api/v[0-9]+/.*") // regex pattern
```
### Actions
**Terminating Actions** (stop processing):
| Command | Description |
| ------------------------ | ---------------------- |
| `error <code> <message>` | Return HTTP error |
| `redirect <url>` | Redirect to URL |
| `serve <path>` | Serve local files |
| `route <name>` | Route to another route |
| `proxy <url>` | Proxy to upstream |
**Non-Terminating Actions** (modify and continue):
| Command | Description |
| ------------------------------ | ---------------------- |
| `pass` / `bypass` | Pass through unchanged |
| `rewrite <from> <to>` | Rewrite request path |
| `require_auth` | Require authentication |
| `require_basic_auth <realm>` | Basic auth challenge |
| `set <target> <field> <value>` | Set header/variable |
| `add <target> <field> <value>` | Add header/variable |
| `remove <target> <field>` | Remove header/variable |
**Response Actions**:
| Command | Description |
| ------------------------------------------ | ----------------- |
| `log <level> <path> <template>` | Log response |
| `notify <level> <provider> <title> <body>` | Send notification |
## Configuration Surface
### Rule Configuration (YAML)
```yaml
rules:
- name: rule name
on: |
condition1
& condition2
do: |
action1
action2
```
### Condition Syntax
```yaml
# Simple condition
on: path /api/users
# Multiple conditions (AND)
on: |
header Authorization Bearer
& path /api/admin/*
# Negation
on: !path /public/*
# OR within a line
on: method GET | method POST
```
### Variable Substitution
```go
// Static variables
$req_method // Request method
$req_host // Request host
$req_path // Request path
$status_code // Response status
$remote_host // Client IP
// Dynamic variables
$header(Name) // Request header
$header(Name, index) // Header at index
$arg(Name) // Query argument
$form(Name) // Form field
// Environment variables
${ENV_VAR}
```
## Dependency and Integration Map
| Dependency | Purpose |
| ---------------------------- | ------------------------ |
| `internal/route` | Route type definitions |
| `internal/auth` | Authentication handlers |
| `internal/acl` | IP-based access control |
| `internal/notif` | Notification integration |
| `internal/logging/accesslog` | Response logging |
| `pkg/gperr` | Error handling |
| `golang.org/x/net/http2` | HTTP/2 support |
## Observability
### Logs
- **DEBUG**: Rule matching details, variable substitution
- **INFO**: Rule execution, terminating actions
- **ERROR**: Rule parse errors, execution failures
Log context includes: `rule`, `alias`, `match_result`
## Security Considerations
- `require_auth` enforces authentication
- `remote` matcher supports IP/CIDR for access control
- Variables are sanitized to prevent injection
- Path rewrites are validated to prevent traversal
## Failure Modes and Recovery
| Failure | Behavior | Recovery |
| ------------------- | ------------------------- | ---------------------------------- |
| Invalid rule syntax | Route validation fails | Fix YAML syntax |
| Missing variables | Variable renders as empty | Check variable sources |
| Rule timeout | Request times out | Increase timeout or simplify rules |
| Auth failure | Returns 401/403 | Fix credentials |
## Usage Examples
### Basic Pass-Through
```yaml
- name: default
do: pass
```
### Path-Based Routing
```yaml
- name: api proxy
on: path /api/*
do: proxy http://api-backend:8080
- name: static files
on: path /static/*
do: serve /var/www/static
```
### Authentication
```yaml
- name: admin protection
on: path /admin/*
do: require_auth
- name: basic auth for API
on: path /api/*
do: require_basic_auth "API Access"
```
### Path Rewriting
```yaml
- name: rewrite API v1
on: path /v1/*
do: |
rewrite /v1 /api/v1
proxy http://backend:8080
```
### IP-Based Access Control
```yaml
- name: allow internal
on: remote 10.0.0.0/8
do: pass
- name: block external
on: |
!remote 10.0.0.0/8
!remote 192.168.0.0/16
do: error 403 "Access Denied"
```
### WebSocket Support
```yaml
- name: websocket upgrade
on: |
header Connection Upgrade
header Upgrade websocket
do: bypass
```
## Testing Notes
- Unit tests for all matchers and actions
- Integration tests with real HTTP requests
- Parser tests for YAML syntax
- Variable substitution tests
- Performance benchmarks for hot paths

202
internal/route/rules/presets/README.md Normal file
View File

@@ -0,0 +1,202 @@
# Rule Presets
Provides embedded, pre-configured rule sets for common routing patterns.
## Overview
The `internal/route/rules/presets` package provides embedded YAML rule configurations that can be reused across routes. Presets are compiled into the binary and loaded at runtime via `sync.Once` initialization.
### Primary Consumers
- **Route configuration**: Applies preset rules to routes
- **WebUI**: Provides default rules for web applications
- **API gateway**: Common patterns for API routes
### Non-goals
- Does not modify presets at runtime
- Does not provide dynamic preset loading
- Does not support preset inheritance/overriding
### Stability
Internal package. Preset content is stable but may change between versions.
## Public API
### Exported Functions
```go
// GetRulePreset retrieves a preset by name
func GetRulePreset(name string) (rules.Rules, bool)
```
**Contract:**
- Uses `sync.Once` for one-time initialization
- Returns a copy of the preset rules
- Second return value indicates if preset exists
## Architecture
### Core Components
```mermaid
classDiagram
class GetRulePreset {
<<function>>
+sync.Once
+rulePresets map
}
class RulePreset {
<<embedded>>
+fs.FS
}
class webui {
<<preset>>
+Login page rule
+Protected routes rule
+API proxy rule
+Auth proxy rule
}
GetRulePreset --> RulePreset : loads
RulePreset --> webui : contains
```
### Preset Loading Flow
```mermaid
sequenceDiagram
participant App as Application
participant Once as sync.Once
participant FS as Embedded FS
participant Parser as YAML Parser
participant Map as rulePresets
App->>Once: GetRulePreset("webui")
Once->>Once: once.Do(initPresets)
Once->>FS: ReadDir("presets/")
FS-->>Once: List files
loop Each file
Once->>FS: ReadFile(filename)
FS-->>Once: YAML content
Once->>Parser: Parse YAML
Parser-->>Once: rules.Rules
Once->>Map: Store in map
end
Map-->>Once: Loaded presets
Once-->>App: Rules copy
```
## Preset Files
### Directory Structure
```
internal/route/rules/presets/
├── embed.go //go:embed *.yml
├── webui.yml // WebUI preset
└── README.md
```
### webui Preset
The default preset for GoDoxy WebUI:
**What it does:**
1. Allows unauthenticated access to `/login`
1. Requires auth for most paths, excluding static assets and auth endpoints
1. Proxies `/api/v1/*` to backend
1. Rewrites and proxies `/auth/*` to backend
## Dependency and Integration Map
| Dependency | Purpose |
| ------------------------ | ------------------------------- |
| `internal/route/rules` | Rules engine for preset content |
| `internal/serialization` | YAML parsing |
| `sync` | One-time initialization |
## Observability
### Logs
- DEBUG: Preset loading errors
- WARN: Missing preset files
## Security Considerations
- Presets are compiled into binary (immutable at runtime)
- Environment variable substitution (`${VAR}`) supports secure configuration
- No runtime preset modification possible
## Failure Modes and Recovery
| Failure | Behavior | Recovery |
| ------------------- | -------------------- | --------------------- |
| Preset file missing | Returns (nil, false) | Check preset exists |
| YAML parse error | Panic during init | Fix preset YAML |
| Unknown preset name | Returns (nil, false) | Use valid preset name |
## Usage Examples
### Loading a Preset
```go
import "github.com/yusing/godoxy/internal/route/rules/presets"
rules, ok := presets.GetRulePreset("webui")
if !ok {
return fmt.Errorf("preset not found")
}
// Apply rules to a handler
handler := rules.BuildHandler(upstreamHandler)
```
### Creating a Custom Preset
```yaml
# internal/route/rules/presets/api-gateway.yml
- name: cors headers
on: method OPTIONS
do: |
set header Access-Control-Allow-Origin *
set header Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS"
error 204 "No Content"
- name: auth required
on: !path /health
do: require_auth
```
Then load it:
```go
rules, ok := presets.GetRulePreset("api-gateway")
```
### Using Preset in Route Config
```yaml
routes:
webapp:
target: http://localhost:3000
rules: |
- name: include webui preset
do: include webui
- name: additional rule
on: path /custom/*
do: proxy http://custom:8080
```
## Limitations
- Presets are read-only after initialization
- No runtime preset modification
- All presets loaded at first access (no lazy loading)
- No preset merging (caller must handle)

306
internal/route/stream/README.md Normal file
View File

@@ -0,0 +1,306 @@
# Stream Handling
Implements TCP and UDP stream proxying for non-HTTP protocols.
## Overview
The `internal/route/stream` package provides protocol-agnostic proxying of TCP and UDP connections. It enables GoDoxy to handle protocols like SSH, DNS, game servers, and other binary protocols that don't use HTTP.
### Primary Consumers
- **Route layer**: Creates stream routes for TCP/UDP schemes
- **Entry point**: Mounts stream listeners
- **ACL system**: Applies access control to listeners
### Non-goals
- Does not implement HTTP/1.1 or HTTP/2 (handled by reverse proxy)
- Does not handle WebSocket (handled by rules engine)
- Does not provide protocol-specific parsing
### Stability
Internal package with stable `nettypes.Stream` interface.
## Public API
### Exported Types
```go
type TCPTCPStream struct {
network string
listener net.Listener
laddr *net.TCPAddr
dst *net.TCPAddr
preDial nettypes.HookFunc
onRead nettypes.HookFunc
closed atomic.Bool
}
type UDPUDPStream struct {
network string
listener net.PacketConn
laddr *net.UDPAddr
dst *net.TCPAddr
cleanUpTicker *time.Ticker
conns map[string]*udpUDPConn
closed atomic.Bool
mu sync.Mutex
}
```
### Exported Functions
```go
// Create a TCP stream
func NewTCPTCPStream(network, listenAddr, dstAddr string) (nettypes.Stream, error)
// Create a UDP stream
func NewUDPUDPStream(network, listenAddr, dstAddr string) (nettypes.Stream, error)
```
### Stream Interface
```go
type Stream interface {
ListenAndServe(ctx context.Context, preDial, onRead HookFunc)
Close() error
LocalAddr() net.Addr
zerolog.LogObjectMarshaler
}
type HookFunc func(ctx context.Context) error
```
## Architecture
### Core Components
```mermaid
classDiagram
class Stream {
<<interface>>
+ListenAndServe(ctx, preDial, onRead)
+Close() error
+LocalAddr() net.Addr
}
class TCPTCPStream {
+listener net.Listener
+laddr *net.TCPAddr
+dst *net.TCPAddr
+ListenAndServe(ctx, preDial, onRead)
+pipe(conn1, conn2)
}
class UDPUDPStream {
+listener net.PacketConn
+laddr *net.TCPAddr
+dst *net.TCPAddr
+conns map
+ListenAndServe(ctx, preDial, onRead)
+handleUDP()
}
class udpUDPConn {
+srcAddr *net.UDPAddr
+dstConn *net.UDPConn
+lastUsed atomic.Time
+close()
}
Stream <|-- TCPTCPStream
Stream <|-- UDPUDPStream
UDPUDPStream --> udpUDPConn : manages
```
### TCP Stream Flow
```mermaid
sequenceDiagram
participant C as Client
participant L as Listener
participant P as Pipe
participant S as Server
C->>L: TCP Connection
L->>P: Accept connection
P->>S: Dial TCP
S-->>P: Connection established
loop Data Transfer
C->>P: Data
P->>S: Forward data
S->>P: Response
P->>C: Forward response
end
```
### UDP Stream Flow
```mermaid
sequenceDiagram
participant C as Client
participant L as Listener
participant M as Connection Manager
participant S as Server
C->>L: UDP Datagram
L->>M: Get/Create connection
alt New Connection
M->>S: Dial UDP
S-->>M: Connection ready
M->>C: Forward initial packet
else Existing Connection
M->>C: Forward packet
end
loop Response Handler
S->>M: Response
M->>C: Forward response
end
```
### Constants
```go
const (
udpBufferSize = 16 * 1024 // 16KB buffer
udpIdleTimeout = 5 * time.Minute
udpCleanupInterval = 1 * time.Minute
udpReadTimeout = 30 * time.Second
)
```
## Configuration Surface
### Route Configuration
```yaml
routes:
ssh-proxy:
scheme: tcp4
bind: 0.0.0.0 # optional
port: 2222:22 # listening port: target port
dns-proxy:
scheme: udp4
bind: 0.0.0.0 # optional
port: 53:53 # listening port: target port
```
### Docker Labels
```yaml
services:
ssh:
image: alpine/ssh
labels:
proxy.aliases: ssh
proxy.ssh.port: 2222:22 # listening port: target port
```
## Dependency and Integration Map
| Dependency | Purpose |
| -------------------------------- | ---------------------------- |
| `internal/acl` | Access control for listeners |
| `internal/entrypoint` | Proxy protocol support |
| `internal/net/types` | Stream interface definitions |
| `github.com/pires/go-proxyproto` | PROXY protocol header |
| `github.com/yusing/goutils/errs` | Error handling |
## Observability
### Logs
- **INFO**: Stream start/stop, connection accepted
- **DEBUG**: Data transfer, connection details
- **ERROR**: Accept failures, pipe errors
Log context includes: `protocol`, `listen`, `dst`, `action`
## Security Considerations
- ACL wrapping available for TCP and UDP listeners
- PROXY protocol support for original client IP
- No protocol validation (relies on upstream)
- Connection limits managed by OS
## Failure Modes and Recovery
| Failure | Behavior | Recovery |
| ---------------- | --------------------- | ----------------------- |
| Bind fails | Stream creation error | Check port availability |
| Dial fails | Connection error | Fix target address |
| Pipe broken | Connection closed | Client reconnects |
| UDP idle timeout | Connection removed | Client reconnects |
## Usage Examples
### Creating a TCP Stream Route
```go
baseRoute := &route.Route{
LisURL: &url.URL{Scheme: "tcp4", Host: ":2222"},
ProxyURL: &url.URL{Scheme: "tcp", Host: "localhost:22"},
}
streamRoute, err := route.NewStreamRoute(baseRoute)
if err != nil {
return err
}
```
### Programmatic Stream Creation
```go
tcpStream, err := stream.NewTCPTCPStream("tcp", ":8080", "localhost:22")
if err != nil {
return err
}
tcpStream.ListenAndServe(ctx, preDialHook, onReadHook)
```
### Using Hook Functions
```go
stream.ListenAndServe(ctx,
func(ctx context.Context) error {
// Pre-dial: authentication, rate limiting
log.Println("Pre-dial check")
return nil
},
func(ctx context.Context) error {
// On-read: metrics, throttling
return nil
},
)
```
### ACL Integration
```go
stream := tcpStream
if acl := acl.ActiveConfig.Load(); acl != nil {
stream.listener = acl.WrapTCP(stream.listener)
// or for UDP
stream.listener = acl.WrapUDP(stream.listener)
}
```
## Performance Considerations
- **TCP**: Bidirectional pipe with goroutines per connection
- **UDP**: 16KB buffer with sized pool
- **Cleanup**: Periodic cleanup of idle UDP connections
- **Concurrency**: Each connection handled independently
## Limitations
- Load balancing not yet supported
- Coherent scheme required:
- `tcp4`/`tcp6` -> `tcp`
- `udp4`/`udp6` -> `udp`
- No UDP broadcast/multicast support