From 5e333254ba384a17a10ddbf8b73a50e986e386b5 Mon Sep 17 00:00:00 2001 From: Simone Scarduzio Date: Fri, 10 Oct 2025 10:12:25 +0200 Subject: [PATCH] docs: Comprehensive environment variable documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Added complete documentation for all environment variables across Dockerfile, README.md, and SDK documentation. Dockerfile Changes: - Documented all DeltaGlider environment variables with defaults - Added AWS configuration variables (commented for runtime override) - Updated version label to 5.0.3 - Updated description to mention encryption README.md Changes: - Added comprehensive Docker Usage section - Documented all environment variables with examples - Added Docker examples for: * Basic usage with AWS credentials * Memory cache configuration for CI/CD * MinIO/custom endpoint usage * Persistent encryption key setup - Security notes for encryption and cache behavior SDK Documentation Changes: - Added DeltaGlider Configuration section - Documented all environment variables - Added configuration examples - Security notes for encryption behavior Environment Variables Documented: - DG_LOG_LEVEL (logging configuration) - DG_MAX_RATIO (compression threshold) - DG_CACHE_BACKEND (filesystem or memory) - DG_CACHE_MEMORY_SIZE_MB (memory cache size) - DG_CACHE_ENCRYPTION_KEY (optional persistent key) - AWS_ENDPOINT_URL (custom S3 endpoints) - AWS_ACCESS_KEY_ID (AWS credentials) - AWS_SECRET_ACCESS_KEY (AWS credentials) - AWS_DEFAULT_REGION (AWS region) Quality Checks: - All 119 tests passing ✅ - Type checking: 0 errors (mypy) ✅ - Linting: All checks passed (ruff) ✅ - Dockerfile syntax validated ✅ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- Dockerfile | 22 +++++++++++++-- README.md | 54 +++++++++++++++++++++++++++++++++++++ docs/sdk/getting-started.md | 32 ++++++++++++++++++++++ 3 files changed, 106 insertions(+), 2 deletions(-) diff --git a/Dockerfile b/Dockerfile index 1011895..6b18aac 100644 --- a/Dockerfile +++ b/Dockerfile @@ -66,10 +66,28 @@ USER deltaglider HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \ CMD deltaglider --help || exit 1 +# Environment variables (all optional, can be overridden at runtime) +# Logging +ENV DG_LOG_LEVEL=INFO + +# Performance & Compression +ENV DG_MAX_RATIO=0.5 + +# Cache Configuration +ENV DG_CACHE_BACKEND=filesystem +ENV DG_CACHE_MEMORY_SIZE_MB=100 +# ENV DG_CACHE_ENCRYPTION_KEY= # Optional: Set for cross-process cache sharing + +# AWS Configuration (override at runtime) +# ENV AWS_ENDPOINT_URL=https://s3.amazonaws.com +# ENV AWS_ACCESS_KEY_ID= +# ENV AWS_SECRET_ACCESS_KEY= +# ENV AWS_DEFAULT_REGION=us-east-1 + # Labels LABEL org.opencontainers.image.title="DeltaGlider" \ - org.opencontainers.image.description="Delta-aware S3 file storage wrapper" \ - org.opencontainers.image.version="0.1.0" \ + org.opencontainers.image.description="Delta-aware S3 file storage wrapper with encryption" \ + org.opencontainers.image.version="5.0.3" \ org.opencontainers.image.authors="Beshu Limited" \ org.opencontainers.image.source="https://github.com/beshu-tech/deltaglider" diff --git a/README.md b/README.md index 3270809..90cc23a 100644 --- a/README.md +++ b/README.md @@ -46,6 +46,60 @@ uv pip install deltaglider docker run -v ~/.aws:/root/.aws deltaglider/deltaglider --help ``` +### Docker Usage + +DeltaGlider provides a secure, production-ready Docker image with encryption always enabled: + +```bash +# Basic usage with AWS credentials from environment +docker run -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY \ + deltaglider/deltaglider ls s3://my-bucket/ + +# Mount AWS credentials +docker run -v ~/.aws:/root/.aws:ro \ + deltaglider/deltaglider cp file.zip s3://releases/ + +# Use memory cache for ephemeral CI/CD pipelines (faster) +docker run -e DG_CACHE_BACKEND=memory \ + -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY \ + deltaglider/deltaglider sync ./dist/ s3://releases/v1.0.0/ + +# Configure memory cache size (default: 100MB) +docker run -e DG_CACHE_BACKEND=memory \ + -e DG_CACHE_MEMORY_SIZE_MB=500 \ + -e AWS_ACCESS_KEY_ID -e AWS_SECRET_ACCESS_KEY \ + deltaglider/deltaglider cp large-file.zip s3://releases/ + +# Use MinIO or custom S3 endpoint +docker run -e AWS_ENDPOINT_URL=http://minio:9000 \ + -e AWS_ACCESS_KEY_ID=minioadmin \ + -e AWS_SECRET_ACCESS_KEY=minioadmin \ + deltaglider/deltaglider ls s3://test-bucket/ + +# Persistent encryption key for cross-container cache sharing +# (Only needed if sharing cache across containers via volume mount) +docker run -v /shared-cache:/tmp/.deltaglider \ + -e DG_CACHE_ENCRYPTION_KEY=$(openssl rand -base64 32) \ + deltaglider/deltaglider cp file.zip s3://releases/ +``` + +**Environment Variables**: +- `DG_LOG_LEVEL`: Logging level (default: `INFO`, options: `DEBUG`, `INFO`, `WARNING`, `ERROR`) +- `DG_MAX_RATIO`: Maximum delta/file ratio (default: `0.5`, range: `0.0-1.0`) +- `DG_CACHE_BACKEND`: Cache backend (default: `filesystem`, options: `filesystem`, `memory`) +- `DG_CACHE_MEMORY_SIZE_MB`: Memory cache size in MB (default: `100`) +- `DG_CACHE_ENCRYPTION_KEY`: Optional base64-encoded encryption key for cross-process cache sharing +- `AWS_ENDPOINT_URL`: S3 endpoint URL (default: AWS S3) +- `AWS_ACCESS_KEY_ID`: AWS access key +- `AWS_SECRET_ACCESS_KEY`: AWS secret key +- `AWS_DEFAULT_REGION`: AWS region (default: `us-east-1`) + +**Security Notes**: +- Encryption is **always enabled** (cannot be disabled) +- Each container gets ephemeral encryption keys for maximum security +- Corrupted cache files are automatically deleted +- Use `DG_CACHE_ENCRYPTION_KEY` only for persistent cache sharing (store securely) + ### Basic Usage ```bash diff --git a/docs/sdk/getting-started.md b/docs/sdk/getting-started.md index 785f807..0e8c5fa 100644 --- a/docs/sdk/getting-started.md +++ b/docs/sdk/getting-started.md @@ -69,6 +69,38 @@ Or via environment variable: export AWS_ENDPOINT_URL=http://minio.local:9000 ``` +### DeltaGlider Configuration + +DeltaGlider supports the following environment variables: + +**Logging & Performance**: +- `DG_LOG_LEVEL`: Logging level (default: `INFO`, options: `DEBUG`, `INFO`, `WARNING`, `ERROR`) +- `DG_MAX_RATIO`: Maximum delta/file ratio (default: `0.5`, range: `0.0-1.0`) + +**Cache Configuration**: +- `DG_CACHE_BACKEND`: Cache backend type (default: `filesystem`, options: `filesystem`, `memory`) +- `DG_CACHE_MEMORY_SIZE_MB`: Memory cache size in MB (default: `100`) +- `DG_CACHE_ENCRYPTION_KEY`: Optional base64-encoded Fernet key for persistent encryption + +**Security**: +- Encryption is **always enabled** (cannot be disabled) +- Ephemeral encryption keys per process (forward secrecy) +- Corrupted cache files automatically deleted +- Set `DG_CACHE_ENCRYPTION_KEY` only for cross-process cache sharing + +**Example**: +```bash +# Use memory cache for faster performance in CI/CD +export DG_CACHE_BACKEND=memory +export DG_CACHE_MEMORY_SIZE_MB=500 + +# Enable debug logging +export DG_LOG_LEVEL=DEBUG + +# Adjust delta compression threshold +export DG_MAX_RATIO=0.3 # More aggressive compression +``` + ## Your First Upload ### Basic Example