deltaglider/docs/DOCKER.md

Docker Support for DeltaGlider

This document describes how to build, run, and publish Docker images for DeltaGlider.

Quick Start

Pull and run the latest image

docker pull beshultd/deltaglider:latest
docker run --rm beshultd/deltaglider:latest --help

Run with AWS credentials

docker run --rm \
  -e AWS_ACCESS_KEY_ID=your_key \
  -e AWS_SECRET_ACCESS_KEY=your_secret \
  -e AWS_DEFAULT_REGION=us-east-1 \
  beshultd/deltaglider:latest ls s3://your-bucket/

Run with MinIO (local S3 alternative)

# Start MinIO
docker run -d \
  -p 9000:9000 -p 9001:9001 \
  -e MINIO_ROOT_USER=minioadmin \
  -e MINIO_ROOT_PASSWORD=minioadmin \
  --name minio \
  minio/minio server /data --console-address ":9001"

# Use DeltaGlider with MinIO
docker run --rm \
  -e AWS_ENDPOINT_URL=http://host.docker.internal:9000 \
  -e AWS_ACCESS_KEY_ID=minioadmin \
  -e AWS_SECRET_ACCESS_KEY=minioadmin \
  -e AWS_DEFAULT_REGION=us-east-1 \
  beshultd/deltaglider:latest ls

Building Locally

Build with current git version

VERSION=$(git describe --tags --always --abbrev=0 | sed 's/^v//')
docker build --build-arg VERSION=${VERSION} -t beshultd/deltaglider:${VERSION} .

Build with custom version

docker build --build-arg VERSION=6.0.2 -t beshultd/deltaglider:6.0.2 .

Multi-platform build

# Create a buildx builder (one-time setup)
docker buildx create --name deltaglider-builder --use

# Build for multiple platforms
docker buildx build \
  --platform linux/amd64,linux/arm64 \
  --build-arg VERSION=6.0.2 \
  -t beshultd/deltaglider:6.0.2 \
  --push \
  .

Testing the Image

Basic functionality test

# Check version
docker run --rm beshultd/deltaglider:test --version

# Check help
docker run --rm beshultd/deltaglider:test --help

# List available commands
docker run --rm beshultd/deltaglider:test

Integration test with MinIO

# 1. Start MinIO
docker run -d \
  -p 9000:9000 -p 9001:9001 \
  -e MINIO_ROOT_USER=minioadmin \
  -e MINIO_ROOT_PASSWORD=minioadmin \
  --name minio \
  minio/minio server /data --console-address ":9001"

# 2. Create a test file
echo "Hello DeltaGlider" > test.txt

# 3. Upload to S3/MinIO
docker run --rm \
  -v $(pwd):/data \
  -w /data \
  -e AWS_ENDPOINT_URL=http://host.docker.internal:9000 \
  -e AWS_ACCESS_KEY_ID=minioadmin \
  -e AWS_SECRET_ACCESS_KEY=minioadmin \
  -e AWS_DEFAULT_REGION=us-east-1 \
  beshultd/deltaglider:test cp test.txt s3://test-bucket/

# 4. List bucket contents
docker run --rm \
  -e AWS_ENDPOINT_URL=http://host.docker.internal:9000 \
  -e AWS_ACCESS_KEY_ID=minioadmin \
  -e AWS_SECRET_ACCESS_KEY=minioadmin \
  -e AWS_DEFAULT_REGION=us-east-1 \
  beshultd/deltaglider:test ls s3://test-bucket/

# 5. Get statistics
docker run --rm \
  -e AWS_ENDPOINT_URL=http://host.docker.internal:9000 \
  -e AWS_ACCESS_KEY_ID=minioadmin \
  -e AWS_SECRET_ACCESS_KEY=minioadmin \
  -e AWS_DEFAULT_REGION=us-east-1 \
  beshultd/deltaglider:test stats test-bucket

# 6. Cleanup
docker stop minio && docker rm minio
rm test.txt

Publishing to Docker Hub

Manual Publishing

# 1. Log in to Docker Hub
docker login

# 2. Build the image
VERSION=$(git describe --tags --always --abbrev=0 | sed 's/^v//')
docker build --build-arg VERSION=${VERSION} \
  -t beshultd/deltaglider:${VERSION} \
  -t beshultd/deltaglider:latest \
  .

# 3. Push to Docker Hub
docker push beshultd/deltaglider:${VERSION}
docker push beshultd/deltaglider:latest

Multi-platform Publishing

# Create builder (one-time setup)
docker buildx create --name deltaglider-builder --use

# Build and push for multiple platforms
VERSION=$(git describe --tags --always --abbrev=0 | sed 's/^v//')
docker buildx build \
  --platform linux/amd64,linux/arm64 \
  --build-arg VERSION=${VERSION} \
  -t beshultd/deltaglider:${VERSION} \
  -t beshultd/deltaglider:latest \
  --push \
  .

GitHub Actions Automation

The repository includes a GitHub Action workflow (.github/workflows/docker-publish.yml) that automatically builds and publishes Docker images.

Automatic Publishing Triggers

• On main branch push: Tags as latest
• On develop branch push: Tags as develop
• On version tag push (e.g., v6.0.2): Tags with semver patterns:
  • 6.0.2 (full version)
  • 6.0 (major.minor)
  • 6 (major)
• On pull request: Builds but doesn't push (testing only)

Required GitHub Secrets

Set these secrets in your GitHub repository settings (Settings > Secrets and variables > Actions):

1. DOCKERHUB_USERNAME: Your Docker Hub username (e.g., beshultd)
2. DOCKERHUB_TOKEN: Docker Hub access token (create at https://hub.docker.com/settings/security)

Manual Workflow Trigger

You can manually trigger the Docker build workflow from the GitHub Actions tab:

1. Go to Actions tab
2. Select Build and Publish Docker Images
3. Click Run workflow
4. Select branch and click Run workflow

Docker Image Details

Image Layers

The Dockerfile uses a multi-stage build:

1. Builder stage: Installs UV and Python dependencies
2. Runtime stage: Minimal Python 3.12-slim with only runtime dependencies

Image Features

• Size: ~150MB (compressed)
• Platforms: linux/amd64, linux/arm64
• User: Runs as non-root user deltaglider (UID 1000)
• Base: Python 3.12-slim (Debian)
• Dependencies:
  • Python 3.12
  • xdelta3 (binary diff tool)
  • All Python dependencies from pyproject.toml

Environment Variables

The image supports the following environment variables:

# Logging
DG_LOG_LEVEL=INFO              # DEBUG, INFO, WARNING, ERROR

# Performance & Compression
DG_MAX_RATIO=0.5               # Max delta/file ratio (0.0-1.0)

# Cache Configuration
DG_CACHE_BACKEND=filesystem    # filesystem or memory
DG_CACHE_MEMORY_SIZE_MB=100    # Memory cache size
DG_CACHE_ENCRYPTION_KEY=       # Optional encryption key

# AWS Configuration
AWS_ENDPOINT_URL=              # S3 endpoint (for MinIO/LocalStack)
AWS_ACCESS_KEY_ID=             # AWS access key
AWS_SECRET_ACCESS_KEY=         # AWS secret key
AWS_DEFAULT_REGION=us-east-1   # AWS region

Health Check

The image includes a health check that runs every 30 seconds:

docker inspect --format='{{.State.Health.Status}}' <container-id>

Troubleshooting

Build Issues

"setuptools-scm was unable to detect version"

Cause: Git metadata not available during build.

Solution: Always use the VERSION build arg:

docker build --build-arg VERSION=6.0.2 -t beshultd/deltaglider:6.0.2 .

Cache issues

Cause: Docker build cache causing stale builds.

Solution: Use --no-cache flag:

docker build --no-cache --build-arg VERSION=6.0.2 -t beshultd/deltaglider:6.0.2 .

Runtime Issues

"unauthorized: access token has insufficient scopes"

Cause: Not logged in to Docker Hub or invalid credentials.

Solution:

docker login
# Enter your Docker Hub credentials

"Cannot connect to MinIO/LocalStack"

Cause: Using localhost instead of host.docker.internal from inside container.

Solution: Use host.docker.internal for Mac/Windows or 172.17.0.1 for Linux:

# Mac/Windows
-e AWS_ENDPOINT_URL=http://host.docker.internal:9000

# Linux
-e AWS_ENDPOINT_URL=http://172.17.0.1:9000

Docker Compose

For local development with MinIO:

version: '3.8'

services:
  minio:
    image: minio/minio:latest
    ports:
      - "9000:9000"
      - "9001:9001"
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
    command: server /data --console-address ":9001"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 10s
      timeout: 5s
      retries: 5

  deltaglider:
    image: beshultd/deltaglider:latest
    environment:
      AWS_ENDPOINT_URL: http://minio:9000
      AWS_ACCESS_KEY_ID: minioadmin
      AWS_SECRET_ACCESS_KEY: minioadmin
      AWS_DEFAULT_REGION: us-east-1
      DG_LOG_LEVEL: DEBUG
    depends_on:
      - minio
    volumes:
      - ./data:/data
    working_dir: /data
    command: ["--help"]

Run with:

docker-compose up -d
docker-compose run --rm deltaglider ls

Best Practices

1. Always specify version: Use --build-arg VERSION=x.y.z when building
2. Use multi-stage builds: Keeps final image small
3. Tag with semantic versions: Follow semver (major.minor.patch)
4. Test before pushing: Run integration tests locally
5. Use secrets: Never hardcode credentials in images
6. Multi-platform builds: Support both amd64 and arm64
7. Update README: Keep Docker Hub description in sync with README.md

Additional Resources

• Docker Hub Repository
• GitHub Repository
• MinIO Documentation
• Docker Buildx Documentation