starred/deltaglider
1
0
Fork 0
mirror of https://github.com/beshu-tech/deltaglider.git synced 2026-03-30 13:51:52 +02:00
Code Issues 1 Packages Projects Releases Wiki Activity

feat: add put_bucket_acl and get_bucket_acl support

Browse Source 
Add boto3-compatible bucket ACL operations as pure S3 passthroughs,
following the existing create_bucket/delete_bucket pattern. Includes
CLI commands (put-bucket-acl, get-bucket-acl), 7 integration tests,
and documentation updates (method count 21→23).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Simone Scarduzio
2026-02-07 15:53:33 +01:00
parent 20053acb5f
commit 6b3245266e
8 changed files with 474 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
BOTO3_COMPATIBILITY.md
View File

@@ -2,7 +2,7 @@
DeltaGlider implements a **subset** of boto3's S3 client API, focusing on the most commonly used operations. This is **not** a 100% drop-in replacement, but covers the core functionality needed for most use cases.
## ✅ Implemented Methods (21 core methods)
## ✅ Implemented Methods (23 core methods)
### Object Operations
- ✅ `put_object()` - Upload objects (with automatic delta compression)
@@ -17,6 +17,8 @@ DeltaGlider implements a **subset** of boto3's S3 client API, focusing on the mo
- ✅ `create_bucket()` - Create buckets
- ✅ `delete_bucket()` - Delete empty buckets
- ✅ `list_buckets()` - List all buckets
- ✅ `put_bucket_acl()` - Set bucket ACL (passthrough to S3)
- ✅ `get_bucket_acl()` - Get bucket ACL (passthrough to S3)
### Presigned URLs
- ✅ `generate_presigned_url()` - Generate presigned URLs
@@ -46,8 +48,6 @@ DeltaGlider implements a **subset** of boto3's S3 client API, focusing on the mo
- ❌ `list_parts()`
### Access Control (ACL)
- ❌ `get_bucket_acl()`
- ❌ `put_bucket_acl()`
- ❌ `get_object_acl()`
- ❌ `put_object_acl()`
- ❌ `get_public_access_block()`
@@ -135,9 +135,9 @@ DeltaGlider implements a **subset** of boto3's S3 client API, focusing on the mo
## Coverage Analysis
**Implemented:** ~21 methods
**Implemented:** ~23 methods
**Total boto3 S3 methods:** ~100+ methods
**Coverage:** ~20%
**Coverage:** ~23%
## What's Covered

14
CLAUDE.md
View File

@@ -79,12 +79,14 @@ deltaglider stats test-bucket # Get bucket statistics
### Available CLI Commands
```bash
cp # Copy files to/from S3 (AWS S3 compatible)
ls # List S3 buckets or objects (AWS S3 compatible)
rm # Remove S3 objects (AWS S3 compatible)
sync # Synchronize directories with S3 (AWS S3 compatible)
stats # Get bucket statistics and compression metrics
verify # Verify integrity of delta file
cp # Copy files to/from S3 (AWS S3 compatible)
ls # List S3 buckets or objects (AWS S3 compatible)
rm # Remove S3 objects (AWS S3 compatible)
sync # Synchronize directories with S3 (AWS S3 compatible)
stats # Get bucket statistics and compression metrics
verify # Verify integrity of delta file
put-bucket-acl # Set bucket ACL (s3api compatible passthrough)
get-bucket-acl # Get bucket ACL (s3api compatible passthrough)
```
## Architecture

10
docs/aws-s3-cli-compatibility.md
View File

@@ -12,6 +12,8 @@ DeltaGlider provides AWS S3 CLI compatible commands with automatic delta compres
- `deltaglider migrate <source> <destination>` - Migrate S3 buckets with compression and EC2 cost warnings
- `deltaglider stats <bucket>` - Get bucket statistics and compression metrics
- `deltaglider verify <s3_url>` - Verify file integrity
- `deltaglider put-bucket-acl <bucket>` - Set bucket ACL (s3api compatible)
- `deltaglider get-bucket-acl <bucket>` - Get bucket ACL (s3api compatible)
### Current Usage Examples
```bash
@@ -23,6 +25,14 @@ deltaglider cp s3://bucket/path/to/file.zip .
# Verify integrity
deltaglider verify s3://bucket/path/to/file.zip.delta
# Set bucket ACL
deltaglider put-bucket-acl my-bucket --acl public-read
deltaglider put-bucket-acl my-bucket --acl private
deltaglider put-bucket-acl my-bucket --grant-read id=12345
# Get bucket ACL
deltaglider get-bucket-acl my-bucket
```
## Target State: AWS S3 CLI Compatibility

145
src/deltaglider/app/cli/main.py
View File

@@ -1053,6 +1053,151 @@ def purge(
sys.exit(1)
@cli.command("put-bucket-acl")
@click.argument("bucket")
@click.option(
"--acl",
type=click.Choice(["private", "public-read", "public-read-write", "authenticated-read"]),
help="Canned ACL to apply",
)
@click.option("--grant-full-control", help="Grants full control (e.g., id=account-id)")
@click.option("--grant-read", help="Allows grantee to list objects (e.g., id=account-id)")
@click.option("--grant-read-acp", help="Allows grantee to read the bucket ACL")
@click.option("--grant-write", help="Allows grantee to create objects in the bucket")
@click.option("--grant-write-acp", help="Allows grantee to write the ACL for the bucket")
@click.option("--access-control-policy", help="Full ACL policy as JSON string")
@click.option("--endpoint-url", help="Override S3 endpoint URL")
@click.option("--region", help="AWS region")
@click.option("--profile", help="AWS profile to use")
@click.pass_obj
def put_bucket_acl(
service: DeltaService,
bucket: str,
acl: str | None,
grant_full_control: str | None,
grant_read: str | None,
grant_read_acp: str | None,
grant_write: str | None,
grant_write_acp: str | None,
access_control_policy: str | None,
endpoint_url: str | None,
region: str | None,
profile: str | None,
) -> None:
"""Set the access control list (ACL) for an S3 bucket.
BUCKET can be specified as:
- s3://bucket-name
- bucket-name
Examples:
deltaglider put-bucket-acl my-bucket --acl private
deltaglider put-bucket-acl my-bucket --acl public-read
deltaglider put-bucket-acl my-bucket --grant-read id=12345
"""
from ...client import DeltaGliderClient
# Recreate service with AWS parameters if provided
if endpoint_url or region or profile:
service = create_service(
log_level=os.environ.get("DG_LOG_LEVEL", "INFO"),
endpoint_url=endpoint_url,
region=region,
profile=profile,
)
try:
# Parse bucket from S3 URL if needed
if is_s3_path(bucket):
bucket, _prefix = parse_s3_url(bucket)
if not bucket:
click.echo("Error: Invalid bucket name", err=True)
sys.exit(1)
client = DeltaGliderClient(service=service)
kwargs: dict[str, Any] = {}
if acl is not None:
kwargs["ACL"] = acl
if grant_full_control is not None:
kwargs["GrantFullControl"] = grant_full_control
if grant_read is not None:
kwargs["GrantRead"] = grant_read
if grant_read_acp is not None:
kwargs["GrantReadACP"] = grant_read_acp
if grant_write is not None:
kwargs["GrantWrite"] = grant_write
if grant_write_acp is not None:
kwargs["GrantWriteACP"] = grant_write_acp
if access_control_policy is not None:
kwargs["AccessControlPolicy"] = json.loads(access_control_policy)
client.put_bucket_acl(Bucket=bucket, **kwargs)
click.echo(f"ACL updated for bucket: {bucket}")
except json.JSONDecodeError as e:
click.echo(f"Error: Invalid JSON for --access-control-policy: {e}", err=True)
sys.exit(1)
except Exception as e:
click.echo(f"Error: {e}", err=True)
sys.exit(1)
@cli.command("get-bucket-acl")
@click.argument("bucket")
@click.option("--endpoint-url", help="Override S3 endpoint URL")
@click.option("--region", help="AWS region")
@click.option("--profile", help="AWS profile to use")
@click.pass_obj
def get_bucket_acl(
service: DeltaService,
bucket: str,
endpoint_url: str | None,
region: str | None,
profile: str | None,
) -> None:
"""Get the access control list (ACL) for an S3 bucket.
BUCKET can be specified as:
- s3://bucket-name
- bucket-name
Examples:
deltaglider get-bucket-acl my-bucket
deltaglider get-bucket-acl s3://my-bucket
"""
from ...client import DeltaGliderClient
# Recreate service with AWS parameters if provided
if endpoint_url or region or profile:
service = create_service(
log_level=os.environ.get("DG_LOG_LEVEL", "INFO"),
endpoint_url=endpoint_url,
region=region,
profile=profile,
)
try:
# Parse bucket from S3 URL if needed
if is_s3_path(bucket):
bucket, _prefix = parse_s3_url(bucket)
if not bucket:
click.echo("Error: Invalid bucket name", err=True)
sys.exit(1)
client = DeltaGliderClient(service=service)
response = client.get_bucket_acl(Bucket=bucket)
# Output as JSON like aws s3api get-bucket-acl
click.echo(json.dumps(response, indent=2, default=str))
except Exception as e:
click.echo(f"Error: {e}", err=True)
sys.exit(1)
def main() -> None:
"""Main entry point."""
cli()

59
src/deltaglider/client.py
View File

@@ -28,9 +28,11 @@ from .client_operations import (
find_similar_files as _find_similar_files,
generate_presigned_post as _generate_presigned_post,
generate_presigned_url as _generate_presigned_url,
get_bucket_acl as _get_bucket_acl,
get_bucket_stats as _get_bucket_stats,
get_object_info as _get_object_info,
list_buckets as _list_buckets,
put_bucket_acl as _put_bucket_acl,
upload_batch as _upload_batch,
upload_chunked as _upload_chunked,
)
@@ -1120,6 +1122,63 @@ class DeltaGliderClient:
"""
return _list_buckets(self, **kwargs)
def put_bucket_acl(
self,
Bucket: str,
ACL: str | None = None,
AccessControlPolicy: dict[str, Any] | None = None,
GrantFullControl: str | None = None,
GrantRead: str | None = None,
GrantReadACP: str | None = None,
GrantWrite: str | None = None,
GrantWriteACP: str | None = None,
**kwargs: Any,
) -> dict[str, Any]:
"""Set the ACL for an S3 bucket (boto3-compatible passthrough).
Args:
Bucket: Bucket name
ACL: Canned ACL (private, public-read, public-read-write, authenticated-read)
AccessControlPolicy: Full ACL policy dict
GrantFullControl: Grants full control to the grantee
GrantRead: Allows grantee to list objects in the bucket
GrantReadACP: Allows grantee to read the bucket ACL
GrantWrite: Allows grantee to create objects in the bucket
GrantWriteACP: Allows grantee to write the ACL for the bucket
**kwargs: Additional S3 parameters (for compatibility)
Returns:
Response dict with status
"""
return _put_bucket_acl(
self,
Bucket,
ACL=ACL,
AccessControlPolicy=AccessControlPolicy,
GrantFullControl=GrantFullControl,
GrantRead=GrantRead,
GrantReadACP=GrantReadACP,
GrantWrite=GrantWrite,
GrantWriteACP=GrantWriteACP,
**kwargs,
)
def get_bucket_acl(
self,
Bucket: str,
**kwargs: Any,
) -> dict[str, Any]:
"""Get the ACL for an S3 bucket (boto3-compatible passthrough).
Args:
Bucket: Bucket name
**kwargs: Additional S3 parameters (for compatibility)
Returns:
Response dict with Owner and Grants
"""
return _get_bucket_acl(self, Bucket, **kwargs)
def _parse_tagging(self, tagging: str) -> dict[str, str]:
"""Parse URL-encoded tagging string to dict."""
tags = {}

4
src/deltaglider/client_operations/__init__.py
View File

@@ -8,7 +8,7 @@ This package contains modular operation implementations:
"""
from .batch import download_batch, upload_batch, upload_chunked
from .bucket import create_bucket, delete_bucket, list_buckets
from .bucket import create_bucket, delete_bucket, get_bucket_acl, list_buckets, put_bucket_acl
from .presigned import generate_presigned_post, generate_presigned_url
from .stats import (
estimate_compression,
@@ -21,7 +21,9 @@ __all__ = [
# Bucket operations
"create_bucket",
"delete_bucket",
"get_bucket_acl",
"list_buckets",
"put_bucket_acl",
# Presigned operations
"generate_presigned_url",
"generate_presigned_post",

100
src/deltaglider/client_operations/bucket.py
View File

@@ -4,6 +4,8 @@ This module contains boto3-compatible bucket operations:
- create_bucket
- delete_bucket
- list_buckets
- put_bucket_acl
- get_bucket_acl
"""
from typing import Any
@@ -173,3 +175,101 @@ def list_buckets(
raise RuntimeError(f"Failed to list buckets: {e}") from e
else:
raise NotImplementedError("Storage adapter does not support bucket listing")
def put_bucket_acl(
client: Any, # DeltaGliderClient (avoiding circular import)
Bucket: str,
ACL: str | None = None,
AccessControlPolicy: dict[str, Any] | None = None,
GrantFullControl: str | None = None,
GrantRead: str | None = None,
GrantReadACP: str | None = None,
GrantWrite: str | None = None,
GrantWriteACP: str | None = None,
**kwargs: Any,
) -> dict[str, Any]:
"""Set the ACL for an S3 bucket (boto3-compatible passthrough).
Args:
client: DeltaGliderClient instance
Bucket: Bucket name
ACL: Canned ACL (private, public-read, public-read-write, authenticated-read)
AccessControlPolicy: Full ACL policy dict
GrantFullControl: Grants full control to the grantee
GrantRead: Allows grantee to list objects in the bucket
GrantReadACP: Allows grantee to read the bucket ACL
GrantWrite: Allows grantee to create objects in the bucket
GrantWriteACP: Allows grantee to write the ACL for the bucket
**kwargs: Additional S3 parameters (for compatibility)
Returns:
Response dict with status
Example:
>>> client = create_client()
>>> client.put_bucket_acl(Bucket='my-bucket', ACL='public-read')
"""
storage_adapter = client.service.storage
if hasattr(storage_adapter, "client"):
try:
params: dict[str, Any] = {"Bucket": Bucket}
if ACL is not None:
params["ACL"] = ACL
if AccessControlPolicy is not None:
params["AccessControlPolicy"] = AccessControlPolicy
if GrantFullControl is not None:
params["GrantFullControl"] = GrantFullControl
if GrantRead is not None:
params["GrantRead"] = GrantRead
if GrantReadACP is not None:
params["GrantReadACP"] = GrantReadACP
if GrantWrite is not None:
params["GrantWrite"] = GrantWrite
if GrantWriteACP is not None:
params["GrantWriteACP"] = GrantWriteACP
storage_adapter.client.put_bucket_acl(**params)
return {
"ResponseMetadata": {
"HTTPStatusCode": 200,
},
}
except Exception as e:
raise RuntimeError(f"Failed to set bucket ACL: {e}") from e
else:
raise NotImplementedError("Storage adapter does not support bucket ACL operations")
def get_bucket_acl(
client: Any, # DeltaGliderClient (avoiding circular import)
Bucket: str,
**kwargs: Any,
) -> dict[str, Any]:
"""Get the ACL for an S3 bucket (boto3-compatible passthrough).
Args:
client: DeltaGliderClient instance
Bucket: Bucket name
**kwargs: Additional S3 parameters (for compatibility)
Returns:
Response dict with Owner and Grants
Example:
>>> client = create_client()
>>> response = client.get_bucket_acl(Bucket='my-bucket')
>>> print(response['Owner'])
>>> print(response['Grants'])
"""
storage_adapter = client.service.storage
if hasattr(storage_adapter, "client"):
try:
response: dict[str, Any] = storage_adapter.client.get_bucket_acl(Bucket=Bucket)
return response
except Exception as e:
raise RuntimeError(f"Failed to get bucket ACL: {e}") from e
else:
raise NotImplementedError("Storage adapter does not support bucket ACL operations")

144
tests/integration/test_bucket_management.py
View File

@@ -308,6 +308,150 @@ class TestBucketManagement:
with pytest.raises(NotImplementedError):
client.list_buckets()
def test_put_bucket_acl_with_canned_acl(self):
"""Test setting a canned ACL on a bucket."""
service = create_service()
mock_storage = Mock()
service.storage = mock_storage
mock_boto3_client = Mock()
mock_boto3_client.put_bucket_acl.return_value = None
mock_storage.client = mock_boto3_client
client = DeltaGliderClient(service)
response = client.put_bucket_acl(Bucket="test-bucket", ACL="public-read")
assert response["ResponseMetadata"]["HTTPStatusCode"] == 200
mock_boto3_client.put_bucket_acl.assert_called_once_with(
Bucket="test-bucket", ACL="public-read"
)
def test_put_bucket_acl_with_grants(self):
"""Test setting ACL with grant parameters."""
service = create_service()
mock_storage = Mock()
service.storage = mock_storage
mock_boto3_client = Mock()
mock_boto3_client.put_bucket_acl.return_value = None
mock_storage.client = mock_boto3_client
client = DeltaGliderClient(service)
response = client.put_bucket_acl(
Bucket="test-bucket",
GrantRead="id=12345",
GrantWrite="id=67890",
)
assert response["ResponseMetadata"]["HTTPStatusCode"] == 200
mock_boto3_client.put_bucket_acl.assert_called_once_with(
Bucket="test-bucket", GrantRead="id=12345", GrantWrite="id=67890"
)
def test_put_bucket_acl_with_access_control_policy(self):
"""Test setting ACL with a full AccessControlPolicy dict."""
service = create_service()
mock_storage = Mock()
service.storage = mock_storage
mock_boto3_client = Mock()
mock_boto3_client.put_bucket_acl.return_value = None
mock_storage.client = mock_boto3_client
policy = {
"Grants": [
{
"Grantee": {"Type": "CanonicalUser", "ID": "abc123"},
"Permission": "FULL_CONTROL",
}
],
"Owner": {"ID": "abc123"},
}
client = DeltaGliderClient(service)
response = client.put_bucket_acl(
Bucket="test-bucket", AccessControlPolicy=policy
)
assert response["ResponseMetadata"]["HTTPStatusCode"] == 200
mock_boto3_client.put_bucket_acl.assert_called_once_with(
Bucket="test-bucket", AccessControlPolicy=policy
)
def test_put_bucket_acl_failure(self):
"""Test that put_bucket_acl raises RuntimeError on boto3 failure."""
service = create_service()
mock_storage = Mock()
service.storage = mock_storage
mock_boto3_client = Mock()
mock_boto3_client.put_bucket_acl.side_effect = Exception("AccessDenied")
mock_storage.client = mock_boto3_client
client = DeltaGliderClient(service)
with pytest.raises(RuntimeError, match="Failed to set bucket ACL"):
client.put_bucket_acl(Bucket="test-bucket", ACL="public-read")
def test_put_bucket_acl_no_boto3_client(self):
"""Test that put_bucket_acl raises NotImplementedError without boto3 client."""
service = create_service()
mock_storage = Mock()
service.storage = mock_storage
delattr(mock_storage, "client")
client = DeltaGliderClient(service)
with pytest.raises(NotImplementedError):
client.put_bucket_acl(Bucket="test-bucket", ACL="private")
def test_get_bucket_acl_success(self):
"""Test getting bucket ACL successfully."""
service = create_service()
mock_storage = Mock()
service.storage = mock_storage
acl_response = {
"Owner": {"DisplayName": "test-user", "ID": "abc123"},
"Grants": [
{
"Grantee": {
"Type": "CanonicalUser",
"DisplayName": "test-user",
"ID": "abc123",
},
"Permission": "FULL_CONTROL",
}
],
}
mock_boto3_client = Mock()
mock_boto3_client.get_bucket_acl.return_value = acl_response
mock_storage.client = mock_boto3_client
client = DeltaGliderClient(service)
response = client.get_bucket_acl(Bucket="test-bucket")
assert response["Owner"]["DisplayName"] == "test-user"
assert len(response["Grants"]) == 1
assert response["Grants"][0]["Permission"] == "FULL_CONTROL"
mock_boto3_client.get_bucket_acl.assert_called_once_with(Bucket="test-bucket")
def test_get_bucket_acl_failure(self):
"""Test that get_bucket_acl raises RuntimeError on boto3 failure."""
service = create_service()
mock_storage = Mock()
service.storage = mock_storage
mock_boto3_client = Mock()
mock_boto3_client.get_bucket_acl.side_effect = Exception("NoSuchBucket")
mock_storage.client = mock_boto3_client
client = DeltaGliderClient(service)
with pytest.raises(RuntimeError, match="Failed to get bucket ACL"):
client.get_bucket_acl(Bucket="nonexistent-bucket")
def test_complete_bucket_lifecycle(self):
"""Test complete bucket lifecycle: create, use, delete."""
service = create_service()