Documentation
¶
Overview ¶
Package s3util contains methods for interacting with S3.
Package s3util contains utilities for working with S3 ¶
Package s3util contains methods for interacting with S3.
Package s3util contains methods for interacting with S3.
Index ¶
Constants ¶
const (
// cn- is a prefix for China region
ChinaRegionPrefix = "cn-"
)
Variables ¶
This section is empty.
Functions ¶
func GetS3CrossRegionCapableSession ¶
Returns a Session capable of performing cross-region S3 bucket accesses (i.e. the bucket region may be different from the instance's home region). The session is initialized to work with the specified bucket, and should not be used to access other buckets.
When initializing the session, we make a best-effort attempt to determine the region in which the bucket resides. The session is initialized with the correct region for the bucket if the region was successfully determined, or with the instance region.
The session also has a Handler chain and custom HTTP RoundTripper that follow cross-region redirect responses from S3. These work as follows:
- The custom RoundTripper (s3BucketRegionHeaderCapturingTransport) extracts the bucket region information from S3 redirect responses and stores them in a cache.
- The Retry Handler, which is invoked before each retry, checks to see whether a bucket -> region mapping exists for the request's bucket, and if so, fixes up the request to point to the correct region.
- The Validation Handler, which is invoked before the first attempt, similarly checks for a bucket -> region mapping for the request's bucket, and if one is found, fixes up the request to point to the correct region.
In most cases, the best-effort attempt will initialize the session with the correct region, and the custom Transport and Handler chain will not need to make any changes.
Types ¶
type AmazonS3URL ¶
type AmazonS3URL struct {
IsValidS3URI bool
IsPathStyle bool
Bucket string
Key string
Region string
}
AmazonS3URL holds interesting pieces after parsing a s3 URL
func ParseAmazonS3URL ¶
func ParseAmazonS3URL(log log.T, s3URL *url.URL) (output AmazonS3URL)
ParseAmazonS3URL parses an HTTP/HTTPS URL for an S3 resource and returns an AmazonS3URL object.
S3 URLs come in two flavors: virtual hosted-style URLs and path-style URLs. Virtual hosted-style URLs have the bucket name as the first component of the hostname, e.g.
https://mybucket.s3.us-east-1.amazonaws.com/a/b/c
Path-style URLs have the bucket name as the first component of the path, e.g.
https://s3.us-east-1.amazonaws.com/mybucket/a/b/c
S3 supports a few features that affect how the URL is formed:
- Website endpoints - "s3.$REGION" becomes "s3-website[-.]$REGION"
- Transfer acceleration - "s3" becomes "s3-accelerate"
- Dual-stack (IPv4/IPv6) - "s3" becomes "s3.dualstack"
- Can be used with acceleration - "s3-accelerate.dualstack"
- VPC endpoints - "s3.$REGION.amazonaws.com" becomes "bucket.$VPC_ENDPOINT_ID.s3.$REGION.vpce.amazonaws.com"
References: https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/WebsiteEndpoints.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/transfer-acceleration-getting-started.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/dual-stack-endpoints.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/privatelink-interface-endpoints.html
func (AmazonS3URL) IsBucketAndKeyPresent ¶
func (output AmazonS3URL) IsBucketAndKeyPresent() bool
IsBucketAndKeyPresent checks the AmazonS3URL if it contains both bucket and key
func (AmazonS3URL) String ¶
func (output AmazonS3URL) String() string
String returns the string representation of the AmazonS3URL
type AmazonS3Util ¶
type AmazonS3Util struct {
// contains filtered or unexported fields
}
func NewAmazonS3Util ¶
func NewAmazonS3Util(context context.T, bucketName string) (res *AmazonS3Util, err error)
func (*AmazonS3Util) IsBucketEncrypted ¶
IsBucketEncrypted checks if the bucket is encrypted
type HttpProviderImpl ¶
type HttpProviderImpl struct {
// contains filtered or unexported fields
}
HttpProviderImpl provides http capabilities
Source Files