Documentation
¶
Overview ¶
Copyright 2017 Applatix, Inc.
Index ¶
- func GetBucketRegion(bucket string) (string, error)
- type AWSBillingBucket
- func (billbuck *AWSBillingBucket) Download(key string, downloadPath string, skipIfSizeIdentical bool) error
- func (billbuck *AWSBillingBucket) GetManifest(key string) (*Manifest, error)
- func (billbuck *AWSBillingBucket) GetManifestPaths() ([]string, error)
- func (billbuck *AWSBillingBucket) ListDir(dir string) ([]string, error)
- type Manifest
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func GetBucketRegion ¶
GetBucketRegion determines the bucket region using a HTTP HEAD request against a formulated S3 URL. We need to correctly match the client region to the bucket's region, otherwise bucket operations fail with: BucketRegionError: incorrect region, the bucket is not in 'us-east-1' region. A previous implementation used the GetBucketLocation API, but this would intermittently hit AWS error: "Insufficient permissions. Check policy configuration" Based on the discussions in https://github.com/aws/aws-sdk-go/issues/720, the best way to determine a bucket's region is an anonymous HEAD request against the bucket URL and read the "x-amz-bucket-region" from the HTTP response headers. Note, however, that anonymous HTTP requests to AWS are subject to much stricter AWS API rate limits, and can hit "SlowDown - Please reduce your request rate" errors.
Types ¶
type AWSBillingBucket ¶
type AWSBillingBucket struct {
Bucket string
Region string
ReportPath string
Session *session.Session
S3Client *s3.S3
// contains filtered or unexported fields
}
AWSBillingBucket is the object representation of a S3 bucket containing AWS Cost & Usage reports
func NewAWSBillingBucket ¶
func NewAWSBillingBucket(awsAccessKeyID, awsSecretAccessKey, bucket, region, reportPath string) (*AWSBillingBucket, error)
NewAWSBillingBucket returns an AWSBillingBucket
func (*AWSBillingBucket) Download ¶
func (billbuck *AWSBillingBucket) Download(key string, downloadPath string, skipIfSizeIdentical bool) error
Download will download the file into given directory or file path, creating directory structure if necessary. Optionally skips files which are same size
func (*AWSBillingBucket) GetManifest ¶
func (billbuck *AWSBillingBucket) GetManifest(key string) (*Manifest, error)
GetManifest returns a manifest object for the manifest at the given key
func (*AWSBillingBucket) GetManifestPaths ¶
func (billbuck *AWSBillingBucket) GetManifestPaths() ([]string, error)
GetManifestPaths returns the paths to all manifests in the bucket (e.g. report/path/20161201-20170101/hourly2-Manifest.json)
type Manifest ¶
type Manifest struct {
AssemblyID string `json:"assemblyId,omitempty"`
Account string `json:"account,omitempty"`
Bucket string `json:"bucket,omitempty"`
BillingPeriod map[string]string `json:"billingPeriod,omitempty"`
Charset string `json:"charset,omitempty"`
Columns []map[string]string `json:"columns,omitempty"`
Compression string `json:"compression,omitempty"`
ContentType string `json:"contentType,omitempty"`
ReportID string `json:"reportId,omitempty"`
ReportName string `json:"reportName,omitempty"`
ReportKeys []string `json:"reportKeys,omitempty"`
AdditionalArtifactKeys []interface{} `json:"additionalArtifactKeys,omitempty"`
}
Manifest is the object representation of an AWS Cost & Usage Report Manifest JSON Manifest contents look like
{
"assemblyId":"aa1ddccb-abcd-1234-b849-57a32b6864a9",
"account":"012345678910",
"columns":[{
"category":"identity",
"name":"LineItemId"
},{ // ...
...
},{
"category":"reservation",
"name":"TotalReservedUnits"
},{
"category":"reservation",
"name":"UnitsPerReservation"
}],
"charset":"UTF-8",
"compression":"GZIP",
"contentType":"text/csv",
"reportId":"a5e3b9e6e25825d3591a5bcb9d662104cb4a76a636eb027459266ab31a06d1c3",
"reportName":"hourly2",
"billingPeriod":{
"start":"20161101T000000.000Z",
"end":"20161201T000000.000Z"
},
"bucket":"billing-bucket",
"reportKeys":[
"report/path/20161101-20161201/aa1ddccb-abcd-1234-b849-57a32b6864a9/hourly2-1.csv.gz",
"report/path/20161101-20161201/aa1ddccb-abcd-1234-b849-57a32b6864a9/hourly2-2.csv.gz"
],
"additionalArtifactKeys":[{
"artifactType":"RedshiftCommands",
"name":"report/path/20161101-20161201/aa1ddccb-abcd-1234-b849-57a32b6864a9/hourly2-RedshiftCommands.sql"
},{
"artifactType":"RedshiftManifest",
"name":"report/path/20161101-20161201/aa1ddccb-abcd-1234-b849-57a32b6864a9/hourly2-RedshiftManifest.json"
}]
}
func (*Manifest) BillingPeriodString ¶
BillingPeriodString returns a string representing the billing period (e.g. 20161201-20170101)
func (*Manifest) ReportPath ¶
ReportPath returns the reportPath from the reportKey (e.g. "report/path")
Source Files