README
¶
S3
s3 provides signing of HTTP requests to Amazon S3, along with parallelized and streamed upload and download of S3 objects.
Features
- Streaming: Parallel and streaming upload and download for efficient operations.
- Integrity checks: Integrity checks are done during multi-part upload.
- Retries: Every call to s3 are retried according to Amazon S3 recommendations.
- Memory conscious: s3 tries to make a concious usage of memory during upload and download.
Installation
Download and install:
$ go get github.com/cyberdelia/aws/s3
Add it to your code:
import "github.com/cyberdelia/aws/s3"
Usage
w, err := s3.Create("s3://s3-us-west-2.amazonaws.com/bucket_name/file.txt", nil, nil)
if err != nil {
//...
}
r, _, err := s3.Open("s3://s3-us-west-2.amazonaws.com/bucket_name/file.txt", nil)
if err != nil {
//...
}
walkFn := func(name string, info os.FileInfo) error {
if info.IsDir() {
return s3.SkipDir
}
return nil
}
if err := s3.Walk("s3://s3-us-west-2.amazonaws.com/bucket_name/", walkFn, nil); err != nil {
// ...
}
See also
Documentation
¶
Index ¶
- Variables
- func Create(uri string, h http.Header, c *http.Client) (io.WriteCloser, error)
- func Get(url string) (resp *http.Response, err error)
- func Head(url string) (resp *http.Response, err error)
- func Open(uri string, c *http.Client) (io.ReadCloser, http.Header, error)
- func Post(url string, contentType string, body io.Reader) (resp *http.Response, err error)
- func PostForm(url string, data url.Values) (resp *http.Response, err error)
- func Remove(uri string, c *http.Client) error
- func Walk(uri string, walkFn WalkFunc, client *http.Client) error
- type Object
- type WalkFunc
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var DefaultClient = &http.Client{ Transport: DefaultSigner.Transport(), }
DefaultClient is the default http.Client used for all requests.
var DefaultSigner = &aws.V4Signer{ Region: os.Getenv("AWS_REGION"), AccessKey: os.Getenv("AWS_ACCESS_KEY_ID"), SecretKey: os.Getenv("AWS_SECRET_ACCESS_KEY"), SecurityToken: os.Getenv("AWS_SECURITY_TOKEN"), Service: "s3", }
DefaultSigner is the default Signer and is use for all requests.
var SkipDir = errors.New("skip this directory")
SkipDir is used as a return value from WalkFuncs to indicate that the directory named in the call is to be skipped. It is not returned as an error by any function.
Functions ¶
func Create ¶
Create creates an S3 object at url and sends multipart upload requests as data is written.
Example ¶
f, _ := os.Open("file.txt")
w, err := Create("s3://s3-us-west-2.amazonaws.com/bucket_name/file.txt", nil, nil)
if err != nil {
return
}
io.Copy(w, f)
Output:
func Open ¶
Open opens an S3 object at url and return a io.ReadCloser.
Example ¶
r, _, err := Open("s3://s3-us-west-2.amazonaws.com/bucket_name/file.txt", nil)
if err != nil {
return
}
f, _ := os.Open("file.txt")
io.Copy(f, r)
Output:
func PostForm ¶
PostForm issues a POST to the specified URL, with data's keys and values URL-encoded as the request body.
func Remove ¶
Remove removes the given object.
Example ¶
if err := Remove("s3://s3-us-west-2.amazonaws.com/bucket_name/object.txt", nil); err != nil {
return
}
Output:
func Walk ¶
Walk walks the bucket starting at prefix, calling walkFn for each objects in the bucket.
Example ¶
walkFn := func(name string, info os.FileInfo) error {
if info.IsDir() {
return SkipDir
}
return nil
}
if err := Walk("s3://s3-us-west-2.amazonaws.com/bucket_name/", walkFn, nil); err != nil {
return
}
Output:
Source Files