Documentation
¶
Index ¶
- Constants
- type File
- func (f *File) Close() (rerr error)
- func (f *File) CopyToFile(targetFile vfs.File) error
- func (f *File) CopyToLocation(location vfs.Location) (vfs.File, error)
- func (f *File) Delete() error
- func (f *File) Exists() (bool, error)
- func (f *File) LastModified() (*time.Time, error)
- func (f *File) Location() vfs.Location
- func (f *File) MoveToFile(targetFile vfs.File) error
- func (f *File) MoveToLocation(location vfs.Location) (vfs.File, error)
- func (f *File) Name() string
- func (f *File) Path() string
- func (f *File) Read(p []byte) (n int, err error)
- func (f *File) Seek(offset int64, whence int) (int64, error)
- func (f *File) Size() (uint64, error)
- func (f *File) String() string
- func (f *File) URI() string
- func (f *File) Write(data []byte) (res int, err error)
- type FileSystem
- type Location
- func (l *Location) ChangeDir(relativePath string) error
- func (l *Location) DeleteFile(fileName string) error
- func (l *Location) Exists() (bool, error)
- func (l *Location) FileSystem() vfs.FileSystem
- func (l *Location) List() ([]string, error)
- func (l *Location) ListByPrefix(prefix string) ([]string, error)
- func (l *Location) ListByRegex(regex *regexp.Regexp) ([]string, error)
- func (l *Location) NewFile(filePath string) (vfs.File, error)
- func (l *Location) NewLocation(relativePath string) (vfs.Location, error)
- func (l *Location) Path() string
- func (l *Location) String() string
- func (l *Location) URI() string
- func (l *Location) Volume() string
Constants ¶
const Scheme = "s3"
Scheme defines the filesystem type.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type File ¶
type File struct {
// contains filtered or unexported fields
}
File implements vfs.File interface for S3 fs.
func (*File) Close ¶
Close cleans up underlying mechanisms for reading from and writing to the file. Closes and removes the local temp file, and triggers a write to s3 of anything in the f.writeBuffer if it has been created.
func (*File) CopyToFile ¶
CopyToFile puts the contents of File into the targetFile passed. Uses the S3 CopyObject method if the target file is also on S3, otherwise uses io.Copy.
func (*File) CopyToLocation ¶
CopyToLocation creates a copy of *File, using the file's current name as the new file's name at the given location. If the given location is also s3, the AWS API for copying files will be utilized, otherwise, standard io.Copy will be done to the new file.
func (*File) Delete ¶
Delete clears any local temp file, or write buffer from read/writes to the file, then makes a DeleteObject call to s3 for the file. Returns any error returned by the API.
func (*File) Exists ¶
Exists returns a boolean of whether or not the object exists on s3, based on a call for the object's HEAD through the s3 API.
func (*File) LastModified ¶
LastModified returns the LastModified property of a HEAD request to the s3 object.
func (*File) Location ¶
Location returns a vfs.Location at the location of the object. IE: if file is at s3://bucket/here/is/the/file.txt the location points to s3://bucket/here/is/the/
func (*File) MoveToFile ¶
MoveToFile puts the contents of File into the targetFile passed using File.CopyToFile. If the copy succeeds, the source file is deleted. Any errors from the copy or delete are returned.
func (*File) MoveToLocation ¶
MoveToLocation works by first calling File.CopyToLocation(vfs.Location) then, if that succeeds, it deletes the original file, returning the new file. If the copy process fails the error is returned, and the Delete isn't called. If the call to Delete fails, the error and the file generated by the copy are both returned.
func (*File) Name ¶
Name returns the name portion of the file's key property. IE: "file.txt" of "s3://some/path/to/file.txt
func (*File) Path ¶
Path return the directory portion of the file's key. IE: "path/to" of "s3://some/path/to/file.txt
func (*File) Read ¶
Read implements the standard for io.Reader. For this to work with an s3 file, a temporary local copy of the file is created, and reads work on that. This file is closed and removed upon calling f.Close()
func (*File) Seek ¶
Seek implements the standard for io.Seeker. A temporary local copy of the s3 file is created (the same one used for Reads) which Seek() acts on. This file is closed and removed upon calling f.Close()
func (*File) Size ¶
Size returns the ContentLength value from an s3 HEAD request on the file's object.
func (*File) String ¶
String implement fmt.Stringer, returning the file's URI as the default string.
func (*File) Write ¶
Write implements the standard for io.Writer. A buffer is added to with each subsequent write. When f.Close() is called, the contents of the buffer are used to initiate the PutObject to s3. The underlying implementation uses s3manager which will determine whether it is appropriate to call PutObject, or initiate a multi-part upload.
type FileSystem ¶
FileSystem implements vfs.Filesystem for the S3 filesystem.
func NewFileSystem ¶
func NewFileSystem(client s3iface.S3API) (*FileSystem, error)
NewFileSystem intializer for fileSystem struct accepts aws-sdk s3iface.S3API client and returns Filesystem or error.
func (FileSystem) NewLocation ¶
NewLocation function returns the s3 implementation of vfs.Location.
func (FileSystem) Scheme ¶
func (fs FileSystem) Scheme() string
Scheme return "s3" as the initial part of a file URI ie: s3://
type Location ¶
type Location struct {
// contains filtered or unexported fields
}
Location implements the vfs.Location interface specific to S3 fs.
func (*Location) ChangeDir ¶
ChangeDir takes a relative path, and modifies the underlying Location's path. The caller is modified by this so the only return is any error. For this implementation there are no errors.
func (*Location) DeleteFile ¶
DeleteFile removes the file at fileName path.
func (*Location) Exists ¶
Exists returns true if the bucket exists, and the user in the underlying s3.fileSystem.Client has the appropriate permissions. Will receive false without an error if the bucket simply doesn't exist. Otherwise could receive false and any errors passed back from the API.
func (*Location) FileSystem ¶
func (l *Location) FileSystem() vfs.FileSystem
FileSystem returns a vfs.fileSystem interface of the location's underlying fileSystem.
func (*Location) List ¶
List calls the s3 API to list all objects in the location's bucket, with a prefix automatically set to the location's path. This will make a call to the s3 API for every 1000 keys to return. If you have many thousands of keys at the given location, this could become quite expensive.
func (*Location) ListByPrefix ¶
ListByPrefix calls the s3 API with the location's prefix modified relatively by the prefix arg passed to the function. The resource considerations of List() apply to this function as well.
func (*Location) ListByRegex ¶
ListByRegex retrieves the keys of all the files at the location's current path, then filters out all those that don't match the given regex. The resource considerations of List() apply here as well.
func (*Location) NewFile ¶
NewFile uses the properties of the calling location to generate a vfs.File (backed by an s3.File). The filePath argument is expected to be a relative path to the location's current path.
func (*Location) NewLocation ¶
NewLocation makes a copy of the underlying Location, then modifies its path by calling ChangeDir with the relativePath argument, returning the resulting location. The only possible errors come from the call to ChangeDir, which, for the s3 implementation doesn't ever result in an error.
Source Files