extract

package module
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 10, 2025 License: MPL-2.0 Imports: 32 Imported by: 0

README

go-extract

Perform tests on unix and windows GoDoc

This library provides secure decompression and extraction for formats like 7-Zip, Brotli, Bzip2, GZip, LZ4, Rar (excluding symlinks), Snappy, Tar, Xz, Zip, Zlib, and Zstandard. It safeguards against resource exhaustion, path traversal, and symlink attacks. Additionally, it offers various configuration options and collects telemetry data during extraction.

Installation Instructions

Add hashicorp/go-extract as a dependency to your project:

go get github.com/hashicorp/go-extract

Build hashicorp/go-extract from source and install it to the system as a command-line utility:

git clone [email protected]:hashicorp/go-extract.git
cd go-extract
make
make test
make install

Install hashicorp/go-extract directly from GitHub:

go install github.com/hashicorp/go-extract/cmd/goextract@latest

Usage Examples

These examples demonstrate how to use hashicorp/go-extract both as a library and as a command-line utility.

Library

The simplest way to use the library is to call the extract.Unpack function with the default configuration. This function extracts the contents from an io.Reader to the specified destination on the local filesystem.

// Unpack the archive
if err := extract.Unpack(ctx, dst, archive, config.NewConfig()); err != nil {
    // Handle error
    log.Fatalf("Failed to unpack archive: %v", err)
}
Command-line Utility

The goextract command-line utility offers all available configuration options via dedicated flags.

$ goextract -h
Usage: goextract <archive> [<destination>] [flags]

A secure extraction utility

Arguments:
  <archive>          Path to archive. ("-" for STDIN)
  [<destination>]    Output directory/file.

Flags:
  -h, --help                               Show context-sensitive help.
  -C, --continue-on-error                  Continue extraction on error.
  -S, --continue-on-unsupported-files      Skip extraction of unsupported files.
  -c, --create-destination                 Create destination directory if it does not exist.
      --custom-create-dir-mode=750         File mode for created directories, which are not listed in the archive. (respecting umask)
      --custom-decompress-file-mode=640    File mode for decompressed files. (respecting umask)
  -D, --deny-symlinks                      Deny symlink extraction.
  -d, --drop-file-attributes               Drop file attributes (mode, modtime, access time).
      --insecure-traverse-symlinks         Traverse symlinks to directories during extraction.
      --max-files=100000                   Maximum files (including folder and symlinks) that are extracted before stop. (disable check: -1)
      --max-extraction-size=1073741824     Maximum extraction size that allowed is (in bytes). (disable check: -1)
      --max-extraction-time=60             Maximum time that an extraction should take (in seconds). (disable check: -1)
      --max-input-size=1073741824          Maximum input size that allowed is (in bytes). (disable check: -1)
  -N, --no-untar-after-decompression       Disable combined extraction of tar.gz.
  -O, --overwrite                          Overwrite if exist.
  -P, --pattern=PATTERN,...                Extracted objects need to match shell file name pattern.
  -p, --preserve-owner                     Preserve owner and group of files from archive (only root/uid:0 on unix systems for tar files).
  -T, --telemetry                          Print telemetry data to log after extraction.
  -t, --type=""                            Type of archive. (7z, br, bz2, gz, lz4, rar, sz, tar, tgz, xz, zip, zst, zz)
  -v, --verbose                            Verbose logging.
  -V, --version                            Print release version information.

Configuration

When calling the extract.Unpack(..) function, we need to provide config object that contains all available configuration.

  cfg := extract.NewConfig(
    extract.WithContinueOnError(..),
    extract.WithContinueOnUnsupportedFiles(..),
    extract.WithCreateDestination(..),
    extract.WithCustomCreateDirMode(..),
    extract.WithCustomDecompressFileMode(..),
    extract.WithDenySymlinkExtraction(..),
    extract.WithDropFileAttributes(..),
    extract.WithExtractType(..),
    extract.WithInsecureTraverseSymlinks(..),
    extract.WithLogger(..),
    extract.WithMaxExtractionSize(..),
    extract.WithMaxFiles(..),
    extract.WithMaxInputSize(..),
    extract.WithNoUntarAfterDecompression(..),
    extract.WithOverwrite(..),
    extract.WithPatterns(..),
    extract.WithPreserveOwner(..),
    extract.WithTelemetryHook(..),
  )

[..]

  if err := extract.Unpack(ctx, dst, archive, cfg); err != nil {
    log.Println(fmt.Errorf("error during extraction: %w", err))
    os.Exit(-1)
  }

Telemetry

Telemetry data can be collected by specifying a telemetry hook in the configuration. This hook receives the collected telemetry data at the end of each extraction.

// create new config
cfg := NewConfig(
  WithTelemetryHook(func(ctx context.Context, m *telemetry.Data) {
    // handle telemetry data
  }),
)

Here is an example collected telemetry data for the extraction of terraform-aws-iam-5.34.0.tar.gz:

{
  "last_extraction_error": "",
  "extracted_dirs": 51,
  "extraction_duration": 55025584,
  "extraction_errors": 0,
  "extracted_files": 241,
  "extraction_size": 539085,
  "extracted_symlinks": 0,
  "extracted_type": "tar.gz",
  "input_size": 81477,
  "pattern_mismatches": 0,
  "unsupported_files": 0,
  "last_unsupported_file": ""
}

Extraction targets

Disk

Interact with the local operating system to create files, directories, and symlinks. Extracted entries can be accessed later using the os.* API calls.

// prepare destination and config
d := extract.NewTargetDisk()
dst := "output/"
cfg := config.NewConfig()

// unpack
if err := extract.UnpackTo(ctx, d, dst, archive, cfg); err != nil {
    // handle error
}

// Walk the local filesystem
localFs := os.DirFS(dst)
if err := fs.WalkDir(localFs, ".", func(path string, d fs.DirEntry, err error) error {
    // process path, d and err
    return nil
}); err != nil {
    // handle error
}
Memory

Extract archives directly into memory, supporting files, directories, and symlinks. Note that file permissions are not validated. Access the extracted entries by converting the target to io/fs.FS.

// prepare destination and config
m   = extract.NewMemory()     // create a new in-memory filesystem
dst = ""                      // root of in-memory filesystem
cfg = extract.NewConfig()     // custom config for extraction

// unpack
if err := extract.UnpackTo(ctx, m, dst, archive, cfg); err != nil {
    // handle error
}

// Walk the memory filesystem
if err := fs.WalkDir(m, ".", func(path string, d fs.DirEntry, err error) error {
    fmt.Println(path)
    return nil
}); err != nil {
    fmt.Printf("failed to walk memory filesystem: %s", err)
    return
}

Errors

If the extraction fails, you can check for specific errors returned by the extract.Unpack function:

if err := extract.Unpack(ctx, dst, archive, cfg); err != nil {
  switch {
  case errors.Is(err, extract.ErrNoExtractorFound):
    // handle no extractor found
  case errors.Is(err, extract.ErrUnsupportedFileType):
    // handle unsupported file type
  case errors.Is(err, extract.ErrFailedToReadHeader):
    // handle failed to read header
  case errors.Is(err, extract.ErrFailedToUnpack):
    // handle failed to unpack
  default:
    // handle other error
  }
}

Documentation

Overview

Package extract provides a function to extract files from a reader to a destination.

The extraction process is determined by the file type, with support for various formats that can be output to the underlying OS, in-memory, or a custom filesystem target.

Configuration is done using the Config, which is a configuration struct that can be used to set the extraction type, the logger, the telemetry hook, and the maximum input size. Telemetry data is captured during the extraction process. The collection of TelemetryData is done using the telemetry package.

Example

Demonstrates how to extract an "example.zip" source archive to an "output" directory on disk with the default configuration options. 

var (
	ctx = context.Background()      // context for cancellation
	src = openFile("example.zip")   // source reader
	dst = createDirectory("output") // create destination directory
	cfg = extract.NewConfig()       // custom config for extraction
)

err := extract.Unpack(ctx, dst, src, cfg)
if err != nil {
	switch {
	case errors.Is(err, extract.ErrNoExtractorFound):
		// handle no extractor found
	case errors.Is(err, extract.ErrUnsupportedFileType):
		// handle unsupported file type
	case errors.Is(err, extract.ErrFailedToReadHeader):
		// handle failed to read header
	case errors.Is(err, extract.ErrFailedToUnpack):
		// handle failed to unpack
	default:
		// handle other error
	}
}

content, err := os.ReadFile(filepath.Join(dst, "example.txt"))
if err != nil {
	// handle error
}

fmt.Println(string(content))

Output:

example content

Index

Examples

Constants

This section is empty.

Variables

View Source 
var (
	// ErrNoExtractorFound is returned when no extractor is found for the given file type.
	ErrNoExtractorFound = fmt.Errorf("extract: no extractor found for file type")

	// ErrUnsupportedFileType is returned when the file type is not supported.
	ErrUnsupportedFileType = fmt.Errorf("extract: unsupported file type")

	// ErrFailedToReadHeader is returned when the header of the file cannot be read.
	ErrFailedToReadHeader = fmt.Errorf("extract: failed to read header")

	// ErrFailedToExtract is returned when the file cannot be extracted.
	ErrFailedToUnpack = fmt.Errorf("extract: failed to unpack")

	// ErrUnsupportedFile is an error that indicates that the file is not supported.
	ErrUnsupportedFile = fmt.Errorf("extract: unsupported file")

	// ErrMaxFilesExceeded indicates that the maximum number of files is exceeded.
	ErrMaxFilesExceeded = fmt.Errorf("extract: maximum files exceeded")

	// ErrMaxExtractionSizeExceeded indicates that the maximum size is exceeded.
	ErrMaxExtractionSizeExceeded = fmt.Errorf("extract: maximum extraction size exceeded")
)

Functions

func HasKnownArchiveExtension added in v1.0.0 

func HasKnownArchiveExtension(name string) bool

HasKnownArchiveExtension returns true if the given name has a known archive extension.

Example

Demonstrates how to check if a given file has a known archive extension. 

var (
	testFile = "example.zip" // source file
)

if extract.HasKnownArchiveExtension(testFile) {
	fmt.Println("test file is an archive")
}

Output:

test file is an archive

func Unpack 

func Unpack(ctx context.Context, dst string, src io.Reader, cfg *Config) error

Unpack unpacks the given source to the destination, according to the given configuration, using the default OS If cfg is nil, the default configuration is used for extraction. If an error occurs, it is returned.

Example 
var (
	ctx = context.Background()      // context for cancellation
	dst = createDirectory("output") // create destination directory
	src = openFile("example.zip")   // source reader
	cfg = extract.NewConfig()       // custom config for extraction
)

// unpack
if err := extract.Unpack(ctx, dst, src, cfg); err != nil {
	// handle error
}

// read extracted file
content, err := os.ReadFile(filepath.Join(dst, "example.txt"))
if err != nil {
	// handle error
}
fmt.Println(string(content))

Output:

example content

func UnpackTo added in v1.0.0 

func UnpackTo(ctx context.Context, t Target, dst string, src io.Reader, cfg *Config) error

UnpackTo unpacks the given source to the destination, according to the given configuration, using the given Target. If cfg is nil, the default configuration is used for extraction. If an error occurs, it is returned.

Example 
var (
	ctx = context.Background()      // context for cancellation
	tm  = extract.NewTargetMemory() // create a new in-memory filesystem
	dst = ""                        // root of in-memory filesystem
	src = openFile("example.zip")   // source reader
	cfg = extract.NewConfig()       // custom config for extraction
)

// unpack
if err := extract.UnpackTo(ctx, tm, dst, src, cfg); err != nil {
	// handle error
}

// read extracted file using fs package
content, err := fs.ReadFile(tm, "example.txt")
if err != nil {
	// handle error
}
fmt.Println(string(content))

Output:

example content

Types

type Config added in v1.0.0 

type Config struct {
	// contains filtered or unexported fields
}

Config provides a configuration struct and options to adjust the configuration.

The configuration struct holds all configuration options for the extraction process. The configuration options can be adjusted using the option pattern style.

The default configuration is designed to be secure by default and prevent exhaustion, path traversal and symlink attacks.

func NewConfig added in v1.0.0 

func NewConfig(opts ...ConfigOption) *Config

NewConfig is a generator option that takes opts as adjustments of the default configuration in an option pattern style.

func (*Config) CacheInMemory added in v1.0.0 

func (c *Config) CacheInMemory() bool

CacheInMemory returns true if caching in memory is enabled. This applies only to the extraction of zip archives, which are provided as a stream.

If set to false, the cache is stored on disk to avoid memory exhaustion.

func (*Config) CheckExtractionSize added in v1.0.0 

func (c *Config) CheckExtractionSize(fileSize int64) error

CheckExtractionSize checks if fileSize exceeds configured maximum. If the maximum is exceeded, a ErrMaxExtractionSizeExceeded error is returned.

func (*Config) CheckMaxFiles added in v1.0.0 

func (c *Config) CheckMaxFiles(counter int64) error

CheckMaxFiles checks if counter exceeds the configured maximum. If the maximum is exceeded, a ErrMaxFilesExceeded error is returned.

func (*Config) ContinueOnError added in v1.0.0 

func (c *Config) ContinueOnError() bool

ContinueOnError returns true if the extraction should continue on error.

func (*Config) ContinueOnUnsupportedFiles added in v1.0.0 

func (c *Config) ContinueOnUnsupportedFiles() bool

ContinueOnUnsupportedFiles returns true if unsupported files, e.g., FIFO, block or character devices, should be skipped.

If symlinks are not allowed and a symlink is found, it is considered an unsupported file.

func (*Config) CreateDestination added in v1.0.0 

func (c *Config) CreateDestination() bool

CreateDestination returns true if the destination directory should be created if it does not exist.

func (*Config) CustomCreateDirMode added in v1.0.0 

func (c *Config) CustomCreateDirMode() fs.FileMode

CustomCreateDirMode returns the file mode for created directories, that are not defined in the archive. (respecting umask)

func (*Config) CustomDecompressFileMode added in v1.0.0 

func (c *Config) CustomDecompressFileMode() fs.FileMode

CustomDecompressFileMode returns the file mode for a decompressed file. (respecting umask)

func (*Config) DenySymlinkExtraction added in v1.0.0 

func (c *Config) DenySymlinkExtraction() bool

DenySymlinkExtraction returns true if symlinks are NOT allowed.

func (*Config) DropFileAttributes added in v1.1.0 

func (c *Config) DropFileAttributes() bool

DropFileAttributes returns true if the file attributes should be dropped.

func (*Config) ExtractType added in v1.0.0 

func (c *Config) ExtractType() string

ExtractType returns the specified extraction type.

func (*Config) Logger added in v1.0.0 

func (c *Config) Logger() logger

Logger returns the logger.

func (*Config) MaxExtractionSize added in v1.0.0 

func (c *Config) MaxExtractionSize() int64

MaxExtractionSize returns the maximum size over all decompressed and extracted files.

func (*Config) MaxFiles added in v1.0.0 

func (c *Config) MaxFiles() int64

MaxFiles returns the maximum of files (including folder and symlinks) in an archive.

func (*Config) MaxInputSize added in v1.0.0 

func (c *Config) MaxInputSize() int64

MaxInputSize returns the maximum size of the input.

func (*Config) NoUntarAfterDecompression added in v1.0.0 

func (c *Config) NoUntarAfterDecompression() bool

NoUntarAfterDecompression returns true if tar.gz should NOT be untared after decompression.

func (*Config) Overwrite added in v1.0.0 

func (c *Config) Overwrite() bool

Overwrite returns true if files should be overwritten in the destination.

func (*Config) Patterns added in v1.0.0 

func (c *Config) Patterns() []string

Patterns returns a list of unix-filepath patterns to match files to extract Patterns are matched using filepath.Match(https://golang.org/pkg/path/filepath/#Match).

func (*Config) PreserveOwner added in v1.1.0 

func (c *Config) PreserveOwner() bool

PreserveOwner returns true if the owner of the extracted files should be preserved. This option is only available on Unix systems requiring root privileges and tar archives as input.

func (*Config) SetNoUntarAfterDecompression added in v1.0.0 

func (c *Config) SetNoUntarAfterDecompression(b bool)

SetNoUntarAfterDecompression sets the noUntarAfterDecompression flag. If true, tar.gz files are not untared after decompression.

func (*Config) TelemetryHook added in v1.0.0 

func (c *Config) TelemetryHook() TelemetryHook

TelemetryHook returns the telemetry hook. 

func (c *Config) TraverseSymlinks() bool

TraverseSymlinks returns true if symlinks should be traversed during extraction.

type ConfigOption added in v1.0.0 

type ConfigOption func(*Config)

ConfigOption is a function pointer to implement the option pattern

func WithCacheInMemory added in v1.0.0 

func WithCacheInMemory(cache bool) ConfigOption

WithCacheInMemory options pattern function to enable/disable caching in memory. This applies only to the extraction of zip archives, which are provided as a stream.

If set to false, the cache is stored on disk to avoid memory exhaustion.

func WithContinueOnError added in v1.0.0 

func WithContinueOnError(yes bool) ConfigOption

WithContinueOnError options pattern function to continue on error during extraction. If set to true, the error is logged and the extraction continues. If set to false, the extraction stops and returns the error.

func WithContinueOnUnsupportedFiles added in v1.0.0 

func WithContinueOnUnsupportedFiles(ctd bool) ConfigOption

WithContinueOnUnsupportedFiles options pattern function to enable/disable skipping unsupported files. An unsupported file is a file that is not supported by the extraction algorithm. If symlinks are not allowed and a symlink is found, it is considered an unsupported file.

func WithCreateDestination added in v1.0.0 

func WithCreateDestination(create bool) ConfigOption

WithCreateDestination options pattern function to create destination directory if it does not exist.

func WithCustomCreateDirMode added in v1.0.0 

func WithCustomCreateDirMode(mode fs.FileMode) ConfigOption

WithCustomCreateDirMode options pattern function to set the file mode for created directories, that are not defined in the archive. (respecting umask)

func WithCustomDecompressFileMode added in v1.0.0 

func WithCustomDecompressFileMode(mode fs.FileMode) ConfigOption

WithCustomDecompressFileMode options pattern function to set the file mode for a decompressed file. (respecting umask)

func WithDenySymlinkExtraction added in v1.0.0 

func WithDenySymlinkExtraction(deny bool) ConfigOption

WithDenySymlinkExtraction options pattern function to deny symlink extraction.

func WithDropFileAttributes added in v1.1.0 

func WithDropFileAttributes(drop bool) ConfigOption

WithDropFileAttributes options pattern function to drop the file attributes of the extracted files.

func WithExtractType added in v1.0.0 

func WithExtractType(extractionType string) ConfigOption

WithExtractType options pattern function to set the extraction type in the Config. 

func WithInsecureTraverseSymlinks(traverse bool) ConfigOption

WithInsecureTraverseSymlinks options pattern function to traverse symlinks during extraction.

func WithLogger added in v1.0.0 

func WithLogger(logger logger) ConfigOption

WithLogger options pattern function to set a custom logger.

func WithMaxExtractionSize added in v1.0.0 

func WithMaxExtractionSize(maxExtractionSize int64) ConfigOption

WithMaxExtractionSize options pattern function to set maximum size over all decompressed 

and extracted files. (-1 to disable check)

func WithMaxFiles added in v1.0.0 

func WithMaxFiles(maxFiles int64) ConfigOption

WithMaxFiles options pattern function to set maximum number of extracted, files, directories and symlinks during the extraction. (-1 to disable check)

func WithMaxInputSize added in v1.0.0 

func WithMaxInputSize(maxInputSize int64) ConfigOption

WithMaxInputSize options pattern function to set MaxInputSize for extraction input file. (-1 to disable check)

func WithNoUntarAfterDecompression added in v1.0.0 

func WithNoUntarAfterDecompression(disable bool) ConfigOption

WithNoUntarAfterDecompression options pattern function to enable/disable combined tar.gz extraction.

func WithOverwrite added in v1.0.0 

func WithOverwrite(enable bool) ConfigOption

WithOverwrite options pattern function specify if files should be overwritten in the destination.

func WithPatterns added in v1.0.0 

func WithPatterns(pattern ...string) ConfigOption

WithPatterns options pattern function to set filepath pattern, that files need to match to be extracted. Patterns are matched using pkg/path/filepath.Match.

func WithPreserveOwner added in v1.1.0 

func WithPreserveOwner(preserve bool) ConfigOption

WithPreserveOwner options pattern function to preserve the owner of the extracted files. This option is only available on Unix systems requiring root privileges and tar archives as input.

func WithTelemetryHook added in v1.0.0 

func WithTelemetryHook(hook TelemetryHook) ConfigOption

WithTelemetryHook options pattern function to set a [telemetry.TelemetryHook], which is called after extraction.

type Target added in v1.0.0 

type Target interface {
	// CreateFile creates a file at the specified path with src as content. The mode parameter is the file mode that
	// should be set on the file. If the file already exists and overwrite is false, an error should be returned. If the
	// file does not exist, it should be created. The size of the file should not exceed maxSize. If the file is created
	// successfully, the number of bytes written should be returned. If an error occurs, the number of bytes written
	// should be returned along with the error. If maxSize < 0, the file size is not limited.
	CreateFile(path string, src io.Reader, mode fs.FileMode, overwrite bool, maxSize int64) (int64, error)

	// CreateDir creates at the specified path with the specified mode. If the directory already exists, nothing is done.
	// The function returns an error if there's a problem creating the directory. If the function completes successfully,
	// it returns nil.
	CreateDir(path string, mode fs.FileMode) error

	// CreateSymlink creates a symbolic link from newname to oldname. If newname already exists and overwrite is false,
	// the function returns an error. If newname already exists and overwrite is true, the function may overwrite the
	// existing symlink.
	CreateSymlink(oldname string, newname string, overwrite bool) error

	// Lstat see docs for os.Lstat. Main purpose is to check for symlinks in the extraction path
	// and for zip-slip attacks.
	Lstat(path string) (fs.FileInfo, error)

	// Stat see docs for os.Stat. Main purpose is to check if a symlink is pointing to a file or directory.
	Stat(path string) (fs.FileInfo, error)

	// Chmod see docs for os.Chmod. Main purpose is to set the file mode of a file or directory.
	Chmod(name string, mode fs.FileMode) error

	// Chtimes see docs for os.Chtimes. Main purpose is to set the file times of a file or directory.
	Chtimes(name string, atime, mtime time.Time) error

	// Lchtimes see docs for os.Lchtimes. Main purpose is to set the file times of a file or directory.
	Lchtimes(name string, atime, mtime time.Time) error

	// Chown see docs for os.Chown. Main purpose is to set the file owner and group of a file or directory.
	Chown(name string, uid, gid int) error
}

Target specifies all function that are needed to be implemented to extract contents from an archive

type TargetDisk added in v1.0.0 

type TargetDisk struct{}

TargetDisk is the struct type that holds all information for interacting with the filesystem

func NewTargetDisk added in v1.0.0 

func NewTargetDisk() *TargetDisk

NewTargetDisk creates a new Os and applies provided options from opts

Example 
var (
	ctx = context.Background()    // context for cancellation
	td  = extract.NewTargetDisk() // local filesystem
	dst = createDirectory("out")  // create destination directory
	src = openFile("example.zip") // source reader
	cfg = extract.NewConfig()     // custom config for extraction
)

// unpack
if err := extract.UnpackTo(ctx, td, dst, src, cfg); err != nil {
	// handle error
}

// read extracted file
content, err := os.ReadFile(filepath.Join(dst, "example.txt"))
if err != nil {
	// handle error
}
fmt.Println(string(content))

Output:

example content

func (*TargetDisk) Chmod added in v1.1.0 

func (d *TargetDisk) Chmod(name string, mode fs.FileMode) error

Chmod changes the mode of the named file to mode.

func (*TargetDisk) Chown added in v1.1.0 

func (d *TargetDisk) Chown(name string, uid, gid int) error

Chown changes the numeric uid and gid of the named file.

func (*TargetDisk) Chtimes added in v1.1.0 

func (d *TargetDisk) Chtimes(name string, atime, mtime time.Time) error

Chtimes changes the access and modification times of the named file.

func (*TargetDisk) CreateDir added in v1.0.0 

func (d *TargetDisk) CreateDir(path string, mode fs.FileMode) error

CreateDir creates a directory at the specified path with the specified mode. If the directory already exists, nothing is done.

func (*TargetDisk) CreateFile added in v1.0.0 

func (d *TargetDisk) CreateFile(path string, src io.Reader, mode fs.FileMode, overwrite bool, maxSize int64) (int64, error)

CreateFile creates a file at the specified path with src as content. The mode parameter is the file mode that should be set on the file. If the file already exists and overwrite is false, an error should be returned. If the file does not exist, it should be created. The size of the file should not exceed maxSize. If the file is created successfully, the number of bytes written should be returned. If an error occurs, the number of bytes written should be returned along with the error. If maxSize < 0, the file size is not limited. 

func (d *TargetDisk) CreateSymlink(oldname string, newname string, overwrite bool) error

CreateSymlink creates a symbolic link from newname to oldname. If newname already exists and overwrite is false, an error should be returned.

func (*TargetDisk) Lchtimes added in v1.1.0 

func (d *TargetDisk) Lchtimes(name string, atime, mtime time.Time) error

Lchtimes changes the access and modification times of the named file.

func (*TargetDisk) Lstat added in v1.0.0 

func (d *TargetDisk) Lstat(name string) (fs.FileInfo, error)

Lstat returns the FileInfo structure describing the named file. If there is an error, it will be of type *PathError.

func (*TargetDisk) Stat added in v1.0.0 

func (d *TargetDisk) Stat(name string) (os.FileInfo, error)

Stat returns the FileInfo structure describing the named file. If there is an error, it will be of type *PathError.

type TargetMemory added in v1.0.0 

type TargetMemory struct {
	fs.FS
	// contains filtered or unexported fields
}

TargetMemory is an in-memory filesystem implementation that can be used to create, read, and write files in memory. It can also be used to create directories and symlinks. Permissions (such as owner or group) are not enforced.

func NewTargetMemory added in v1.0.0 

func NewTargetMemory() *TargetMemory

NewTargetMemory creates a new in-memory filesystem.

Example 
var (
	ctx = context.Background()      // context for cancellation
	tm  = extract.NewTargetMemory() // create a new in-memory filesystem
	dst = ""                        // root of in-memory filesystem
	src = openFile("example.zip")   // source reader
	cfg = extract.NewConfig()       // custom config for extraction
)

// unpack
if err := extract.UnpackTo(ctx, tm, dst, src, cfg); err != nil {
	// handle error
}

if err := fs.WalkDir(tm, ".", func(path string, d fs.DirEntry, err error) error {
	if path == "." {
		return nil
	}
	fmt.Println(path)
	return nil
}); err != nil {
	fmt.Printf("failed to walk memory filesystem: %s", err)
	return
}

Output:

example.txt

func (*TargetMemory) Chmod added in v1.1.0 

func (m *TargetMemory) Chmod(path string, mode fs.FileMode) error

Chmod changes the mode of the file at the given path. If the file does not exist, an error is returned.

func (*TargetMemory) Chown added in v1.1.0 

func (m *TargetMemory) Chown(path string, uid, gid int) error

Chown changes the owner and group of the file at the given path. If the file does not exist, an error is returned.

func (*TargetMemory) Chtimes added in v1.1.0 

func (m *TargetMemory) Chtimes(path string, atime time.Time, mtime time.Time) error

Chtime changes the access and modification times of the file at the given path. If the file does not exist, an error is returned.

func (*TargetMemory) CreateDir added in v1.0.0 

func (m *TargetMemory) CreateDir(path string, mode fs.FileMode) error

CreateDir creates a new directory in the in-memory filesystem. If the directory already exists, nothing is done. If the directory does not exist, it is created. The directory is created with the given mode. If the directory is created successfully, nil is returned.

func (*TargetMemory) CreateFile added in v1.0.0 

func (m *TargetMemory) CreateFile(path string, src io.Reader, mode fs.FileMode, overwrite bool, maxSize int64) (int64, error)

CreateFile creates a new file in the in-memory filesystem. The file is created with the given mode. If the overwrite flag is set to false and the file already exists, an error is returned. If the overwrite flag is set to true, the file is overwritten. The maxSize parameter can be used to limit the size of the file. If the file exceeds the maxSize, an error is returned. If the file is created successfully, the number of bytes written is returned. 

func (m *TargetMemory) CreateSymlink(oldName string, newName string, overwrite bool) error

CreateSymlink creates a new symlink in the in-memory filesystem. If the overwrite flag is set to false and the symlink already exists, an error is returned. If the overwrite flag is set to true, the symlink is overwritten. If the symlink is created successfully, nil is returned.

func (*TargetMemory) Glob added in v1.0.0 

func (m *TargetMemory) Glob(pattern string) ([]string, error)

Glob implements the io/fs.GlobFS interface. It returns the names of all files matching pattern.

func (*TargetMemory) Lchtimes added in v1.1.0 

func (m *TargetMemory) Lchtimes(path string, atime time.Time, mtime time.Time) error

Lchtimes changes the access and modification times of the file at the given path. If the file does not exist, an error is returned.

func (*TargetMemory) Lstat added in v1.0.0 

func (m *TargetMemory) Lstat(path string) (fs.FileInfo, error)

Lstat returns the FileInfo for the given path. If the path is a symlink, the FileInfo for the symlink is returned. If the path does not exist, an error is returned.

golang/go#49580 proposes adding a standard io/fs.SymlinkFS interface to the io/fs package, which discusses if the Lstat method should be moved to the io/fs.SymlinkFS interface. ref: https://github.com/golang/go/issues/49580#issuecomment-2344253737

func (*TargetMemory) Open added in v1.0.0 

func (m *TargetMemory) Open(path string) (fs.File, error)

Open implements the io/fs.FS interface. It opens the file at the given path. If the file does not exist, an error is returned. If the file is a directory, an error is returned. If the file is a symlink, the target of the symlink is returned.

func (*TargetMemory) ReadDir added in v1.0.0 

func (m *TargetMemory) ReadDir(path string) ([]fs.DirEntry, error)

ReadDir implements the io/fs.ReadDirFS interface. It reads the directory named by dirname and returns a list of directory entries sorted by filename.

func (*TargetMemory) ReadFile added in v1.0.0 

func (m *TargetMemory) ReadFile(path string) ([]byte, error)

ReadFile implements the io/fs.ReadFileFS interface. It reads the file named by filename and returns the contents. 

func (m *TargetMemory) Readlink(path string) (string, error)

Readlink returns the target of the symlink at the given path. If the path is not a symlink, an error is returned.

golang/go#49580 proposes adding a standard io/fs.SymlinkFS interface to the io/fs package. If this proposal is accepted, the Readlink method will be moved to the io/fs.SymlinkFS interface. Until then, the Readlink method is kept not exposed.

func (*TargetMemory) Remove added in v1.0.0 

func (m *TargetMemory) Remove(path string) error

Remove removes the file or directory at the given path. If the path is invalid, an error is returned. If the path does not exist, no error is returned.

func (*TargetMemory) Stat added in v1.0.0 

func (m *TargetMemory) Stat(path string) (fs.FileInfo, error)

Stat implements the io/fs.File and io/fs.StatFS interfaces. It returns the io/fs.FileInfo for the given path.

func (*TargetMemory) Sub added in v1.0.0 

func (m *TargetMemory) Sub(subPath string) (fs.FS, error)

Sub implements the io/fs.SubFS interface. It returns a new FS representing the subtree rooted at dir.

type TelemetryData added in v1.0.0 

type TelemetryData struct {
	// ExtractedDirs is the number of extracted directories
	ExtractedDirs int64 `json:"extracted_dirs"`

	// ExtractionDuration is the time it took to extract the archive
	ExtractionDuration time.Duration `json:"extraction_duration"`

	// ExtractionErrors is the number of errors during extraction
	ExtractionErrors int64 `json:"extraction_errors"`

	// ExtractedFiles is the number of extracted files
	ExtractedFiles int64 `json:"extracted_files"`

	// ExtractionSize is the size of the extracted files
	ExtractionSize int64 `json:"extraction_size"`

	// ExtractedSymlinks is the number of extracted symlinks
	ExtractedSymlinks int64 `json:"extracted_symlinks"`

	// ExtractedType is the type of the archive
	ExtractedType string `json:"extracted_type"`

	// InputSize is the size of the input
	InputSize int64 `json:"input_size"`

	// LastExtractionError is the last error during extraction
	LastExtractionError error `json:"last_extraction_error"`

	// PatternMismatches is the number of skipped files
	PatternMismatches int64 `json:"pattern_mismatches"`

	// UnsupportedFiles is the number of skipped unsupported files
	UnsupportedFiles int64 `json:"unsupported_files"`

	// LastUnsupportedFile is the last skipped unsupported file
	LastUnsupportedFile string `json:"last_unsupported_file"`
}

TelemetryData holds all telemetry data of an extraction.

func (TelemetryData) MarshalJSON added in v1.0.0 

func (m TelemetryData) MarshalJSON() ([]byte, error)

MarshalJSON implements the encoding/json.Marshaler interface.

func (TelemetryData) String added in v1.0.0 

func (m TelemetryData) String() string

String returns a string representation of TelemetryData.

type TelemetryHook added in v1.0.0 

type TelemetryHook func(context.Context, *TelemetryData)

TelemetryHook is a function type that performs operations on TelemetryData after an extraction has finished which can be used to submit the TelemetryData to a telemetry service, for example.

type UnsupportedFileError added in v1.0.0 

type UnsupportedFileError struct {
	// contains filtered or unexported fields
}

UnsupportedFileError is an error that indicates that the file is not supported.

func (UnsupportedFileError) Error added in v1.0.0 

func (e UnsupportedFileError) Error() string

Error returns the error message.

func (*UnsupportedFileError) Filename added in v1.0.0 

func (e *UnsupportedFileError) Filename() string

Filename returns the filename of the unsupported file.

func (*UnsupportedFileError) Unwrap added in v1.0.0 

func (e *UnsupportedFileError) Unwrap() error

Unwrap returns the underlying error.

Source Files

View all Source files

Directories

Path Synopsis
cmd
goextract

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.
JackTT - Gopher 🇻🇳