importer

package
v0.0.0-...-2963f48 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 17, 2015 License: MIT Imports: 9 Imported by: 0

Documentation

Overview

package importer implements utilities used to create ipfs DAGs from files and readers

Index

Constants

This section is empty.

Variables

View Source
var BlockSizeLimit = 1048576 // 1 MB

BlockSizeLimit specifies the maximum size an imported block can have.

View Source
var DefaultLinksPerBlock = (roughLinkBlockSize / roughLinkSize)

DefaultLinksPerBlock governs how the importer decides how many links there will be per block. This calculation is based on expected distributions of:

  • the expected distribution of block sizes
  • the expected distribution of link sizes
  • desired access speed

For now, we use:

var roughLinkBlockSize = 1 << 13 // 8KB
var roughLinkSize = 288          // sha256 + framing + name
var DefaultLinksPerBlock = (roughLinkBlockSize / roughLinkSize)

See calc_test.go

View Source
var ErrSizeLimitExceeded = fmt.Errorf("object size limit exceeded")

ErrSizeLimitExceeded signals that a block is larger than BlockSizeLimit.

View Source
var IndirectBlockDataSize = 0

IndirectBlocksCopyData governs whether indirect blocks should copy over data from their first child, and how much. If this is 0, indirect blocks have no data, only links. If this is larger, Indirect blocks will copy as much as (maybe less than) this many bytes.

This number should be <= (BlockSizeLimit - (DefaultLinksPerBlock * LinkSize)) Note that it is not known here what the LinkSize is, because the hash function could vary wildly in size. Exercise caution when setting this option. For safety, it will be clipped to (BlockSizeLimit - (DefaultLinksPerBlock * 256))

Functions

func BuildDagFromFile

func BuildDagFromFile(fpath string, ds dag.DAGService, mp pin.ManualPinner) (*dag.Node, error)

Builds a DAG from the given file, writing created blocks to disk as they are created

func BuildDagFromReader

func BuildDagFromReader(r io.Reader, ds dag.DAGService, mp pin.ManualPinner, spl chunk.BlockSplitter) (*dag.Node, error)

Types

This section is empty.

Directories

Path Synopsis
package chunk implements streaming block splitters
package chunk implements streaming block splitters

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
JackTT - Gopher 🇻🇳