storage

package
v0.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 17, 2024 License: GPL-3.0 Imports: 16 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BloomConfig

type BloomConfig struct {
	Size     uint
	HashFunc []HasherType
}

BloomConfig holds the configurable elements of a bloom filter

type BloomFilter

type BloomFilter interface {
	//Add adds the value to the bloom filter
	Add([]byte)
	// MayContain checks if the value is in in the set. If it returns 'false',
	//the item is definitely not in the DB
	MayContain([]byte) bool
	//Clear sets all the bits from the filter to 0
	Clear()
}

BloomFilter provides services for filtering database requests

func NewBloomFilter

func NewBloomFilter(conf BloomConfig) (BloomFilter, error)

NewBloomFilter creates a new bloom filter from bloom filter config

type CacheConfig

type CacheConfig struct {
	Size   uint32
	Type   CacheType
	Shards uint32
}

CacheConfig holds the configurable elements of a cache

type CacheType

type CacheType string

CacheType represents the type of the supported caches

const (
	LRUCache         CacheType = "LRU"
	FIFOShardedCache CacheType = "FIFOSharded"
)

LRUCache is currently the only supported Cache type

type Cacher

type Cacher interface {
	// Clear is used to completely clear the cache.
	Clear()
	// Put adds a value to the cache.  Returns true if an eviction occurred.
	Put(key []byte, value interface{}) (evicted bool)
	// Get looks up a key's value from the cache.
	Get(key []byte) (value interface{}, ok bool)
	// Has checks if a key is in the cache, without updating the
	// recent-ness or deleting it for being stale.
	Has(key []byte) bool
	// Peek returns the key value (or undefined if not found) without updating
	// the "recently used"-ness of the key.
	Peek(key []byte) (value interface{}, ok bool)
	// HasOrAdd checks if a key is in the cache  without updating the
	// recent-ness or deleting it for being stale,  and if not, adds the value.
	// Returns whether found and whether an eviction occurred.
	HasOrAdd(key []byte, value interface{}) (ok, evicted bool)
	// Remove removes the provided key from the cache.
	Remove(key []byte)
	// RemoveOldest removes the oldest item from the cache.
	RemoveOldest()
	// Keys returns a slice of the keys in the cache, from oldest to newest.
	Keys() [][]byte
	// Len returns the number of items in the cache.
	Len() int
	// RegisterHandler registers a new handler to be called when a new data is added
	RegisterHandler(func(key []byte))
}

Cacher provides caching services

func NewCache

func NewCache(cacheType CacheType, size uint32, shards uint32) (Cacher, error)

NewCache creates a new cache from a cache config TODO: add a cacher factory or a cacheConfig param instead

type DBConfig

type DBConfig struct {
	FilePath string
	Type     DBType
}

DBConfig holds the configurable elements of a database

type DBType

type DBType string

DBType represents the type of the supported databases

const (
	LvlDB    DBType = "LvlDB"
	BadgerDB DBType = "BadgerDB"
	BoltDB   DBType = "BoltDB"
)

LvlDB currently the only supported DBs More to be added

type HasherType

type HasherType string

HasherType represents the type of the supported hash functions

const (
	// Keccak is the string representation of the keccak hashing function
	Keccak HasherType = "Keccak"
	// Blake2b is the string representation of the blake2b hashing function
	Blake2b HasherType = "Blake2b"
	// Fnv is the string representation of the fnv hashing function
	Fnv HasherType = "Fnv"
)

func (HasherType) NewHasher

func (h HasherType) NewHasher() (hashing.Hasher, error)

NewHasher will return a hasher implementation form the string HasherType

type Persister

type Persister interface {
	// Put add the value to the (key, val) persistance medium
	Put(key, val []byte) error
	// Get gets the value associated to the key
	Get(key []byte) ([]byte, error)
	// Has returns true if the given key is present in the persistance medium
	Has(key []byte) error
	// Init initializes the persistance medium and prepares it for usage
	Init() error
	// Close closes the files/resources associated to the persistance medium
	Close() error
	// Remove removes the data associated to the given key
	Remove(key []byte) error
	// Destroy removes the persistance medium stored data
	Destroy() error
}

Persister provides storage of data services in a database like construct

func NewDB

func NewDB(dbType DBType, path string) (Persister, error)

NewDB creates a new database from database config

type Storer

type Storer interface {
	Put(key, data []byte) error
	Get(key []byte) ([]byte, error)
	Has(key []byte) error
	HasOrAdd(key []byte, value []byte) error
	Remove(key []byte) error
	ClearCache()
	DestroyUnit() error
}

Storer provides storage services in a two layered storage construct, where the first layer is represented by a cache and second layer by a persitent storage (DB-like)

type Unit

type Unit struct {
	// contains filtered or unexported fields
}

Unit represents a storer's data bank holding the cache, persistance unit and bloom filter

func NewStorageUnit

func NewStorageUnit(c Cacher, p Persister) (*Unit, error)

NewStorageUnit is the constructor for the storage unit, creating a new storage unit from the given cacher and persister.

func NewStorageUnitFromConf

func NewStorageUnitFromConf(cacheConf CacheConfig, dbConf DBConfig, bloomFilterConf BloomConfig) (*Unit, error)

NewStorageUnitFromConf creates a new storage unit from a storage unit config

func NewStorageUnitWithBloomFilter

func NewStorageUnitWithBloomFilter(c Cacher, p Persister, b BloomFilter) (*Unit, error)

NewStorageUnitWithBloomFilter is the constructor for the storage unit, creating a new storage unit from the given cacher, persister and bloom filter.

func (*Unit) ClearCache

func (s *Unit) ClearCache()

ClearCache cleans up the entire cache

func (*Unit) DestroyUnit

func (s *Unit) DestroyUnit() error

DestroyUnit cleans up the bloom filter, the cache, and the db

func (*Unit) Get

func (s *Unit) Get(key []byte) ([]byte, error)

Get searches the key in the cache. In case it is not found, it searches for the key in bloom filter first and if found it further searches it in the associated database. In case it is found in the database, the cache is updated with the value as well.

func (*Unit) Has

func (s *Unit) Has(key []byte) error

Has checks if the key is in the Unit. It first checks the cache. If it is not found, it checks the bloom filter and if present it checks the db

func (*Unit) HasOrAdd

func (s *Unit) HasOrAdd(key []byte, value []byte) error

HasOrAdd checks if the key is present in the storage and if not adds it. it updates the cache either way it returns if the value was originally found

func (*Unit) Put

func (s *Unit) Put(key, data []byte) error

Put adds data to both cache and persistance medium and updates the bloom filter

func (*Unit) Remove

func (s *Unit) Remove(key []byte) error

Remove removes the data associated to the given key from both cache and persistance medium

type UnitConfig

type UnitConfig struct {
	CacheConf CacheConfig
	DBConf    DBConfig
	BloomConf BloomConfig
}

UnitConfig holds the configurable elements of the storage unit

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
JackTT - Gopher 🇻🇳