snapshotter

package
v0.10.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 7, 2025 License: Apache-2.0 Imports: 29 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	DiscoverSnapshotsMsg = "discover_snapshots"

	ProtocolIDSnapshotCatalog protocol.ID = "/kwil/snapcat/1.0.0"
	ProtocolIDSnapshotChunk   protocol.ID = "/kwil/snapchunk/1.0.0"
	ProtocolIDSnapshotMeta    protocol.ID = "/kwil/snapmeta/1.0.0"

	SnapshotCatalogNS = "snapshot-catalog" // namespace on which snapshot catalogs are advertised
)
View Source 
const (
	DefaultSnapshotFormat = 0

	CreateSchema   = "CREATE SCHEMA"
	CreateTable    = "CREATE TABLE"
	CreateFunction = "CREATE FUNCTION"
)
View Source 
const HashLen int = 32

Variables

This section is empty.

Functions

This section is empty.

Types

type BlockStore 

type BlockStore interface {
	GetByHeight(height int64) (types.Hash, *ktypes.Block, *ktypes.CommitInfo, error)
	Best() (height int64, blkHash, appHash types.Hash, stamp time.Time)
}

type DBSnapshotter 

type DBSnapshotter interface {
	CreateSnapshot(ctx context.Context, height uint64, snapshotID string, schemas, excludeTables []string, excludeTableData []string) (*Snapshot, error)
}

type NamespaceManager 

type NamespaceManager interface {
	ListPostgresSchemasToDump() []string
}

type Snapshot 

type Snapshot struct {
	Height       uint64          `json:"height"`
	Format       uint32          `json:"format"`
	ChunkHashes  [][HashLen]byte `json:"chunk_hashes"`
	ChunkCount   uint32          `json:"chunk_count"`
	SnapshotHash []byte          `json:"hash"`
	SnapshotSize uint64          `json:"size"`
}

Snapshot is the header of a snapshot file representing the snapshot of the database at a certain height. It contains the height, format, chunk count, hash, size, and name of the snapshot. WARNING: This struct CAN NOT be changed without breaking functionality, since it is used for communication between nodes.

func (*Snapshot) MarshalBinary 

func (s *Snapshot) MarshalBinary() ([]byte, error)

func (*Snapshot) SaveAs 

func (s *Snapshot) SaveAs(file string) error

SaveAs saves the snapshot header to a file.

func (*Snapshot) UnmarshalBinary 

func (s *Snapshot) UnmarshalBinary(data []byte) error

type SnapshotChunkReq 

type SnapshotChunkReq struct {
	Height uint64
	Format uint32
	Index  uint32
	Hash   types.Hash // TODO: Is this required? maybe providers serve the chunk only if the snapshot hash matches
}

SnapshotChunkReq is for ProtocolIDSnapshotChunk "/kwil/snapchunk/1.0.0"

func (SnapshotChunkReq) MarshalBinary 

func (r SnapshotChunkReq) MarshalBinary() ([]byte, error)

func (*SnapshotChunkReq) ReadFrom 

func (r *SnapshotChunkReq) ReadFrom(rd io.Reader) (int64, error)

func (*SnapshotChunkReq) UnmarshalBinary 

func (r *SnapshotChunkReq) UnmarshalBinary(data []byte) error

func (SnapshotChunkReq) WriteTo 

func (r SnapshotChunkReq) WriteTo(w io.Writer) (int64, error)

type SnapshotConfig 

type SnapshotConfig struct {
	// Snapshot store configuration
	Enable          bool
	SnapshotDir     string
	MaxSnapshots    int
	RecurringHeight uint64
	DBConfig        *config.DBConfig
}

type SnapshotKey 

type SnapshotKey [sha256.Size]byte

SnapshotKey is a snapshot key used for lookups.

type SnapshotMetadata 

type SnapshotMetadata struct {
	Height      uint64     `json:"height"`
	Format      uint32     `json:"format"`
	Chunks      uint32     `json:"chunk_count"`
	Hash        []byte     `json:"hash"`
	Size        uint64     `json:"size"`
	ChunkHashes [][32]byte `json:"chunk_hashes"`

	AppHash []byte `json:"app_hash"`
}

func (*SnapshotMetadata) Key 

func (s *SnapshotMetadata) Key() SnapshotKey

Key generates a snapshot key, used for lookups. It takes into account not only the height and format, but also the chunks, snapshot hash and chunk hashes in case peers have generated snapshots in a non-deterministic manner. All fields must be equal for the snapshot to be considered the same.

func (*SnapshotMetadata) String 

func (sm *SnapshotMetadata) String() string

type SnapshotReq 

type SnapshotReq struct {
	Height uint64
	Format uint32
}

SnapshotReq is for ProtocolIDSnapshotMeta "/kwil/snapmeta/1.0.0"

func (SnapshotReq) MarshalBinary 

func (r SnapshotReq) MarshalBinary() ([]byte, error)

func (*SnapshotReq) ReadFrom 

func (r *SnapshotReq) ReadFrom(rd io.Reader) (int64, error)

func (*SnapshotReq) UnmarshalBinary 

func (r *SnapshotReq) UnmarshalBinary(data []byte) error

func (SnapshotReq) WriteTo 

func (r SnapshotReq) WriteTo(w io.Writer) (int64, error)

type SnapshotStore 

type SnapshotStore struct {
	// contains filtered or unexported fields
}

func NewSnapshotStore 

func NewSnapshotStore(cfg *SnapshotConfig, bs BlockStore, ns NamespaceManager, logger log.Logger) (*SnapshotStore, error)

func (*SnapshotStore) CreateSnapshot 

func (s *SnapshotStore) CreateSnapshot(ctx context.Context, height uint64, snapshotID string, schemas, excludedTables []string, excludeTableData []string) error

CreateSnapshot creates a new snapshot at the given height and snapshot ID. SnapshotStore ensures that the number of snapshots does not exceed the maximum configured snapshots. If exceeds, it deletes the oldest snapshot. It takes a list of schemas, excludedTables and excludeTableData args to specify the contents of the snapshot. schemas: list of schemas to include in the snapshot excludedTables: list of tables to exclude from the snapshot excludeTableData: list of tables to include schema but exclude data from the snapshot

func (*SnapshotStore) Enabled 

func (s *SnapshotStore) Enabled() bool

func (*SnapshotStore) GetSnapshot 

func (s *SnapshotStore) GetSnapshot(height uint64, _ uint32) *Snapshot

func (*SnapshotStore) IsSnapshotDue 

func (s *SnapshotStore) IsSnapshotDue(height uint64) bool

IsSnapshotDue checks if a snapshot is due at the given height.

func (*SnapshotStore) ListSnapshots 

func (s *SnapshotStore) ListSnapshots() []*Snapshot

List snapshots lists all the registered snapshots in the snapshot store.

func (*SnapshotStore) LoadSnapshotChunk 

func (s *SnapshotStore) LoadSnapshotChunk(height uint64, format uint32, chunkIdx uint32) ([]byte, error)

LoadSnapshotChunk loads a snapshot chunk at the given height and chunk index of given format. It returns the snapshot chunk as a byte slice of max size 16MB. errors if the chunk of chunkIndex corresponding to snapshot at given height and format does not exist.

func (*SnapshotStore) RegisterSnapshot 

func (s *SnapshotStore) RegisterSnapshot(snapshot *Snapshot) error

RegisterSnapshot registers the existing snapshot in the snapshot store. It ensures that the number of snapshots does not exceed the maximum configured snapshots. If exceeds, it deletes the oldest snapshot.

func (*SnapshotStore) RegisterSnapshotStreamHandlers 

func (s *SnapshotStore) RegisterSnapshotStreamHandlers(ctx context.Context, host host.Host, discovery discovery.Discovery)

RegisterSnapshotStreamHandlers registers the snapshot stream handlers if snapshotting is enabled.

type Snapshotter 

type Snapshotter struct {
	// contains filtered or unexported fields
}

func NewSnapshotter 

func NewSnapshotter(cfg *config.DBConfig, dir string, namespaceMgr NamespaceManager, logger log.Logger) *Snapshotter

func (*Snapshotter) CreateSnapshot 

func (s *Snapshotter) CreateSnapshot(ctx context.Context, height uint64, snapshotID string, schemas, excludeTables []string, excludeTableData []string) (*Snapshot, error)

CreateSnapshot creates a snapshot at the given height and snapshotID

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.
JackTT - Gopher 🇻🇳