README
¶
isodb - isolated db (experimental)
isodb could be considered as a simple document database build on top of a git-like data strcture.
Operations always take a commit pointer (a sha256 hash) with a root folder, then, document keys are mapped into sub-folders of this root until the leaf file is found.
At this point, the leaf file has an edge named blob which points to the content of that document.
The whole datastructure is immutable and allow for multiple writers to work on the same database. If their changes are different they will end-up with different commits (different sha256 hash). This will trigger a conflict resolution (not yet implemented).
The database also allows for any sha256 reference to have a human readable name. Updating this ref is atomic and has cas semantics.
Why?
Because I like to work on systems where the nodes might spend hours,days,weeks or even months, without having a chance to sync their writes. Allowing applications to have some guarantees even in such scenarios is very uself (think of a farmer without 3G or satellite connection). They should be able to have sane way to keep information about their systems while they are disconnected.
Eventually isodb will have a turing interpreter with its code being stored as another document. So in this scenario conflict resolution could run a pre-defined script which given the same inputs will generate the same output regardless of where they are executed.
On top of that, commits might be signed so not only commits are immutable but can be verified without having to sync with the person who created it in the first place.
Why not?
The database must not be the fastest one and if you don't have latency issues between your process and your datastorage and using locks to protect the documents/rows is feasible, then there is not reason to use isodb.
Just answer the question: Should your system being able to accept writes even when the host is completely disconnected from others? If yes, then maybe isodb is for you.
How it works?
Check repo_test.go to have an idea of how to use it directly (API needs polishing, so don't judge)
Prior art
- CouchDB
- Couchbase
- Noms
- Git
- Fossil
CouchDB and Couchbase aren't specifically desigend to support the days/weeks/months of disconnected work.
Documentation
¶
Index ¶
- Constants
- type Blob
- type BlobRef
- type BlobRefList
- type Changeset
- type CheckFn
- type Commit
- type DocumentKey
- type Edge
- type EdgeList
- type File
- type HashAlg
- type Idx
- type KV
- type Repo
- func (r *Repo) Apply(cs *Changeset) (BlobRef, error)
- func (r *Repo) GetBlob(ref BlobRef) (Blob, error)
- func (r *Repo) GetCommit(ref BlobRef) (Commit, error)
- func (r *Repo) GetContentAtKey(commitRef BlobRef, key DocumentKey) (Blob, error)
- func (r *Repo) GetFile(ref BlobRef) (File, error)
- func (r *Repo) GetPointer(ptr string) (BlobRef, error)
- func (r *Repo) UpdatePointer(ptr string, newRef, oldRef BlobRef) error
Constants ¶
const ( // ErrInvalidOldRef the expected value for the pointer is old ErrInvalidOldRef = strErr("isodb: invalid old reference") // ErrDocumentNotFound document not found ErrDocumentNotFound = strErr("isodb: document not found") )
const (
// ErrCASNotExecuted indicates that a KV CAS operation didn't work
ErrCASNotExecuted = strErr("isodb: unable to perform CAS operation")
)
const (
// ErrInvalidHashAlgorithm indicates a invalid value for HashAlg
ErrInvalidHashAlgorithm = strErr("isodb: invalid hash algorithm")
)
const ( // IdxNotFound indicates when a value wasn't found in the array IdxNotFound = Idx(-1) )
const ( // Sha256 indicates Sha256 algorithm Sha256 = HashAlg("sha256") )
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Blob ¶
type Blob struct {
// Content of this blob
Content []byte
}
Blob holds raw bytes without any know structure
func NewBlobString ¶
NewBlobString returns a new blob from the given string
func (Blob) Ref ¶
Ref returns the default hash version of this blob, it is just a syntatic sugar for RefAlg
type BlobRef ¶
type BlobRef struct {
// Algorithm used to compute the hash
Alg HashAlg
// Value of the hash itself
Value string
}
BlobRef holds the reference to any blob
type BlobRefList ¶
type BlobRefList []BlobRef
BlobRefList implements a sorted slice of BlobRef objects. Sorted by Alg and Value
func (BlobRefList) Contains ¶
func (bl BlobRefList) Contains(r BlobRef) bool
Contains returns true if the parent is present
func (*BlobRefList) Insert ¶
func (bl *BlobRefList) Insert(r BlobRef) bool
Insert the ref in the list or do nothing if the item is there already.
Returns true if a new item was added
func (BlobRefList) Len ¶
func (bl BlobRefList) Len() int
func (BlobRefList) Less ¶
func (bl BlobRefList) Less(i, j int) bool
func (BlobRefList) Swap ¶
func (bl BlobRefList) Swap(i, j int)
type Changeset ¶
type Changeset struct {
// contains filtered or unexported fields
}
Changeset is used to prepare a commit before actually commiting to it.
Changeset shouldn't be shared between different goroutines as it is not a goroutine-safe structre
func NewChangeset ¶
NewChangeset returns the changeset with the given parents as the previous commit.
It is safe to not provide any parents
func (*Changeset) Put ¶
func (c *Changeset) Put(k DocumentKey, b Blob)
Put the document in the changeset to be later added to the commit
type Commit ¶
type Commit struct {
// Folder points to the root direction of this commit
Folder BlobRef
// Parents points to the list of previous commit before this one
Parents BlobRefList
}
Commit represents a single snapshot of the entire database
type DocumentKey ¶
type DocumentKey struct {
// Set to group documents into a collection
Set string
// Id of the document (32-bit k-sorted ids)
K ksuid.KSUID
}
DocumentKey contains the identification of a document
func NewRandomKey ¶
func NewRandomKey(set string) DocumentKey
NewRandomKey returns a new random key for the given set
type EdgeList ¶
type EdgeList []Edge
EdgeList is a list of edges sorted by Name
func (EdgeList) FindByName ¶
FindByName the edge with the given name and returns it (if found), its index.
Returns -1 if the edge isn't found
func (EdgeList) Insert ¶
Insert will update this EdgeList to include the given edge, returns the updated list if the edge is new or the same list if the edge already exists.
The array is sorted after this operation regardless of it being changed or not.
func (EdgeList) SortInPlace ¶
func (el EdgeList) SortInPlace()
SortInPlace this list, always calls this before serializing to keep the hash consistent
type File ¶
File contains links to sub-folders or files
func (*File) GetFileContent ¶
GetFileContent returns the blob edge link for the given file or empty.
func (*File) SetFileContent ¶
SetFileContent creates a copy of this file with a different blob content
type HashAlg ¶
type HashAlg string
HashAlg lists the available hash algorithms to use
func (HashAlg) ComputeBytes ¶
ComputeBytes the hash using the given algorithm. It is just a syntatic sugar to ComputeReader
type KV ¶
type KV interface {
io.Closer
// PutNew if the key isn't found in the database
PutNew(k string, b Blob) (bool, error)
// Put the given key in the database
Put(k string, b Blob) error
// PutIf updates k if the old version passes the given check function
PutIf(k string, b Blob, check CheckFn) (bool, error)
// CAS update the key if old value matches the expected old (syntaic sugar for PutIf)
CAS(k string, old, new Blob) (bool, error)
// Get returns the value for the given key
Get(k string) (Blob, error)
// Has returns if the key is present in the database
Has(k string) (bool, error)
}
KV abstraction used to perform atomic operations. Empty keys are not allowed.
func NewPersistentKV ¶
NewPersistentKV returns a kv-implementation using badger
type Repo ¶
type Repo struct {
// contains filtered or unexported fields
}
Repo contains all the commits/changes written to the database
func NewPersistentRepo ¶
NewPersistentRepo returns a new Repo with a persistent stored in `folder`.
func NewRepoWithKV ¶
NewRepoWithKV returns a new Repo using the given KV
func (*Repo) Apply ¶
Apply the provided Changeset to the repository and returns the reference to the new commit
func (*Repo) GetContentAtKey ¶
func (r *Repo) GetContentAtKey(commitRef BlobRef, key DocumentKey) (Blob, error)
GetContentAtKey returns the blob at the given key or null if they key does not exist
func (*Repo) GetPointer ¶
GetPointer returns the ref from the given pointer
Source Files