acid

acid

An embedded in-memory key-value store with unlimited indexes. Don't use this thing in production unless you have a really good use case for ephemeral data. Totally use it in your tests, though.

Installation

Go 1.18+ required, as acid uses generics.

go get -u codeberg.org/ess/acid

Core Concepts

Buckets

A bucket is a key-value store. Think of it as a thread-safe array with secondary indexes.

The Bucket type must be instantiated with a comparable type. That covers most types that one can imagine, so you should be good.

In acid, a bucket can have any number of indexes, and they can be either plain ol' indexes, or they can be unique indexes.

By default, a bucket has no indexes, which means that you can't really query it, so you should add some indexes.

Indexes

As is the case in any system that allows indexing, an index is effectively a map that connects a specific data type to the exact location of an item in the system.

In acid, a bucket can have any number of indexes, and they can be either plain ol' indexes, or they can be unique indexes.

There are two ways to add indexes to a bucket. You can do it during instantiation by chaining:

data := acid.Bucket[Person]().
  WithIndex("id", acid.UniqueIndex[int]()).
  WithIndex("name", acid.Index[string]()).
  WithIndex("age", acid.Index[int]).
  Finalize()

You can also add indexes procedurally:

data := acid.Bucket[Person]()
data.WithIndex("id", acid.UniqueIndex[int]())
data.WithIndex("name", acid.Index[string]())
data.WithIndex("age", acid.Index[int]())
data.Finalize()

As you'll note, the last step on each of these is to Finalize() the bucket. This lets the bucket know that it's ready and we're not planning on adding any further configuration to it.

You can still attempt to make WithIndex calls after finalization, but they will not actually do anything.

Querying

Most data-related actions on a bucket require a Query. This is just a fancy way to let the action know what stored items you're interested in. For example, when adding the hypothetical Person items that are referenced above, you would do something like this:

data.Add(
  somePerson,
  acid.Query().
    Where("id", somePerson.ID).
    Where("name", somePerson.Name).
    Where("age", somePerson.Age))

Note: The where clauses of a query are always an AND relationship.

Now that you have an item added (and you indexed it), you can query for it. There are two ways to do this: All and Get.

With Get, the idea is that you want to retrive exactly one item from the bucket, and your query should narrow that down as far as you see fit. This is much less error-prone if you specify a unique index in your bucket and query, but it's not required. The bucket will complain if your query resolves to zero or more than one item.

person, err := data.Get(acid.Query(Where("id", 1)))

On the other hand, All will get you all items in the bucket that match your query:

// To get literally all items, give it an empty query
all := data.All(acid.Query())

// To get some items, give it a query, but think about omiting any
// unique index references
some := data.All(acid.Query().Where("age", 20))

Full Usage Example

package main

import (
	"fmt"

	"codeberg.org/ess/acid"
	"codeberg.org/ess/acid/storage"
)

func main() {
	cohort := newPeople()

	p1 := &person{ID: 1, Name: "Jack Randomhacker", Age: 33}
	p2 := &person{ID: 2, Name: "Jill Randomhacker", Age: 23}
	p3 := &person{ID: 3, Name: "Rando Bystandarius", Age: 23}

	for _, p := range []*person{p1, p2, p3} {
		err := cohort.Add(p)
		if err != nil {
			panic(err)
		}
	}

	jills := cohort.Named("Jill Randomhacker")
	if len(jills) == 0 {
		panic("there are no jills!")
	}

	fmt.Println("jills:")
	if len(jills) == 0 {
		fmt.Println("\t:sadface: no jills")
	} else {
		for _, p := range jills {
			fmt.Println("\t* ", p)
		}
	}

	fmt.Println("\nerrybody:")
	for _, p := range cohort.All() {
		fmt.Println("\t* ", p)
	}

	fmt.Println()

	err := cohort.Add(&person{ID: 1, Name: "Badius Actorius", Age: 12})
	if err == nil {
		panic("should have failed to add a duplicate ID, but instead we got a bad actor")
	}

	fmt.Println("bad actor avoided")
}

type person struct {
	ID   int
	Name string
	Age  int
}

func (p *person) String() string {
	return fmt.Sprintf("[%d] %s who is %d years old", p.ID, p.Name, p.Age)
}

type people struct {
	bucket *storage.Bucket[*person]
}

func newPeople() *people {
	bucket := acid.Bucket[*person]().
		WithIndex("id", acid.UniqueIndex[int]()).
		WithIndex("name", acid.Index[string]()).
		WithIndex("age", acid.Index[int]()).
		Finalize()

	return &people{bucket: bucket}
}

func (repo *people) Add(candidate *person) error {
	return repo.bucket.Add(
		candidate,
		acid.Query().
			Where("id", candidate.ID).
			Where("name", candidate.Name).
			Where("age", candidate.Age))
}

func (repo *people) Get(id int) (*person, error) {
	return repo.bucket.Get(acid.Query().Where("id", id))
}

func (repo *people) Named(name string) []*person {
	return repo.bucket.All(acid.Query().Where("name", name))
}

func (repo *people) All() []*person {
	return repo.bucket.All(acid.Query())
}

History

• v1.0.2 - Bucket.Remove now works
• v1.0.1 - Corrected some bucket errors
• v1.0.0 - Initial Release