Documentation
¶
Overview ¶
Package simulation implements a full fledged Cosmos SDK application used for executing simulation test suites.
Simulation App ¶
The SimApp type defines an application used for running extensive simulation testing suites. It contains all core modules, including governance, staking, slashing, and distribution.
Simulation is executed with various inputs including the number of blocks to simulate, the block size, whether the app should commit or not, the invariant checking period, and a seed which is used as a source of pseudo-randomness.
In addition to the various inputs, simulation runs mainly in three modes:
1. Completely random where the initial state, module parameters and simulation parameters are pseudo-randomly generated.
2. From a genesis file where the initial state and the module parameters are defined. This mode is helpful for running simulations on a known state such as a live network export where a new (mostly likely breaking) version of the application needs to be tested.
3. From a params file where the initial state is pseudo-randomly generated but the module and simulation parameters can be provided manually. This allows for a more controlled and deterministic simulation setup while allowing the state space to still be pseudo-randomly simulated.
The simulation test suite also supports testing determinism and import/export functionality.
Randomness ¶
Currently, simulation uses a single seed (integer) as a source for a PRNG by which all random operations are executed from. Any call to the PRNG changes all future operations as the internal state of the PRNG is modified. For example, if a new message type is created and needs to be simulated, the new introduced PRNG call will change all subsequent operations.
This may can often be problematic when testing fixes to simulation faults. One current solution to this is to use a params file as mentioned above. In the future the simulation suite is expected to support a series of PRNGs that can be used uniquely per module and simulation component so that they will not effect each others state execution outcome.
Usage ¶
To execute a completely pseudo-random simulation:
$ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
-run=TestFullAppSimulation \
-Enabled=true \
-NumBlocks=100 \
-BlockSize=200 \
-Commit=true \
-Seed=99 \
-Period=5 \
-v -timeout 24h
To execute simulation from a genesis file:
$ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
-run=TestFullAppSimulation \
-Enabled=true \
-NumBlocks=100 \
-BlockSize=200 \
-Commit=true \
-Seed=99 \
-Period=5 \
-Genesis=/path/to/genesis.json \
-v -timeout 24h
To execute simulation from a simulation params file:
$ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
-run=TestFullAppSimulation \
-Enabled=true \
-NumBlocks=100 \
-BlockSize=200 \
-Commit=true \
-Seed=99 \
-Period=5 \
-Params=/path/to/params.json \
-v -timeout 24h
To export the simulation params to a file at a given block height:
$ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
-run=TestFullAppSimulation \
-Enabled=true \
-NumBlocks=100 \
-BlockSize=200 \
-Commit=true \
-Seed=99 \
-Period=5 \
-ExportParamsPath=/path/to/params.json \
-ExportParamsHeight=50 \
-v -timeout 24h
To export the simulation app state (i.e genesis) to a file:
$ go test -mod=readonly github.com/cosmos/cosmos-sdk/simapp \
-run=TestFullAppSimulation \
-Enabled=true \
-NumBlocks=100 \
-BlockSize=200 \
-Commit=true \
-Seed=99 \
-Period=5 \
-ExportStatePath=/path/to/genesis.json \
v -timeout 24h
Params ¶
Params that are provided to simulation from a JSON file are used to used to set both module parameters and simulation parameters. See sim_test.go for the full set of parameters that can be provided.
Index ¶
- Constants
- func DeriveRand(r *rand.Rand) *rand.Rand
- func GetMemberOfInitialState(r *rand.Rand, weights []int) int
- func RandIntBetween(r *rand.Rand, min, max int) int
- func RandPositiveInt(r *rand.Rand, max sdk.Int) (sdk.Int, error)
- func RandStringOfLength(r *rand.Rand, n int) string
- func RandSubsetCoins(r *rand.Rand, coins sdk.Coins) sdk.Coins
- func RandTimestamp(r *rand.Rand) time.Time
- func RandomAmount(r *rand.Rand, max sdk.Int) sdk.Int
- func RandomDecAmount(r *rand.Rand, max sdk.Dec) sdk.Dec
- func RandomFees(r *rand.Rand, ctx sdk.Context, spendableCoins sdk.Coins) (sdk.Coins, error)
- func RandomRequestBeginBlock(r *rand.Rand, params Params, validators mockValidators, pastTimes []time.Time, ...) abci.RequestBeginBlock
- type Account
- type AppParams
- type AppStateFn
- type Config
- type ContentSimulatorFn
- type DummyLogWriter
- type EventStats
- type FutureOperation
- type LogWriter
- type Operation
- type OperationEntry
- func BeginBlockEntry(height int64) OperationEntry
- func EndBlockEntry(height int64) OperationEntry
- func MsgEntry(height, order int64, opMsg OperationMsg) OperationEntry
- func NewOperationEntry(entry string, height, order int64, op json.RawMessage) OperationEntry
- func QueuedMsgEntry(height int64, opMsg OperationMsg) OperationEntry
- type OperationMsg
- type OperationQueue
- type ParamChange
- type ParamSimulator
- type Params
- type SimValFn
- type StandardLogWriter
- type TransitionMatrix
- type WeightedOperation
- type WeightedOperations
- type WeightedProposalContent
Constants ¶
const (
BeginBlockEntryKind = "begin_block"
EndBlockEntryKind = "end_block"
MsgEntryKind = "msg"
QueuedMsgEntryKind = "queued_msg"
)
entry kinds for use within OperationEntry
Variables ¶
This section is empty.
Functions ¶
func DeriveRand ¶ added in v0.34.0
func DeriveRand(r *rand.Rand) *rand.Rand
DeriveRand derives a new Rand deterministically from another random source. Unlike rand.New(rand.NewSource(seed)), the result is "more random" depending on the source and state of r.
NOTE: not crypto safe.
func GetMemberOfInitialState ¶
func GetMemberOfInitialState(r *rand.Rand, weights []int) int
GetMemberOfInitialState takes an initial array of weights, of size n. It returns a weighted random number in [0,n).
func RandIntBetween ¶ added in v0.34.5
func RandIntBetween(r *rand.Rand, min, max int) int
RandIntBetween returns a random int between two numbers inclusively.
func RandPositiveInt ¶ added in v0.36.0
func RandPositiveInt(r *rand.Rand, max sdk.Int) (sdk.Int, error)
RandPositiveInt get a rand positive sdk.Int
func RandStringOfLength ¶ added in v0.34.0
func RandStringOfLength(r *rand.Rand, n int) string
RandStringOfLength generates a random string of a particular length
func RandSubsetCoins ¶ added in v0.38.0
func RandSubsetCoins(r *rand.Rand, coins sdk.Coins) sdk.Coins
returns random subset of the provided coins will return at least one coin unless coins argument is empty or malformed i.e. 0 amt in coins
func RandTimestamp ¶ added in v0.34.0
func RandTimestamp(r *rand.Rand) time.Time
RandTimestamp generates a random timestamp
func RandomAmount ¶ added in v0.34.0
func RandomAmount(r *rand.Rand, max sdk.Int) sdk.Int
RandomAmount generates a random amount Note: The range of RandomAmount includes max, and is, in fact, biased to return max as well as 0.
func RandomDecAmount ¶ added in v0.34.0
func RandomDecAmount(r *rand.Rand, max sdk.Dec) sdk.Dec
RandomDecAmount generates a random decimal amount Note: The range of RandomDecAmount includes max, and is, in fact, biased to return max as well as 0.
func RandomFees ¶ added in v0.38.0
func RandomFees(r *rand.Rand, ctx sdk.Context, spendableCoins sdk.Coins) (sdk.Coins, error)
RandomFees returns a random fee by selecting a random coin denomination and amount from the account's available balance. If the user doesn't have enough funds for paying fees, it returns empty coins.
func RandomRequestBeginBlock ¶
func RandomRequestBeginBlock(r *rand.Rand, params Params,
validators mockValidators, pastTimes []time.Time,
pastVoteInfos [][]abci.VoteInfo,
event func(route, op, evResult string), header abci.Header) abci.RequestBeginBlock
RandomRequestBeginBlock generates a list of signing validators according to the provided list of validators, signing fraction, and evidence fraction
Types ¶
type Account ¶ added in v0.34.0
type Account struct {
PrivKey crypto.PrivKey
PubKey crypto.PubKey
Address sdk.AccAddress
}
Account contains a privkey, pubkey, address tuple eventually more useful data can be placed in here. (e.g. number of coins)
func FindAccount ¶ added in v0.38.0
func FindAccount(accs []Account, address sdk.Address) (Account, bool)
FindAccount iterates over all the simulation accounts to find the one that matches the given address
func RandomAcc ¶ added in v0.34.0
func RandomAcc(r *rand.Rand, accs []Account) (Account, int)
RandomAcc picks and returns a random account from an array and returs its position in the array.
func RandomAccounts ¶ added in v0.34.0
func RandomAccounts(r *rand.Rand, n int) []Account
RandomAccounts generates n random accounts
type AppParams ¶ added in v0.36.0
type AppParams map[string]json.RawMessage
AppParams defines a flat JSON of key/values for all possible configurable simulation parameters. It might contain: operation weights, simulation parameters and flattened module state parameters (i.e not stored under it's respective module name).
func (AppParams) GetOrGenerate ¶ added in v0.36.0
func (sp AppParams) GetOrGenerate(cdc *codec.Codec, key string, ptr interface{}, r *rand.Rand, ps ParamSimulator)
GetOrGenerate attempts to get a given parameter by key from the AppParams object. If it exists, it'll be decoded and returned. Otherwise, the provided ParamSimulator is used to generate a random value or default value (eg: in the case of operation weights where Rand is not used).
type AppStateFn ¶ added in v0.34.0
type AppStateFn func(r *rand.Rand, accs []Account, config Config) (
appState json.RawMessage, accounts []Account, chainId string, genesisTimestamp time.Time,
)
AppStateFn returns the app state json bytes and the genesis accounts
type Config ¶ added in v0.38.0
type Config struct {
GenesisFile string // custom simulation genesis file; cannot be used with params file
ParamsFile string // custom simulation params file which overrides any random params; cannot be used with genesis
ExportParamsPath string // custom file path to save the exported params JSON
ExportParamsHeight int //height to which export the randomly generated params
ExportStatePath string //custom file path to save the exported app state JSON
ExportStatsPath string // custom file path to save the exported simulation statistics JSON
Seed int64 // simulation random seed
InitialBlockHeight int // initial block to start the simulation
NumBlocks int // number of new blocks to simulate from the initial block height
BlockSize int // operations per block
ChainID string // chain-id used on the simulation
Lean bool // lean simulation log output
Commit bool // have the simulation commit
OnOperation bool // run slow invariants every operation
AllInvariants bool // print all failed invariants if a broken invariant is found
}
Config contains the necessary configuration flags for the simulator
type ContentSimulatorFn ¶ added in v0.38.0
type ContentSimulatorFn func(r *rand.Rand, ctx sdk.Context, accs []Account) govtypes.Content
ContentSimulatorFn defines a function type alias for generating random proposal content.
type EventStats ¶
type EventStats map[string]map[string]map[string]int
EventStats defines an object that keeps a tally of each event that has occurred during a simulation.
func NewEventStats ¶
func NewEventStats() EventStats
NewEventStats creates a new empty EventStats object
func (EventStats) ExportJSON ¶
func (es EventStats) ExportJSON(path string)
ExportJSON saves the event stats as a JSON file on a given path
type FutureOperation ¶ added in v0.34.0
type FutureOperation struct {
BlockHeight int
BlockTime time.Time
Op Operation
}
FutureOperation is an operation which will be ran at the beginning of the provided BlockHeight. If both a BlockHeight and BlockTime are specified, it will use the BlockHeight. In the (likely) event that multiple operations are queued at the same block height, they will execute in a FIFO pattern.
type LogWriter ¶
type LogWriter interface {
AddEntry(OperationEntry)
PrintLogs()
}
log writter
func NewLogWriter ¶
func NewLogWriter(testingmode bool) LogWriter
LogWriter - return a dummy or standard log writer given the testingmode
type Operation ¶ added in v0.34.0
type Operation func(r *rand.Rand, app *baseapp.BaseApp,
ctx sdk.Context, accounts []Account, chainID string) (
OperationMsg OperationMsg, futureOps []FutureOperation, err error)
Operation runs a state machine transition, and ensures the transition happened as expected. The operation could be running and testing a fuzzed transaction, or doing the same for a message.
For ease of debugging, an operation returns a descriptive message "action", which details what this fuzzed state machine transition actually did.
Operations can optionally provide a list of "FutureOperations" to run later These will be ran at the beginning of the corresponding block.
type OperationEntry ¶
type OperationEntry struct {
EntryKind string `json:"entry_kind" yaml:"entry_kind"`
Height int64 `json:"height" yaml:"height"`
Order int64 `json:"order" yaml:"order"`
Operation json.RawMessage `json:"operation" yaml:"operation"`
}
OperationEntry - an operation entry for logging (ex. BeginBlock, EndBlock, XxxMsg, etc)
func BeginBlockEntry ¶
func BeginBlockEntry(height int64) OperationEntry
BeginBlockEntry - operation entry for begin block
func EndBlockEntry ¶
func EndBlockEntry(height int64) OperationEntry
EndBlockEntry - operation entry for end block
func MsgEntry ¶
func MsgEntry(height, order int64, opMsg OperationMsg) OperationEntry
MsgEntry - operation entry for standard msg
func NewOperationEntry ¶
func NewOperationEntry(entry string, height, order int64, op json.RawMessage) OperationEntry
NewOperationEntry creates a new OperationEntry instance
func QueuedMsgEntry ¶
func QueuedMsgEntry(height int64, opMsg OperationMsg) OperationEntry
QueuedMsgEntry creates an operation entry for a given queued message.
func (OperationEntry) MustMarshal ¶
func (oe OperationEntry) MustMarshal() json.RawMessage
MustMarshal marshals the operation entry, panic on error.
type OperationMsg ¶ added in v0.34.0
type OperationMsg struct {
Route string `json:"route" yaml:"route"` // msg route (i.e module name)
Name string `json:"name" yaml:"name"` // operation name (msg Type or "no-operation")
Comment string `json:"comment" yaml:"comment"` // additional comment
OK bool `json:"ok" yaml:"ok"` // success
Msg json.RawMessage `json:"msg" yaml:"msg"` // JSON encoded msg
}
OperationMsg - structure for operation output
func NewOperationMsg ¶ added in v0.34.0
func NewOperationMsg(msg sdk.Msg, ok bool, comment string) OperationMsg
NewOperationMsg - create a new operation message from sdk.Msg
func NewOperationMsgBasic ¶ added in v0.34.0
func NewOperationMsgBasic(route, name, comment string, ok bool, msg []byte) OperationMsg
NewOperationMsgBasic creates a new operation message from raw input.
func NoOpMsg ¶ added in v0.34.0
func NoOpMsg(route string) OperationMsg
NoOpMsg - create a no-operation message
func (OperationMsg) LogEvent ¶ added in v0.34.0
func (om OperationMsg) LogEvent(eventLogger func(route, op, evResult string))
LogEvent adds an event for the events stats
func (OperationMsg) MustMarshal ¶ added in v0.34.0
func (om OperationMsg) MustMarshal() json.RawMessage
MustMarshal Marshals the operation msg, panic on error
type OperationQueue ¶
type OperationQueue map[int][]Operation
OperationQueue defines an object for a queue of operations
func NewOperationQueue ¶
func NewOperationQueue() OperationQueue
NewOperationQueue creates a new OperationQueue instance.
type ParamChange ¶
type ParamChange struct {
Subspace string
Key string
SimValue SimValFn
}
ParamChange defines the object used for simulating parameter change proposals
func NewSimParamChange ¶
func NewSimParamChange(subspace, key string, simVal SimValFn) ParamChange
NewSimParamChange creates a new ParamChange instance
func (ParamChange) ComposedKey ¶
func (spc ParamChange) ComposedKey() string
ComposedKey creates a new composed key for the param change proposal
type ParamSimulator ¶ added in v0.36.0
type ParamSimulator func(r *rand.Rand)
ParamSimulator creates a parameter value from a source of random number
type Params ¶
type Params struct {
PastEvidenceFraction float64
NumKeys int
EvidenceFraction float64
InitialLivenessWeightings []int
LivenessTransitionMatrix TransitionMatrix
BlockSizeTransitionMatrix TransitionMatrix
}
Params define the parameters necessary for running the simulations
func RandomParams ¶
func RandomParams(r *rand.Rand) Params
RandomParams returns random simulation parameters
func SimulateFromSeed ¶
func SimulateFromSeed(
tb testing.TB, w io.Writer, app *baseapp.BaseApp,
appStateFn AppStateFn, ops WeightedOperations,
blackListedAccs map[string]bool, config Config,
) (stopEarly bool, exportedParams Params, err error)
SimulateFromSeed tests an application by running the provided operations, testing the provided invariants, but using the provided config.Seed. TODO: split this monster function up
type SimValFn ¶ added in v0.38.0
type SimValFn func(r *rand.Rand) string
SimValFn function to generate the randomized parameter change value
type StandardLogWriter ¶
type StandardLogWriter struct {
OpEntries []OperationEntry `json:"op_entries" yaml:"op_entries"`
}
log writter
type TransitionMatrix ¶
type TransitionMatrix struct {
// contains filtered or unexported fields
}
TransitionMatrix is _almost_ a left stochastic matrix. It is technically not one due to not normalizing the column values. In the future, if we want to find the steady state distribution, it will be quite easy to normalize these values to get a stochastic matrix. Floats aren't currently used as the default due to non-determinism across architectures
func CreateTransitionMatrix ¶
func CreateTransitionMatrix(weights [][]int) (TransitionMatrix, error)
CreateTransitionMatrix creates a transition matrix from the provided weights. TODO: Provide example usage
type WeightedOperation ¶
type WeightedOperation struct {
Weight int
Op Operation
}
WeightedOperation is an operation with associated weight. This is used to bias the selection operation within the simulator.
func NewWeightedOperation ¶
func NewWeightedOperation(weight int, op Operation) WeightedOperation
NewWeightedOperation creates a new WeightedOperation instance
type WeightedOperations ¶
type WeightedOperations []WeightedOperation
WeightedOperations is the group of all weighted operations to simulate.
type WeightedProposalContent ¶
type WeightedProposalContent struct {
AppParamsKey string // key used to retrieve the value of the weight from the simulation application params
DefaultWeight int // default weight
ContentSimulatorFn ContentSimulatorFn // content simulator function
}
WeightedProposalContent defines a common struct for proposal contents defined by external modules (i.e outside gov)