GoTezos Q1-Q2 Update

Summary

This document serves as a technical update on the progress of the GoTezos software library for quarters Q1-Q2.

Milestones

Improvements

The focus of quarter one and two was to harden the previous version of the library by making it more secure, thoroughly unit/integration tested, to introduce a CI pipeline to ensure code integrity, and to start on smart contact interoperability. The library was completely redesigned with idiomatic go as a focus, and also includes additional RPC support.

Unit Testing

There are currently 201 passing test cases for GoTezos covering 86.3% code base; up from below 40% previously. The plan is to increase this even farther in the future.

Each unit-test in GoTezos is driven by a mock HTTP server, mocked with real data from the Tezos network. The mock server is custom built based on the chain of events that happen in a function. This is done by defining HTTP handlers for each RPC supported. You can find these handlers here.

Each unit-test around an RPC related function usually goes through, but is not limited to, 3(+) basic tests. The first is a test to make sure it handles an RPC error properly. The second test(s) check to see if the function is bubbling up and handling other errors properly. Finally the third test(s) usually consist of testing a successful execution with different parameters. A good example of this is the testing for func (t *GoTezos) Cycle(cycle int) (*Cycle, error) .

There is also thorough coverage around forging operations, creating wallets, and general cryptographic related functions.

Integration Testing

In addition to unit testing GoTezos now includes integration tests. The integration tests can be configured to point to your preferred node. These integration tests specifically serve to make sure data is being serialized correctly into and from the Tezos RPC and Go.

CI Pipeline

The one thing equally as important as the test coverage itself, is the execution of these tests before things get merged and released. GoTezos now has a CI pipeline that must pass before code is allowed to be merged. This CI pipeline is required even for the administrator(s) of the repository. The stages of the pipeline that must pass are:

1. Formatting: checks to make sure the code has the proper formatting expected by gofmt.
2. Lint: checks to make sure the code follows the proper coding style with golint.
3. Staticcheck: analyzes the code to detect bugs, suggest code simplifications, point out dead code, and more with go-staticcheck.
4. Unit-Tests: checks to make sure the code in the current commit passes all available unit-tests.
5. Integration-Tests: checks to make sure the code in the current commit passes all integration tests.
6. Race Conditions: checks to make sure the code doesn’t contain race conditions.

Redesign

Forging Operations : In the previous version of GoTezos only one operation was supported and it was the transfer operation. The library was using the RPC to forge the operation. While the Tezos RPC is safe to use, it is not always safe to trust the node you’re making requests to. This means that that the old version of Go Tezos was vulnerable to what’s called a blind signature attack.

The new version of GoTezos is no longer vulnerable to a blind signature attack on the following operations: reveal, origination, delegation, and transfer. GoTezos also supports forging batch transfers, which allows a developer to execute multiple transfers in a single operation.

Forging operations with the RPC is still available but there are additional security mechanisms built in the function. The operation hash as a result of the forge is verified by the UnforgeOperationWithRPC function. This makes sure that the operation hash returned by the RPC returned the correct contents requested. Additionally, an option was added into the ForgeOperationWithRPC function to allow the caller to use two different nodes for both the forging and unforging of the operation.

Custom Unmarshaling: The Tezos RPC representation of mutez is a big number in string format. Originally GoTezos structs unmarshaled mutez fields as strings, like they are represented in JSON. This can be inconvenient because the likely reason a developer will access a mutez related field is to do mutez related math. The new version of GoTezos adds a custom unmarshaler to mutez fields so that the JSON strings get unmarsheled into a wrapper around go’s *big.Int. You can find documentation of go’s math/big package here.

Race Conditions: In the previous state of GoTezos there were a lot of functions that required a reference point to the chain (i.e. block hash). The library assumed the user wanted the head block hash for most functions. As an example on why this is bad, imagine a developer wanted to get a list of delegations and get the balance of each. As each function retrieves it’s own reference to head, the state of the chain could change, resulting in a string of queries with references to different heads (or states).

To fix this the newer version of Go Tezos requires the user to pass the state to each function. As an example if you wanted to get the list of delegations and their balances you would do something like this:

head, err := gt.Head()
// check err (we're not animals)
delegations, err := gt.DelegatedContracts(head.Hash, "tz1...")
// check err again
var balances []*big.Int
for _, delegation := range *delegations {
    balance, err := gt.Balance(head.Hash, delegation)
    // check err again
    balances = append(balances, balance)
}

The important thing to recognize here is that both the DelegatedContracts()and Balance()functions have the same reference to head.

Documentation: In addition to some major design changes, I also improved the in code documentation or godoc. Previously there wasn’t much documentation, and the documentation that existed was incomplete. Now each and every exported function or exported structure has documentation related to the RPC it’s for, a description of what it does, and a list of parameters and their purposes.

A good example of this is the the BakingRights function.

Michelson And Smart Contracts:

In addition to the above improvements a branch has been made and is continuously worked on around the easy consumption of smart contract storage. It is important for GoTezos to be able to convert Micheslon Types into Go Types and to convert contract storage into a workable structure. Because Go requires a strict type to unmarshal into, the best data structure to achieve the above is a tree representation of the storage. The tree contains children of the MichelsonType, a go defined interface which allows you to retrieve the type of the data, and do comparisons if possible. You can find the continued work here.

Future Smart Contract Work

In addition to being able to consume the storage of Michelson contracts, I would like to add the ability to get the code of a contract, and run it within the go library itself. This will allow Go developers to simulate how a contract might behave on chain pragmatically before forming an operation to interact with a contract. The scope of this work could balloon to be large, because it is essentially a reimplementation of the Michelson rules in go, but would be a valuable feature to have started in Q3.

A Payout Tool: Tzpay

Outside of the library itself, the consumption of the library is used live in production by my own personal bakery and several other bakeries that are using the tool. Tzpay is a payout software that is easily manageable and dockerized based on common devops practices. It allows for the batch payment of baker payouts offline, using the local forge operations of GoTezos, as well as many other features. You can find more information on tzpay here.

Conclusion

I am very much looking forward to adding and refining smart contract features into the next quarter. If there is a feature you would like to see added to GoTezos don’t hesitate to add it the the issues page on github.