Adding a new protocol environment¶
The economic protocols of Tezos are compiled against a restricted set of libraries. This is for security reasons (so that, e.g., the protocol can never be leveraged for accessing files of the host machine) and safety reasons (so that, e.g., the protocol is immune to some categories of errors). See the general overview in Protocol environment.
The set of libraries a protocol is compiled against is called the protocol environment, or simply, the environment.
Each protocol is compiled against a given environment, as declared in the protocol’s
TEZOS_PROTOCOL manifest file. In order to ensure that old protocols can still be compiled in the future, all the environments are set in stone: their interface cannot ever be modified. Consequently, when new features are needed for new protocols, a whole new environment must be created. (Otherwise, if no new feature is needed, a protocol can use the same environment as its predecessor’s.)
This page details the process of creating a new environment by copying the latest environment and building upon it. In the rest of this page,
<N> is the version number for the new environment you are creating and
<N-1> is the version number for the existing environment that you are copying from.
The following steps are roughly the steps taken in the V6 bootstrap MR
Copy the existing environment files:
Copy the directory
Copy the file
Make the new environment buildable by updating
make -C manifest
Copy the existing compatibility layer if any (see details in Struct compatibility layer).
src/lib_protocol_environment/structs/tezos_protocol_environment_structs.mlto add a new submodule
V<N>by copying the submodule
Copy and adapt the environment functor:
Change any reference from
V<N>in all those copied files the
If the protocol signature is expected to change then copy and adapt it otherwise leave it as is:
Environment_protocol_T_V<X>is the current protocol signature and
<X>is equal to the environment version that introduce it.
Add references to the new environment version number in the rest of the code:
Add references to
Adapt demo protocols to the new environment:
Modify the required environment in
Verify they both compile with
dune build @src/proto_demo_noops/runtest_compile_protocoland
dune build @src/proto_demo_counter/runtest_compile_protocol
Commit all those changes and open an MR with your changes.
It is recommended that you test your work more comprehensively offline. To that end, follow the instructions below on how to activate the environment, and then run the protocol tests locally. Do not commit the changes or at least, do not push the changes.
Struct compatibility layer¶
The struct compatibility layer is for providing compatibility between a signature of the protocol environment (which is set in stone) and the interface of an external library that provides it (which might change from version to version). E.g., at the time of the V0 environment the OCaml Stdlib did not include an
Option module and so a custom one was provided in the whole of the Tezos project including the protocol environment; later, when the Tezos project switched to the now available and standard
Stdlib.Option module, the struct compatibility module
src/lib_protocol_environment/structs/v0_option.ml was added.
More recent protocol environments generally need less struct compatibility modules. Occasionally, the most recent environment needs no compatibility layer at all. You can know if this is the case by checking the file
src/lib_protocol_environment/structs/tezos_protocol_environment_structs.ml: if the submodule
V<N> exists and is not empty then there is a compatibility layer, otherwise there isn’t.
Either way, the instructions in the list above are sufficient for creating the new environment.
Activating the environment¶
The new environment as it stands now is not activated. More precisely, it cannot be used by any protocol. A few more changes are needed before it can be used.
When to activate¶
This is on purpose: we do not want to release an unfinished environment because it interferes with the distributed nature of Tezos protocol development. Specifically, if an unfinished protocol was made available in a release of the Octez suite, then anyone could propose a protocol built upon this version. But then further work on the protocol (to finish it) would create multiple different environment that have the same name. To avoid this, we only activate the environment once it is safe.
The new environment should only be activated after the last release that precedes injection of the protocol that uses it. Don’t worry too much about this, simply reach out to a release manager and work with them on the schedule.
How to activate¶
To activate the environment you will need to change the following files, adding references to
V<N> to match the references to
manifest/main.ml to embed the new environment version in
embedded_cmis.ml, and then run
make -C manifest.
Bump environment version in
src/bin_client/test/proto_test_injection/TEZOS_PROTOCOL and in the embedded
TEZOS_PROTOCOL found in
tezt/tests/voting.ml. Update the corresponding test in the multiple
And finally, bump environment version in
For an example, check the MR in which the environment V6 was activated.
Making changes in the environment¶
You can make changes to the newly created environment until it is released. For this purpose release candidates do not count. Below are examples of changes from previous work on the environment.
Add the interface file
Add a reference to the file in
Resultmodule in the functor in
Provide backwards compatibility layers for older environments
Replace some of the environment modules with a new one (remove old files)
Remove struct compatibility module (the new interface is identical to the one in the most recent library)