Building parachains with Go-cumulus
Authored by Timothy Hao Chi Ho
ChainSafe awarded a new Web3 Foundation grant for Go-cumulus
"Specialization breeds optimization." Robert Habermeier
ChainSafe has been awarded a new grant from the Web3 Foundation to build a Go equivalent of the Cumulus extension to Substrate! The goal of this grant is to enable Gossamer-built nodes (a Polkadot Host in Go) to become Polkadot-compatible parachains. We are thrilled to have the opportunity to continue our Gossamer work in parallel with this new grant, serving as a natural extension to Gossamer.
If all of that sounded like technical mumble-jumble to you, fret not. We will briefly go over fundamental concepts in the next section to prime you for an understanding of the grant. We will then take you through the milestones for our Go-cumulus grant, which will serve as key junctures for assessing our progress on the road to a full rollout of the Gossamer package.
If you are already familiar with concepts in Polkadot, feel free to skip directly to the milestones section!
[For more on Gossamer, check out The future of Polkadot in Golang: Gossamer and Gossamer: Into the Polkadot-verse Pt. 1.]
What are parachains?
To understand the scope of our work with this new grant, it is important to first have an understanding of parachains and their role in the Polkadot network. From the 10,000 ft view, the Relay Chain sits at the centre of Polkadot. The Relay Chain is designed to provide minimal functionality outside of providing security and consensus for the entire network. The design of this protocol allows sovereign, application-specific blockchains to hook into the Relay Chain, execute their specialized state transition functions (like smart contract logic), and share security with the Relay Chain.
These application-specific blockchains are known as parallelized chains, or "parachains". 100 parachain slots are currently estimated to be made available for lease on the Relay Chain, and when the entire network is operational, every parachain will be able to parallelize transaction processing. In essence, this is how Polkadot achieves scalability and security at the same time.
[See A brief overview of the Scalability Trilemma for a helpful explainer on the problem of scaling in blockchain.]
Parachains are some of the last pieces of core functionality to be implemented from the Polkadot whitepaper. They are currently being tested on Polkadot's Rococo testnet. Once they are vetted thoroughly for functionality and stability, they will be officially launched on Polkadot via its on-chain governance mechanism.
What is Substrate? What is Cumulus?
Substrate is a framework for building blockchains using the Rust programming language, created by Parity Technologies. In fact, the Polkadot Relay Chain was built using the Substrate framework. However, Substrate should not be conflated with Polkadot the network. It merely serves as a framework that provides developers with a toolset to build their own blockchains. This saves developers from having to expend the effort of researching, building, and implementing - from scratch - fundamental blockchain components like networking, consensus, cryptography, and the state trie.
However, on its own, blockchains built using Substrate do not have the ability to hook into the Polkadot Relay Chain. As a result, the Parity team built the Cumulus library, which serves as an extension to the Substrate framework and allows conversion of Substrate runtimes into parachain runtimes. In essence, Cumulus provides the "glue code" to Polkadot-host-chains (built using Substrate) to become Polkadot-compatible parachains.
[A runtime is the core state transition logic of a blockchain.]
Here's a helpful analogy: imagine Cumulus as a Network Card that connects your computer (i.e. blockchain) to the internet (i.e. the Relay Chain). By building a blockchain with Substrate+Cumulus, developers have the ability to deploy as parachains on one of Polkadot's 100 leasable parachain slots, and thereby connecting and transmitting data & value amongst an internet of parachains.
So then wtf is Gossamer?
ChainSafe has been building an alternative implementation of the Polkadot Host in the Go programming language, called Gossamer. In simple terms, you can think of Gossamer as an alternative to Substrate for building blockchains. The difference lies in the language used (Go instead of Rust). Outside of this, Gossamer nodes can interoperate with Substrate nodes on the Polkadot network, or any other chain built with Substrate, forming a resilient multi-client blockchain.
But what if we wanted to take Gossamer nodes and deploy them as parachains? Could we just tack on the existing Cumulus library onto Gossamer? Unfortunately not. As the existing Cumulus extension was architected using the Rust language for compatibility with Substrate, there would need to be some equivalent built in Golang.
Hence, the grant for this work!
The road to Go-cumulus
We have set out 5 overarching milestones to be completed for Go-cumulus:
-
Implement Go-cumulus' networking layer
-
Implement the Cumulus Consensus engine that follows a Polkadot Relay Chain
-
Implement a Cumulus Collator to author parachain blocks
-
Integration testing
-
Documentation
Each milestone will carry its own corresponding sub-components and completion criteria which we will use to assess the progress of Go-cumulus.
Milestone 0 - Implement Cumulus Networking
Cumulus' network layer is implemented with libp2p. The purpose of the network layer is to allow communication between the parachain collators and the Relay Chain validators. See Joe Petrowski's article The Path of a Parachain Block for a great breakdown of their roles.
[Relay Chain validators receive proofs from parachain collators and verifies the state transitions of parachains in order to advance the Relay Chain.]
[Parachain collators receive new blocks from the Relay Chain and propose block candidates and state transition proofs on behalf of parachains to be sent to validators from the Relay Chain.]
The crux of this milestone includes ensuring collators can send a proof-of-validity to a validator, and that parachain collators can receive candidate messages from validators as well as block announcements from the Relay Chain.
Milestone 1 - Implement Cumulus Consensus
The purpose of Cumulus's consensus layer is to ensure Go-cumulus parachains follow the Relay Chain, and that upon new Relay Chain blocks, the heads of the parachain will get updated. Similarly, when Relay Chain blocks reach finality, the corresponding updates should also be made on the finalized blocks of the parachain. Finally, the consensus layer tells the collator which chain to treat as the canonical chain.
Milestone 2 - Implement Cumulus Collator
The purpose of the Cumulus collator is to run a block production algorithm for the parachain. It notifies each collator when to build a parachain block. The blocks that collators create need to contain the block header, body, witness data, and outgoing messages, including the candidate message proving it will likely be included in the Relay Chain. This is referred to as a proof-of-validity (PoV) block.
Milestone 3 - Integration Testing
Upon completion of the first three milestones, we will then test the implementation of Go-cumulus by integrating it with a Gossamer-based devnet. This will include creating a Gossamer-based Relay Chain using the Rococo runtime; and creating and connecting a Gossamer-based parachain to the corresponding Relay Chain.
Some key tests of success here will be:
-
Do our parachains track the canonical chain of the Relay Chain?
-
Do they track finalized blocks?
-
Are the collators able to communicate with parachain validators to know when to build a block and when to submit it?
-
Are the parachain blocks being validated by parachain validators properly and getting included in the Relay Chain?
-
Are messages being passed from parachain to parachain via the Relay Chain?
Milestone 4 - Documentation
In the final milestone, we will provide documentation on how to use Go-cumulus, as well as how to create a Cumulus runtime and how to create a parachain on a devnet using Gossamer/Go-cumulus.
As progress on this grant develops, we will be sure to share it with the wider community along with our Gossamer updates!
Get Involved
Want to work with ChainSafe? COME JOIN US!!! Check out the new Careers section of our website and our open positions, and get in touch with us at careers@chainsafe.io if you are interested!
If you are interested in getting involved and contributing to the project, check out our Github. If you would like to get in contact with one of the Gossamer team members, feel free to drop by on Chainsafe's Discord, or email info@chainsafe.io. We would love to know more about you, your team and your project!
For more details on Gossamer, please head to our !!!new!!! documentation site.
Learn more about ChainSafe by visiting our website, through our Medium, via Twitter, or by visiting the ChainSafe GitHub.
Acknowledgements
Thank you to Elizabeth Binks, Dustin Brickwood, & Amer Ameen. Your contributions were invaluable to the making of this article.
