Lodestar Updates

Authored by Colin Schwarz
The Lodestar team is excited to provide another update on our recent work. It has been about two months since our last update so we have a lot of exciting progress to share!

Written by Colin Schwarz, Gregory Markou, Cayman Nava and Marin Petrunić

Spec updates

Shortly after we released our last update, we were able to update Lodestar's state transition to the 0.7.1 spec. After the 0.8 spec froze on June 30th, we immediately began to get the various components of Lodestar up to date with the latest spec.

We're pleased to finally be fully migrated to the 0.8.1 spec. We're now passing all 0.8.1 spec tests: ssz, bls, sanity, and state transitions, in both the minimal and mainnet configurations.

A substantial part of the update was bringing ssz in line with the spec. From 0.7 to 0.8, ssz includes two new features: explicit limits on dynamic lists and new bitlist/bitvector types. To that end, we created a new utility library called bit-utils that implements these two new types: bitvector and bitlist. Our most-recent ssz and bit-utils packages are both available on npm and github.

Technical Progress

In addition to keeping up with the specs, we have made a ton of technical progress in the last two months. A lot has happened, but we'll try to hit the highlights here:

We split out runtime parameters from constants, allowing us to run both the mainnet and minimal spec tests and easily experiment with testnet configurations. We've also decided to move to a "monorepo" style for managing our surplus of published eth2.0 packages, and we think this will help us move quickly and reuse as much work as we can as we start work on our light client. We've organized the Lodestar repo to use lerna to manage the various shared modules across Lodestar, ssz utilities, and light client work. In the coming weeks, we're planning on migrating our auxiliary repos (ssz, bls, bit-utils, testing utilities, etc) into the Lodestar monorepo.

We refactored our CLI options to be more consistent across the codebase, refactored our database code to remove code duplication, fixed choosing the Eth1 best vote, finished the deposit contract/deposit handling with updates to a Merkle tree, completed our ENR implementation and are well on the way to a Discv5 implementation.

More recently we have been working with the libp2p team on fixing issues with gossipsub. We have already published gossipsub v0.0.3, and we are working with the libp2p team to integrate gossipsub as the default pubsub implementation in js-libp2p; expect another release in the coming weeks! We're also still working on integrating Hobbits and helping to flesh out the spec to suit our needs.

Another avenue of experimentation is around migrating various smaller components to AssemblyScript. We've made a few announcements about AssemblyScript in the past, but briefly, AssemblyScript is a language that looks like TypeScript but compiles to WebAssembly. We think it will be valuable to create readable, accessible code that can run with a higher level of speed and reliability. A few weeks ago we started working on LMD Ghost and an implementation of SHA256 in AssemblyScript.

We've also built out some infrastructure for optimizing our code. We now have a simple framework for running benchmarks, and we've also made it easy to trace CPU usage across our spec tests. This will be invaluable as we bring Lodestar and its utilities towards production use.

Collaborations

One of the main objectives of Lodestar is to create and contribute to state of the art tooling to make developer experience as easy and intuitive as possible. To this end we are continuing forward with several productive collaborations. Through building Lodestar, the team has continued its fruitful collaboration with the folks at libp2p, both drawing from and contributing to this great and emerging ecosystem. As of June, Yeeth and Lodestar are also collaborating. The Yeeth team will be building out AssemblyScript components that will be applicable to Lodestar. We have also taken our collaboration with SSZ to the next level and have created simpleserialize.com. This website provides a browser ready version of SSZ written in TypeScript and offers a glimpse of what can be achieved on the web for Ethereum 2.0. The website is still on the old version of the spec, but we're expecting to update it in the coming week.

Licence

A few weeks ago we updated the Lodestar licence to make our implementation more widely accessible. We did a massive ChainSafe-wide migration from GPLv3 to LGPLv3. Under our previous licence, proprietary websites would not have been able to use Lodestar, now they can! This will greatly increase the number of cases for which Lodestar can be used, making it more widely available without decreasing its protection.

Future Direction

We're now in a position where we have most of the major elements of the beacon chain and validator clients in place. So where's the testnet? We're working on it. Our future focus is on "productionizing" Lodestar, finishing out missing components, and, most importantly, on light client research/development.

We are now working on splitting out Lodestar packages for easier reuse in other projects. Upcoming packages that we will publish include TypeScript typings for eth2.0 data structures and ssz utility packages for easily de/serializing eth2.0 data structures.

We also have a lot of work to do around optimizing Lodestar. We currently have correct code, but it needs to be honed for use in production. We'll be building off our current benchmarking and tracing infrastructure to identify key sticking points and optimize accordingly. We're also working on adding monitoring of various application and system metrics using Prometheus / Grafana.

We've been relatively quiet about our plans for a light client, but we think this will be the place where Lodestar can ultimately shine its brightest. We've been developing Lodestar as a series of pluggable modules, and we plan on reusing as much code as possible to accelerate our light client development. As the light client research settles, we will be shifting focus from our full beacon / validator node towards a browser-ready light client. Stay tuned for future updates on this!

Funding

Finally, we are happy to announce that we were recently awarded a grant by the Moloch DAO. Many thanks to those who voted in our favour! The grant will help fund the R&D for an in-browser light client (i.e removing the infura dependency on @metamask_io).