A Lodestar for Ethereum Consensus #2
Authored by Phil Ngo
A year-end update for ChainSafe's TypeScript implementation of Ethereum Consensus
Welcome to another edition of "A Lodestar for Ethereum Consensus," a series where we give periodic updates on ChainSafe's TypeScript implementation of Ethereum Consensus (formerly Eth2).
There have been a lot of big Lodestar milestones recently. We proposed our first mainnet blocks, stabilized Lodestar, grew our dedicated team, presented at numerous conferences, achieved light client syncs to mainnet chain head, renewed our Ethereum Foundation grant, and bootstrapped Kintsugi 🍵 merge devnets. Let's recap!
Lodestar is now production-ready!
One of our biggest achievements this year is being able to actively participate on mainnet with all the other consensus clients. To give you an idea of how far we've come, in January 2021, we attempted to join the Pyrmont testnet, and our node couldn't process block one after five minutes past genesis. Today, Lodestar can handle hundreds of keys in Prater with a regular desktop.
Starting with v0.33.0, Lodestar will have stabilized releases and a thorough beta testing process for new releases to ensure the stability of our client. Rest assured, we will be working diligently to improve the user experience as Lodestar gains adoption.
We've also now adopted a "maturity release strategy" where we periodically select a commit to release on a separate branch from our :nightly master branch tagged as beta.x and deploy it for quality assurance over at least three days to ensure stability. Pending the performance of the beta and any other fixes that need to be ported, we will then publicly release the update.
These updates will be announced in ChainSafe's Discord under the channel #lodestar-announcements, on Twitter, and through our new mailing list, which will be used for important updates. Subscribe now!
Speaking of production-ready, we hit a huge milestone recently when we officially proposed our first block on Mainnet with v0.32.0 of Lodestar! 🎉
After months of hard work, it was great to finally see our production validator get assigned a block for proposal to, in fact, prove that we're mainnet capable.
This was a team effort, and it was easily the highlight of 2021 for Lodestar. And this is just the start! Progress with our client translates to even greater benefits for the Ethereum ecosystem, as demonstrated by our initiatives with light clients.
CSCON[1]: Realizing the light client dream
Our Team Lead Dapplion presented at CSCON[1] this year to update the community about our light client progress. The presentation included a demo of our updated light client which now syncs to the chain head on mainnet (thanks, Altair!), and our proposal to standardize a REST server proposal for light client routes!
Why are light clients so important, you may ask?
One of the biggest complaints about Ethereum is its inability to scale. If we want the most-used smart contract platform to be accessible to more people and more devices, we must figure out how to scale without sacrificing decentralization or security.
One of the core tenets of an effective blockchain is being able to verify the chain and data yourself, not trusting someone else to serve you unverified data. When interacting with Ethereum today, you're stuck making some hard decisions on whether you want to run your own trusted Ethereum node where you can guarantee that trust or trusting a centralized provider to do the work on your behalf.
Luckily in Ethereum, you can still run your own trusted node with a simple desktop computer or on something as small as a Raspberry Pi 4. But realistically, it's not something every user will do - especially users seeking to interact with Ethereum on lower-end devices. So how do you provide that trust at scale? How do you verify the data you're getting is true without verifying everything yourself?
Light clients work by providing enough trust from "witnesses" at the protocol level without having to verify the whole chain yourself or outright trusting a centralized provider. Light clients are in their infancy, and the Lodestar team is actively working on these problems so we can stop relying on centralized providers for data that may or may not be correct.
Library improvements
This year, we've made it a point to take advantage of research in other parts of the Ethereum ecosystem. For example, by having weak subjectivity sync within our client, we are able to have users warp right to the head of the beacon chain through a trusted checkpoint. This was inspired by Vitalik's piece on weak subjectivity sync. And it means that one of the main benefits of starting with Lodestar is you won't have to sync from genesis!
Our whole gossip engine was also rewritten to have "queues," which ensure we process important gossip, and it is structured to easily handle different forks. We made enormous progress on performance optimization and continue to make our client more efficient.
How do we do that? Metrics.
We now have Grafana metrics for important processes to help diagnose and analyze issues. You must know what the problem is before you can fix it! This improvement was key in helping us with some of the issues mentioned in the Metrics & Analytics section of our previous update.
Kintsugi 🍵 Merge Interop Devnets
We've been active participants alongside other execution and consensus clients to move Ethereum to a Proof of Stake consensus engine. The last two months were focused on ensuring we were compatible with as many execution clients as possible by bootstrapping devnets simulating the upcoming Merge.
Thanks to our team and, specifically, one of our newest members, Gajinder Singh. In a short time, his contributions have made Lodestar compatible with Geth and Nethermind on the Merge devnets. Overall, we're happy to be at the forefront of development alongside other clients and provide Ethereum users with an alternative client to run for The Merge. Follow along with us and find out how you can help with #TestingTheMerge through our simple devnet scripts.
Ethereum Foundation Grant 🤝
In November of 2021, ChainSafe renewed a grant with the Ethereum Foundation to continue our work on Lodestar, Ethereum light clients, and our various JS/TS tooling libraries such as SSZ, BLS, gossipsub, and noise. We will continue to work towards creating accessible, world-class developer tooling for Ethereum Consensus and adding extensive value to the Ethereum ecosystem as a whole.
Our milestones for this grant include advancing our light client initiatives, such as standardizing light client server APIs with the community, alongside improving and demonstrating capabilities that can be enabled with light clients. Whether it's collaborating with the Portal Network to create a decentralized p2p network to expose the JSON-RPC API, or collaborating with execution clients to pull data such as balances into our demo, we will be there to help advance it.
Additionally, we are focused on furthering Ethereum's overall objectives like The Merge and greater client diversity. We are truly thankful for being a part of the Ethereum Foundation's (EF) recently announced Client Incentive Program.
The EF's generous funding will help Lodestar continue to build performant software and maintain the long-term health of the Ethereum network. We will analyze every opportunity to optimize and enhance our client to ensure we can achieve greater adoption for diversifying the ecosystem. There's so much more to come!
The team is growing!
This year brought many new opportunities, including the ability to expand our team. Lodestar now has a dedicated full-time project manager, Phil Ngo, and we've recently added two more engineers to the fold!
Gajinder Singh is a talented software engineer with extensive experience running a consultancy and managing development within payments and marketplaces. He has contributed to Lodestar by improving our client performance, metrics, libraries and leads our participation in the Kintsugi Merge Interop.
Dadepo Aderemi is our newest hire, building robust and scalable software solutions. His experience in full-stack development, networking, cryptography, and distributed systems will provide Lodestar with the ability to optimize and scale our client for the future.
We're always looking to bring more talent into the ChainSafe family and add contributors to our open-source repository. We have many tough problems to solve, but the collective intelligence and skill of those working in the Ethereum ecosystem make us optimistic about the future.
If you're interested in getting involved with the project, check out our Github. If you'd like to get in contact with the Lodestar team, feel free to drop by Chainsafe's Discord on the #lodestar-general channel or email info@chainsafe.io. We would love to know more about you, your team, and your project!
For more details on Lodestar, please head to our documentation site. You can also check out our open positions and get in touch ➡️➡️ careers@chainsafe.io
Notes:
Light Client:
-
Browser-based light client tracks consensus head
-
Proposed standard for REST-based server light-client
Critical performance improvements:
-
Use BLST for signature verification in worker-threads, #FreeTheMainThread
-
Re-do optimizing almost all beacon state transition functions and beacon state data structures
-
Use extensive caches around the beacon state + epoch context + persistent vectors
-
Run benchmarks on every hot path + track in CI
For better networking:
-
Refactor sync module: Concurrent, more resilient syncing
-
Refactor peer manager: Faster peer discovery and better management, scoring
-
Refactor req resp: Better handling of memory leaks, timeouts, other edge cases
-
Refactor gossipsub: Use of queues for memory ceiling + batch processing
-
Refactor gossip validation: Optimize and audit
-
Refactor discv5: Faster peer discovery + caching to prevent lookups
-
Refactor network parent module: Better DX, maintainability
Metrics:
-
From 0 to 1000. Now metrics track almost everything
-
Added validator monitor for detailed performance tracking
Forks:
-
Generic allForks types, incredible DX to implement forks fast
-
Delivered Altair 🙌
-
Delivering The Merge 🥁
API:
-
Single declaration for client and server
-
Reduce tons of boilerplate, easy to maintain and extend
-
Refactored validator client with more optimized flow
About ChainSafe
ChainSafe is a leader in blockchain development and infrastructure solutions for Web3. We're working toward a more decentralized future by building client implementations for Ethereum, Filecoin, Polkadot, and Mina. We're also maintaining the Web3.js library, working on a gaming SDK that connects NFTs to Unity, building a distributed cloud storage application, and bootstrapping a multi-chain bridge. To learn more, click here.
