NodeWatch: The open source node explorer for Ethereum Consensus (Eth2)
Authored by Timothy Hao Chi Ho
Building an Ethereum Consensus node crawler for the Ethereum Foundation
"Night gathers and now my watch begins..."
ChainSafe is pleased to announce the completion of an engagement with the Ethereum Foundation for the release of NodeWatch, an open source node explorer for Ethereum Consensus nodes.
[Editor: Ethereum Consensus is the preferred nomenclature from the core devs team at the EF. This is to avoid confusion over Eth1 and Eth2 being separate networks - it is not. Instead, we should look at them as two layers before the merging of the two. As such, the Ethereum Consensus layer refers to how the network will be achieving consensus in the future-state Proof-of-Stake paradigm, while the Ethereum Execution layer refers to Eth1 under the current Proof-of-Work paradigm that is executing in-place.]
NodeWatch can be considered a replacement and an extension to the original Ethernodes crawler for Ethereum Execution nodes. The current release contains a backend service that crawls, stores, and provides access to valuable information about nodes such as location, client, operating system, and sync status, as well as a basic front-end UI that displays the data collected.
Why do we need a tool to watch nodes?
Crawlers are conventionally known in the web2 paradigm as programs that visit websites, read the content, and create entries for a search engine index. Similarly, a crawler that collects information about nodes on a blockchain network queries those same nodes, collects high level data about them in aggregate, and returns them to be used in some way.
In order to collect and assemble this data, the crawler crawls Ethereum's peer discovery network (which uses the devp2p stack) with a node discovery protocol called discv5. The collected data can be accessed through the NodeWatch API. With this public tooling built, we invite others to piggyback off this data to build their own graphQL queries and network visualizations.
What kind of data is served?
The current iteration of NodeWatch serves up a number of different metrics in three different categories.
General Information
In this section, the crawler returns information on the number of nodes that are connectable, the percentages of the network that are synced (or unsynced) within 256 epochs (about 1 day) of the head of the chain, and the count of nodes over the past 7 days as displayed on a line graph.
Regional Information
In this section, the crawler returns information on the diversity of countries that are participating in consensus on the network, as well as the ratio of hosted to non-hosted (cloud service vs. residential or business) nodes. Finally, a heatmap shows the concentration of where nodes are located globally.
The regional node count (below) illustrates the number of nodes located in each country as well as the percentage of the network they represent globally.
Node Statistics
In this section, various statistics that pertain to node software are displayed, including: client type, operating systems, network types, and versioning across clients.
This section is notably important, as you will get to see Et[here](https://blog.ethereum.org/2020/09/01/eth2-quick-update-no-15/)um Consensus' client diversity metrics. As Danny Ryan has alluded to before here and here, client diversity is a must for network health and resiliency. If the majority of the network runs on a majority client and that client goes down, validators stand to lose as a result of "inactivity leaks". [Editor: Try ChainSafe's Lodestar!]
How can you access NodeWatch?
Check out the current release of NodeWatch here.
You can access the NodeWatch API here.
We have also made public the NodeWatch codebase on this Github repo here.
The NodeWatch front-end UI codebase can be found in this repo here.
Next Steps
What should you do with NodeWatch?
Use it!
Our vision for this tool is for developers, researchers, security devs, node operators, and media reporters to use this tool and gleam insights from the graphs. What do you think of the client diversity numbers on Ethereum Consensus ? How does it differ from Ethereum Execution? How can we build a more robust and antifragile Ethereum network given the data provided? What other data would be useful?
We welcome discussions on NodeWatch under our GitHub issues here.
Finally, since the codebase is completely open source, we invite others to fork the code and build on top of what we have built.
Get Involved
Want to work with ChainSafe? **** Come join us! Check out the new Careers section of our website and our open positions, and get in touch with us at careers@chainsafe.io if you are interested.
Make sure to join in the conversation on ChainSafe's Discord. Learn more about ChainSafe by visiting our website, through Medium, via Twitter, or by visiting the ChainSafe GitHub.
Acknowledgments
Thank you to Tanmoy Basak Anjan, Cindy Chau, Sadiqur Rahman, Danny Ryan, David T and Taha Ben Esmael. Your contributions were invaluable to the making of this article.
