Back Into The Forest
Authored by Ben Adar
The latest from ChainSafe's Filecoin implementation in Rust
Forest is ChainSafe's Rust implementation of the Filecoin protocol. Rust provides a foundation for building blockchains with memory safety and performance as a priority. Filecoin is an open-source blockchain-based decentralized storage network. Check out this primer by former Forest Tech Lead Austin Abell to learn more about the project.
Also, be sure to check out Amer Ameen's high-level overview of Forest at April's Filecoin Virtual Community Meetup!
Major Developments
This month Forest has seen some huge milestones!
Starting With The Audit
Since the end of April, Sigma Prime has been performing a security audit of the Forest codebase. Before the Audit started, the team was updating the documentation for the audit team, which will be ready for public use after the audit is complete.
The team will share the final audit report once it is ready. This will cover any issues Sigma Prime found and how they will be addressed.
Getting Forest ready for Auditing
Leading up to the Sigma Prime Audit, the team was getting Forest production-ready. This involved improving syncing and the message pool. The team refactored the message pool entirely and implemented Optimal Message Selection, which provides message selection based on a miner's ticket quality.
On the syncing side of things, the team improved the robustness and performance of the ChainSyncer by refactoring and simplifying the flow and logic by implementing a Rust Futures-driven state machine. Things are looking stable as the team has been able to run a node with a few weeks of uptime. Also, as part of improving the syncing process, the team fixed a memory leak. With that, Forest has been able to achieve relative stability on the memory side. In fact, Forest has had a node running for a few weeks using only 7 GB of RAM!
Before the audit, the Forest team was also working on the final steps for a production-ready RPC, including updating error handling and JWT (JSON Web Token) for RPC and matching it to the Lotus RPC. The team also added an RPC method for an encrypted Keystore.
Prior to adding the RPC method, keys were not encrypted before being saved. Now, users can take advantage of a secure keystore using a password to decrypt their private keys upon startup. The encrypted keystore increases the security for users while ensuring ease of use and accessibility.
With improving syncing, refactoring the message pool, and implementing the RPC interface, a production-ready Forest was ready to be audited!
State Migration
State migration is a process on the Filecoin network where the state tree's contents transform from an older form to a newer one.
The Forest team started implementing the state migration to fully sync from genesis instead of relying on a recent snapshot. With the state migration implemented, Forest will have full archival node capabilities to query historical state information!
Currently, Forest nodes cannot process the tipset at the network upgrade epoch, only the tipsets before and after that epoch. Manual intervention is required, which involves importing a snapshot post the migration epoch and continuing from there.
Improving CLI and User Experience
The team has also spent time improving the Forest CLI, so users will have a way to interact with the Filecoin blockchain through Forest seamlessly. The goal is to make Forest more usable by an end-user, including commands for the State, Message Pool, and P2P CLI. The team is also working on getting all currently implemented commands in the Forest CLI working with the RPC backend, eventually adding RPC commands for the Network RPC.
By adding more commands and matching them with their Lotus counterparts, Forest will become much easier for users to use and increase interoperability between other Filecoin implementations.
Performance Analysis
For the last few months, the team has been looking to add instrumentation in Forest to measure and analyze performance metrics. Currently, the team does not have a reliable way of understanding how the Forest node performs and where the bottlenecks are.
The team started collecting metrics in Forest, including, a tipset processing time, Gossipsub messages stats, a counter of tipsets received through Gossipsub which fail validation, a gauge of rocksDB size on disk, a count of TipsetRangeSyncer future errors, and libp2p metrics.
Next, the team will pull these metrics through a dashboard using Grafana. Once all the metrics can be displayed all neat and tidy, the team will analyze what needs improvement in Forest, further strengthening the Filecoin Rust implementation!
What's Next?
Moving forward, the team will continue improving the CLI and will finish implementing the rest of the state migrations. The team is also pushing for more user documentation as they inch closer to Forest's first release. The main goal is to make it easy for users to run Forest!
Also very exciting to note, the team is close to finishing the storage and retrieval markets integration!
Get Involved!
If you are interested in getting involved and contributing to the project, check out our Github. If you would like to contact one of the Forest team members, please drop by on Chainsafe's Discord or email forest@chainsafe.io. We would love to know more about you, your team and your project!
For more details on Forest, please head to our documentation site.
Learn more about ChainSafe by visiting our website, through our Medium, via Twitter, or visiting the ChainSafe GitHub.
