Highlights of Ethereum's All Core Devs Meeting (ACDE) #207

The Execution Layer Meeting 207 focused on resolving ongoing issues with the Holesky Testnet, particularly its non-finality problems and the massive validator exit queue. The discussion centered on how to move forward with testing, with most participants leaning toward launching a new testnet, Hoodi, rather than implementing risky fixes on Holesky. Developers debated various approaches, including a shadow fork, using an existing devnet, or modifying the exit queue via a hard fork. The meeting also covered the timeline for Hoodi’s rollout, ensuring it would be a reliable testing ground for staking providers and client teams while minimizing disruptions to Ethereum’s upgrade schedule.

• Holesky Finalized
• Hoodi Testnet
• Pectra Mainnet Plan
• Fusaka Updates
• System Contract Error Handling
• RPC Changes
• Stateless Clients

Holesky Finalized

Holesky suffered from non-finality for nearly two weeks, leading to instability across the network. This issue created significant challenges for client stability, as nodes struggled to synchronize, and for map builders who were unable to properly test their applications. On Monday, the team managed to restore finality, stabilizing the network.

However, the prolonged period of non-finality had secondary effects, such as mass validator slashings and a large validator exit queue, which currently includes one million validators. Due to these slashings and forced exits, Holesky’s exit queue has become too long to be realistically usable for testing Pectra exits.

To address these issues, the core developers considered four possible solutions.

• The first option, Plan A, was to deploy a completely new testnet that would run parallel to Holesky. While Holesky would remain active, it would no longer be relied upon for testing validator exits. A new testnet would provide a clean slate, free from the legacy issues of Holesky.
• Plan B proposed using Devnet 6 as a temporary testnet, which was already active and operational. However, it was not considered a viable long-term solution.
• Plan C suggested implementing a shadow fork, a short-term solution where a temporary fork would run for about a month, allowing Pectra exits to be tested without launching a new testnet. Once Pectra went live on Mainnet, this fork would be deprecated.
• Plan D involved implementing a hard fork to reset the validator exit queue on Holesky. This would allow Pectra exit testing to continue without requiring a new testnet. However, modifying code paths to affect only Holesky’s exit queue raised concerns about unintended effects on Mainnet. Some core developers were hesitant about implementing such a change, as it could introduce unexpected issues.

After gathering feedback from developers and the community, most participants supported Plan A, favoring the creation of a new testnet. However, some staking providers, including Lido and Eigen Layer, preferred Plan D, as their existing tools and integrations were built around Holesky.

Deploying a new testnet would require them to spend three to four weeks redeploying contracts and reconfiguring off-chain tools, oracles, monitoring systems, and interfaces. This made the transition more challenging for them.

Despite these concerns, the primary argument against Plan D was the potential risk to Mainnet from implementing a hardcoded exit queue reset. Additionally, executing Plan D would be more complex than simply setting up a fresh testnet.

Hoodi Testnet

Holesky has become unreliable for exit testing due to the massive validator exit queue, prompting the introduction of a new testnet, Hoodi. To ensure realistic testing, Hoodi will closely mirror Mainnet by using 32 ETH validators at launch. If Mainnet undergoes significant validator consolidations, Hoodi will adjust accordingly to maintain consistency.

Ensuring high validator participation and network stability is a key priority for Hoodi. The network must achieve consistent finalization with high participation rates to test attestation packing effectively. Performance tracking will help detect any bottlenecks, ensuring that validators, staking providers, and infrastructure teams can operate seamlessly.

Since many off-chain tools, oracles, and integrations currently depend on Holesky, these must be reconfigured for Hoodi, a process estimated to take between three to eight weeks depending on the complexity of modifications. Testing on Hoodi will be conducted in phases to allow for a smooth transition.

Staking providers and infrastructure teams will be given at least ten days before the Electra activation to complete contract deployments and tooling adjustments. This structured approach ensures that no critical infrastructure is rushed, reducing the likelihood of unexpected issues once Hoodi is fully operational.

Pectra Mainnet Plan

The discussions focused on determining the steps required to safely launch Pectra on Mainnet. The agreed-upon milestones include the Hoodi testnet fork scheduled for March 26, 2025, which will serve as a final rehearsal for Pectra. This will allow staking protocols and clients to test key features before a possible mainnet launch. Following this, a minimum stability testing period of approximately 30 days is required to ensure that Hoodi finalizes without issues and that infrastructure providers verify full compatibility.

The earliest possible mainnet fork is expected in late April or early May 2025, with the absolute earliest date being at least one month after Hoodi’s fork. The estimated timeframe for Pectra’s launch on Mainnet falls between the last week of April and the first week of May.

For the transition to be smooth, certain conditions must be met. Hoodi must consistently finalize blocks, staking protocols such as Lido and Eigen Layer must complete their testing, and attestation packing should be verified. Slashing mechanisms must be tested, and infrastructure providers need to have their tooling fully prepared.

The general consensus was that Pectra would not be launched on Mainnet until Hoodi has completed a full stability cycle. If Hoodi remains stable within 30 days, the launch will take place in late April. If issues arise, the fork may be delayed by a few weeks. The mainnet fork will not happen before April 26, ensuring that client teams have enough time to finalize fixes and infrastructure providers can complete their testing.

The shift to Hoodi as the main testnet has a direct impact on client teams, as they need to balance their resources between testing and upcoming protocol upgrades. The primary priorities for client teams include ensuring a smooth Hoodi launch, implementing any necessary fixes for Pectra, and preparing for the Fusaka upgrade cycle.

To ensure a stable launch for Pectra, several client-side fixes must be implemented and tested. The sentiment among client teams was that time should not be wasted on short-term Holesky fixes and should instead be spent preparing for upcoming upgrades.

Fusaka Updates

Fusaka is one of the upcoming Ethereum upgrades, and discussions focused on finalizing the proposal scope, reviewing critical Ethereum Improvement Proposals (EIPs), and ensuring proper testing before implementation. The timeline for Fusaka remains flexible, but teams aim to finalize the upgrade scope by April 10.

The discussion revolved around three major technical topics: blob transaction formatting, execution API adjustments, and handling of the EVM Object Format (EOF) within Fusaka.

One of the significant changes in Fusaka involves blob transactions. A proposal was reviewed to improve transaction formatting for Ethereum blobs and address potential edge cases in how blobs are handled within the execution layer. The proposal was approved pending a final technical review from client teams.

A corresponding execution API pull request was discussed, and it was agreed that it should be merged alongside the blob transaction changes. The final decision was that the execution API changes would be incorporated with the blob transaction pull request, and client teams would need to verify compatibility with their implementations.

The discussion focused on how Fusaka should handle EOF, whether it should be enabled or restricted in this upgrade, and its potential impact on smart contracts and existing infrastructure. Some teams suggested temporarily disabling EOF in Fusaka to avoid unexpected complications and ensure that further testing is completed before full deployment. A flag will be added to restrict EOF transactions in Fusaka, and once EOF is fully tested and validated, it will be enabled in a future fork.

System Contract Error Handling

A critical discussion revolved around how Ethereum should handle failures in system contracts, particularly when a contract runs out of gas, fails, or reverts unexpectedly. Currently, some EIPs specify a gas limit of 30 million, but there is no standardized failure-handling mechanism across all clients.

The development team explored multiple potential solutions.

• One approach suggested marking blocks as invalid if a system contract failure occurs, ensuring strong consistency but possibly leading to chain reorgs.
• Another alternative was to handle failures on a per-transaction basis, where only the failing transaction is reverted instead of invalidating the entire block. While this approach is more graceful, it could lead to unintended state changes.
• A third option was to codify a standardized failure response, ensuring that all Ethereum clients handle failures in a uniform way.

To move forward, testing teams were assigned the task of adding failure-handling test cases, while developers were asked to analyze how different clients currently process these failures.

A decision on this matter will be finalized in an upcoming testing call. The group reached a consensus that a clear failure response is necessary, and a proposal will be drafted to standardize behavior across Ethereum clients.

RPC Changes

Another key issue that was discussed involved how Ethereum nodes should respond when a user requests block data that has been pruned. Currently, when a client requests historical data that no longer exists due to pruning, the responses vary across different execution clients, leading to inconsistencies.

To address this, a new RPC error code will be introduced. This error code will explicitly indicate when the requested data is no longer available, helping users differentiate between data that has been removed versus data that never existed. The change aims to make responses from Ethereum clients more predictable and user-friendly.

Developers have been given a two-week period to review the proposal before it is finalized. Once the review is complete, the error code will be standardized and implemented across all execution clients. There was a general agreement that this standardization will improve error handling in Ethereum’s RPC system, making it easier for users and developers to work with historical blockchain data.

Stateless Clients

The topic of stateless clients remains an ongoing area of research within Ethereum, with the primary goal of reducing node storage requirements, improving sync times for new nodes, and increasing decentralization by lowering hardware demands. Ethereum currently stores its state data in the Merkle Patricia Trie (MPT), and for stateless clients to be feasible, optimizations to the MPT structure must be implemented.

During the discussion, developers considered whether MPT could be optimized for lower storage while maintaining efficient verification. Another key question was how to handle state proofs efficiently, as well as the trade-offs between proof size and verification speed. The consensus was that further research is needed before any major changes can be implemented.

Related Articles

1. Highlights of Ethereum's All Core Devs Meeting (ACDC) #152
2. Highlights of Ethereum's All Core Devs Meeting (ACDE) #206
3. Highlights of Ethereum's All Core Devs Meeting (ACDE) #205
4. Highlights of Ethereum's All Core Devs Meeting (ACDC) #150
5. Highlights of Ethereum's All Core Devs Meeting (ACDE) #204