EIP-7688 - Forward compatible consensus data structures

Created	2024-04-15
Status	Review
Category	Core
Type	Standards Track
Authors
Requires	EIP-6110 EIP-7002 EIP-7251 EIP-7495 EIP-7549 EIP-7569

Abstract

This EIP defines the changes needed to adopt StableContainer from EIP-7495 in consensus data structures.

Motivation

Ethereum's consensus data structures make heavy use of Simple Serialize (SSZ) Container, which defines how they are serialized and merkleized. The merkleization scheme allows application implementations to verify that individual fields (and partial fields) have not been tampered with. This is useful, for example, in smart contracts of decentralized staking pools that wish to verify that participating validators have not been slashed.

While SSZ Container defines how data structures are merkleized, the merkleization is prone to change across the different forks. When that happens, e.g., because new features are added or old features get removed, existing verifier implementations need to be updated to be able to continue processing proofs.

StableContainer, of EIP-7495, is a forward compatible alternative that guarantees a forward compatible merkleization scheme. By transitioning consensus data structures to use StableContainer, smart contracts that contain verifier logic no longer have to be maintained in lockstep with Ethereum's fork schedule as long as the underlying features that they verify don't change. For example, as long as the concept of slashing is represented using the boolean slashed field, existing verifiers will not break when unrelated features get added or removed. This is also true for off-chain verifiers, e.g., in hardware wallets or in operating systems for mobile devices that are on a different software update cadence than Ethereum.

Specification

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 and RFC 8174.

Conversion procedure

For each converted data structure, a new fork agnostic StableContainer type B is introduced that serves as the primary definition of each data structure.

Each StableContainer is assigned a capacity to represent its potential design space that SHALL NOT change across future forks; if it is later determined that it is insufficient, a new field can be added to contain additional fields in a sub-container.
The StableContainer starts as a copy of the latest fork's Container equivalent, except that all field types T are wrapped into Optional[T].
To guarantee forward and backward compatibility, new fields from future forks MUST only be appended to the StableContainer definition.
The type of existing fields MUST NOT change, including the capacity of List/Bitlist. If a change is necessary, the old field SHALL NOT be used anymore and a new field with a new name SHALL be considered. It is important to anticipate potential future extensions when deciding on the capacities of the StableContainer itself and of the various lists.
For List/Bitlist, the opportunity SHOULD be used to re-evaluate their design space capacity. If the design space is increased, application logic SHALL check the fork specific length limit; the SSZ library solely defines the merkleization limit, not the serialization limit.
The conversion process is repeated for each field type. All field types referred to by the StableContainer MUST be StableContainer themselves, or are considered immutable.

Subsequently, for each StableContainer base type B, a fork specific Profile[B] type is introduced that matches the latest fork's Container equivalent. The old Container is no longer necessary. The SSZ serialization of Profile is compatible with Container, but the merkleization and hash_tree_root are computed differently. Furthermore, Profile MAY use fields of Optional type if necessary.

Subsequent forks specify a new Profile.

If new fields of type T are added, they are appended to the StableContainer as Optional[T] to register them with the stable merkleization scheme. In the new fork's Profile, the new field MAY be T (required), or Optional[T] (optional).
If old fields are deprecated, they are kept in the StableContainer to retain the stable merkleization scheme. In the new fork's Profile, the field is omitted from the definition. The SSZ library guarantees that hash_tree_root and all generalized indices remain the same.
Other fields MAY be changed between T (required) and Optional[T] (optional) in the new fork's Profile. No changes to the StableContainer are required for such changes.

Immutable types

These types are used as part of the StableContainer definitions, and, as they are not StableContainer themselves, are considered to have immutable Merkleization. If a future fork requires changing these types in an incompatible way, a new type SHALL be defined and assigned a new field name.

Type	Description
`Slot`	Slot number on the beacon chain
`Epoch`	Epoch number on the beacon chain, a group of slots
`CommitteeIndex`	Index of a committee within a slot
`ValidatorIndex`	Unique index of a beacon chain validator
`Gwei`	Amount in Gwei (1 ETH = 10^9 Gwei = 10^18 Wei)
`Root`	Byte vector containing an SSZ Merkle root
`Hash32`	Byte vector containing an opaque 32-byte hash
`Version`	Consensus fork version number
`BLSPubkey`	Cryptographic type representing a BLS12-381 public key
`BLSSignature`	Cryptographic type representing a BLS12-381 signature
`KZGCommitment`	G1 curve point for the KZG polynomial commitment scheme
`Fork`	Consensus fork information
`Checkpoint`	Tuple referring to the most recent beacon block up through an epoch's start slot
`Validator`	Information about a beacon chain validator
`AttestationData`	Vote that attests to the availability and validity of a particular consensus block
`Eth1Data`	Target tracker for importing deposits from transaction logs
`DepositData`	Log data emitted as part of a transaction's receipt when depositing to the beacon chain
`BeaconBlockHeader`	Consensus block header
`ProposerSlashing`	Tuple of two equivocating consensus block headers
`Deposit`	Tuple of deposit data and its inclusion proof
`VoluntaryExit`	Consensus originated request to exit a validator from the beacon chain
`SignedVoluntaryExit`	Tuple of voluntary exit request and its signature
`SyncAggregate`	Cryptographic type representing an aggregate sync committee signature
`ExecutionAddress`	Byte vector containing an account address on the execution layer
`Transaction`	Byte list containing an RLP encoded transaction
`WithdrawalIndex`	Unique index of a withdrawal from any validator's balance to the execution layer
`Withdrawal`	Withdrawal from a beacon chain validator's balance to the execution layer
`DepositRequest`	Tuple of flattened deposit data and its sequential index
`WithdrawalRequest`	Execution originated request to withdraw from a validator to the execution layer
`ConsolidationRequest`	Execution originated request to consolidate two beacon chain validators
`BLSToExecutionChange`	Request to register the withdrawal account address of a beacon chain validator
`SignedBLSToExecutionChange`	Tuple of withdrawal account address registration request and its signature
`ParticipationFlags`	Participation tracker of a beacon chain validator within an epoch
`HistoricalSummary`	Tuple combining a historical block root and historical state root
`PendingDeposit`	Pending operation for depositing to a beacon chain validator
`PendingPartialWithdrawal`	Pending operation for withdrawing from a beacon chain validator
`PendingConsolidation`	Pending operation for consolidating two beacon chain validators

`StableContainer` capacities

Name	Value	Description
`MAX_ATTESTATION_FIELDS`	`uint64(2**3)` (= 8)	Maximum number of fields to which `StableAttestation` can ever grow in the future
`MAX_INDEXED_ATTESTATION_FIELDS`	`uint64(2**3)` (= 8)	Maximum number of fields to which `StableIndexedAttestation` can ever grow in the future
`MAX_EXECUTION_PAYLOAD_FIELDS`	`uint64(2**6)` (= 64)	Maximum number of fields to which `StableExecutionPayload` can ever grow in the future
`MAX_EXECUTION_REQUESTS_FIELDS`	`uint64(2**4)` (= 16)	Maximum number of fields to which `StableExecutionRequests` can ever grow in the future
`MAX_BEACON_BLOCK_BODY_FIELDS`	`uint64(2**6)` (= 64)	Maximum number of fields to which `StableBeaconBlockBody` can ever grow in the future
`MAX_BEACON_STATE_FIELDS`	`uint64(2**7)` (= 128)	Maximum number of fields to which `StableBeaconState` can ever grow in the future

Maximum proof depth:

StableBeaconState > validators (1 + 7) > <item> (1 + 40) > pubkey (3) > <chunk> (1) = 53 bits
StableBeaconBlockBody > execution_payload (1 + 6) > transactions (1 + 6) > <item> (1 + 20) > <chunk> (1 + 25) = 61 bits

Fork-agnostic `StableContainer` definitions

These type definitions are fork independent and shared across all forks. They are not exchanged over libp2p.

class StableAttestation(StableContainer[MAX_ATTESTATION_FIELDS]):
    aggregation_bits: Optional[Bitlist[MAX_VALIDATORS_PER_COMMITTEE * MAX_COMMITTEES_PER_SLOT]]
    data: Optional[AttestationData]
    signature: Optional[BLSSignature]
    committee_bits: Optional[Bitvector[MAX_COMMITTEES_PER_SLOT]]

class StableIndexedAttestation(StableContainer[MAX_INDEXED_ATTESTATION_FIELDS]):
    attesting_indices: Optional[List[ValidatorIndex, MAX_VALIDATORS_PER_COMMITTEE * MAX_COMMITTEES_PER_SLOT]]
    data: Optional[AttestationData]
    signature: Optional[BLSSignature]

class StableAttesterSlashing(Container):
    attestation_1: StableIndexedAttestation
    attestation_2: StableIndexedAttestation

class StableExecutionPayload(StableContainer[MAX_EXECUTION_PAYLOAD_FIELDS]):
    parent_hash: Optional[Hash32]
    fee_recipient: Optional[ExecutionAddress]  # 'beneficiary' in the yellow paper
    state_root: Optional[Bytes32]
    receipts_root: Optional[Bytes32]
    logs_bloom: Optional[ByteVector[BYTES_PER_LOGS_BLOOM]]
    prev_randao: Optional[Bytes32]  # 'difficulty' in the yellow paper
    block_number: Optional[uint64]  # 'number' in the yellow paper
    gas_limit: Optional[uint64]
    gas_used: Optional[uint64]
    timestamp: Optional[uint64]
    extra_data: Optional[ByteList[MAX_EXTRA_DATA_BYTES]]
    base_fee_per_gas: Optional[uint256]
    block_hash: Optional[Hash32]  # Hash of execution block
    transactions: Optional[List[Transaction, MAX_TRANSACTIONS_PER_PAYLOAD]]
    withdrawals: Optional[List[Withdrawal, MAX_WITHDRAWALS_PER_PAYLOAD]]  # [New in Capella]
    blob_gas_used: Optional[uint64]  # [New in Deneb:EIP4844]
    excess_blob_gas: Optional[uint64]  # [New in Deneb:EIP4844]

class StableExecutionPayloadHeader(StableContainer[MAX_EXECUTION_PAYLOAD_FIELDS]):
    parent_hash: Optional[Hash32]
    fee_recipient: Optional[ExecutionAddress]
    state_root: Optional[Bytes32]
    receipts_root: Optional[Bytes32]
    logs_bloom: Optional[ByteVector[BYTES_PER_LOGS_BLOOM]]
    prev_randao: Optional[Bytes32]
    block_number: Optional[uint64]
    gas_limit: Optional[uint64]
    gas_used: Optional[uint64]
    timestamp: Optional[uint64]
    extra_data: Optional[ByteList[MAX_EXTRA_DATA_BYTES]]
    base_fee_per_gas: Optional[uint256]
    block_hash: Optional[Hash32]  # Hash of execution block
    transactions_root: Optional[Root]
    withdrawals_root: Optional[Root]  # [New in Capella]
    blob_gas_used: Optional[uint64]  # [New in Deneb:EIP4844]
    excess_blob_gas: Optional[uint64]  # [New in Deneb:EIP4844]

class StableExecutionRequests(StableContainer[MAX_EXECUTION_REQUESTS_FIELDS]):
    deposits: Optional[List[DepositRequest, MAX_DEPOSIT_REQUESTS_PER_PAYLOAD]]  # [New in Electra:EIP6110]
    withdrawals: Optional[List[WithdrawalRequest, MAX_WITHDRAWAL_REQUESTS_PER_PAYLOAD]]  # [New in Electra:EIP7002:EIP7251]
    consolidations: Optional[List[ConsolidationRequest, MAX_CONSOLIDATION_REQUESTS_PER_PAYLOAD]]  # [New in Electra:EIP7251]

class StableBeaconBlockBody(StableContainer[MAX_BEACON_BLOCK_BODY_FIELDS]):
    randao_reveal: Optional[BLSSignature]
    eth1_data: Optional[Eth1Data]  # Eth1 data vote
    graffiti: Optional[Bytes32]  # Arbitrary data
    proposer_slashings: Optional[List[ProposerSlashing, MAX_PROPOSER_SLASHINGS]]
    attester_slashings: Optional[List[StableAttesterSlashing, MAX_ATTESTER_SLASHINGS_ELECTRA]]  # [Modified in Electra:EIP7549]
    attestations: Optional[List[StableAttestation, MAX_ATTESTATIONS_ELECTRA]]  # [Modified in Electra:EIP7549]
    deposits: Optional[List[Deposit, MAX_DEPOSITS]]
    voluntary_exits: Optional[List[SignedVoluntaryExit, MAX_VOLUNTARY_EXITS]]
    sync_aggregate: Optional[SyncAggregate]  # [New in Altair]
    execution_payload: Optional[StableExecutionPayload]  # [New in Bellatrix]
    bls_to_execution_changes: Optional[List[SignedBLSToExecutionChange, MAX_BLS_TO_EXECUTION_CHANGES]]  # [New in Capella]
    blob_kzg_commitments: Optional[List[KZGCommitment, MAX_BLOB_COMMITMENTS_PER_BLOCK]]  # [New in Deneb:EIP4844]
    execution_requests: Optional[StableExecutionRequests]  # [New in Electra]

class StableBeaconState(StableContainer[MAX_BEACON_STATE_FIELDS]):
    # Versioning
    genesis_time: Optional[uint64]
    genesis_validators_root: Optional[Root]
    slot: Optional[Slot]
    fork: Optional[Fork]
    # History
    latest_block_header: Optional[BeaconBlockHeader]
    block_roots: Optional[Vector[Root, SLOTS_PER_HISTORICAL_ROOT]]
    state_roots: Optional[Vector[Root, SLOTS_PER_HISTORICAL_ROOT]]
    historical_roots: Optional[List[Root, HISTORICAL_ROOTS_LIMIT]]  # Frozen in Capella, replaced by historical_summaries
    # Eth1
    eth1_data: Optional[Eth1Data]
    eth1_data_votes: Optional[List[Eth1Data, EPOCHS_PER_ETH1_VOTING_PERIOD * SLOTS_PER_EPOCH]]
    eth1_deposit_index: Optional[uint64]
    # Registry
    validators: Optional[List[Validator, VALIDATOR_REGISTRY_LIMIT]]
    balances: Optional[List[Gwei, VALIDATOR_REGISTRY_LIMIT]]
    # Randomness
    randao_mixes: Optional[Vector[Bytes32, EPOCHS_PER_HISTORICAL_VECTOR]]
    # Slashings
    slashings: Optional[Vector[Gwei, EPOCHS_PER_SLASHINGS_VECTOR]]  # Per-epoch sums of slashed effective balances
    # Participation
    previous_epoch_participation: Optional[List[ParticipationFlags, VALIDATOR_REGISTRY_LIMIT]]  # [Modified in Altair]
    current_epoch_participation: Optional[List[ParticipationFlags, VALIDATOR_REGISTRY_LIMIT]]  # [Modified in Altair]
    # Finality
    justification_bits: Optional[Bitvector[JUSTIFICATION_BITS_LENGTH]]  # Bit set for every recent justified epoch
    previous_justified_checkpoint: Optional[Checkpoint]
    current_justified_checkpoint: Optional[Checkpoint]
    finalized_checkpoint: Optional[Checkpoint]
    # Inactivity
    inactivity_scores: Optional[List[uint64, VALIDATOR_REGISTRY_LIMIT]]  # [New in Altair]
    # Sync
    current_sync_committee: Optional[SyncCommittee]  # [New in Altair]
    next_sync_committee: Optional[SyncCommittee]  # [New in Altair]
    # Execution
    latest_execution_payload_header: Optional[StableExecutionPayloadHeader]  # [New in Bellatrix]
    # Withdrawals
    next_withdrawal_index: Optional[WithdrawalIndex]  # [New in Capella]
    next_withdrawal_validator_index: Optional[ValidatorIndex]  # [New in Capella]
    # Deep history valid from Capella onwards
    historical_summaries: Optional[List[HistoricalSummary, HISTORICAL_ROOTS_LIMIT]]  # [New in Capella]
    deposit_requests_start_index: Optional[uint64]  # [New in Electra:EIP6110]
    deposit_balance_to_consume: Optional[Gwei]  # [New in Electra:EIP7251]
    exit_balance_to_consume: Optional[Gwei]  # [New in Electra:EIP7251]
    earliest_exit_epoch: Optional[Epoch]  # [New in Electra:EIP7251]
    consolidation_balance_to_consume: Optional[Gwei]  # [New in Electra:EIP7251]
    earliest_consolidation_epoch: Optional[Epoch]  # [New in Electra:EIP7251]
    pending_deposits: Optional[List[PendingDeposit, PENDING_DEPOSITS_LIMIT]]  # [New in Electra:EIP7251]
    # [New in Electra:EIP7251]
    pending_partial_withdrawals: Optional[List[PendingPartialWithdrawal, PENDING_PARTIAL_WITHDRAWALS_LIMIT]]
    pending_consolidations: Optional[List[PendingConsolidation, PENDING_CONSOLIDATIONS_LIMIT]]  # [New in Electra:EIP7251]

Fork-specific `Profile` definitions

The consensus type definitions specific to the fork that introduces this EIP are updated to inherit the Merkleization of the StableContainer definitions. Fields are kept as is.

class Attestation(Profile[StableAttestation]):
    ...

class IndexedAttestation(Profile[StableIndexedAttestation]):
    ...

class ExecutionPayload(Profile[StableExecutionPayload]):
    ...

class ExecutionPayloadHeader(Profile[StableExecutionPayloadHeader]):
    ...

class ExecutionRequests(Profile[StableExecutionRequests]):
    ...

class BeaconBlockBody(Profile[StableBeaconBlockBody]):
    ...

class BeaconState(Profile[StableBeaconState]):
    ...

Rationale

Best timing?

Applying this EIP breaks hash_tree_root and Merkle tree verifiers a single time, while promising forward compatibility from the fork going forward. It is best to apply it before merkleization would be broken by different changes. Merkleization is broken by a Container reaching a new power of 2 in its number of fields.

Can this be applied retroactively?

While Profile serializes in the same way as the legacy Container, the merkleization and hash_tree_root of affected data structures changes. Therefore, verifiers that wish to process Merkle proofs of legacy variants still need to support the corresponding legacy schemes.

Immutability

Once a field in a StableContainer has been published, its name can no longer be used to represent a different type in the future. This includes list types with a higher capacity than originally intended. This is in line with historical management of certain cases:

Phase0: BeaconState contained previous_epoch_attestations / current_epoch_attestations
Altair: BeaconState replaced these fields with previous_epoch_participation / current_epoch_participation

Furthermore, new fields have to be appended at the end of StableContainer. This is in line with historical management of other cases:

Capella appended historical_summaries to BeaconState instead of squeezing the new field next to historical_roots

With StableContainer, stable Merkleization requires these rules to become strict.

Backwards Compatibility

Existing Merkle proof verifiers need to be updated to support the new Merkle tree shape. This includes verifiers in smart contracts on different blockchains and verifiers in hardware wallets, if applicable.

Note that backwards compatibility is also broken when one of the converted Container data structures would reach a new power of 2 in its number of fields.

Security Considerations

None