ProposerVM

Every chain in an Avalanche node runs inside the ProposerVM. This wrapper connects the inner virtual machine (Subnet-EVM, Coreth, the PlatformVM itself) to Avalanche consensus and to the P-Chain. The wrapper decides who may build the next block and when, and it stamps every block with the P-Chain context of that decision. When an L1 stops producing blocks and the cause is hard to diagnose, the cause usually sits in this layer.

How ProposerVM wraps your VM

The ProposerVM intercepts the block pipeline: consensus talks to the wrapper, and the wrapper talks to your VM. Each wrapped block records two P-Chain heights with different jobs:

Do not confuse the two. Proposer eligibility comes from the parent's embedded height. The epoch height feeds the inner VM and warp verification. Every block carries both, and both advance only when a block is produced. An idle chain's view of the P-Chain stands still while the P-Chain itself moves on.

Proposer selection and slots

For each height, the wrapper samples a stake-weighted proposer list from the validator set at the parent block's embedded P-Chain height (vms/proposervm/proposer/windower.go). Each sampled proposer owns one slot. If a slot passes without a block, the schedule moves to the next sampled validator. The constants, from windower.go:

The practical consequence for validator set migrations: some operations change which identities exist on the P-Chain (a subnet to L1 conversion, a validator replacement, a long outage plus churn). Through any such operation, keep at least one identity from the current tip's embedded validator set alive until a new block lands. If you do not, the schedule rotates through validators that no longer exist, forever.

How a node decides to build

A node does not poll for build opportunities. It computes when its next proposer slot arrives and sleeps until then (WaitForEvent). It plans up to 720 slots (one hour) ahead. Two properties follow:

• A node that is not in the validator set at the parent's embedded height never calls the VM's BuildBlock. There is no error to log: from the node's point of view, it is simply never its turn. The avalanche_meterchainvm_build_block_count metric stays at exactly zero.
• The gauge avalanche_proposervm_block_building_slot climbs by one for every 5 seconds of tip staleness. A reading of 27,000 means the tip is roughly a day and a half old. A chain can therefore look fully able to build (pending transactions, healthy node, live validator) and still produce nothing, with no error anywhere.

Epochs (ACP-181)

ACP-181 gave every chain a coarser, per-chain P-Chain clock: the epoch. The first block whose timestamp is at or past the epoch's start time plus D seals the epoch. The next block starts the new epoch, and the new epoch adopts the sealing block's embedded height. Epochs never skip: the first block after a gap advances the number by exactly one, no matter how long the gap. There is no upper bound on an epoch's duration.

Two boundaries worth keeping sharp:

• Epochs feed verification context, not proposer sampling. Warp message verification and in-VM validator set reads use the epoch's pinned height. The proposer schedule uses the parent's embedded height. The two heal at different moments after an outage: one block restores building, the next block restores the epoch.
• Epoched views activated at the Granite upgrade (AvalancheGo v1.14.0). Before Granite, proposervm.getCurrentEpoch returns all zeros. The embedded height and the proposer schedule existed before epochs.

The RPC surface

Each wrapped chain exposes two ProposerVM endpoints at /ext/bc/<blockchainID>/proposervm (vms/proposervm/service.go):

No RPC reads an arbitrary historical block's embedded height. The two methods above are the whole per-chain surface. For validator sets at a historical height, use platform.getValidatorsAt (it accepts any depth and the keyword proposed).

Troubleshooting: my L1 stopped producing blocks

The signature of a frozen ProposerVM view, all at once:

• The mempool accepts transactions with correct nonces, and they stay pending.
• The chain height is flat; the tip's timestamp ages.
• avalanche_meterchainvm_build_block_count is exactly 0; avalanche_proposervm_block_building_slot climbs.
• No errors in the logs, and every health check is green: bootstrapped, Validating, peers connected.

How a healthy view goes stale

Three causes lead to the same frozen state. A halt: every validator goes offline for long enough. When the chain returns, its view is whatever the tip recorded before the halt. A conversion on unsynced validators: a subnet to L1 conversion hands the chain to a validator set that never synced it. The real history stays in the old nodes' databases, while the P-Chain now lists a new validator set. A restore: a database restored to an old tip brings back the old view, no matter how current the node's own P-Chain is.

Diagnose a frozen view

curl -s -X POST -H 'content-type: application/json' \
  --data '{"jsonrpc":"2.0","id":1,"method":"eth_getBlockByNumber","params":["latest", false]}' \
  http://127.0.0.1:9650/ext/bc/<blockchainID>/rpc | jq -r '.result.timestamp'

curl -s -X POST -H 'content-type: application/json' \
  --data '{"jsonrpc":"2.0","id":1,"method":"proposervm.getCurrentEpoch","params":{}}' \
  http://127.0.0.1:9650/ext/bc/<blockchainID>/proposervm

curl -s -X POST -H 'content-type: application/json' \
  --data '{"jsonrpc":"2.0","id":1,"method":"platform.getValidatorsAt","params":{"height":<FROZEN_HEIGHT>,"subnetID":"<SUBNET_ID>"}}' \
  http://127.0.0.1:9650/ext/bc/P

curl -s -X POST -H 'content-type: application/json' \
  --data '{"jsonrpc":"2.0","id":1,"method":"platform.getValidatorsAt","params":{"height":"proposed","subnetID":"<SUBNET_ID>"}}' \
  http://127.0.0.1:9650/ext/bc/P

curl -s http://127.0.0.1:9650/ext/metrics \
  | grep -E 'proposervm_block_building_slot|meterchainvm_build_block(_err)?_count'

Recover

The fix is one registration and one block. Put an identity the frozen view already knows back on the live P-Chain. Let it build once. The view then heals itself.

cat > sig-agg-config.json <<'EOF'
{
  "log-level": "debug",
  "info-api":    { "base-url": "https://api.avax-test.network" },
  "p-chain-api": { "base-url": "https://api.avax-test.network" },
  "api-port": 8080,
  "metrics-port": 8081,
  "tracked-subnet-ids": ["<YOUR_SUBNET_ID>"]
}
EOF

docker run --rm -p 8080:8080 -p 8081:8081 \
  -v "$(pwd)/sig-agg-config.json:/config.json" \
  avaplatform/signature-aggregator:v0.6.0-fuji \
  --config-file /config.json

curl -s http://localhost:8080/aggregate-signatures \
  -H 'content-type: application/json' \
  --data '{"message":"<HEX_UNSIGNED_WARP_MESSAGE>","signing-subnet-id":"<SUBNET_ID>","quorum-percentage":67}'

A worked example

A two validator subnet (weights 80 and 20) misses a network upgrade and converts to an L1 with a single fresh node at weight 100. The fresh node was never synced. Alone in its validator set, it bootstraps against itself and decides that its own empty database is the network. It quietly builds a new chain from genesis, with perfectly normal logs. The operator notices that contracts are missing, stops the node, and restores the original database. Now the real history is back, but the restored tip predates the conversion. Its embedded height resolves the old set (weights 80 and 20), and neither of those is a P-Chain validator anymore. The only live validator (the 100) is invisible to the schedule. Transactions queue, build counters sit at zero, and the slot counter climbs.

The fix took one registration and one block: re-register the surviving 20 identity at weight 19 through the ValidatorManager. That identity was visible to both the frozen view and the live P-Chain, so it built the unlocking block. The block after that restored the epoch.

All numbers are illustrative.

Related pitfalls

Prevention checklist

• Before any ConvertSubnetToL1Tx, confirm the incoming validator is bootstrapped for the chain (not just the P-Chain) and that its tip hash matches an existing node at the same height.
• Never leave a fresh sole validator alone with an empty database: it will build its own chain rather than wait.
• Through every conversion or restore, keep at least one identity from the current tip's validator set registered on the P-Chain until a new block lands.
• Take cold database backups before you swap data directories. Run the set intersection check before you declare a restore done.
• Alert on production silence: block_building_slot climbs while build_block_count stays flat, and the tip age grows. These alerts catch this class in minutes instead of days.
• Keep validator continuous fee balances funded: inactive weight cannot sign but still counts against every quorum.
• Run current releases: the v1.15.0 line logs the offending parent P-Chain height when a node cannot determine its proposer slot.

Related