Accidental chain split · March 11, 2013
March 2013: the accidental chain split
A block that Bitcoin 0.8 accepted and older versions could not process split the network for about six hours, until large pools switched back to the older software.
- Split at
- Block 225,430
- Lasted
- About 6 hours, 24 blocks
- Rejoined
- Block 225,454, 06:19 UTC
- Double spend
- About $10,000 against OKPay
- Write-up
- BIP 50
Timeline (UTC)
- Mar 11, 2013 · 22:39 UTCBlock 225,430 is mined (time from its header). Nodes running 0.8 accept it; many older nodes cannot process it and keep building another chain.
- About 23:30 UTCThe split is noticed. The 0.8 chain has about 60% of the hashrate, so it will not heal on its own.
- OvernightDevelopers ask the largest 0.8 pools to return to 0.7. BTCGuild and Slush do, giving the older chain the majority.
- Mar 12, 2013 · 06:19 UTCThe chains rejoin at block 225,454. 24 blocks on the 0.8 side are orphaned.
- Aug 16, 2013Block 252,451 is accepted by the network, forking off nodes that never took the patch.
What broke
Versions before 0.8 stored the block chain in Berkeley DB, configured with too few database locks. A block touching more than about 5,000 transactions needed more locks than they had, so they rejected it. Version 0.8 had moved to LevelDB, which has no such limit, and accepted it. A database setting had silently become a consensus rule.
How it was fixed
Miners chose the chain every node could read: the largest 0.8 pools downgraded and the 0.8 nodes reorganised onto the older chain. Version 0.8.1 then refused blocks likely to need more than 10,000 locks and capped new blocks at 500,000 bytes for two months, and a patch raised the lock limit for older versions.
Impact
At least one large double spend happened during the split, about $10,000 against OKPay. Large merchants paused deposits until the chain settled.
Why it matters
Any difference in behaviour between node versions is part of consensus, whether anyone intended it or not. The incident is why Bitcoin developers treat consensus code as something to change only with extreme care.