Scalability FAQ: Difference between revisions
It was recommended that there be a FAQ to avoid retyping the same answers on Reddit and elsewhere |
→What is the short history of the block size limit?: Corrections |
||
(21 intermediate revisions by 5 users not shown) | |||
Line 7: | Line 7: | ||
=== What is the short history of the block size limit? === | === What is the short history of the block size limit? === | ||
Bitcoin was initially released with a dynamic block "change" limit equivalent to approximately 500kB worth of normal transactions; there was a "hard" size limit of 32 MiB, but it was effectively impractical to hit unless one crafted a spam block. | |||
Around 15 July 2010, Satoshi Nakamoto changed Bitcoin’s mining code so that it would never create any blocks larger than 990,000 bytes.<ref>[https://github.com/bitcoin/bitcoin/commit/a30b56ebe76ffff9f9cc8a6667186179413c6349 Bitcoin commit a30b56e], Satoshi Nakamoto, 15 July 2010</ref> | |||
Two months later on 7 September 2010, Nakamoto changed Bitcoin’s consensus rules to reject blocks larger than 1,000,000 bytes (1 megabyte) if their block height was higher than 79,400.<ref>[https://github.com/bitcoin/bitcoin/commit/8c9479c6bbbc38b897dc97de9d04e4d5a5a36730 Bitcoin commit 8c9479c], Satoshi Nakamoto, 7 September 2010</ref> (Block 79,400 was later produced on 12 September 2010.<ref>[https://explorer.blockstream.com/block/000000000021d821ec06be7173f413690bc5c4bc648dfa70b3b6763236f055b7 Block height 79400], dated 12 September 2010</ref>) However, the effective ~500kB limit remained in place. | |||
Neither the July nor the September commit message explains the reason for the limit, although the careful attempt to prevent a fork may indicate Nakamoto didn’t consider it an emergency. | |||
Nakamoto’s subsequent warning not to run a blocksize increase patch which would have destructively forked the network said that '''if''' users needed to increase the blocksize, it '''could''' be accomplished with a planned hardfork at a future time<ref>[https://bitcointalk.org/index.php?topic=1347.msg15139#msg15139 Increasing the block size if needed], Satoshi Nakamoto, 3 October 2010</ref>, but he never specified which conditions might require one. | |||
Statements by Nakamoto in the summer of 2010 indicate he believed Bitcoin could scale to block sizes far larger than 1 megabyte. For example, on 5 August 2010, he wrote that "[W]hatever size micropayments you need will eventually be practical. I think in 5 or 10 years, the bandwidth and storage will seem trivial" and "[microtransactions] can become more practical ... if I implement client-only mode and the number of network nodes consolidates into a smaller number of professional server farms" <ref>[https://bitcointalk.org/index.php?topic=532.msg6306#msg6306 the number of network nodes consolidates into a smaller number of professional server farms]</ref>. Satoshi's reasoning was likely based on his belief that SPV would be a primary scaling mechanism. It's since been discovered that SPV security requires strong fraud proofs and the fallback requires ''full node logic.'' <ref>[https://en.bitcoin.it/wiki/User:Gmaxwell/features#Proofs Greg Maxwell's outline of fraud proof logic] Greg Maxwell's wiki page, 22 March, 2013</ref><ref>[https://bitcointalk.org/index.php?topic=96644.msg1064601#msg1064601 Greg Maxwell's early post on fraud proofs] Greg Maxwell on fraud proofs, 30 July, 2012</ref> | |||
In one of Nakamoto's final public messages, he wrote that "Bitcoin users might get increasingly tyrannical about limiting the size of the chain so it's easy for lots of users and small devices."<ref>[https://bitcointalk.org/index.php?topic=1790.msg28917#msg28917 BitDNS and Generalizing Bitcoin], Satoshi Nakamoto, 10 December 2010</ref> This statement suggests that Nakamoto was aware of the value of limiting the block size and that he correctly predicted it was an issue that Bitcoin users would be dealing with in the future---a future Nakamoto may have known would not include him. This was in the context of Satoshi attempting to redirect to a subchain non-Bitcoin uses like DNS data which used direct on-chain block space. | |||
=== What is this Transactions Per Second (TPS) limit? === | === What is this Transactions Per Second (TPS) limit? === | ||
Line 32: | Line 34: | ||
Both old estimates<ref>[[Scalability]], Bitcoin Wiki, retrieved 7 July | Both old estimates<ref>[[Scalability]], Bitcoin Wiki, retrieved 7 July | ||
2015</ref> and | 2015</ref> and newer estimates<ref name="garzik_bip" /> placed the pre-segwit | ||
theoretical | theoretical transaction speed at about 7 TPS. | ||
=== What do devs mean by the scaling expressions O(1), O(n), O(n<sup>2</sup>), etc…? === | === What do devs mean by the scaling expressions O(1), O(n), O(n<sup>2</sup>), etc…? === | ||
Line 49: | Line 50: | ||
==== O(1) block propagation ==== | ==== O(1) block propagation ==== | ||
Bitcoin Core | Bitcoin Core relays unconfirmed transactions and then later reconstructs blocks when they are mined while avoiding redundant transaction propagation. The prior transaction double-relay redundancy was eliminated with the invention of a novel block propagation algorithm called Compact Blocks <ref>[https://bitcoincore.org/en/2016/06/07/compact-blocks-faq/ Compact Blocks] Compact Blocks, BIP0152</ref> to allow miners to propagate large blocks very quickly to active network nodes and also significantly reduced miners' need for powerful bandwidth. Miners use a network<ref name="block_relay_net">[http://bitcoinrelaynetwork.org/ Matt Corallo's block relay network]</ref> that is designed for high-bandwidth Wirehair-based<ref>[https://github.com/catid/wirehair Wirehair Github Repository] Wirehair</ref> which is slightly faster still than stock Bitcoin Core. | ||
==== O(n<sup>2</sup>) network total validation resource requirements ==== | ==== O(n<sup>2</sup>) network total validation resource requirements with decentralization level held constant ==== | ||
Each on-chain Bitcoin transaction needs to be processed by each full node. If we assume that a certain percentage of users run full nodes (''n'') and that each user creates a certain number of transactions on average (''n'' again), then the network’s total resource requirements are <code>n<sup>2</sup> = n * n</code>. In short, this means that the aggregate cost of | [[File:On2_scaling_illustrated.png|thumb|right|visualization of O(n<sup>2</sup>) scaling]] | ||
While the validation effort required per full node simply grows in O(n), the combined validation effort of all nodes grows by O(n<sup>2</sup>) '''in the domain of resource consumption'', decentralization being held constant. For a single node, it takes twice as many resources to process the transactions of twice as many users, while for all nodes it takes a combined four times as many CPU resources to process the transactions of twice as many users assuming the number of full nodes increases in proportion to the number of users. | |||
Each on-chain Bitcoin transaction needs to be processed by each full node. If we assume that a certain percentage of users run full nodes (''n'') and that each user creates a certain number of transactions on average (''n'' again), then the network’s total resource requirements are <code>n<sup>2</sup> = n * n</code>. In short, this means that the aggregate cost of handling all transactions on-chain quadruples each time the number of users doubles.<ref>[http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-June/009199.html Total system cost], Dr. Adam Back, 28 June 2015</ref> For example, | |||
* Imagine a network starts with 100 users and 2 total nodes (a 2% ratio). | * Imagine a network starts with 100 users and 2 total nodes (a 2% ratio). | ||
* The network doubles in users to 200. The number of nodes also doubles to 4 (maintaining a 2% ratio of decentralized low-trust security). However, double the number of users means double the number of transactions, and each node needs to process ''every'' transaction---so each node now does double work with its bandwidth and its CPU. With double the number of nodes and double the amount of work per node, total work increases by four times. | * The network doubles in users to 200. The number of nodes also doubles to 4 (maintaining a 2% ratio of decentralized low-trust security). However, double the number of users means double the number of transactions, and each node needs to process ''every'' transaction---so each node now does double work with its bandwidth and its CPU. With double the number of nodes and double the amount of work per node, total work increases by four times. | ||
* The network doubles again: 400 users, 8 nodes (still 2% of total), and 4 times the original workload per node for a total of 16 times as much aggregate work as the original. | * The network doubles again: 400 users, 8 nodes (still 2% of total), and 4 times the original workload per node for a total of 16 times as much aggregate work as the original. | ||
* Another doubling: 800 users, 16 nodes (still 2%), and 8 times the original workload per node for a total of 64 times aggregate work as the original. | * Another doubling: 800 users, 16 nodes (still 2%), and 8 times the original workload per node for a total of 64 times aggregate work as the original. | ||
* Another doubling: 1,600 users—sixteen times the original---with 32 nodes (still 2%), and 16 times the original workload per node. The aggregate total work done by nodes is 256 times higher than it was originally. | * Another doubling: 1,600 users—sixteen times the original---with 32 nodes (still 2%), and 16 times the original workload per node. The aggregate total work done by nodes is 256 times higher than it was originally. | ||
===== Criticisms ===== | |||
Emphasis on the total validation resource requirements is suggested by some individuals to be misleading, as they claim it obfuscates the growth of the supporting base of full nodes that the total validation resource effort is split amongst. The validation resource effort made by each individual full node increases at O(n), and critics say this is the only pertinent fact with respect to scaling. Some critics also point out that the O(n<sup>2</sup>) network total validation resource requirements claim is assuming decentralization must be held constant as the network scales, and that this is not a founding principle of Bitcoin. Of course, as quoted above, Satoshi knew that decentralization would be a critical value of the safety of the system in the first place by demonstrating he could foresee multiple paths for the future of Bitcoin. | |||
=== What’s the difference between on-chain and off-chain transactions? === | === What’s the difference between on-chain and off-chain transactions? === | ||
Line 68: | Line 81: | ||
* '''Exchange transactions:''' when you buy or sell bitcoins, the exchange tracks ownership in a database without putting data in the block chain. Only when you deposit or withdraw bitcoins does the transaction appear on the block chain. | * '''Exchange transactions:''' when you buy or sell bitcoins, the exchange tracks ownership in a database without putting data in the block chain. Only when you deposit or withdraw bitcoins does the transaction appear on the block chain. | ||
* '''Web wallet internal payments:''' many web wallets allow users of the service to pay other users of the same service using off-chain payments. For example, when one Coinbase user pays another Coinbase user. | * '''Web wallet internal payments:''' many web wallets allow users of the service to pay other users of the same service using off-chain payments. For example, when one Coinbase user pays another Coinbase user. '''WARNING: Web wallets are extremely unsafe and many users have had funds stolen from them worth many hundreds of millions of dollars in today's exchange rates.''' | ||
* '''Tipping services:''' most tipping services | * '''Tipping services:''' most tipping services todayvuse off-chain transactions for everything except deposits and withdrawals from their services. | ||
* '''Payment channels:''' channels are started using one on-chain transaction and ended using a second on-chain transaction.<ref>[https://bitcoin.org/en/developer-guide#micropayment-channel Micropayment channels], Bitcoin.org Developer Guide</ref> In between can be an essentially unlimited number of off-chain transactions as small as a single satoshi (1/100,000,000th of a bitcoin). Payment channels include those that | * '''Payment channels:''' channels are started using one on-chain transaction and ended using a second on-chain transaction.<ref>[https://bitcoin.org/en/developer-guide#micropayment-channel Micropayment channels], Bitcoin.org Developer Guide</ref> In between can be an essentially unlimited number of off-chain transactions as small as a single satoshi (1/100,000,000th of a bitcoin). Payment channels include those that were baked into the protocol at release as well as proposed hub-and-spoke channels and the more advanced [http://lightning.network/ Lightning network]. | ||
The vast majority of all bitcoin-denominated payments today are made off-chain.<ref>[http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-June/008844.html Pure off-chain is a weak form of layer 2], Dr. Adam Back, 19 June 2015</ref> | |||
=== What is a fee market? === | === What is a fee market? === | ||
When you create a Bitcoin transaction, you have the option to pay a [[transaction fee]]. If your software is flexible, you can pay anything from zero fees (a free transaction) to 100% of the value of the transaction (spend-to-fees). | When you create a Bitcoin transaction, you have the option to pay a [[transaction fee]]. If your software is flexible, you can pay anything from zero fees (a free transaction) to 100% of the value of the transaction (spend-to-fees). This fee is comparable to a tip. The higher it is, the bigger the incentive of the miners to incorporate your transaction into the next block. | ||
When miners assemble a block, they are free to include whatever transactions they wish. They usually include as many as possible up to the maximum block size and then prioritize the transactions that pay the most fees per kilobyte of data.<ref name="fee_pri">[https://bitcointalk.org/index.php?topic=95837.0 Fee prioritization patch], Gavin Andresen, 26 July 2012</ref> This usually results in higher-fee transactions confirming before lower-fee transactions. If blocks become full on a regular basis, users who pay a fee that's too small will have to wait a longer and longer time for their transactions to confirm. At this point, a demand-driven fee market will arise where transactions that pay higher fees get confirmed significantly faster than transactions that pay low fees.<ref name="todd_coinwallet">[https://gist.github.com/petertodd/8e87c782bdf342ef18fb Comments on the CoinWallet.eu tx-flood stress-test], Peter Todd, 22 June 2015</ref><ref name="garzik_bip" /> | |||
In a competitive market, prices are driven by supply and demand and the emerging equilibrium price asymptotically approaches marginal costs over time. However, the current version of segwit-enabled Bitcoin limits the supply to 4 MW (4 million weight units) per block, allowing only demand to adjust. It turns out that a fee market can stabilize as long as there is a block size limit.<ref>[https://medium.com/@bergealex4/bitcoin-is-unstable-without-the-block-size-size-limit-70db07070a54 Bitcoin is unstable without a block size limit], Alex B., 24 Oct 2016</ref> Simply allowing individual miners to include as many transactions as they want is problematic due to the externalities<ref name="exernality">[https://en.wikipedia.org/wiki/Externality Definition of externality], Wikipedia, 26 January 2016</ref> of including additional blocks and explained further in section [[#Should miners be allowed to decide the block size?|'should miners be allowed to decide the block size?']] . | |||
=== | === The naive way to scale Bitcoin === | ||
If Bitcoin decentralization is abandoned, whatever is left could scale limitlessly as a plain database-backed ledger. Mining uses enormous amounts of electricity to provide a secure, verifiable decentralized ledger and full nodes use an enormous amount of bandwidth and CPU time keeping miners honest. | |||
To wit, if users instead decided to hand authority over to someone they trusted, mining and keeping miners honest wouldn’t be necessary. This is how Visa, MasterCard, PayPal, and the rest of the centralized payment systems works—users trust them, and they have no special difficulty scaling to [[scalability|millions of transactions an hour]]. It’s comparatively energy-efficient; it isn’t decentralized, and thus isn't efficient to the ''purpose that Bitcoin was designed to accommodate.'' | |||
=== What is a hard fork, and how does it differ from other types of forks? === | === What is a hard fork, and how does it differ from other types of forks? === | ||
The word fork is badly overloaded in Bitcoin development. | The word fork is badly overloaded in Bitcoin development. The following terms are used, for example: | ||
* '''[[hardfork|Hard fork]]:''' a change to the system which is not backwards compatible. Everyone needs to upgrade or things | * '''[[hardfork|Hard fork]]:''' a change to the system which is not backwards compatible. Everyone needs to upgrade or things will go wrong. | ||
* '''[[softfork|Soft fork]]:''' a change to the system which is backwards compatible as long as a majority of miners enforce it. Full nodes that don’t upgrade have | * '''[[softfork|Soft fork]]:''' a change to the system which is backwards compatible as long as a majority of miners enforce it. Full nodes that don’t upgrade potentially but not necessarily have reduced security. | ||
* '''Chain fork:''' when there are two are more blocks at the same height on a block chain. This typically happens a few times a week by accident, but it can also indicate more severe problems. | * '''Chain fork:''' when there are two are more blocks at the same height on a block chain. This typically happens a few times a week by accident, but it can also indicate more severe problems. | ||
* '''[https://en.wikipedia.org/wiki/Fork_%28software_development%29 Software fork]:''' using the code from an open source project to create a different project. | * '''[https://en.wikipedia.org/wiki/Fork_%28software_development%29 Software fork]:''' using the code from an open source project to create a different project. | ||
Line 104: | Line 117: | ||
Bitcoin Core has historically come pre-configured to limit blocks it mines to certain sizes.<ref name="fee_pri" /> These “limits” aren’t restrictions placed on miners by other miners or nodes, but rather configuration options that help miners produce reasonably-sized blocks. | Bitcoin Core has historically come pre-configured to limit blocks it mines to certain sizes.<ref name="fee_pri" /> These “limits” aren’t restrictions placed on miners by other miners or nodes, but rather configuration options that help miners produce reasonably-sized blocks. | ||
Miners can change their soft limit size (up to | Miners can change their soft limit size (up to 4 MW) using <code>-blockmaxweight=<size></code> | ||
Default soft limits at various times: | Default soft limits at various times: | ||
Line 111: | Line 124: | ||
* '''November 2013:''' Raised to 750KB<ref>[https://github.com/bitcoin/bitcoin/commit/ad898b40aaf06c1cc7ac12e953805720fc9217c0 Bitcoin Core commit ad898b40], Gavin Andresen, 27 November 2013</ref> | * '''November 2013:''' Raised to 750KB<ref>[https://github.com/bitcoin/bitcoin/commit/ad898b40aaf06c1cc7ac12e953805720fc9217c0 Bitcoin Core commit ad898b40], Gavin Andresen, 27 November 2013</ref> | ||
* '''June 2015:''' Raise to 1 MB suggested<ref>[https://github.com/bitcoin/bitcoin/pull/6231 Bitcoin Core pull #6231], Chris Wheeler, 4 June 2015</ref> | * '''June 2015:''' Raise to 1 MB suggested<ref>[https://github.com/bitcoin/bitcoin/pull/6231 Bitcoin Core pull #6231], Chris Wheeler, 4 June 2015</ref> | ||
* '''August 2017:''' Segwit activates at block 481824<ref>[https://explorer.blockstream.com/block/0000000000000000001c8018d9cb3b742ef25114f27563e3fc4a1902167f9893 Block 481824] August 23, 2017</ref> | |||
== General Block Size Increase Theory == | == General Block Size Increase Theory == | ||
==== Why are some people in favour of keeping the block size conservative forever? ==== | |||
It is commonly claimed<ref name="garzik_bip" /><ref>[https://epicenterbitcoin.com/podcast/082/ Epicenter Bitcoin podcast #82], Mike Hearn (interviewed), 8 June 2015</ref> that there are people opposed to ever raising the maximum block size limit, but no Bitcoin developers have suggested a permanent blocksize.<ref name="maxwell_not_proposing" /> | |||
It is commonly claimed<ref name="garzik_bip" /><ref>[https://epicenterbitcoin.com/podcast/082/ Epicenter Bitcoin podcast #82], Mike Hearn (interviewed), 8 June 2015</ref> that there are people opposed to ever raising the maximum block size limit, but no Bitcoin developers have suggested | |||
All developers support raising the maximum size at some point<ref>[https://www.reddit.com/r/Bitcoin/comments/39kqzs/how_about_a_softfork_optin_blocksize_increase/cs5hj8l Nothing magic about 1MB], Dr. Adam Back, 13 June 2015</ref><ref>[http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-May/008328.html Regarding size-independent new block propagation], Pieter Wuille, 28 May 2015</ref>—they just disagree about whether now is the correct time. | All developers support raising the maximum size at some point<ref>[https://www.reddit.com/r/Bitcoin/comments/39kqzs/how_about_a_softfork_optin_blocksize_increase/cs5hj8l Nothing magic about 1MB], Dr. Adam Back, 13 June 2015</ref><ref>[http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-May/008328.html Regarding size-independent new block propagation], Pieter Wuille, 28 May 2015</ref>—they just disagree about whether now is the correct time. | ||
Line 127: | Line 138: | ||
A block may include as little as a single transaction, so miners can always restrict block size. Letting miners choose the maximum block size is more problematic for several reasons: | A block may include as little as a single transaction, so miners can always restrict block size. Letting miners choose the maximum block size is more problematic for several reasons: | ||
# '''Miners profit, others pay the cost:''' bigger blocks earn miners more fees, but miners | # '''Miners profit, others pay the cost:''' bigger blocks earn miners more fees, but technically miners can successfully mine without storing block data almost at all.<ref>[https://github.com/bitcoin/bitcoin/pull/5863 Bitcoin Core pull #5863 adding pruning functionality], Suhas Daftuar et al., 6 March 2015</ref> Some miners will SPV mine optimistically on other miners' blocks without even validating anyting. Other users who want full validation security, or who provide services to lightweight wallets, pay the costs of downloading and storing those larger blocks. | ||
# '''Bigger miners can afford more bandwidth:''' each miner needs to download every transaction that | # '''Bigger miners can afford more bandwidth:''' each miner needs to download every transaction that they want to include in blocks,<ref name="maxwell_correcting_assumptions" /> which means that they all need high-speed connections. However, a miner with 8.3% of total hash rate can take that cost out of the about 300 BTC they make a day, while a miner with only 0.7% of total hash rate has to take that cost out of only 25 BTC a day. This means bigger miners may logically choose to make bigger blocks even though it may further centralize mining. | ||
# '''Centralized hardware production:''' only a few companies in the world produce efficient mining equipment, and many of them have chosen to stop selling to the public.<ref>[http://blogs.wsj.com/venturecapital/2014/09/05/for-kncminer-time-to-forget-selling-bitcoin-machines-to-at-home-miners/ KnCMiner to stop selling to home miners], Wall Street Journal, Yuliya Chernova, 5 September 2014</ref> This prevents ordinary Bitcoin users from participating in the process even if they were willing to pay for mining equipment. | # '''Centralized hardware production:''' only a few companies in the world produce efficient mining equipment, and many of them have historically chosen to stop selling to the public.<ref>[http://blogs.wsj.com/venturecapital/2014/09/05/for-kncminer-time-to-forget-selling-bitcoin-machines-to-at-home-miners/ KnCMiner to stop selling to home miners], Wall Street Journal, Yuliya Chernova, 5 September 2014</ref> This prevents ordinary Bitcoin users from participating in the process even if they were willing to pay for mining equipment. | ||
# '''Voting by hash rate | # '''Voting by hash rate favours larger miners:''' voting based on hash rate allows the current hash rate majority (or super-majority) to enforce conditions on the minority. This is similar to a lobby of large businesses being able to get laws passed that may hurt smaller businesses.<ref>[https://www.reddit.com/r/Bitcoin/comments/39kzyt/draft_bip_100_soft_fork_block_size_increase/cs5gvak Miners choosing the block size limit is ill-advised], Meni Rosenfeld, 13 June 2015</ref> This is less of an issue now that miner-signalled activation has been abandoned. | ||
# '''Miners | # '''Miners do not care about users:''' this was demonstrated during the [[July 2015 Forks]] where even after miners lost over $50,000 USD in income from mining invalid chains, and even after they were told that long forks harm users, they continued to mine invalid blocks.<ref>[https://bitcointalk.org/index.php?topic=700411.msg11790734#msg11790734 Intention to continue SPV mining], Wang Chun, 4 July 2015</ref> | ||
However, there are also possible justifications for letting miners choose the block size: | However, there are also possible justifications for letting miners choose the block size: | ||
# '''Authenticated participants:''' miners are active participants in Bitcoin, and they can prove it by adding data to the blocks they create.<ref>[https://bitcoin.org/en/developer-reference#term-coinbase-field Coinbase field], Bitcoin.org Developer Reference</ref> It is much harder to prove people on Reddit, BitcoinTalk, and elsewhere are real people with an active stake in Bitcoin. | # '''Authenticated participants:''' miners are active participants in Bitcoin, and they can prove it by adding data to the blocks they create.<ref>[https://bitcoin.org/en/developer-reference#term-coinbase-field Coinbase field], Bitcoin.org Developer Reference</ref> It is much harder to prove people on Reddit, BitcoinTalk, and elsewhere are real people with an active stake in Bitcoin. | ||
# '''Bigger blocks may cost miners:''' small miners and poorly-connected miners are at a disadvantage if larger blocks are produced<ref name="wuille_centralization" />, and even large and well-connected miners may find that large blocks reduce their profit percentage if the cost of bandwidth rises too high or their stale rate increases.<ref>[http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-May/008355.html F2Pool block size concerns], Chun Wang, 29 May 2015</ref> | # '''Bigger blocks may cost smaller miners:''' small miners and poorly-connected miners are at a disadvantage if larger blocks are produced<ref name="wuille_centralization" />, and even large and well-connected miners may find that large blocks reduce their profit percentage if the cost of bandwidth rises too high or their stale rate increases.<ref>[http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-May/008355.html F2Pool block size concerns], Chun Wang, 29 May 2015</ref> | ||
==== How could a block size increase affect user security? ==== | ==== How could a block size increase affect user security? ==== | ||
Line 142: | Line 153: | ||
Bitcoin’s security is highly dependent on the number of active users who protect their bitcoins with a full node like Bitcoin Core. The more active users of full nodes, the harder it is for miners to trick users into accepting fake bitcoins or other frauds.<ref>[https://medium.com/@lopp/bitcoin-nodes-how-many-is-enough-9b8e8f6fd2cf Bitcoin nodes: how many is enough?], Jameson Lopp, 7 June 2015</ref> | Bitcoin’s security is highly dependent on the number of active users who protect their bitcoins with a full node like Bitcoin Core. The more active users of full nodes, the harder it is for miners to trick users into accepting fake bitcoins or other frauds.<ref>[https://medium.com/@lopp/bitcoin-nodes-how-many-is-enough-9b8e8f6fd2cf Bitcoin nodes: how many is enough?], Jameson Lopp, 7 June 2015</ref> | ||
Full nodes need to download and verify every block, and most nodes also store blocks plus relay transactions, | Full nodes need to download and verify every block, and most nodes also store blocks plus relay transactions, blocks, and filtered blocks to other users on the network. The bigger blocks become, the more difficult it becomes to do all this, so it is expected that bigger blocks will both reduce the number of users who currently run full nodes and suppress the number of users who decide to start running a full node later.<ref>[https://www.reddit.com/r/Bitcoin/comments/3bae2d/amir_taaki_on_raising_the_block_size_a_lot_of/cslcgmw Block size increases-only will create problems], Dr. Adam Back, 28 June 2015</ref> | ||
In addition, full blocks may increase mining centralization<ref name="wuille_centralization" /> at a time when mining is already so centralized that it makes it easy to reverse transactions which have been confirmed multiple times.<ref>[http://bitcoin.stackexchange.com/questions/32562/when-to-worry-about-1-confirmation-payments/32564#32564 When to worry about one-confirmation payments], David A. Harding, 17 November 2014</ref> | In addition, full blocks may increase mining centralization<ref name="wuille_centralization" /> at a time when mining is already so centralized that it makes it easy to reverse transactions which have been confirmed multiple times.<ref>[http://bitcoin.stackexchange.com/questions/32562/when-to-worry-about-1-confirmation-payments/32564#32564 When to worry about one-confirmation payments], David A. Harding, 17 November 2014</ref> | ||
Line 156: | Line 167: | ||
=== What happens if blocks aren't big enough to include all pending transactions? === | === What happens if blocks aren't big enough to include all pending transactions? === | ||
This is already the case | This is already often the case, so we know that miners will simply queue transactions. The transactions eligible for block inclusion that pay the highest fee per kiloweight will be confirmed earlier than transactions that pay comparatively lower fees.<ref name="fee_pri" /> | ||
What happens from there is debated: | What happens from there is debated: | ||
* BitcoinJ lead developer Mike Hearn | * BitcoinJ lead developer Mike Hearn believed in 2015 there would be “crashing nodes, a swelling transaction backlog, a sudden spike in double spending, [and] skyrocketing fees.”<ref>[https://medium.com/@octskyward/crash-landing-f5cc19908e32 Crash Landing], Mike Hearn, 7 May 2015</ref> Clearly this didn't happen, as of late 2020. | ||
* Bitcoin Foundation chief scientist Gavin Andresen | * Bitcoin Foundation chief scientist Gavin Andresen believed in 2015 that the “average transaction fee paid will rise, people or applications unwilling or unable to pay the rising fees will stop submitting transactions, [and] people and businesses will shelve plans to use Bitcoin, stunting growth and adoption.” <ref>[https://github.com/bitcoin/bitcoin/pull/6341 Bitcoin Core pull #6341], Gavin Andresen, 26 June 2015</ref> Clearly, this didn't happen either, as of late 2020. | ||
* Bitcoin Core developer Pieter Wuille | * Bitcoin Core developer Pieter Wuille replied in 2015 to Andresen’s comment above: “Is it fair to summarize this as ‘Some use cases won’t fit any more, people will decide to no longer use the blockchain for these purposes, and the fees will adapt.’? I think that is already happening […] I don’t think we should be afraid of this change or try to stop it.” <ref>[http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-June/009098.html The need for larger blocks], Pieter Wuille, 26 June 2015</ref> | ||
== Segregated Witness == | |||
[https://en.bitcoin.it/wiki/Segregated_Witness Segregated Witness], a block size increase proposal implemented as a soft-fork and which increased the block size to 4 megabytes (actually 4 mega-weight units but strictly-speaking that is contained within 4 megabytes maximum) was adopted as the Bitcoin blocksize increase in August 2017 in block 481824<ref>[https://explorer.blockstream.com/block/0000000000000000001c8018d9cb3b742ef25114f27563e3fc4a1902167f9893 Block 481824] Block 481824</ref>. It was most favoured by the user community and the Bitcoin developers, in spite of significant political pressure against it by signalling miners and attackers. Segwit was eventually forced into activation by James Hilliard's BIP0091<ref>[https://github.com/bitcoin/bips/blob/master/bip-0091.mediawiki BIP 0091], James Hilliard, May 22, 2017.</ref> | |||
== Specific Scaling Proposals == | == Specific Scaling Proposals (for historical interest) == | ||
=== Andresen-Hearn Block Size Increase Proposals === | === Andresen-Hearn Block Size Increase Proposals === | ||
Questions about the series of related proposals by Gavin Andresen and Mike Hearn to use a hard fork to raise the maximum block size, thereby allowing miners to include more on-chain transactions. As of July 2015, the current focus | Questions about the series of related proposals by Gavin Andresen and Mike Hearn to use a hard fork to raise the maximum block size, thereby allowing miners to include more on-chain transactions. As of July 2015, the current focus was [https://github.com/bitcoin/bips/pull/163 BIP101]. | ||
==== What | ==== What was the major advantage of this proposal? ==== | ||
''Bitcoin can support many more users.'' Assuming earliest possible adoption, 250 bytes per on-chain transaction, 144 blocks per day, and 2 transaction a day per user, Bitcoin can support about, | ''Bitcoin can support many more users.'' Assuming earliest possible adoption, 250 bytes per on-chain transaction, 144 blocks per day, and 2 transaction a day per user, Bitcoin can support about, | ||
Line 209: | Line 223: | ||
==== What are the dangers of the proposed hard fork? ==== | ==== What are the dangers of the proposed hard fork? ==== | ||
Under the current proposal, 75% of miners must indicate that they agree to accept blocks up to 8 MB in size.<ref name="bip101">[https://github.com/bitcoin/bips/pull/163 BIP101], Gavin Andresen, 24 June 2016</ref> After the change goes into effect, if any miner creates a block larger than 1 MB, all nodes which have not upgraded yet will reject the block.<ref>[https://bitcoin.org/en/developer-guide#consensus-rule-changes Consensus rule changes], Bitcoin.org Developer Guide</ref> | Under the then-current current proposal, 75% of miners must indicate that they agree to accept blocks up to 8 MB in size.<ref name="bip101">[https://github.com/bitcoin/bips/pull/163 BIP101], Gavin Andresen, 24 June 2016</ref> After the change goes into effect, if any miner creates a block larger than 1 MB, all nodes which have not upgraded yet will reject the block.<ref>[https://bitcoin.org/en/developer-guide#consensus-rule-changes Consensus rule changes], Bitcoin.org Developer Guide</ref> | ||
If all full nodes have upgraded, or very close to all nodes, there should be no practical effect. If a significant number of full nodes have not upgraded, they will continue using a different block chain than the upgraded users, with the following consequences: | If all full nodes have upgraded, or very close to all nodes, there should be no practical effect. If a significant number of full nodes have not upgraded, they will continue using a different block chain than the upgraded users, with the following consequences: | ||
Line 264: | Line 278: | ||
Pieter Wuille, 18 June 2015</ref> | Pieter Wuille, 18 June 2015</ref> | ||
If Garzik's proposal gains strong consensus, it will as safe as possible | If Garzik's proposal gains strong consensus, it will be as safe as possible | ||
for a hard fork and it will allow increasing the block size up to 32 MB | for a hard fork and it will allow increasing the block size up to 32 MB | ||
without any additional risk of a fork. | without any additional risk of a fork. | ||
Line 300: | Line 314: | ||
Using the numbers from the Lightning presentation<ref name="lightning_slides">[http://lightning.network/lightning-network.pdf Lightning presentation], Joseph Poon, 23 February 2015</ref>, if people open and close a plausible two channels a year, Lightning can support about 52 million users with the current one-megabyte limit—and each user can make an unlimited number of transactions. Currently, assuming people make an average of only two Bitcoin transactions a day, basic Bitcoin can support only 288,000 users. | Using the numbers from the Lightning presentation<ref name="lightning_slides">[http://lightning.network/lightning-network.pdf Lightning presentation], Joseph Poon, 23 February 2015</ref>, if people open and close a plausible two channels a year, Lightning can support about 52 million users with the current one-megabyte limit—and each user can make an unlimited number of transactions. Currently, assuming people make an average of only two Bitcoin transactions a day, basic Bitcoin can support only 288,000 users. | ||
Under these assumptions, Lightning is 180 times ( | Under these assumptions, Lightning is 180 times (17,900%) more efficient than basic Bitcoin. | ||
Presumably, if around 30 million people are using Bitcoin via Lightning so that capacity is called into question, it will not be difficult to find consensus to double the block size to two megabytes and bring user capacity up to 105 million. The same goes for later increases to 200 million (4MB), 400 million (8MB), 800 million (16MB), 1.6 billion (32 MB), 3.2 billion (64MB), and all 7 billion living humans (133MB). In each case, all Lightning network participants get unlimited transactions to the other participants. | Presumably, if around 30 million people are using Bitcoin via Lightning so that capacity is called into question, it will not be difficult to find consensus to double the block size to two megabytes and bring user capacity up to 105 million. The same goes for later increases to 200 million (4MB), 400 million (8MB), 800 million (16MB), 1.6 billion (32 MB), 3.2 billion (64MB), and all 7 billion living humans (133MB). In each case, all Lightning network participants get unlimited transactions to the other participants. |
Latest revision as of 00:36, 16 July 2021
Answers to commonly-asked questions and concerns about scaling Bitcoin, including “level 1” solutions such as increasing the block size and “level 2” solutions such as the proposed Lightning network.
Background
Questions about how Bitcoin currently works (related to scaling) as well as questions about the technical terminology related to the scaling discussion.
What is the short history of the block size limit?
Bitcoin was initially released with a dynamic block "change" limit equivalent to approximately 500kB worth of normal transactions; there was a "hard" size limit of 32 MiB, but it was effectively impractical to hit unless one crafted a spam block.
Around 15 July 2010, Satoshi Nakamoto changed Bitcoin’s mining code so that it would never create any blocks larger than 990,000 bytes.[1]
Two months later on 7 September 2010, Nakamoto changed Bitcoin’s consensus rules to reject blocks larger than 1,000,000 bytes (1 megabyte) if their block height was higher than 79,400.[2] (Block 79,400 was later produced on 12 September 2010.[3]) However, the effective ~500kB limit remained in place.
Neither the July nor the September commit message explains the reason for the limit, although the careful attempt to prevent a fork may indicate Nakamoto didn’t consider it an emergency.
Nakamoto’s subsequent warning not to run a blocksize increase patch which would have destructively forked the network said that if users needed to increase the blocksize, it could be accomplished with a planned hardfork at a future time[4], but he never specified which conditions might require one.
Statements by Nakamoto in the summer of 2010 indicate he believed Bitcoin could scale to block sizes far larger than 1 megabyte. For example, on 5 August 2010, he wrote that "[W]hatever size micropayments you need will eventually be practical. I think in 5 or 10 years, the bandwidth and storage will seem trivial" and "[microtransactions] can become more practical ... if I implement client-only mode and the number of network nodes consolidates into a smaller number of professional server farms" [5]. Satoshi's reasoning was likely based on his belief that SPV would be a primary scaling mechanism. It's since been discovered that SPV security requires strong fraud proofs and the fallback requires full node logic. [6][7]
In one of Nakamoto's final public messages, he wrote that "Bitcoin users might get increasingly tyrannical about limiting the size of the chain so it's easy for lots of users and small devices."[8] This statement suggests that Nakamoto was aware of the value of limiting the block size and that he correctly predicted it was an issue that Bitcoin users would be dealing with in the future---a future Nakamoto may have known would not include him. This was in the context of Satoshi attempting to redirect to a subchain non-Bitcoin uses like DNS data which used direct on-chain block space.
What is this Transactions Per Second (TPS) limit?
The current block size limit is 1,000,000 bytes (1 megabyte)[9], although a small amount of that space (such as the block header) is not available to store transactions.[10]
Bitcoin transactions vary in size depending on multiple factors, such as whether they’re spending single-signature inputs or a multiple signature inputs, and how many inputs and outputs they have.
The simple way to calculate the number of Transactions Per Second (TPS) Bitcoin can handle is to divide the block size limit by the expected average size of transactions, divided by the average number of seconds between blocks (600). For example, if you assume average transactions are 250 bytes,
6.6 TPS = 1,000,000 / 250 / 600
There seems to be general agreement that Bitcoin in 2015 can handle about 3 TPS with the current average size of transactions.[11][12]
Both old estimates[13] and newer estimates[14] placed the pre-segwit theoretical transaction speed at about 7 TPS.
What do devs mean by the scaling expressions O(1), O(n), O(n2), etc…?
Big O notation is a shorthand used by computer scientists to describe how well a system scales. Such descriptions are rough approximations meant to help predict potential problems and evaluate potential solutions; they are not usually expected to fully capture all variables.
- O(1) means a system has roughly the same properties no matter how big it gets.
- O(n) means that a system scales linearly: doubling the number of things (users, transactions, etc.) doubles the amount of work.
- O(n2) means that a system scales quadratically : doubling (2x) the number of things quadruples (4x) the amount of work. Often written O(n^2) is places without convenient superscripts.
- Additional examples may be found in the Wikipedia article
The following subsections show cases where big O notation has been applied to the scaling Bitcoin transaction volume.
O(1) block propagation
Bitcoin Core relays unconfirmed transactions and then later reconstructs blocks when they are mined while avoiding redundant transaction propagation. The prior transaction double-relay redundancy was eliminated with the invention of a novel block propagation algorithm called Compact Blocks [15] to allow miners to propagate large blocks very quickly to active network nodes and also significantly reduced miners' need for powerful bandwidth. Miners use a network[16] that is designed for high-bandwidth Wirehair-based[17] which is slightly faster still than stock Bitcoin Core.
O(n2) network total validation resource requirements with decentralization level held constant
While the validation effort required per full node simply grows in O(n), the combined validation effort of all nodes grows by O(n2) 'in the domain of resource consumption, decentralization being held constant. For a single node, it takes twice as many resources to process the transactions of twice as many users, while for all nodes it takes a combined four times as many CPU resources to process the transactions of twice as many users assuming the number of full nodes increases in proportion to the number of users.
Each on-chain Bitcoin transaction needs to be processed by each full node. If we assume that a certain percentage of users run full nodes (n) and that each user creates a certain number of transactions on average (n again), then the network’s total resource requirements are n2 = n * n
. In short, this means that the aggregate cost of handling all transactions on-chain quadruples each time the number of users doubles.[18] For example,
- Imagine a network starts with 100 users and 2 total nodes (a 2% ratio).
- The network doubles in users to 200. The number of nodes also doubles to 4 (maintaining a 2% ratio of decentralized low-trust security). However, double the number of users means double the number of transactions, and each node needs to process every transaction---so each node now does double work with its bandwidth and its CPU. With double the number of nodes and double the amount of work per node, total work increases by four times.
- The network doubles again: 400 users, 8 nodes (still 2% of total), and 4 times the original workload per node for a total of 16 times as much aggregate work as the original.
- Another doubling: 800 users, 16 nodes (still 2%), and 8 times the original workload per node for a total of 64 times aggregate work as the original.
- Another doubling: 1,600 users—sixteen times the original---with 32 nodes (still 2%), and 16 times the original workload per node. The aggregate total work done by nodes is 256 times higher than it was originally.
Criticisms
Emphasis on the total validation resource requirements is suggested by some individuals to be misleading, as they claim it obfuscates the growth of the supporting base of full nodes that the total validation resource effort is split amongst. The validation resource effort made by each individual full node increases at O(n), and critics say this is the only pertinent fact with respect to scaling. Some critics also point out that the O(n2) network total validation resource requirements claim is assuming decentralization must be held constant as the network scales, and that this is not a founding principle of Bitcoin. Of course, as quoted above, Satoshi knew that decentralization would be a critical value of the safety of the system in the first place by demonstrating he could foresee multiple paths for the future of Bitcoin.
What’s the difference between on-chain and off-chain transactions?
On-chain Bitcoin transactions are those that appear in the Bitcoin block chain. Off-chain transactions are those that transfer ownership of bitcoins without putting a transaction on the block chain.
Common and proposed off-chain transactions include:
- Exchange transactions: when you buy or sell bitcoins, the exchange tracks ownership in a database without putting data in the block chain. Only when you deposit or withdraw bitcoins does the transaction appear on the block chain.
- Web wallet internal payments: many web wallets allow users of the service to pay other users of the same service using off-chain payments. For example, when one Coinbase user pays another Coinbase user. WARNING: Web wallets are extremely unsafe and many users have had funds stolen from them worth many hundreds of millions of dollars in today's exchange rates.
- Tipping services: most tipping services todayvuse off-chain transactions for everything except deposits and withdrawals from their services.
- Payment channels: channels are started using one on-chain transaction and ended using a second on-chain transaction.[19] In between can be an essentially unlimited number of off-chain transactions as small as a single satoshi (1/100,000,000th of a bitcoin). Payment channels include those that were baked into the protocol at release as well as proposed hub-and-spoke channels and the more advanced Lightning network.
The vast majority of all bitcoin-denominated payments today are made off-chain.[20]
What is a fee market?
When you create a Bitcoin transaction, you have the option to pay a transaction fee. If your software is flexible, you can pay anything from zero fees (a free transaction) to 100% of the value of the transaction (spend-to-fees). This fee is comparable to a tip. The higher it is, the bigger the incentive of the miners to incorporate your transaction into the next block.
When miners assemble a block, they are free to include whatever transactions they wish. They usually include as many as possible up to the maximum block size and then prioritize the transactions that pay the most fees per kilobyte of data.[21] This usually results in higher-fee transactions confirming before lower-fee transactions. If blocks become full on a regular basis, users who pay a fee that's too small will have to wait a longer and longer time for their transactions to confirm. At this point, a demand-driven fee market will arise where transactions that pay higher fees get confirmed significantly faster than transactions that pay low fees.[22][14]
In a competitive market, prices are driven by supply and demand and the emerging equilibrium price asymptotically approaches marginal costs over time. However, the current version of segwit-enabled Bitcoin limits the supply to 4 MW (4 million weight units) per block, allowing only demand to adjust. It turns out that a fee market can stabilize as long as there is a block size limit.[23] Simply allowing individual miners to include as many transactions as they want is problematic due to the externalities[24] of including additional blocks and explained further in section 'should miners be allowed to decide the block size?' .
The naive way to scale Bitcoin
If Bitcoin decentralization is abandoned, whatever is left could scale limitlessly as a plain database-backed ledger. Mining uses enormous amounts of electricity to provide a secure, verifiable decentralized ledger and full nodes use an enormous amount of bandwidth and CPU time keeping miners honest.
To wit, if users instead decided to hand authority over to someone they trusted, mining and keeping miners honest wouldn’t be necessary. This is how Visa, MasterCard, PayPal, and the rest of the centralized payment systems works—users trust them, and they have no special difficulty scaling to millions of transactions an hour. It’s comparatively energy-efficient; it isn’t decentralized, and thus isn't efficient to the purpose that Bitcoin was designed to accommodate.
What is a hard fork, and how does it differ from other types of forks?
The word fork is badly overloaded in Bitcoin development. The following terms are used, for example:
- Hard fork: a change to the system which is not backwards compatible. Everyone needs to upgrade or things will go wrong.
- Soft fork: a change to the system which is backwards compatible as long as a majority of miners enforce it. Full nodes that don’t upgrade potentially but not necessarily have reduced security.
- Chain fork: when there are two are more blocks at the same height on a block chain. This typically happens a few times a week by accident, but it can also indicate more severe problems.
- Software fork: using the code from an open source project to create a different project.
- Git/GitHub fork: a way for developers to write and test new features before contributing them to a project.
(There are other types of software-related forks, but they don’t tend to cause as much confusion.)
What are the block size soft limits?
Bitcoin Core has historically come pre-configured to limit blocks it mines to certain sizes.[21] These “limits” aren’t restrictions placed on miners by other miners or nodes, but rather configuration options that help miners produce reasonably-sized blocks.
Miners can change their soft limit size (up to 4 MW) using -blockmaxweight=<size>
Default soft limits at various times:
- July 2012:
-blockmaxsize
option created and set to default 250 KB soft limit[25] - November 2013: Raised to 750KB[26]
- June 2015: Raise to 1 MB suggested[27]
- August 2017: Segwit activates at block 481824[28]
General Block Size Increase Theory
Why are some people in favour of keeping the block size conservative forever?
It is commonly claimed[14][29] that there are people opposed to ever raising the maximum block size limit, but no Bitcoin developers have suggested a permanent blocksize.[12]
All developers support raising the maximum size at some point[30][31]—they just disagree about whether now is the correct time.
Should miners be allowed to decide the block size?
A block may include as little as a single transaction, so miners can always restrict block size. Letting miners choose the maximum block size is more problematic for several reasons:
- Miners profit, others pay the cost: bigger blocks earn miners more fees, but technically miners can successfully mine without storing block data almost at all.[32] Some miners will SPV mine optimistically on other miners' blocks without even validating anyting. Other users who want full validation security, or who provide services to lightweight wallets, pay the costs of downloading and storing those larger blocks.
- Bigger miners can afford more bandwidth: each miner needs to download every transaction that they want to include in blocks,[33] which means that they all need high-speed connections. However, a miner with 8.3% of total hash rate can take that cost out of the about 300 BTC they make a day, while a miner with only 0.7% of total hash rate has to take that cost out of only 25 BTC a day. This means bigger miners may logically choose to make bigger blocks even though it may further centralize mining.
- Centralized hardware production: only a few companies in the world produce efficient mining equipment, and many of them have historically chosen to stop selling to the public.[34] This prevents ordinary Bitcoin users from participating in the process even if they were willing to pay for mining equipment.
- Voting by hash rate favours larger miners: voting based on hash rate allows the current hash rate majority (or super-majority) to enforce conditions on the minority. This is similar to a lobby of large businesses being able to get laws passed that may hurt smaller businesses.[35] This is less of an issue now that miner-signalled activation has been abandoned.
- Miners do not care about users: this was demonstrated during the July 2015 Forks where even after miners lost over $50,000 USD in income from mining invalid chains, and even after they were told that long forks harm users, they continued to mine invalid blocks.[36]
However, there are also possible justifications for letting miners choose the block size:
- Authenticated participants: miners are active participants in Bitcoin, and they can prove it by adding data to the blocks they create.[37] It is much harder to prove people on Reddit, BitcoinTalk, and elsewhere are real people with an active stake in Bitcoin.
- Bigger blocks may cost smaller miners: small miners and poorly-connected miners are at a disadvantage if larger blocks are produced[38], and even large and well-connected miners may find that large blocks reduce their profit percentage if the cost of bandwidth rises too high or their stale rate increases.[39]
How could a block size increase affect user security?
Bitcoin’s security is highly dependent on the number of active users who protect their bitcoins with a full node like Bitcoin Core. The more active users of full nodes, the harder it is for miners to trick users into accepting fake bitcoins or other frauds.[40]
Full nodes need to download and verify every block, and most nodes also store blocks plus relay transactions, blocks, and filtered blocks to other users on the network. The bigger blocks become, the more difficult it becomes to do all this, so it is expected that bigger blocks will both reduce the number of users who currently run full nodes and suppress the number of users who decide to start running a full node later.[41]
In addition, full blocks may increase mining centralization[38] at a time when mining is already so centralized that it makes it easy to reverse transactions which have been confirmed multiple times.[42]
How could larger blocks affect proof-of-work (POW) security?
Proof of work security is dependent on how much money miners spend on mining equipment. However, to effectively mine blocks, miners also need to spend money on bandwidth to receive new transactions and blocks created by other miners; CPUs to validate received transactions and blocks; and bandwidth to upload new blocks. These additional costs don’t directly contribute to POW security.[33]
As block sizes increase, the amount of bandwidth and CPU required also increases. If block sizes increase faster than the costs of bandwidth and CPU decrease, miners will have less money for POW security relative to the gross income they earn.
In addition, larger blocks have a higher risk of becoming stale (orphaned)[43], which directly correlates to lost POW security. For example, if the network average stale rate is 10%, then 10% of POW performed isn't protecting transactions on the block chain.
What happens if blocks aren't big enough to include all pending transactions?
This is already often the case, so we know that miners will simply queue transactions. The transactions eligible for block inclusion that pay the highest fee per kiloweight will be confirmed earlier than transactions that pay comparatively lower fees.[21]
What happens from there is debated:
- BitcoinJ lead developer Mike Hearn believed in 2015 there would be “crashing nodes, a swelling transaction backlog, a sudden spike in double spending, [and] skyrocketing fees.”[44] Clearly this didn't happen, as of late 2020.
- Bitcoin Foundation chief scientist Gavin Andresen believed in 2015 that the “average transaction fee paid will rise, people or applications unwilling or unable to pay the rising fees will stop submitting transactions, [and] people and businesses will shelve plans to use Bitcoin, stunting growth and adoption.” [45] Clearly, this didn't happen either, as of late 2020.
- Bitcoin Core developer Pieter Wuille replied in 2015 to Andresen’s comment above: “Is it fair to summarize this as ‘Some use cases won’t fit any more, people will decide to no longer use the blockchain for these purposes, and the fees will adapt.’? I think that is already happening […] I don’t think we should be afraid of this change or try to stop it.” [46]
Segregated Witness
Segregated Witness, a block size increase proposal implemented as a soft-fork and which increased the block size to 4 megabytes (actually 4 mega-weight units but strictly-speaking that is contained within 4 megabytes maximum) was adopted as the Bitcoin blocksize increase in August 2017 in block 481824[47]. It was most favoured by the user community and the Bitcoin developers, in spite of significant political pressure against it by signalling miners and attackers. Segwit was eventually forced into activation by James Hilliard's BIP0091[48]
Specific Scaling Proposals (for historical interest)
Andresen-Hearn Block Size Increase Proposals
Questions about the series of related proposals by Gavin Andresen and Mike Hearn to use a hard fork to raise the maximum block size, thereby allowing miners to include more on-chain transactions. As of July 2015, the current focus was BIP101.
What was the major advantage of this proposal?
Bitcoin can support many more users. Assuming earliest possible adoption, 250 bytes per on-chain transaction, 144 blocks per day, and 2 transaction a day per user, Bitcoin can support about,
- 288,000 users in 2015
- 2,304,000 users in 2016 (800% increase)
- 4,608,000 in 2018 (1,600%)
- 9,216,000 in 2020 (3,200%)
- 18,432,000 in 2022 (6,400%)
- 36,864,000 in 2024 (12,800%)
- 73,728,000 in 2026 (25,600%)
- 147,456,000 in 2028 (51,200%)
- 294,912,000 in 2030 (102,400%)
- 589,824,000 in 2032 (204,800%)
- 1,179,648,000 in 2034 (409,600%
- 2,359,296,000 in 2036 (819,200%)
What is the major disadvantage of this proposal?
The total cost of remaining decentralized will be high. Assuming earliest possible adoption and that the ratio of total users to full node operators remains the same as today, the total estimated amount of work necessary to maintain the decentralized system under the O(2) scaling model will be,
- 100% of today’s work in 2015
- 6,400% in 2016
- 25,600% in 2018
- 102,400% in 2020
- 409,600% in 2022
- 1,638,400% in 2024
- 6,553,600% in 2026
- 26,214,400% in 2028
- 104,857,600% in 2030
- 419,430,400% in 2032
- 1,677,721,600% in 2034
- 6,710,886,400% in 2036
For example, if the total amount spent on running full nodes today (not counting mining) is $100 thousand per year, the estimated cost for the same level of decentralization in 2036 will be $6.7 trillion per year if validation costs stay the same. (They'll certainly go down, but algorithmic and hardware improvements probably won't eliminate an approximate 6,710,886,400% cost increase.)
What are the dangers of the proposed hard fork?
Under the then-current current proposal, 75% of miners must indicate that they agree to accept blocks up to 8 MB in size.[49] After the change goes into effect, if any miner creates a block larger than 1 MB, all nodes which have not upgraded yet will reject the block.[50]
If all full nodes have upgraded, or very close to all nodes, there should be no practical effect. If a significant number of full nodes have not upgraded, they will continue using a different block chain than the upgraded users, with the following consequences:
- Bitcoins received from before the fork can be spent twice, once on both sides of the fork. This creates a high double spend risk.
- Bitcoins received after the fork are only guaranteed to be spendable on the side of the fork they were received on. This means some users will have to lose money to restore Bitcoin to a single chain.
Users of hosted wallets (Coinbase, BlockChain.info, etc.) will use whatever block chain their host chooses. Users of lightweight P2P SPV wallets will use the longest chain they know of, which may not be the actual longest chain if they only connect to nodes on the shorter chain. Users of non-P2P SPV wallets, like Electrum, will use whatever chain is used by the server they connect to.
If a bad hard fork like this happens, it will likely cause large-scale confusion and make Bitcoin very hard to use until the situation is resolved.
What is the deployment schedule for BIP 101?
- The patch needs to be accepted into Bitcoin Core, Bitcoin XT, or both.
- If accepted into Bitcoin Core, miners using that software will need to upgrade (this probably requires a new released version of Bitcoin Core). If accepted only into Bitcoin XT, miners will need to switch to using XT.
- After 75% of miners have upgraded, 8MB blocks will become allowed at the the latest of (1) 11 January 2016 or (2) two weeks after the 75% upgrade threshold.[49]
- After the effective date has passed, any miner may create a block larger than 8MB. When a miner does this, upgraded full nodes will accept that block and non-upgraded full nodes will reject it, forking the network.[49] See hard fork dangers for what his might mean.
Will miners go straight from 1 MB blocks to 8 MB blocks?
That would be highly unlikely. Uploading an 8 MB block currently takes significantly more time than uploading a 1 MB block[43], even with the roughly 25x reduction in size available from use of Matt Corallo’s block relay network.[16]
The longer it takes to upload a block, the higher the risk it will become a stale block—meaning the miner who created it will not receive the block subsidy or transaction fees (currently about 25 BTC per block).
However, if technology like Invertible Bloom Lookup Tables (IBLTs) is deployed and found to work as expected[51], or if mining centralizes further, adding additional transactions may not have a significant enough cost to discourage miners from creating full blocks. In that case, blocks will be filled as soon as there are enough people wanting to make transactions paying the default relay fee[52] of 0.00001000 BTC/kilobyte.
- 20MB block processing (Gavin Andresen) “if we increased the maximum block size to 20 megabytes tomorrow, and every single miner decided to start creating 20MB blocks and there was a sudden increase in the number of transactions on the network to fill up those blocks… the 0.10.0 version of [Bitcoin Core] would run just fine.”[53] (ellipses in original)
- Mining & relay simulation (Gavin Andresen) “if blocks take 20 seconds to propagate, a network with a miner that has 30% of the hashing power will get 30.3% of the blocks.”[54]
- Mining on a home DSL connection (Rusty Russell) “Using the rough formula of 1-exp(-t/600), I would expect orphan rates of 10.5% generating 1MB blocks, and 56.6% with 8MB blocks; that’s a huge cut in expected profits.”[43]
- Mining centralization pressure (Pieter Wuille) “[This] does very clearly show the effects of larger blocks on centralization pressure of the system.”[38]
Garzik Miner Block Size Voting Proposal (AKA BIP100)
Note, BIP 100 is the marketing name for this proposal. No BIP number has yet been publicly requested, and the number assigned is unlikely to be 100.[55]
Questions related to Jeff Garzik's proposal[14] to allow miners to vote on raising and lowering the maximum block size.
What does this proposal do?
- It creates a one-time hard fork that does not automatically change the maximum block size.
- It allows miners to vote to increase or decrease the maximum block size within the range of 1MB to 32MB. These changes are neither hard forks nor soft forks, but simply rules that all nodes should know about after the initial hard fork.
Does it reduce the risk of a hard fork?
Hard forks are most dangerous when they're done without strong consensus.[14][56]
If Garzik's proposal gains strong consensus, it will be as safe as possible for a hard fork and it will allow increasing the block size up to 32 MB without any additional risk of a fork.
Why is it limited to 32 megabytes?
According to the draft BIP, "the 32 MB hard fork is largely coincidental -- a whole network upgrade at 32 MB was likely needed anyway."
Lightning Network
Questions related to using the proposed Lightning network as a way to scale the number of users Bitcoin can support.
How does transaction security differ from that of Bitcoin?
Transactions on the proposed Lightning network use Bitcoin transactions exclusively to transfer bitcoins[57], and so the same security as regular Bitcoin transactions is provided when both a Lightning hub and client are cooperating.
When a Lightning hub or client refuses to cooperate with the other (maybe because they accidentally went offline), the other party can still receive their full 100% bitcoin balance by closing the payment channel—but they must first wait for a defined amount of time mutually chosen by both the hub and client.[58]
If one party waits too long to close the payment channel, the other party may be able to steal bitcoins. However, it is possible to delegate the task of closing the channel in an emergency to an unlimited number of hubs around the Internet, so closing the channel on time should be easy.[59]
In summary, provided that payment channels are closed on time, the worst that can happen to users is that they may have to wait a few weeks for a channel to fully close before spending the bitcoins they received from it. Their security is otherwise the same as with regular Bitcoin.
For Lightning hubs, security is slightly worse in the sense that they need to keep a potentially-large amount of funds in a hot wallet[60], so they can’t take advantage of the anti-hacking protection offered by cold wallets. Otherwise, Lightning hubs have the same security as clients.
When will Lightning be ready for use?
Lightning requires a basic test implementation (work underway[61]), several soft-fork changes to the Bitcoin protocol (work underway[62][63], and wallets need to be updated to support the Lightning network protocol.[59]
Dr. Adam Back has said, "I expect we can get something running inside a year."[64]
Doesn’t Lightning require bigger blocks anyway?
Lightning is block-size neutral. It requires one on-chain transaction to open a channel between a hub and a client, and one on-chain transaction to close a channel.[57] In between it can support an unlimited number of transactions between anyone on the Lightning network.[60]
Using the numbers from the Lightning presentation[65], if people open and close a plausible two channels a year, Lightning can support about 52 million users with the current one-megabyte limit—and each user can make an unlimited number of transactions. Currently, assuming people make an average of only two Bitcoin transactions a day, basic Bitcoin can support only 288,000 users.
Under these assumptions, Lightning is 180 times (17,900%) more efficient than basic Bitcoin.
Presumably, if around 30 million people are using Bitcoin via Lightning so that capacity is called into question, it will not be difficult to find consensus to double the block size to two megabytes and bring user capacity up to 105 million. The same goes for later increases to 200 million (4MB), 400 million (8MB), 800 million (16MB), 1.6 billion (32 MB), 3.2 billion (64MB), and all 7 billion living humans (133MB). In each case, all Lightning network participants get unlimited transactions to the other participants.
For basic Bitcoin to scale to just 2 transactions a day for 7 billion people would require 24-gigabyte blocks.[65]
Sidechains
Real quick, what are sidechains?
Block chains that are separate from Bitcoin’s block chain, but which allow you to receive and spend bitcoins using a two-way peg (2WP).[66] (Although a sidechain can never be more secure than the Bitcoin block chain.[67])
Are sidechains a scaling option?
No. Sidechains have the same fundamental scaling problems as Bitcoin does. Moving some transactions to sidechains simply moves some problems elsewhere on the network—the total difficulty of the problem remains the same.[68]
But couldn’t I create a sidechain that had 100 GB blocks?
Certainly, sidechain code is open source[69]—so you can create your own sidechain. But you’d still have the difficulty of finding people who are willing to validate 100 GB blocks with their own full nodes.
Do sidechains require a hard fork?
No. Federated sidechains, such as have already been implemented[70], don’t require any changes to the Bitcoin consensus rules. Merged-mined sidechains, which have not been implemented yet, do require a backwards-compatible soft fork in order to transfer funds between Bitcoin and the sidechain.[66]
References
- ↑ Bitcoin commit a30b56e, Satoshi Nakamoto, 15 July 2010
- ↑ Bitcoin commit 8c9479c, Satoshi Nakamoto, 7 September 2010
- ↑ Block height 79400, dated 12 September 2010
- ↑ Increasing the block size if needed, Satoshi Nakamoto, 3 October 2010
- ↑ the number of network nodes consolidates into a smaller number of professional server farms
- ↑ Greg Maxwell's outline of fraud proof logic Greg Maxwell's wiki page, 22 March, 2013
- ↑ Greg Maxwell's early post on fraud proofs Greg Maxwell on fraud proofs, 30 July, 2012
- ↑ BitDNS and Generalizing Bitcoin, Satoshi Nakamoto, 10 December 2010
- ↑
MAX_BLOCK_SIZE
, retrieved 2 July 2015 - ↑ Block serialization description, Bitcoin.org Developer Reference
- ↑ Max TPS based on current average transaction size, Mike Hearn, 8 May 2015
- ↑ 12.0 12.1 "No one proposing 3 TPS forever", Gregory Maxwell, 15 June 2015
- ↑ Scalability, Bitcoin Wiki, retrieved 7 July 2015
- ↑ 14.0 14.1 14.2 14.3 14.4 Draft BIP: decentralized economic policy, Jeff Garzik, 12 June 2015 (revised 15 June 2015)
- ↑ Compact Blocks Compact Blocks, BIP0152
- ↑ 16.0 16.1 Matt Corallo's block relay network
- ↑ Wirehair Github Repository Wirehair
- ↑ Total system cost, Dr. Adam Back, 28 June 2015
- ↑ Micropayment channels, Bitcoin.org Developer Guide
- ↑ Pure off-chain is a weak form of layer 2, Dr. Adam Back, 19 June 2015
- ↑ 21.0 21.1 21.2 Fee prioritization patch, Gavin Andresen, 26 July 2012
- ↑ Comments on the CoinWallet.eu tx-flood stress-test, Peter Todd, 22 June 2015
- ↑ Bitcoin is unstable without a block size limit, Alex B., 24 Oct 2016
- ↑ Definition of externality, Wikipedia, 26 January 2016
- ↑ Bitcoin Core commit c555400, Gavin Andresen, 12 July 2012
- ↑ Bitcoin Core commit ad898b40, Gavin Andresen, 27 November 2013
- ↑ Bitcoin Core pull #6231, Chris Wheeler, 4 June 2015
- ↑ Block 481824 August 23, 2017
- ↑ Epicenter Bitcoin podcast #82, Mike Hearn (interviewed), 8 June 2015
- ↑ Nothing magic about 1MB, Dr. Adam Back, 13 June 2015
- ↑ Regarding size-independent new block propagation, Pieter Wuille, 28 May 2015
- ↑ Bitcoin Core pull #5863 adding pruning functionality, Suhas Daftuar et al., 6 March 2015
- ↑ 33.0 33.1 Correcting wrong assumptions about mining, Gregory Maxwell, 15 June 2015
- ↑ KnCMiner to stop selling to home miners, Wall Street Journal, Yuliya Chernova, 5 September 2014
- ↑ Miners choosing the block size limit is ill-advised, Meni Rosenfeld, 13 June 2015
- ↑ Intention to continue SPV mining, Wang Chun, 4 July 2015
- ↑ Coinbase field, Bitcoin.org Developer Reference
- ↑ 38.0 38.1 38.2 Mining centralization pressure from non-uniform propagation speed, Pieter Wuille, 12 June 2015
- ↑ F2Pool block size concerns, Chun Wang, 29 May 2015
- ↑ Bitcoin nodes: how many is enough?, Jameson Lopp, 7 June 2015
- ↑ Block size increases-only will create problems, Dr. Adam Back, 28 June 2015
- ↑ When to worry about one-confirmation payments, David A. Harding, 17 November 2014
- ↑ 43.0 43.1 43.2 Mining on a home DSL connection test results, Rusty Russell, 19 June 2015
- ↑ Crash Landing, Mike Hearn, 7 May 2015
- ↑ Bitcoin Core pull #6341, Gavin Andresen, 26 June 2015
- ↑ The need for larger blocks, Pieter Wuille, 26 June 2015
- ↑ Block 481824 Block 481824
- ↑ BIP 0091, James Hilliard, May 22, 2017.
- ↑ 49.0 49.1 49.2 BIP101, Gavin Andresen, 24 June 2016
- ↑ Consensus rule changes, Bitcoin.org Developer Guide
- ↑ O(1) block propagation, Gavin Andresen, 8 August 2014
- ↑ Bitcoin Core pull #3305 dropping default fees, Mike Hearn, 22 November 2013
- ↑ Twenty megabyte block testing, Gavin Andresen, 20 January 2015
- ↑ Are bigger blocks better for bigger miners?, Gavin Andresen, 22 May 2015
- ↑ [http://lists.linuxfoundation.org/pipermail/bitcoin-dev/2014-May/005756.html Prenumbered BIP naming], Gregory Maxwell, 12 May 2014
- ↑ Consensus changes are not about making a change to the software, Pieter Wuille, 18 June 2015
- ↑ 57.0 57.1 Lightning Networks: Part I, Rusty Russell, 30 March 2015
- ↑ Lightning Networks: Part II, Rusty Russell, 1 April 2015
- ↑ 59.0 59.1 Lightning Networks: Part IV, Rusty Russell, 8 April 2015
- ↑ 60.0 60.1 Lightning Network (Draft 0.5), Section 6.3: Coin Theft via Hacking, Joseph Poon and Thaddeus Dryja
- ↑ Lightning first steps, Rusty Russell, 13 June 2015
- ↑ BIP65: CheckLockTimeVerify, Peter Todd, 1 October 2014
- ↑ BIP68: Consensus-enforced transaction replacement, Mark Friedenbach, 28 May 2015
- ↑ Reply to Andresen regarding Lightning, Dr. Adam Back, 28 June 2015
- ↑ 65.0 65.1 Lightning presentation, Joseph Poon, 23 February 2015
- ↑ 66.0 66.1 Sidechains paper, Adam Back et al., 22 October 2014
- ↑ Security limitations of pegged chains, Gregory Maxwell, 14 June 2015
- ↑ Sidechains not a direct means for solving any of the scaling problems Bitcoin has, Pieter Wuille, 13 June 2015
- ↑ Elements Project
- ↑ Elements Alpha