Latest revision as of 18:35, 28 January 2020

This article was written in March 2016 to clear up misconceptions common in the bitcoin community at the time.

Myth: There are only about 5500 full nodes worldwide

This number comes from sites like this and it measured by trying to connect to every nodes on their open ports.

Problem is, not all nodes actually have open ports that can be probed. Either because they are behind firewalls or because their users have configured them to not listen for connections.

Nobody knows exactly how many full nodes there are, since many people don't know how to forward ports behind a firewall, and bandwidth can be costly, its quite likely that the number of nodes with closed ports is at least another several thousand.

Nodes with open ports are able to upload blocks to new full nodes. In all other ways they are the same as nodes with closed ports. But because open-port-nodes can be measured and closed-port-nodes cannot, some members of the bitcoin community have been mistaken into believing that open-port-nodes represent the full count.

Very roughly estimating the total node count

(Original research by theymos on June 6, 2016 18:30 UTC.)

According to getaddr.bitnodes.io, there are 5834 listening full nodes. Assuming they're all using default Bitcoin Core settings, they'll each provide 117 connection slots to the network on net (125 provided minus 8 consumed). SPV nodes typically use 4 connection slots, and full nodes typically use 8. So the network can support a maximum of around 170,644 SPV nodes or 85,322 non-listening full nodes at one time. This is roughly the upper limit for the number of wallets that are online and connected to the Bitcoin network at any one time. (If there were more people online at once than that, people would start seeing various issues.) This doesn't include wallets that don't actually connect to the Bitcoin network, of course.

If I look at my long-running listening full node, I currently have incoming connections from 40 full nodes and 10 SPV nodes via the Internet, and 11 full nodes via my Tor hidden service. So if I ignore the hidden service connections because they are somewhat unusual, and I assume that my node is typical otherwise, it seems reasonable to make a very rough guess that there are at this moment (40*5834)/8 = 29,170 full nodes (5834 listening) and (10*5834)/4 = 14,585 lightweight nodes connected to the network.

Luke's script

Luke-jr has a script which attempts to count all full nodes: http://luke.dashjr.org/programs/bitcoin/files/charts/software.html

Myth: The number of these nodes matters and/or is too low.

Nodes with open ports are useful to the bitcoin network because they help bootstrap new nodes by uploading historical blocks -- they are a measure of the number of redundant copies of the blockchain available for synchronizing with. There has been no shortage of bandwidth capacity for simply syncing wallets from the available nodes, and if there were, bandwidth might be added by renting cloud servers.

Trust, security, and privacy are what matters right now. Full nodes are able to check that all of bitcoin's rules are being followed. Rules like following the inflation schedule, no spending the same coin twice, no spending of coins that don't belong to the holder of the private key and all the other rules required to make bitcoin work (e.g. difficulty)

Full nodes are required to make bitcoin trustless. No longer do you have to trust a financial institution like a bank or paypal, you can simply run software on your own computer. To put simply, the only node that matters is the one you use.

Myth: There is no incentive to run nodes so the network relies on altruism

It is very much in the individual bitcoin's users rational self interest to run a full node and use it as their wallet.

Trustless

Petertodd alt text. — An example of a lightweight client happily accepting a fake transaction. If the attacker had some hashpower then this transaction could be made to have confirmations too.

Running a full node as your wallet is the only way to know for sure that none of bitcoin's rules have been broken. Rules like no coins were spent not belonging to the owner, that no coins were spent twice, that no inflation happens outside of the schedule and that all the rules needed to make the system work are followed (e.g. Difficulty.) All other kinds of wallet involve trusting a third party server.

If bitcoin is digital gold, then a full node wallet is your own personal goldsmith who checks for you that received payments are genuine. You wouldn't accept a cash banknote or gold coin without checking it is genuine, the same applies for bitcoin.

Security

All these checks done by full nodes also increase the security. There are many attacks possible against lightweight wallets that do not affect full node wallets.

This is not just mindless paranoia, there have been real world examples where full node users were unaffected by turmoil in the rest of the bitcoin ecosystem. The 4th July 2015 accidental chain fork effected many kinds of wallets. Here is the wiki page on this event July_2015_chain_forks#Wallet_Advice

Notice how node software updated months ago was completely unaffected by the fork. All other wallets required either extra confirmations or checking that the third-party institution was running the correct version. A bitcoin business, like an exchange, marketplace or online store should always use a full node for security.

Privacy

Full nodes wallets are also currently the most private way to use Bitcoin, with nobody else learning which bitcoin addresses belong to you. All other lightweight wallets leak information about which addresses are yours because they must query third-party servers. The Electrum servers will know which addresses belong to you and can link them together. Despite bloom filtering, lightweight wallets based on BitcoinJ do not provide much privacy against nodes who connected directly to the wallet or wiretappers.^[1]

For some use cases, such privacy may not be required. But an important reason to run a full node and use it as a wallet is to get the full privacy benefits.

Myth: I can just set up a node on a cloud server instance and leave it

To get the benefits of running a full node, you must use it as your wallet, preferably on hardware you control.

Most people who do this do not use a full node as their wallet. Unfortunately because Bitcoin has a similar name to Bittorrent, some people believe that upload capacity is the most important thing for a healthy network. As I've explained above: bandwidth and connections are not a problem today; trust, security and privacy are.

Myth: Running a full node is not recommended, most people should use a lightweight client

This was common advice in 2012, but since then the full node software has vastly improved in terms of user experience.

If you cannot spare the disk space to store the blockchain, you can enable pruning. Altogether this should require less than 900MB of hard disk space.

If you cannot spare the bandwidth to upload blocks to other nodes, there are number of options to reduce or eliminate the bandwidth requirement. These include limiting connections, bandwidth targeting and disabling listening. There is also the option -blocksonly, where the node will not download unconfirmed transaction and only download new blocks. This more than halves the bandwidth usage at the expense of not seeing unconfirmed transactions.

Synchronizing the blockchain for a new node has improved since 2012 too. Features like [1] and libsecp256k1 have greatly improved the initial synchronization time.

It can be further improved by setting -dbcache=3000 which keeps more of the UTXO set in memory. It reduces the amount of time reading from disk and therefore speeds up synchronization. Tests showed that the entire blockchain can now be synchronized in less than **3 and a half hours** (Note that you'll need Bitcoin Core 0.12 or later to get all these efficiency improvements) Another example with 2h 25m

How to run a full node as your wallet

I think every moderate user of bitcoin would benefit by running a full node and using it as their wallet. There are several ways to do this.

Run a bitcoin-qt full node.

Use wallet software that is backed by a full node (e.g. Armory, JoinMarket

Use a lightweight wallet that connects only to your full node (e.g. GreenAddress connecting only to your node running at home, Electrum connecting only to your own Electrum server)
- Electrum Personal Server is designed specifically for this use case.

So what are you waiting for? The benefits are many, the downsides are not that bad. The more people do this, the more robust and healthy the bitcoin ecosystem is.

References

↑ http://jonasnick.github.io/blog/2015/02/12/privacy-in-bitcoinj/ Privacy in BitcoinJ

[1] ttp://jonasnick.github.io/blog/2015/02/12/privacy-in-bitcoinj/ Privacy in BitcoinJ

[1]

@@ Line 1: / Line 1: @@
 This article was written in March 2016 to clear up misconceptions common in the bitcoin community at the time.
-It's time to clear up some misconceptions floating around about full nodes.
 =Myth: There are only about 5500 [[Full node|full nodes]] worldwide=
@@ Line 9: / Line 7: @@
 Problem is, not all nodes actually have open ports that can be probed. Either because they are behind firewalls or because their users have configured them to not listen for connections.
-Nobody knows how many [[full node|full nodes]] there are, since many people don't know how to forward ports behind a firewall, and bandwidth can be costly, its quite likely that the number of nodes with closed ports is at least another several thousand.
+Nobody knows exactly how many [[full node|full nodes]] there are, since many people don't know how to forward ports behind a firewall, and bandwidth can be costly, its quite likely that the number of nodes with closed ports is at least another several thousand.
+Nodes with open ports are able to upload blocks to new full nodes. In all other ways they are the same as nodes with closed ports. But because open-port-nodes can be measured and closed-port-nodes cannot, some members of the [[bitcoin]] community have been mistaken into believing that open-port-nodes represent the full count.
+====Very roughly estimating the total node count====
+(Original research by theymos on June 6, 2016 18:30 UTC.)
-Nodes with open ports are able to upload blocks to new full nodes. In all other ways they are the same as nodes with closed ports. But because open-port-nodes can be measured and closed-port-nodes cannot, some members of the [[bitcoin]] community have been mistaken into believing that open-port-nodes are that matters.
+According to getaddr.bitnodes.io, there are 5834 listening full nodes. Assuming they're all using default Bitcoin Core settings, they'll each provide 117 connection slots to the network on net (125 provided minus 8 consumed). SPV nodes typically use 4 connection slots, and full nodes typically use 8. So the network can support a maximum of around 170,644 SPV nodes or 85,322 non-listening full nodes at one time. This is roughly the upper limit for the number of wallets that are online and connected to the Bitcoin network at any one time. (If there were more people online at once than that, people would start seeing various issues.) This doesn't include wallets that don't actually connect to the Bitcoin network, of course.
-=Myth: This number of nodes matters and/or is too low.=
+If I look at my long-running listening full node, I currently have incoming connections from 40 full nodes and 10 SPV nodes via the Internet, and 11 full nodes via my Tor hidden service. So if I ignore the hidden service connections because they are somewhat unusual, and I assume that my node is typical otherwise, it seems reasonable to make a very rough guess that there are at this moment (40*5834)/8 = 29,170 full nodes (5834 listening) and (10*5834)/4 = 14,585 lightweight nodes connected to the network.
-Nodes with open ports are useful to the [[Network|bitcoin network]] because they help [[Network#Initial_block_download|bootstrap new nodes]] by uploading historical blocks, they are a measure of bandwidth capacity. Right now there is no shortage of bandwidth capacity, and if there was it could be easily added by renting cloud servers.
+====Luke's script====
-The problem is not bandwidth or connections, but trust, security and privacy. Let me explain.
+Luke-jr has a script which attempts to count all full nodes: http://luke.dashjr.org/programs/bitcoin/files/charts/software.html
-[[full node|Full nodes]] are able to check that all of bitcoin's rules are being followed. Rules like following the [[Controlled_supply|inflation schedule]], [[Double-spending|no double spending]], no spending of coins that don't belong to the holder of the private key and all the other rules required to make [[bitcoin]] work (e.g. difficulty)
+=Myth: The number of these nodes matters and/or is too low.=
-Full nodes are what make [[bitcoin]] trustless. No longer do you have to trust a financial institution like a bank or paypal, you can simply run software on your own computer. ''To put simply, the only node that matters is the one you use''
+Nodes with open ports are useful to the [[Network|bitcoin network]] because they help [[Network#Initial_block_download|bootstrap new nodes]] by uploading historical blocks -- they are a measure of the number of redundant copies of the blockchain available for synchronizing with. There has been no shortage of bandwidth capacity for simply syncing wallets from the available nodes, and if there were, bandwidth might be added by renting cloud servers.
-=Myth: There is no incentive to run nodes, the network relies on altruism=
+Trust, security, and privacy are what matters right now. [[full node|Full nodes]] are able to check that all of bitcoin's rules are being followed. Rules like following the [[Controlled_supply|inflation schedule]], no spending the same coin twice, no spending of coins that don't belong to the holder of the private key and all the other rules required to make [[bitcoin]] work (e.g. [[difficulty]])
+Full nodes are required to make [[bitcoin]] trustless. No longer do you have to trust a financial institution like a bank or paypal, you can simply run software on your own computer. ''To put simply, the only node that matters is the one you use''.
+=Myth: There is no incentive to run nodes so the network relies on altruism=
 It is very much in the individual bitcoin's users rational self interest to run a [[full node]] and use it as their [[wallet]].
-Using a full node as your [[wallet]] is the only way to know for sure that none of bitcoin's rules have been broken. Rules like no coins were spent not belonging to the owner, that no coins were spent twice, that no inflation happens outside of the schedule and that all the rules needed to make the system work are followed  (e.g. [[Difficulty]].) All other kinds of wallet involve trusting a third party server.
+====Trustless====
+[[File:Petertodd-screenshot-mit2016-talk-gullible-lite-clients.png|500px|thumb|alt=Petertodd alt text.|An example of a lightweight client happily accepting a fake transaction. If the attacker had some hashpower then this transaction could be made to have [[confirmation|confirmations]] too.]]
+Running a full node as your [[wallet]] is the only way to know for sure that none of bitcoin's rules have been broken. Rules like no coins were spent not belonging to the owner, that no coins were spent twice, that no inflation happens outside of the schedule and that all the rules needed to make the system work are followed  (e.g. [[Difficulty]].) All other kinds of wallet involve trusting a third party server.
+If bitcoin is digital gold, then a full node wallet is your own personal goldsmith who checks for you that received payments are genuine. You wouldn't accept a cash banknote or gold coin without checking it is genuine, the same applies for bitcoin.
+====Security====
 All these checks done by full nodes also increase the security. There are many attacks possible against lightweight wallets that do not affect full node wallets.
 This is not just mindless paranoia, there have been real world examples where full node users were unaffected by turmoil in the rest of the bitcoin ecosystem. The 4th July 2015 accidental chain fork effected many kinds of wallets. Here is the wiki page on this event [[July_2015_chain_forks#Wallet_Advice]]
-Notice how node software updated months ago was completely unaffected by the fork. All other wallets required either extra [[confirmations]] or checking that the third-party institution was running the correct version. For example a bitcoin business, like an exchange, marketplace or online store should always use a full node for security.
+Notice how node software updated months ago was completely unaffected by the fork. All other wallets required either extra [[confirmation|confirmations]] or checking that the third-party institution was running the correct version. A bitcoin business, like an exchange, marketplace or online store should always use a full node for security.
-Full nodes wallets are also currently the most private way to use Bitcoin, with nobody else learning which bitcoin [[address|addresses]] belong to you. All other [[Thin_Client_Security|lightweight wallets]] leak information about which [[address|addresses]] are yours because they must query third-party servers. The Electrum servers will know which [[address|addresses]] belong to you and can link them together. Despite [[BIP_0037|bloom filtering]], lightweight wallets based on [[BitcoinJ]] do not provide much privacy against nodes who connected directly to the wallet or wiretappers.<ref>http://jonasnick.github.io/blog/2015/02/12/privacy-in-bitcoinj/ Privacy in BitcoinJ</ref>
+====Privacy====
+Full nodes wallets are also currently the most private way to use Bitcoin, with nobody else learning which bitcoin [[address|addresses]] belong to you. All other [[Thin_Client_Security|lightweight wallets]] leak information about which [[address|addresses]] are yours because they must query third-party servers. The Electrum servers will know which [[address|addresses]] belong to you and can link them together. Despite [[BIP_0037|bloom filtering]], lightweight wallets based on [[bitcoinj|BitcoinJ]] do not provide much privacy against nodes who connected directly to the wallet or wiretappers.<ref>http://jonasnick.github.io/blog/2015/02/12/privacy-in-bitcoinj/ Privacy in BitcoinJ</ref>
 For some use cases, such privacy may not be required. But an important reason to run a full node and use it as a wallet is to get the full privacy benefits.
@@ Line 43: / Line 59: @@
 To get the benefits of running a full node, you must use it as your wallet, preferably on hardware you control.
-Most people who do this do not use a full node as their wallet. Unfortunately because Bitcoin has a similar name to Bittorrent, some people believe that upload capacity is the most important thing for a healthy network. As I've explained above: bandwidth and connections are not a problem today, trust, security and privacy are.
+Most people who do this do not use a full node as their wallet. Unfortunately because Bitcoin has a similar name to Bittorrent, some people believe that upload capacity is the most important thing for a healthy network. As I've explained above: bandwidth and connections are not a problem today; trust, security and privacy are.
 =Myth: Running a full node is not recommended, most people should use a lightweight client=
@@ Line 49: / Line 65: @@
 This was common advice in 2012, but since then the full node software has vastly improved in terms of user experience.
-If you cannot spare the disk space to store the [[blockchain]], you can [https://bitcoin.org/en/release/v0.11.0#block-file-pruning enable pruning]. In Bitcoin Core 0.12, pruning being enabled will leave the wallet enabled. Altogether this should require less than 900MB of hard disk space.
+If you cannot spare the disk space to store the [[blockchain]], you can [https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.12.0.md#wallet-pruning enable pruning]. Altogether this should require less than 900MB of hard disk space.
-If you cannot spare the bandwidth to upload blocks to other nodes, [https://bitcoin.org/en/full-node#reduce-traffic there are number of options to reduce or eliminate the bandwidth requirement]. These include limiting connections, bandwidth targeting and disabling listening. Bitcoin Core 0.12 has the new option [https://bitcointalk.org/index.php?topic=1377345.0 -blocksonly], where the node will not download unconfirmed transaction and only download new blocks. This more than halves the bandwidth usage at the expense of not seeing unconfirmed transactions.
+If you cannot spare the bandwidth to upload blocks to other nodes, [https://btcinformation.org/en/full-node#reduce-traffic there are number of options to reduce or eliminate the bandwidth requirement]. These include limiting connections, bandwidth targeting and disabling listening. There is also the option [https://bitcointalk.org/index.php?topic=1377345.0 -blocksonly], where the node will not download unconfirmed transaction and only download new blocks. This more than halves the bandwidth usage at the expense of not seeing unconfirmed transactions.
-[[Network#Initial_block_download|Synchronizing the blockchain]] for a new node has improved since 2012 too. Features like [https://bitcoin.org/en/release/v0.10.0#faster-synchronization headers-first] and libsecp256k1 have greatly improved the initial synchronization time.
+[[Network#Initial_block_download|Synchronizing the blockchain]] for a new node has improved since 2012 too. Features like [https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.10.0.md#faster-synchronization] and libsecp256k1 have greatly improved the initial synchronization time.
 It can be further improved by setting -dbcache=3000 which keeps more of the [[Data_directory#chainstate_subdirectory|UTXO set]] in memory. It reduces the amount of time reading from disk and therefore speeds up synchronization. Tests showed that the entire [[blockchain]] can now be synchronized in [https://github.com/bitcoin/bitcoin/pull/6954#issuecomment-154993958 less than **3 and a half hours**] (Note that you'll need Bitcoin Core 0.12 or later to get all these efficiency improvements) [https://www.botbot.me/freenode/bitcoin-core-dev/2016-04-13/?msg=64033258&page=1 Another example with 2h 25m]
@@ Line 61: / Line 77: @@
 I think every moderate user of bitcoin would benefit by running a full node and using it as their wallet. There are several ways to do this.
-* Run a [https://bitcoin.org/en/download bitcoin-qt full node].
+* Run a [https://bitcoincore.org/en/download/ bitcoin-qt full node].
-* Use wallet software that is backed by a full node (e.g. [[Armory]], [https://github.com/AdamISZ/JMBinary/#jmbinary JoinMarket])
+* Use wallet software that is backed by a full node (e.g. [[Armory]], [[JoinMarket]]
-* Use a lightweight wallet that connects only to your full node (e.g. [[Multibit]] connecting only to your node running at home, [[Electrum]] connecting only to your own Electrum server)
+* Use a lightweight wallet that connects only to your full node (e.g. [[GreenAddress]] connecting only to your node running at home, [[Electrum]] connecting only to your own Electrum server)
+** [[Electrum#Electrum_Personal_Server|Electrum Personal Server]] is designed specifically for this use case.
 So what are you waiting for? The benefits are many, the downsides are not that bad. The more people do this, the more robust and healthy the bitcoin ecosystem is.
@@ Line 73: / Line 90: @@
 [https://www.reddit.com/r/BitcoinBeginners/comments/3eq3y7/full_node_question/ctk4lnd Pieter Wuille's explaination of full node trustlessness]
+[https://bitcoin.org/en/bitcoin-core/features/validation bitcoin.org's article on validation]
+[https://www.reddit.com/r/Bitcoin/comments/44rx5k/psa_clearing_up_some_misconceptions_about_full/ Reddit comments]
 =References=
@@ Line 78: / Line 99: @@
 [[Category:Technical]]
+[[Category:Educational]]

Clearing Up Misconceptions About Full Nodes: Difference between revisions

Latest revision as of 18:35, 28 January 2020

Contents

Myth: There are only about 5500 full nodes worldwide

Very roughly estimating the total node count

Luke's script

Myth: The number of these nodes matters and/or is too low.

Myth: There is no incentive to run nodes so the network relies on altruism

Trustless

Security

Privacy

Myth: I can just set up a node on a cloud server instance and leave it

Myth: Running a full node is not recommended, most people should use a lightweight client

How to run a full node as your wallet

Further Reading

References

Navigation menu

Clearing Up Misconceptions About Full Nodes: Difference between revisions

Latest revision as of 18:35, 28 January 2020

Myth: There are only about 5500 full nodes worldwide

Very roughly estimating the total node count

Luke's script

Myth: The number of these nodes matters and/or is too low.

Myth: There is no incentive to run nodes so the network relies on altruism

Trustless

Security

Privacy

Myth: I can just set up a node on a cloud server instance and leave it

Myth: Running a full node is not recommended, most people should use a lightweight client

How to run a full node as your wallet

Further Reading

References

Navigation menu

Search